Strategies for Open-Source AI Competitiveness

It's ironic that just last week I was on a debate/panel with James Cham, Pete Skomoroch and others about "Will the next Billon dollar companies be built on closed or open-source AI?” and the strongest argument for closed source models was that savvy builders would build on OpenAI first because it was fastest to get to market and best-in-class terms of performance. (We should probably redo the debate now Alexy Khrabrov?) Jokes aside, with the OpenAI’s turmoil, while GPT-4 remains the best model out there, the instability of the company means that no enterprise or startup can depend solely on OpenAI’s models; everyone must have an open-source plan B, i.e., the new Plan A. Specifically, every team which has so far only built with GPT needs to rethink and diversify their GenAI strategy. And based on my discussions with GenAI builders, this accounts for 70%+ of teams building with GenAI. So if you are in this boat, how do you implement an OSS-based plan B? 1) If not already, get familiar with OSS models ASAP (here’s a very helpful listing: https://lnkd.in/g4SkHZ6V & a Leaderboard: https://chat.lmsys.org/) 2) Deploy up a few different OSS models appropriate for your task 3) Compare performance of selected models on your use case (broad benchmarks are great but not useful for you to pick the one for your use case); use human evals and LLMs 4) Move to an OSS model and build your APIs in such a way that you can switch models whenever needed The last step cannot be understated: models and companies will continue to evolve but a single API allows you to maintain continuity for your business. When we built the Verta GenAI Workbench, our goal was to give GenAI builders the freedom to evaluate and choose any GenAI model, proprietary or open-source, precisely to avoid being wedded to a single model, and so, we baked Steps 2 - 4 into the tool itself. If you are looking to evaluate OSS alternatives in light of the OpenAI news, feel free to reach out, happy to help quickly assess options and formulate an OSS Plan-B.

I often get this question from industry friends and peers - Should we lean on commercial like OpenAI's black-box API or immerse ourselves in open-source LLMs? 🌍🤔 For those in the prototype phase 🛠: **Use OpenAI's API**: - It’s budget-friendly for early-stage projects. 🪙 - A single API key, and voila, integration is complete! 🔑 - This allows for swift product idea validation.✅ However, for long-term visionaries 🚀🌌: **Invest in open-source local LLMs**: - Establish a custom LLM setup within your cloud infrastructure. ☁️ - Focus on curating top-notch datasets that resonate with your business objectives. 📊🎯 Considering this pivot? Here’s why 🤓: 1️⃣ **Optimal Performance & Savings**: Tailored models often surpass giants like GPT4, especially with a specific dataset. They're not only effective but also economically wise. 💡💰 2️⃣ **Guardian of Data**: In our data-driven age, LLMs thrive on premium data. Ensure your data’s privacy by keeping it close. 🛡️🔒 Sending data over third-party channels might expose vulnerabilities. 🚫 3️⃣ **Flexibility in Strategy**: Transitioning back to APIs like OpenAI's is straightforward. Yet, initiating a proprietary LLM can be more complex later on. Hence, current investment paves the way for future adaptability. 🌳🔄 4️⃣ **Customization & Control**: With open-source LLMs, you have the autonomy to tweak and refine models to your heart's content, ensuring they align perfectly with your requirements. 🎛️🔧 5️⃣ **In-House Expertise**: Building in-house capabilities elevates your team's knowledge, making them not just users but experts in LLM technology. 🎓💼 6️⃣ **Future-Proofing**: Technology and regulations are evolving. Having control over your LLM means you can swiftly adapt to changes without waiting for third-party updates. ⏱️📈 7️⃣ **Cost Predictability**: With third-party APIs, costs can surge based on usage. In-house LLMs allow for more predictable budgeting in the long run. 📉💼 Would love to engage in a discussion and get insights from others in the field. Drop your thoughts below! 💭 #llms #languagemodels #openai #genai #deployment #production #datascience #artificialintelligence #largelanguagemodels

When we started to see open source models come out and get more competitive, one of the big potential appeals was that they could be cheaper at scale than OpenAI. That was, of course, dependent on whether you wanted to or were ready to deploy them yourselves. And managing an internal endpoint for an OSS model—even one as good as Mistral’s—is not easiest. What we might not have foreseen, though, was that there would be a proliferation of startups offering serverless endpoints for these performant models that literally require just a handful of lines of code to completely swap out an OpenAI model. And they’re now starting to serve those models. The biggest potential shift is the launch of Mixtral, Mistral’s mixture of experts model that is competitive with GPT-3.5 Turbo. And those endpoints from Anyscale, Perplexity, Together AI, and others can come in as low as half the cost of the input for GPT 3.5-Turbo. Anyscale, one of the lowest, serves Mixtral at $0.50 per 1 million tokens. (GPT-3.5 Turbo is $2.00 per 1 million output.) It turns out that the potential cost savings around those smaller models are indeed very real, and we can see the early signs of it showing up in these endpoints. And we haven’t even gotten to fine tuning, nor an endpoint deployment and GPU management system that is as comically easy to use as OpenAI’s serverless tooling. But it’s starting to show that OSS models are catching up, fast, and they’re going to very quickly become real alternatives to the larger foundation model providers. And that’s going to put a lot of pressure on them to compete with startups that are able to make on-the-fly adjustments to new models, like they did with Mistral’s. #ai #openai #mistral https://lnkd.in/gZ9Z6Bg3