Holistic AI Strategies Beyond Large Language Models

LLM field notes: Where multiple models are stronger than the sum of their parts, an AI diaspora is emerging as a strategic strength... Combining the strengths of different LLMs in a thoughtful, combined architecture can enable capabilities beyond what any individual model can achieve alone, and gives more flexibility today (when new models are arriving virtually every day), and in the long term. Let's dive in. 🌳 By combining multiple, specialized LLMs, the overall system is greater than the sum of its parts. More advanced functions can emerge from the combination and orchestration of customized models. 🌻 Mixing and matching different LLMs allows creating solutions tailored to specific goals. The optimal ensemble can be designed for each use case; ready access to multiple models will make it easier to adopt and adapt to new use cases more quickly. 🍄 With multiple redundant models, the system is not reliant on any one component. Failure of one LLM can be compensated for by others. 🌴 Different models have varying computational demands. A combined diasporic system makes it easier to allocate resources strategically, and find the right price/performance balance per use case. 🌵 As better models emerge, the diaspora can be updated by swapping out components without needing to retrain from scratch. This is going to be the new normal for the next few years as whole new models arrive. 🎋 Accelerated development - Building on existing LLMs as modular components speeds up the development process vs monolithic architectures. 🫛 Model diversity - Having an ecosystem of models creates more opportunities for innovation from many sources, not just a single provider. 🌟 Perhaps the biggest benefit is scale - of operation and capability. Each model can focus on its specific capability rather than trying to do everything. This plays to the models' strengths. Models don't get bogged down trying to perform tasks outside their specialty. This avoids inefficient use of compute resources. The workload can be divided across models based on their capabilities and capacity for parallel processing. Takes a bit to build this way (plan and execute on multiple models, orchestration, model management, evaluation, etc), but that upfront cost will pay off time and again, for every incremental capability you are able to add quickly. Plan accordingly. #genai #ai #aws #artificialintelligence

There's an emerging VC narrative that 4-5 LLMs will rule enterprise AI which is dangerously misguided. Here's why I think investors who adopt this lens into #AI will massively miss out. The human mind is a miracle 🧠. Our brains seamlessly process massive multi-sensory data into high level intelligence, problem-solving, creativity, learning, reasoning, and more. Replicating this functionality with AI models and apps is a galactic challenge. Even the mighty language models developed by OpenAI, Anthropic, and others are just the tip of the iceberg. These models are impressive but a small piece of what's needed to recreate capabilities that exist in all of us.The future is one of AI multiplicity across both models and applications. Foundation Models: One Size Will Not Fit All 🌐 LLMs and other foundation models will proliferate to meet diverse needs. Regional models tuned to local cultures & idioms (eg. Mistral AI). Medical/legal models steeped in domain knowledge (eg. Google DeepMind, Harvey/Clio - Cloud-Based Legal Technology). Proprietary enterprise models for optimization (eg. Collective[i]) Beyond these, specialized models will emerge spanning vision, robotics, forecasting, and beyond - interlocking pieces required for true intelligence. At the Application Layer: Proliferation of Valuable Point Solutions Some apps like Midjourney and Collective[i]'s C[i] for Sales will be powerful standalone offerings. Many will be AI-augmented tools enhancing existing workflows (ChatGPT in CRMs, marketing automation). So while commodity LLMs have a place, the highest value will come from a tapestry of complementary, interoperable models (commodity, open source, and proprietary) with a multitude of applications tailored to diverse business needs. Curious if others agree/disagree... #EnterpriseAI #LLMs #AI #whatsnext https://lnkd.in/gJXccnbs

https://lnkd.in/dizz2Ywy I've revisited this article around ten times, and each time, its perspective increasingly aligns with my observations in the generative AI landscape. The focus shifts from standalone models to the compound systems. In recent discussions with various teams, the necessity for such integrated systems is becoming a common theme. Take, for instance, a scenario where your business critically depends on processing multimodal information to aid search and discovery for hundreds of millions of customers. The question extends to how we approach it: Do we amalgamate all multimodal data into a unified embedding space? Do we create separate embedding representations for different modalities and leverage them accordingly? Or do we extract information and transpose it into a format closely matching user queries? No single approach can fulfill every customer requirement. Meeting these diverse needs requires a solution that embodies all these characteristics, a testament to the inherent complexity of customer demands and the requisite sophistication of the systems designed to cater to them. Consider a company in need of a system adept at handling complex customer inquiries. Should this be addressed through direct responses from large language models (LLMs)? Perhaps a Retrieval-Augmented Generation (RAG) architecture? Or maybe it involves understanding the semantics of queries via LLMs and navigating them through an existing advanced, intricate search system? Addressing customer inquiries effectively necessitates a blend of these strategies, not a reliance on a singular approach. The inherent complexity of customer needs dictates the complexity of the compound systems required. What about businesses that process highly localized queries? Should this involve latitude-longitude prompts through LLMs, or maybe more precise, street-level, or neighborhood-focused inquiries? Perhaps it requires an LLM augmented with a separate geospatial reasoning component and integrated seamlessly? Again, the natural complexity of customer inquiries calls for all solutions, underscoring the need for compound systems. In essence, the rising demand for compound LLM systems across various sectors mirrors the intricate and diverse nature of customer and business requirements.