Evaluating the Limits of AI Scaling

I spend a lot of time with technical founders building AI companies. Many assume that if we just make models bigger and feed them more data, we'll eventually reach true intelligence. I see a different reality. The fundamental limits of transformer architecture run deeper than most founders realize. Transformer models face three architectural barriers that no amount of scale can solve: 1️⃣ The Edge Case Wall An example in autonomous vehicles: Every time you think you've handled all scenarios, reality throws a new one: a child chasing a ball, construction patterns you've never seen, extreme weather conditions. The architecture itself can't generalize to truly novel situations, no matter how much data you feed it. 2️⃣ The Pattern Matching Trap Our portfolio companies building enterprise AI tools hit this constantly. Current models can mimic patterns brilliantly but struggle to reason about new scenarios. It's like having a highly skilled copywriter who can't generate original insights. The limitation isn't in the training—it's baked into how transformers work. 3️⃣ The Semantic Gap LMs process text without truly understanding meaning. We see this clearly in technical domains like software development. Models can generate syntactically perfect code but often miss fundamental logic because they don't grasp what the code actually does. This creates a massive opportunity for technical founders willing to rethink AI architecture from first principles. Some promising directions I'm tracking: → World models that understand causality and physical interaction → Architectures designed for reasoning during inference rather than training → Systems that combine multiple specialized models rather than one large generalist Founders: While others chase marginal improvements through scale, focus on solving the fundamental problems to build the next $100B+ business (and I'll be your first check ;))

Thought provoking and great conversation between Aravind Srinivas (Founder, Perplexity) and Ali Ghodsi (CEO, Databricks) today Perplexity Business Fellowship session sometime back offering deep insights into the practical realities and challenges of AI adoption in enterprises. TL;DR: 1. Reliability is crucial but challenging: Enterprises demand consistent, predictable results. Despite impressive model advancements, ensuring reliable outcomes at scale remains a significant hurdle. 2. Semantic ambiguity in enterprise Data: Ali pointed out that understanding enterprise data—often riddled with ambiguous terms (C meaning calcutta or california etc.)—is a substantial ongoing challenge, necessitating extensive human oversight to resolve. 3. Synthetic data & customized benchmarks: Given limited proprietary data, using synthetic data generation and custom benchmarks to enhance AI reliability is key. Yet, creating these benchmarks accurately remains complex and resource-intensive. 4. Strategic AI limitations: Ali expressed skepticism about AI’s current capability to automate high-level strategic tasks like CEO decision-making due to their complexity and nuanced human judgment required. 5. Incremental productivity, not fundamental transformation: AI significantly enhances productivity in straightforward tasks (HR, sales, finance) but struggles to transform complex, collaborative activities such as aligning product strategies and managing roadmap priorities. 6. Model fatigue and inference-time compute: Despite rapid model improvements, Ali highlighted the phenomenon of "model fatigue," where incremental model updates are becoming less impactful in perception, despite real underlying progress. 7. Human-centric coordination still essential: Even at Databricks, AI hasn’t yet addressed core challenges around human collaboration, politics, and organizational alignment. Human intuition, consensus-building, and negotiation remain central. Overall the key challenges for enterprises as highlighted by Ali are: - Quality and reliability of data - Evals- yardsticks where we can determine the system is working well. We still need best evals. - Extreme high quality data is a challenge (in that domain for that specific use case)- Synthetic data + evals are key. The path forward with AI is filled with potential—but clearly, it's still a journey with many practical challenges to navigate.

TERRIFIC research on limits of LLMs: https://lnkd.in/d5DVCNup TLDR; AI "reasoning" models like Meta’s Llama, DeepSeek AI and OpenAI ‘s o3-mini don't “reason” per say. They just memorize patterns really well. Here's what this team at Apple discovered: (hint: we're not as close to AGI as the hype suggests) —— Instead of using the same old math tests that AI companies love to brag about, Apple created fresh puzzle games. This is in part to avoid contamination. They tested Claude Thinking, DeepSeek-R1, and o3-mini on problems these models had never seen before. The result ↓ —— All "reasoning" models hit a complexity wall where they completely collapse to 0% accuracy. No matter how much computing power you give them, they can't solve harder problems. —— We expect “reasoning” models to get better with more compute and clearer instructions. Instead, they hit hard walls doing sophisticated pattern matching that works great until patterns become too complex and start giving up. Is that intelligence or memorization hitting its limits? —— This research suggests we're not as close to AGI as the hype suggests. Current "reasoning" breakthroughs may be hitting fundamental walls that can't be solved by just adding more data or compute. —— While many of the large AI companies celebrate their models "thinking," I appreciate that this paper is putting more rigor into how we measure and evaluate for intelligence. —— Their use of controllable puzzle environments was great because: • They avoid data contamination • They require pure logical reasoning • They can scale complexity precisely • They reveal where models actually break #openai #deepseek #ai #apple #models #llms #intelligence Summary adapted from Jaime Garcia’s post. Thanks for surfacing.

Ilya Sutskever gave a talk at NeurIPS about the post pretraining world - here's my talk on his talk - Ilya is implying we need to find something else to scale - the brain–body mass ratio graph in the talk showed human intelligence “scaled” better than mammals. LSTMs got out-scaled by transformers - the goal is to "edit" the scaling laws to make it more efficient. Evolution somehow first tried scaling intelligence for mammals, then pushed the frontier up for non-human primates. Large elephants which exceeded the 700g gram wall were extinct in the end. Then hominids came along and broke the wall, and scaled far better. (A) Kaplan et al’s scaling laws shows if we increase TRAINING compute = N (# parameters) * D (# tokens / data), the test loss also decreases in a log-log setting. (A)* Instead of scaling TRAINING compute, Sutskever mentioned we can scale TEST TIME compute through search, or like O1 / QwQ etc. (B) First on D (scaling data). There exists a theoretical “Data Wall” which is when all the data in the world (the internet and everything else) gets consumed by large models. Once we reach that point, we have to find ways to overcome this barrier to make models to continue to scale. This could mean Synthetic Data Generation as Sutskever mentioned - literally using a trained model to augment datasets. The question is if this will plateau or keep scaling. Another approach is to make data scaling more efficient through better filtering like the FineWeb dataset. We can also do more RL & post-training via DPO, PPO etc to squeeze more performance out of the same amount of tokens. (C) Second on N (# of parameters) - the trick is to move to active parameters instead of total parameters. Large labs like OpenAI replaced MLP / FFNs in Dense transformers with MoE layers . Instead of doing huge matrix multiplies, we smartly only select a few column groups to multiply instead, and leave the rest as 0. Coincidentally Meta released multiple papers including one on Byte Latent Transformers and Memory Layers . BLTs edit the scaling laws itself by changing the definition of “tokens” in data scaling and also adding more to the non embedding parameters. (D) Memory Layers are what really interested me! They are essentially sparse lookup tables - first devised as Product Key layers in Lample et al’s paper we replace the FFN MLP with a gigantic learnable matrix of size (100M, d) called V (Values). We then only select the top K rows of V (say 4) via a weighted sum via the softmax. A long post, but my final talk is Ilya is saying we need to find something else to scale. This could be: 1) Scaling instead test time compute via search, agents, O1 style 2) Changing the arch by holding training compute constant like MoEs, Memory+ layers etc 3) Changing the scales for scaling laws ie like BLTs 4) Breaking the Data Wall via Synthetic Data Generation, RL, filtering etc 5) Or something else! You can watch Ilya's talk here: https://lnkd.in/gPS7mtsm