Perplexity’s Post

Building trillion-parameter models is impressive. Making them useful is leadership. Amazon’s latest work on scaling AI infrastructure (via Neuron SDK + AWS EFA) makes it possible to train and deploy trillion-parameter models on the cloud. This isn’t about showing off compute power. It’s about reducing friction between innovation and implementation. And that’s where most companies still get stuck. “Trillion-parameter models aren’t the goal. Business value is. Use scale when it solves a real problem faster, cheaper, or better.” If you work in AI strategy, here’s what this signals: • The hardware excuse is gone. Public cloud infrastructure is now enterprise-ready for LLMs at any scale. • ROI matters more than ever. Bigger models mean higher costs. Use them where they deliver real advantage—speed, quality, or reach. • Start with fit, not size. Most use-cases don’t need trillion-parameter models. Start with lean, proven architectures. Scale later. • Governance is non-negotiable. As model size grows, so does risk. Monitoring, bias detection, and control must grow with it. • Think in stages. Don’t launch into trillion-parameter territory blind. Move in phases. Measure at each step. This kind of infrastructure shift changes what’s possible. The real question is whether your team is ready to make it valuable. Let’s stop talking about size. Let’s talk about use.

This work is a testament to the fact that the next frontier of #LLM performance and scalability lies as much in systems engineering and networking as it does in model architecture. By open sourcing these kernels, you are providing a crucial piece of infrastructure for the community, lowering the barrier to deploying trillion parameter models in a cost-effective and performant manner. This is a substantial contribution that will likely influence how future large scale inference systems are built. Excellent work from the Perplexity team. I look forward to trying the kernels and reading the full paper

Our UCCL-ep supported MoE kernels on AWS EFA since last month: https://lnkd.in/dpJq7vFB. It also supports AMD GPUs and Broadcom NICs: https://lnkd.in/edZidD9R Moreover, UCCL-ep has supported SGlang SGLang (without any line of code change) to run deepseekv3 on AWS H200 VMs. We are also integrating into vllm vLLM now. Will release soon! So, UCCL-EP compared to pplx (1) UCCL-EP runs on AMD and Broadcom, beyond EFA (2) UCCL-EP has better perf with larger # tokens (e.g. 4096), e.g. 2.1ms for dispatch, 4.9ms for combine at EP32, while pplx has 4.7ms and 8.3ms respectively (3) API-compatible with DeepEP, no code changes needed. [My previous comments under this post got mysteriously disappeared. So I reposted here. But who deleted it? 🤔]

Everyone will get the model, but not everyone will get the renewal. Perplexity just showed how to run trillion-parameter models on standard cloud infrastructure. The bigger story is what it signals: model access is normalizing. The most important question for AI-first orgs to ask themselves isn't if they can build a smarter model, but if they can repeatably convert capability into customer value. • A workflow that eliminates effort • A result the customer can measure • Something customers will pay to keep Renewal conversations will tell the answer. https://lnkd.in/geSxh9fT

Datadog just published a new research report! Cloud providers offer an increasing breadth of modern compute services, from serverless to containers to GPUs and beyond. For this report, we examined adoption trends across compute options, autoscaling practices, and approaches to efficiency and optimization. Our findings suggest that most organizations rely on an evolving mix of modern compute technologies, shaped by growing focus on cost optimization, shifting preference of autoscaling tools, and expanding Arm usage. Did you know Karpenter adoption has overtaken Cluster Autoscaler? Find out more in the new "State of Containers and Serverless" report. https://lnkd.in/eyPNGWKn

Yesterday at GTC, Jensen unveiled BlueField-4 - the processor running the operating system of AI Factories. Here's how Oracle Cloud is taking advantage of it: https://lnkd.in/gZBrr9T9

Datadog has just launched the 2025 State of Containers and Serverless report. It's been very interesting to be a small part of putting this together. We've aggregated, analyzed and pulled together data to look at exactly how people are running their applications. From the underlying infrastructure to the managed services provided by cloud providers. My personal favourite fact, 66% of organizations using functions are also running containers. It's re-affirmed a theme I've repeated all year, the technologies are converging. It's no longer about being serverless or not serverless. It's about choosing the right technology for your workload. https://lnkd.in/eGgpR386 #serverless #containers #cloud #architecture #technology

Fact 3: Most workloads use less than half of their requested resources. "Most" in this case means that almost 80% of lambdas are heavily overprovisioned (using <50% of assigned memory and CPU). The same applies to solutions compared across all platforms: Azure Container Apps, Google Cloud Run Functions, Fargate, and k8s containers. Will such measurements get better over time in the report? Is this representative for serverless functions? There is no option for dynamic function sizing, and lambdas can consume items with varying batch sizes and inputs. In those cases, it is better to overprovision than fail. I wonder if the methodology of measuring that shouldn't be different. Again, not a shocker, but it can be an action point to review your workloads!

Oracle announced Oracle Cloud Infrastructure (OCI) Zettascale10, the largest AI supercomputer in the cloud. FInd out more in Data Center Dynamics: https://lnkd.in/eyvCSRrj

Oracle announced Oracle Cloud Infrastructure (OCI) Zettascale10, the largest AI supercomputer in the cloud. FInd out more in Data Center Dynamics: https://lnkd.in/djdik5Ab