Marc Dibeh’s Post

I’ve recently been helping several customers set up GPU-based AI workloads on Azure, and one question keeps coming up: “What CUDA driver version and NVIDIA driver version are required for V100/H100 GPU?" Just wanted to throw some light on this today. When you install the NVIDIA driver on an Azure GPU VM, it already includes the CUDA driver API. There’s no separate “CUDA driver” that needs to be installed. With just the NVIDIA driver, you get everything required to run workloads like PyTorch, TensorFlow, Ollama, or other LLM inference setups. The CUDA Toolkit is separate and only required if someone is compiling CUDA code or building custom GPU kernels. For most customer scenarios—especially LLM inference—installing the toolkit is unnecessary. On GPUs like the V100, A10, or A100, installing the correct NVIDIA driver is typically all it takes to get started. Once the driver is installed, I usually ask customers to run a quick nvidia-smi check to confirm the GPU is recognized and the driver is loaded correctly. If the command shows the GPU model and driver version, the environment is ready for AI workloads.

🧠 The AI Infrastructure Paradox: owning GPUs ≠ operationalizing AI. While enterprises rush to build GPU clusters, many are discovering that the real bottleneck lies in platform complexity: configuring networks, managing resources, and providing self-service access to developers. At AI Infrastructure Field Day, Rafay presented a different approach: ✅ Secure multi-tenancy for shared GPU clusters ✅ Standardization across vSphere, Kubernetes, and hybrid clouds ✅ Application-first delivery via catalog (e.g., Jupyter, inference endpoints) ✅ Governance and cost tracking baked in By providing the missing automation layer between raw hardware and AI services, Rafay aims to help enterprises bridge the 20x cost gap between owning GPUs and renting AI capacity. AI infrastructure isn’t just about compute; it’s about control, consistency, and consumption. #AIInfrastructure #Rafay #GPUs #HybridCloud #PlatformEngineering #AIEnablement #theCUBEResearch #EfficientlyConnected Read the fully analysis by Jack Poller here: https://lnkd.in/eJQ_m3AD

Exciting step forward for Azure Local -- support for the NVIDIA RTX PRO 6000 Blackwell GPU is on the way. This brings high-performance AI and visualization right to the edge, helping customers run advanced workloads wherever they need them. Learn more: https://lnkd.in/gYwycc7f

Unleashing the beast of computing 🚀 NVIDIA's ComputeDomains are set to revolutionize the landscape of Kubernetes, tearing down the complex walls of multi-node GPU orchestration. 🟠 This is your ticket to dynamic, elastic GPU connectivity, with NVLink domains expanding and contracting like the universe itself. 🟠 No more wrestling with static configurations—ComputeDomains smartly manage these interconnections autonomously. 🟠 Forget about manual assignments; your workloads will sail smoothly in their own secure NVLink domains. 🟠 Whether you're after scalable GPU-to-GPU communication or optimal pod placement, bringing AI workloads to scale has never been more seamless. Elevate your Kubernetes experience and let your AI workloads soar. Do you think this will redefine how enterprises prioritize GPU architectures in their infrastructure strategies? 🤔 #Kubernetes #NVIDIA #GPUs #AIDevelopment #CloudComputing #TechTransformation 🔗https://lnkd.in/dTN2JznE 👉 Post of the day: https://lnkd.in/dACBEQnZ 👈

🪙 Microsoft Azure just reached 1,100,948 tokens/sec on ND GB300 v6 racks powered by NVIDIA GB300 NVL72 systems — independently validated by Signal65. NVIDIA GB300 NVL72 brings a step-change lift in generational efficiency — delivering ~10× higher inference performance vs H100 with ~2.5× better power efficiency at the rack. 🔄 More performance per watt → better TCO → a more sustainable path to scaled production AI. 🔗 Learn how datacenters can unlock real #AI inference ROI — delivering record throughput with enterprise-grade governance at scale.

🪙 Microsoft Azure just reached 1,100,948 tokens/sec on ND GB300 v6 racks powered by NVIDIA GB300 NVL72 systems — independently validated by Signal65. NVIDIA GB300 NVL72 brings a step-change lift in generational efficiency — delivering ~10× higher inference performance vs H100 with ~2.5× better power efficiency at the rack. 🔄 More performance per watt → better TCO → a more sustainable path to scaled production AI. 🔗 Learn how datacenters can unlock real #AI inference ROI — delivering record throughput with enterprise-grade governance at scale.

🪙 Microsoft Azure just reached 1,100,948 tokens/sec on ND GB300 v6 racks powered by NVIDIA GB300 NVL72 systems — independently validated by Signal65. NVIDIA GB300 NVL72 brings a step-change lift in generational efficiency — delivering ~10× higher inference performance vs H100 with ~2.5× better power efficiency at the rack. 🔄 More performance per watt → better TCO → a more sustainable path to scaled production AI. 🔗 Learn how datacenters can unlock real #AI inference ROI — delivering record throughput with enterprise-grade governance at scale.

🪙 Microsoft Azure just reached 1,100,948 tokens/sec on ND GB300 v6 racks powered by NVIDIA GB300 NVL72 systems — independently validated by Signal65. NVIDIA GB300 NVL72 brings a step-change lift in generational efficiency — delivering ~10× higher inference performance vs H100 with ~2.5× better power efficiency at the rack. 🔄 More performance per watt → better TCO → a more sustainable path to scaled production AI. 🔗 Learn how datacenters can unlock real #AI inference ROI — delivering record throughput with enterprise-grade governance at scale.

🪙 Microsoft Azure just reached 1,100,948 tokens/sec on ND GB300 v6 racks powered by NVIDIA GB300 NVL72 systems — independently validated by Signal65. NVIDIA GB300 NVL72 brings a step-change lift in generational efficiency — delivering ~10× higher inference performance vs H100 with ~2.5× better power efficiency at the rack. 🔄 More performance per watt → better TCO → a more sustainable path to scaled production AI. 🔗 Learn how datacenters can unlock real #AI inference ROI — delivering record throughput with enterprise-grade governance at scale.

🪙 Microsoft Azure just reached 1,100,948 tokens/sec on ND GB300 v6 racks powered by NVIDIA GB300 NVL72 systems — independently validated by Signal65. NVIDIA GB300 NVL72 brings a step-change lift in generational efficiency — delivering ~10× higher inference performance vs H100 with ~2.5× better power efficiency at the rack. 🔄 More performance per watt → better TCO → a more sustainable path to scaled production AI. 🔗 Learn how datacenters can unlock real #AI inference ROI — delivering record throughput with enterprise-grade governance at scale.