At the Google Cloud Next conference, Google and NVIDIA unveiled a comprehensive hardware and software roadmap specifically engineered to tackle the burgeoning costs associated with AI inference at scale. This strategic partnership introduces advanced bare-metal instances and integrated platforms designed to deliver unprecedented efficiency and performance for demanding AI workloads across various sectors.

Central to this initiative are the new A5X bare-metal instances, which leverage NVIDIA Vera Rubin NVL72 rack-scale systems. Through a meticulous hardware and software codesign approach, this architecture is projected to achieve up to ten times lower inference cost per token compared to previous generations, while simultaneously delivering ten times higher token throughput per megawatt. To support the immense data transfer required by thousands of processors, the A5X instances integrate NVIDIA ConnectX-9 SuperNICs with Google Virgo networking technology, enabling scaling to 80,000 NVIDIA Rubin GPUs within a single site cluster and an astonishing 960,000 GPUs across a multisite deployment. Managing workloads at such a massive scale necessitates sophisticated management systems, ensuring exact synchronization to prevent processing delays and idle compute time. Mark Lohmeyer, VP and GM of AI and Computing Infrastructure at Google Cloud, emphasized the importance of this integrated, AI-optimized infrastructure stack for the next decade of AI.

Beyond raw processing power, data governance remains a critical concern, particularly for enterprise deployments in highly regulated industries like finance and healthcare. To address data sovereignty and proprietary information risks, Google Gemini models, running on NVIDIA Blackwell and Blackwell Ultra GPUs, are entering preview on Google Distributed Cloud. This innovative deployment method allows organizations to securely retain frontier AI models entirely within their controlled environments, alongside their most sensitive data. The architecture is bolstered by NVIDIA Confidential Computing, a hardware-level security protocol that ensures training models operate in a protected environment where prompts and fine-tuning data remain encrypted. This encryption prevents unauthorized access or alteration by any party, including cloud infrastructure operators. For multi-tenant public cloud environments, Confidential G4 VMs, equipped with NVIDIA RTX PRO 6000 Blackwell GPUs, introduce these same cryptographic protections in preview, providing regulated industries with access to high-performance hardware without compromising data privacy standards. This marks the first cloud-based confidential computing offering for NVIDIA Blackwell GPUs.

The development of multi-step agentic AI systems, which connect large language models to complex APIs, demand continuous vector database synchronization, and require active mitigation of algorithmic hallucinations, presents significant operational overhead. To streamline these heavy engineering requirements, NVIDIA Nemotron 3 Super is now available on the Gemini Enterprise Agent Platform. This platform furnishes developers with specialized tools to customize and deploy reasoning and multimodal models tailored for agentic tasks. The broader NVIDIA platform on Google Cloud is optimized for a range of models, including Google’s Gemini and Gemma families, empowering developers to construct systems capable of reasoning, planning, and acting. Training these models at scale introduces further operational complexities, especially in managing cluster sizing and hardware failures during extensive reinforcement learning cycles. To mitigate this, Google Cloud and NVIDIA introduced Managed Training Clusters on the Gemini Enterprise Agent Platform, incorporating a managed reinforcement learning API built with NVIDIA NeMo RL. This system automates cluster sizing, failure recovery, and job execution, enabling data science teams to focus on model quality rather than low-level infrastructure management. CrowdStrike, for instance, actively uses NVIDIA NeMo open libraries, including NeMo Data Designer and NeMo Megatron Bridge, to generate synthetic data and fine-tune models for domain-specific cybersecurity applications, accelerating their automated threat detection and response capabilities when operating on Blackwell GPUs with Managed Training Clusters.

Integrating machine learning into heavy industry and manufacturing poses a distinct set of engineering challenges, often involving connecting digital models to physical factory floors, requiring precise physical simulations, immense compute power, and standardization across legacy data formats. NVIDIA’s AI infrastructure and physical AI libraries are now accessible on Google Cloud, providing a robust foundation for organizations to simulate and automate real-world manufacturing workflows. Major industrial software providers, such as Cadence and Siemens, have made their solutions available on Google Cloud, accelerated by NVIDIA infrastructure, powering the engineering and manufacturing of heavy machinery, aerospace platforms, and autonomous vehicles. Many manufacturing firms operate on decades-old product lifecycle management systems, making the translation of geometry and physics data challenging. By leveraging NVIDIA Omniverse libraries and the open-source NVIDIA Isaac Sim framework via the Google Cloud Marketplace, developers can bypass many of these translation issues, constructing physically accurate digital twins and training robotics simulation pipelines prior to physical deployment. Furthermore, deploying NVIDIA NIM microservices, such as the Cosmos Reason 2 model, to Google Vertex AI and Google Kubernetes Engine, enables vision-based agents and robots to interpret and navigate their physical surroundings. Collectively, these platforms facilitate the advancement from computer-aided design directly to living industrial digital twins.

The impact of these hardware specifications and integrated solutions is evident in how early adopters are utilizing the infrastructure. The broad portfolio offers options scaling from full NVL72 racks down to fractional G4 VMs, providing one-eighth of a GPU, allowing customers to precisely provision acceleration capabilities for mixture-of-experts reasoning and data processing tasks. Thinking Machines Lab, for example, scales its Tinker API on A4X Max VMs to accelerate training, while OpenAI leverages large-scale inference on NVIDIA GB300 and GB200 NVL72 systems on Google Cloud for demanding workloads, including ChatGPT operations. Snap has transitioned its data pipelines to GPU-accelerated Spark on Google Cloud, significantly cutting the extensive costs associated with large-scale A/B testing. In the pharmaceutical sector, Schrödinger utilizes NVIDIA accelerated computing on Google Cloud to compress drug discovery simulations that previously took weeks into a matter of hours. The developer ecosystem supporting these tools has expanded rapidly, with over 90,000 developers joining the joint NVIDIA and Google Cloud developer community within a year. Startups like CodeRabbit and Factory apply NVIDIA Nemotron-based models on Google Cloud to execute code reviews and run autonomous software development agents. Aible, Mantis AI, Photoroom, and Baseten build enterprise data, video intelligence, and generative imagery solutions using the full-stack platform. Together, NVIDIA and Google Cloud are committed to providing a cutting-edge computing foundation designed to advance experimental agents and simulations into robust production systems that secure fleets and optimize factories in the physical world.