Google Cloud rents out NVIDIA fastest AI racks at massive scale
Google Cloud will make NVIDIA Vera Rubin A5X instances generally available in November with plans to scale to 960,000 GPUs.
Business · Source: Unite.AI
What happened
Google Cloud is rolling out its A5X instances in November 2026. These instances run on NVIDIA new Vera Rubin NVL72 rack-scale systems. The cloud provider plans to scale this infrastructure to 960,000 Rubin GPUs across multiple sites. Europe will also get access soon.
The hardware is built for heavy agentic inference and reinforcement learning. Each rack packs 72 Rubin GPUs, 36 Vera CPUs, ConnectX-9 SuperNICs, and BlueField-4 DPUs. Google combines this hardware with its Virgo Network and fourth-generation liquid cooling. NVIDIA claims this setup cuts inference costs per token by up to ten times compared to the previous generation.
Early access is already live. AlphaGo creator David Silver is the first customer through his startup Ineffable Intelligence. They raised a 1.1 billion dollar seed round to build systems that learn continuously from experience. Marketing giant WPP also tested the hardware using NVIDIA Nemotron models and the OpenShell runtime. They ran complex creative agent workflows on a single instance instead of a multi-node cluster.
Key facts
- 960,000 — Rubin GPUs Google Cloud plans to deploy across multiple sites
- 10x — Claimed reduction in inference cost per token versus prior generation
- 72 — Rubin GPUs packed into a single Vera Rubin NVL72 rack
- $1.1 billion — Seed funding raised by first customer Ineffable Intelligence in April
- 440,000 — NVIDIA Vera Rubin chips SpaceX plans to cluster by Q2 2027
Why it matters
Builders running reinforcement learning and complex agent workflows finally get infrastructure designed specifically for continuous feedback loops. The massive reduction in inference costs means you can run agentic systems at scale without burning through your runway. You no longer need to string together multi-node clusters for heavy creative or reasoning tasks. Furthermore, Google is introducing bare-metal confidential computing. This allows you to protect model intellectual property and training data on shared cloud hardware.
The scale of the 960,000 GPU plan shows a massive shift in cloud strategy. Google has its own custom TPU chips but still buys nearly a million NVIDIA GPUs to stay competitive. This creates a brutal supply crunch for everyone else. Companies like SpaceX are already fighting for the exact same Vera Rubin chips to build their own massive clusters. The two companies are even exploring NVIDIA DSX MaxLPS to squeeze more capacity out of fixed data center power budgets.
For builders
Cheaper inference for agentic workflows
NVIDIA claims the Rubin architecture drops inference costs per token by up to ten times. Startups building autonomous agents win big here. Cloud providers absorb the massive capital costs while you pay lower compute bills.
Single instances replace multi node clusters
WPP proved that workloads previously requiring multiple nodes now run on a single A5X instance. This simplifies infrastructure engineering for AI teams. DevOps engineers save time and companies save money on networking overhead.
Rack scale confidential computing
Google Cloud is rolling out bare-metal confidential computing to protect model IP and training data. Enterprise builders handling sensitive data can now use shared cloud hardware safely. Legacy on-premise hardware vendors lose their main security advantage.
My take
Google has its own TPUs but still bends the knee to NVIDIA. Customers want the fastest hardware and the CUDA ecosystem, not cloud-branded alternatives. When a cloud provider commits to buying nearly a million competitor chips, you know who actually controls the AI infrastructure market.
Original reporting: Unite.AI. This is my rewrite and opinion.