Google and Meta open-source TorchTPU to run PyTorch natively
Google and Meta just made TPUs a native PyTorch device, replacing clunky XLA with a direct backend to break Nvidia's software moat.
Tools · Source: MarkTechPost
What happened
Google and Meta just open-sourced TorchTPU at the PyTorch 2026 Conference. It is a native PyTorch backend for Cloud TPUs. You can now target Google silicon by simply writing torch.device("tpu"). This replaces the old PyTorch/XLA stack and its lazy-tensor graphs.
The release includes four open repositories under an Apache 2.0 license. TorchTPU plugs directly into PyTorch through the PrivateUse1 interface. It runs standard PyTorch tensors with no wrappers or subclasses. Developers get three eager modes for debugging, including a Strict mode and a DeferAndFuse mode to cut memory traffic. For production throughput, torch.compile lowers the graph through StableHLO to OpenXLA.
Integrations are already live across the entire AI ecosystem. Hugging Face Transformers v5 supports it out of the box. Google also shipped native TPU backends for vLLM, SGLang, and TorchTitan. The system scales massively. Google tested it on generations from v5e all the way up to 8,192 Ironwood TPU v7x chips. Serving features include FP8 KV caches and speculative decoding.
Key facts
- 52% — Model Flops Utilization reached by TorchTitan training Qwen 3 8B on Ironwood TPUs
- 8,192 — Number of Ironwood TPU v7x chips Google scaled TorchTPU training across
- 11.49s to 2.78s — Drop in fine-tuning step time when moving from out-of-box eager mode to full torch.compile
- 5.97 billion — Daily tokens generated per 1,000 TPU v6e accelerators by Zenteiq using llm-d on GKE
Why it matters
Building on TPUs used to mean rewriting code or fighting with XLA compilers. Developers had to manually mark compilable regions and deal with pure SPMD code limitations. Now, TPUs sit right next to GPUs and CPUs as first-class citizens in PyTorch. You keep your existing hardware-agnostic model code, schedulers, and API servers. This drastically lowers the switching cost for teams wanting to escape GPU shortages or high compute prices.
The second-order effect is massive for the broader AI infrastructure market. The AI chip war is officially a framework war. By teaming up with Meta, Google is directly attacking Nvidia's CUDA moat. They are making it trivial to port major workloads like large language models, ads, and recommendation systems to custom silicon. When developers can switch hardware by changing a single line of code, hardware monopolies start to crack. This forces cloud providers to compete purely on price and performance rather than software lock-in.
For builders
Zero-friction TPU deployment
You can deploy open-weight models from Hugging Face directly to TPUs using device_map="tpu". This saves engineering hours previously spent adapting models for XLA. Startups save money by easily testing workloads on cheaper TPU spot instances without rewriting their deployment pipelines.
Drop-in inference scaling
The vLLM-TorchTPU and SGLang-TorchTPU integrations preserve your upstream API servers. You get continuous batching and quantization on TPUs without changing your inference stack. Teams serving high-traffic agents can cut latency using native prefix caching through RadixAttention.
Massive training orchestration
TorchTitan provides reference recipes for pre-training massive models like Llama 3 405B and Qwen 3 235B MoE on TPUs. Google proved near-linear scaling up to 8,192 chips. Founders building foundation models now have a viable, documented alternative to massive GPU clusters.
My take
Nvidia's biggest advantage was never just silicon. It was the fact that CUDA simply works out of the box. Google and Meta finally woke up and realized they had to make TPUs frictionless too. By turning hardware switching into a single line of PyTorch code, I see them actively dismantling the CUDA monopoly.
Original reporting: MarkTechPost. This is my rewrite and opinion.