Google launches AX to orchestrate billions of AI agents
Google just open-sourced AX, a new orchestrator built specifically to run stateful, bursty AI agents at massive scale.
Tools · Source: Hacker News
What happened
Google released AX. It is an open-source orchestrator built specifically for AI agents. Traditional orchestrators were designed to handle stateless microservices or predictable batch jobs. Agents are a completely different kind of workload. They accumulate state. They require strict isolation to run untrusted code. They are highly bursty. Most importantly, agents spend the vast majority of their time just waiting around for model APIs, external tool servers, or human approval.
AX runs on top of a compute runtime called Agent Substrate. It gives developers four declarative primitives to manage this chaos. Tasks handle isolated execution in sandboxes with strict memory and CPU limits. Workspaces set up the environment the agent needs. Gateways manage network policies and inject credentials. Models centralize configuration so you can rotate keys or pin versions in one place. You write a simple configuration file, and AX handles the heavy lifting.
The system is designed to scale to billions of concurrent tasks per cluster. It achieves this massive density through dense multiplexing and sub-second resumption. When an agent is idle and waiting on a response, AX checkpoints its state and suspends it. Dozens of tasks share the same worker resources. When the response finally arrives, the agent is brought back online in under a second with zero cold-start delay.
Key facts
- Apache 2.0 — Open-source license for the AX project
- Billions — Number of concurrent agent sessions AX can scale to per cluster
- Under a second — Time it takes to resume a suspended agent with zero cold-start delay
Why it matters
Building agent infrastructure today is a massive headache for founders and engineers. You are forced into a bad trade-off. You either overpay for idle compute while your sandboxes sit around waiting for LLM responses, or you waste months building custom suspend-and-resume logic from scratch. AX solves this problem directly at the infrastructure layer. Developers can now run massive fleets of agents without reinventing sandboxing, state management, or network isolation. You get to focus entirely on agent logic and application workflows.
This release marks the beginning of true agent-native infrastructure. Just as Kubernetes standardized how we deploy microservices, tools like AX will standardize how we deploy and manage agent swarms. We will see a rapid shift away from monolithic agent frameworks running on generic cloud instances. Researchers and founders can now spin up massive numbers of reproducible sandboxes. This makes it viable to collect trajectories, run complex reinforcement learning loops, and evaluate agents at scale without breaking the bank.
For builders
Stop paying for idle agent compute
Agents spend most of their lifecycle waiting on network calls and model APIs. AX multiplexes dozens of tasks on shared worker resources and automatically suspends idle agents. You only pay for compute when your agents are actively thinking and running code.
Generative workspace setup saves time
You can define a workspace goal in plain English rather than writing bash scripts. AX hands that goal to an agent on first boot to install necessary toolchains and verify all dependencies. This eliminates the tedious work of writing complex setup scripts for every new agent environment.
Built-in blast radius containment
Agents can easily burn money in an infinite loop if left unchecked. AX provides strict CPU and memory limits per task out of the box. The Gateway primitive also locks down network traffic to explicit allowlists so untrusted agent code cannot compromise your internal systems.
My take
We finally have an orchestrator that understands agents are not microservices. Google open-sourcing the deep infrastructure they use for DeepMind research is a massive gift to the open-source community. Stop writing custom state management loops to save cloud costs and start using purpose-built tools like AX to scale your workloads.
Original reporting: Hacker News. This is my rewrite and opinion.