Cerebras Hits 1500 Tokens/Sec with Qwen 3.8 27B
Cerebras just added Qwen 3.8 27B to its public API at a blistering 1500 tokens per second without pruning the model architecture.
Models · Source: Hacker News
What happened
Cerebras just updated its public inference catalog with two massive additions. Builders can now access Qwen 3.8 27B running at a staggering 1500 tokens per second. The platform also added an OpenAI GPT OSS 120B model that hits 3000 tokens per second. Both models feature massive context windows, offering up to 128k and 131k tokens on paid tiers.
The hardware company is drawing a hard line on model degradation. They explicitly stated that no pruned models run on their public endpoints. Every model served through the shared API retains its original, unpruned architecture. Their experimental REAP pruned models remain quarantined on Hugging Face for research purposes only.
Storage optimization happens behind the scenes without sacrificing runtime precision. Cerebras applies selective weight-only quantization to store models in 16-bit, 8-bit, or 4-bit formats. Sensitive layers stay at full precision and dequantize on the fly during inference. Crucially, the activations, attention heads, and KV cache remain entirely unquantized to guarantee output quality.
Key facts
- 1500 — tokens per second for Qwen 3.8 27B
- 3000 — tokens per second for OpenAI GPT OSS 120B
- 128k — paid context window for Qwen 3.8 27B
- 131k — paid context window for OpenAI GPT OSS 120B
Why it matters
Speed changes the entire product equation for AI applications. When you can generate 1500 tokens per second, synchronous LLM calls feel instantaneous to the end user. This unlocks real-time voice agents, rapid multi-step reasoning chains, and massive data extraction tasks that previously timed out. Builders no longer have to choose between a highly capable 27-billion parameter model and low-latency user experiences. You can build complex agentic workflows without making your users stare at a loading spinner. Fast inference turns batch processes into interactive features.
The explicit guarantee against silent model pruning sets a new baseline for inference providers. Trust in API providers has eroded as companies quietly quantize or prune models to save compute costs, breaking prompts in the process. By documenting exactly how they handle weights, activations, and the KV cache, Cerebras is forcing competitors to be transparent. Inference is rapidly becoming a commodity market. Providers will have to compete on verifiable quality and architectural integrity just as much as they compete on raw speed.
For builders
Real-time voice and video agents
Latency is the only bottleneck for conversational AI. Hitting 1500 tokens per second means text generation is no longer the slow step in your pipeline. Founders building voice agents will win big here, while legacy chatbot wrappers will look even slower by comparison.
Predictable prompt engineering
Cerebras guarantees they will not alter model architectures without notice. This means your prompts will not randomly degrade because the provider decided to prune layers to save money. Engineers save hours of debugging, and inference providers who secretly degrade models will lose customers.
Massive context processing
The Qwen model offers a 128k context window on the paid tier. Combined with extreme inference speeds, you can process entire codebases or legal documents in seconds. Enterprise customers pay a premium for this speed, making it a highly profitable wedge for B2B founders.
My take
Speed is a feature, but transparency is the actual moat. I am tired of inference providers quietly degrading models to pad their margins and breaking my apps in the process. Cerebras putting their exact quantization and pruning policies in writing is exactly what the industry needs right now.
Original reporting: Hacker News. This is my rewrite and opinion.