The landscape of AI infrastructure is undergoing a tectonic shift from monolithic NVIDIA-centric clusters to heterogeneous, purpose-built "best-of-breed" architectures. On March 13, 2026, Amazon (AWS) and Cerebras Systems announced a landmark partnership to deploy Cerebras’ wafer-scale hardware directly within AWS data centers, integrated with Amazon’s proprietary Trainium3 silicon.
Executive Summary
- Technical Summary: AWS is implementing a hybrid inference pipeline that utilizes AWS Trainium3 for prefill operations and the Cerebras CS-3 (Wafer-Scale Engine) for decode operations, enabling ultra-low latency inference for ultra-large models like Qwen3-235B with context windows up to 131K tokens.
- The deal represents a significant departure from standard GPU-based inference, moving toward a split-architecture approach to solve the "memory wall" in LLM decoding.
- Cerebras hardware will sit physically inside AWS data centers, connected via Amazon’s custom networking fabric, marking the first time a third-party "giant chip" is natively integrated into the AWS silicon stack.
- The partnership targets massive open-source models (Qwen, Llama, GPT-OSS), offering a high-speed alternative to NVIDIA H100/H200 clusters for enterprise-grade inference.
1. Technical Architecture: The Prefill-Decode Split
The core innovation of the AWS-Cerebras partnership is the hardware-level bifurcation of the LLM inference process. In a traditional GPU setup, the same chip handles both the Prefill (processing the input prompt) and the Decode (generating the output tokens). This is inefficient because these two stages have diametrically opposed computational requirements.
AWS Trainium3 for Prefill
The prefill stage is highly parallelizable and compute-bound. It requires processing the entire input prompt simultaneously to generate Key-Value (KV) caches.
- Role: AWS Trainium3, Amazon’s custom-built AI accelerator, is optimized for high-throughput tensor operations and scalable memory bandwidth.
- Efficiency: By offloading prefill to Trainium3, AWS leverages its custom silicon's ability to handle massive parallel matrix multiplications at a lower cost-per-token than general-purpose GPUs.
Cerebras CS-3 (WSE-3) for Decode
The decode stage is the bottleneck of LLM inference. It is strictly sequential—one token must be generated before the next can be calculated—making it memory-bandwidth bound rather than compute-bound.
- The Wafer-Scale Advantage: The Cerebras CS-3 uses a single piece of silicon the size of a dinner plate (the Wafer-Scale Engine). Unlike GPUs, which must move data across a slow PCIe bus or HBM (High Bandwidth Memory), the CS-3 keeps the entire model state in on-chip SRAM.
- Latency: This architecture eliminates the "memory wall." Because every core is microns away from its memory, the CS-3 can generate tokens at speeds far exceeding what is possible on a distributed GPU cluster.
- Role: The CS-3 handles the sequential token generation (Decode), receiving the KV-cache from the Trainium3 "prefill" engine via Amazon’s custom low-latency networking.
Custom Networking Integration
A critical technical detail disclosed is that Cerebras chips will be linked to Trainium3 chips using custom networking technology from Amazon. This suggests a deep integration at the rack level, likely utilizing a modified version of Amazon’s Nitro or SRD (Scalable Reliable Datagram) protocol to ensure that the hand-off between the prefill engine (Trainium) and the decode engine (Cerebras) happens in sub-millisecond timeframes.
2. Performance Analysis: Benchmarking the Hybrid Stack
The primary target for this new infrastructure is high-parameter, long-context models. Specifically, the collaboration has announced support for Qwen3-235B, a massive model that pushes the limits of standard hardware.
Benchmark Comparison: Qwen3-235B Performance
| Metric | Standard H100 Cluster (Inference) | AWS Trainium3 + Cerebras CS-3 |
|---|---|---|
| Context Window Support | ~32K - 64K (standard) | 131K (Supported) |
| Tokens Per Second (Decode) | Variable (Latency-limited) | "Fast Inference" (SRAM-optimized) |
| Architecture Type | Multi-node GPU (HBM-bound) | Hybrid Wafer-Scale (SRAM-bound) |
| Context Length (Pages) | ~50–75 pages | 200–300 pages |
| Compute Efficiency | Shared prefill/decode | Segregated prefill/decode |
Note: Specific TFLOPS and exact latency figures for the Trainium3-Cerebras handoff are not yet disclosed, but the context suggests a 4x improvement in context length handling compared to previous-generation cloud offerings.
Model Support
The "Cerebras Fast Inference Cloud" available on AWS Marketplace lists several key models optimized for this architecture:
- Qwen3-235B: The flagship "heavy" model for this stack.
- Llama Series: High-speed inference for Meta’s open-source models.
- OpenAI GPT-OSS: Open-source versions of GPT-style architectures.
3. Technical Implications for the Ecosystem
The End of the "NVIDIA Tax" for Inference?
This partnership is a direct strike at NVIDIA’s dominance in the inference market. By combining their own silicon (Trainium3) with Cerebras’ radical wafer-scale hardware, Amazon is creating a vertically integrated stack that does not rely on NVIDIA’s supply chain or CUDA software layer.
The Rise of Heterogeneous Compute
We are entering an era where a single AI request may touch three different types of silicon:
- CPU: For request routing and logic.
- ASIC (Trainium): For initial prompt processing (Prefill).
- WSE (Cerebras): For token generation (Decode). This "Heterogeneous Pipeline" requires sophisticated orchestration software, which AWS is building natively into its cloud environment.
Diversification of Infrastructure
As noted in the additional context, OpenAI is also pursuing a similar diversification strategy, working with Broadcom and AMD. Amazon's move to include Cerebras signals that "big silicon" (wafer-scale) is finally moving from experimental research labs into production-grade cloud data centers.
4. Limitations and Trade-offs
While the performance gains are substantial, this architecture introduces several technical hurdles:
- Memory Constraints: The Cerebras WSE relies on on-chip SRAM for its speed. While SRAM is incredibly fast, it is significantly more expensive and less dense than the HBM3e found in NVIDIA H100s. This limits the total number of parameters that can fit on a single wafer without off-loading, though the "giant" size of the chip (850,000+ cores) mitigates this to an extent.
- Orchestration Complexity: Splitting a model's execution between Trainium and Cerebras requires a highly specialized compiler and runtime. Developers may find it harder to "lift and shift" their existing CUDA-based workflows to this hybrid stack without significant optimization.
- Physical Power and Cooling: Cerebras units have unique power delivery and liquid cooling requirements that differ from standard 1U/2U rack servers. AWS has had to modify its data center floor plans to accommodate these "giant chips."
5. Expert Perspective
From a senior ML engineering standpoint, this is the most logical evolution of the AI data center. The industry has realized that the "one-size-fits-all" GPU is starting to buckle under the weight of 200B+ parameter models and 100K+ context windows.
The decision to use Trainium for prefill and Cerebras for decode is a "chef's kiss" of architectural design. It treats the LLM inference problem not as a single task, but as a multi-stage pipeline where different physics apply at each stage. This move validates Cerebras as a legitimate contender for the "Inference King" title and proves that Amazon is willing to cannibalize its own internal hardware roadmap (by bringing in a third-party chip) to win the performance war against Microsoft/Azure.
6. Technical FAQ
How does the Trainium3 to Cerebras data transfer work?
While the exact protocol is proprietary, it involves a high-speed, low-latency interconnect using Amazon's custom networking. The Trainium3 cluster processes the prompt, generates the KV-cache, and streams that cache to the Cerebras CS-3, which then takes over the autoregressive decoding process.
Is this stack backwards-compatible with standard v1/v2 APIs?
AWS has integrated this via the "Cerebras Fast Inference Cloud" on the AWS Marketplace. It likely supports standard OpenAI-compatible API endpoints, allowing developers to switch their base_url without rewriting their core application logic, though performance tuning will require using AWS-specific SDKs.
Why not just use more Trainium3 chips for the decode?
Trainium, like all standard architectures, uses traditional memory hierarchies. While powerful, it still faces the sequential bottleneck of decoding where the chip spends most of its time waiting for data from memory. Cerebras’ wafer-scale SRAM allows the entire model to effectively "live" inside the processor, removing that wait time entirely.
7. References
- AWS Marketplace: Cerebras Fast Inference Cloud documentation.
- Cerebras Systems: CS-3 Technical Specifications and Wafer-Scale Engine 3 Overview.
- Amazon Web Services: Trainium3 Performance Whitepapers (Limited Release).
- Qwen3-235B Model Card and Context Window Benchmarks.