Cerebras has published a wafer-scale deployment plan for trillion-parameter models on its official blog, using the open model Kimi K3 (about 3 trillion parameters, 93 layers) as a worked example. At 4-bit quantization, each layer is about 16GB: the 69 linear-attention layers pack two per wafer, while the 24 full-attention layers take one wafer each with room left for the KV cache — 35 plus 24, or 59 wafers in total. Each WSE-3T wafer has 44GB of on-chip memory, so a whole layer can sit directly on the chip. At run time the layers form a pipeline, passing only intermediate results between wafers at about 2 microseconds per hop; Cerebras says models of this class can reach 1,000–2,000+ tokens per second. One clarification matters: K3 is only an example — this is a deployment recipe, not a measured result on 59 wafers. The bigger the model, the more the deployment engineering matters.
Original link: Click to view the original
Comments