SemiAnalysis AgentX · NVIDIA GB300

P/D and Wide-EP: 7.9× throughput per GPU at higher interactivity

Throughput per GPU (cached + uncached input + output tokens) vs. P90 interactivity (1 / P90 inter-token latency). Up and to the right is better.

This post: N × DEP2 prefill + one DEP16 or DEP32 decode (34–64 GPUs), and TP4; Oct 20. First post: TP4 and DEP2 on one node; Oct 2.

This post: P/D + Wide-EP First post: aggregated, one node