Decode · Wide-EP scaling

Only the MoE layers shrink as decode goes wider

One decode step per rank, in ms: 64 requests per rank, 6 tokens per request, 128K context. Output tokens/s per GPU above each bar.

NVIDIA GB200, nsys over 100 steps on every GPU. EPLB with one redundant expert per rank. Experts per rank = (384 + DP) / DP.

Target MegaMoE Rest of the target model DSpark draft model Sampling, input prep, idle