22
→
7
kernels per MoE layer, batch 1
NVFP4, up to 8 requests. Dense layers 0–1 replace the MoE row with a norm and two GEMVs.
one kernel
+ gold
work fused into it