From a rank-0 kernel trace of the final optimized snapshot (TP8, batch 1, one query token). Not shown: a small workspace memset before each attention call. Prefill switches the all-reduce kernels to two-shot. Dense layers 0–2 replace rows 7–13 with cuBLAS gate/up and down GEMMs around one fused SiLU-and-multiply kernel, then the same all-reduce + norm kernel.