A shared Mooncake KV pool with workload-aware routing serves 68% more requests and cuts TTFT P50 from 29.8 s to 4.2 s.
Three DEP2 prefill instances at concurrency 4,800; the same one-layer stand-in decoder in both arms. Requests/s is relative to the per-rank CPU tier arm.
Per-rank CPU tier + session affinityShared Mooncake pool + workload-aware routing