How we trained the fastest DSpark for Kimi-K3 using GB300 NVL72
·9 min read
How Speculators and Mooncake enabled multi-node DSpark training for Kimi K3.
#distributed
2 posts
Menu
Theme
2 posts
How Speculators and Mooncake enabled multi-node DSpark training for Kimi K3.
Distributed Layerwise Offload shards and streams DiT weights across devices, serving a measured 124 GB Cosmos3 model on 64 GB HBM and estimating a path toward 200B+ models.