Shared memory pools

Every prefill rank reads shared KV and Engram pools, so Dynamo can route each request to the least-loaded rank.

Schematic, not to scale.

Mooncake: KV lookup and load over RDMA. Engram: peer-HBM reads over NVLink. NIXL: P→D KV push.

shared pools (this post) router, prefill and decode instances