Engram peers

Splitting the Engram tables across a peer group's HBM frees 378 GiB of host memory per tray.

Schematic. 18 prefill GPUs in one peer group; prefill and decode form separate groups.

Engram: two hashed n-gram tables, 94.4 GiB each, 188.8 GiB per copy. One DEP2 prefill instance = 2 GPUs; one tray = 4 GPUs.

Engram slice in peer HBM (new) host DRAM freed GPU and host DRAM