Splitting a model across racks layer-wise so each rack handles a slice of the forward pass, used heavily in training and selectively in inference.
Tensions
Pipelining solves weight-capacity problems cleanly but does nothing for KV cache (per-GPU activation footprint is invariant) and adds milliseconds of latency per hop. In inference it is often skipped — modern racks already hold a trillion-parameter model — but it lets hardware designers cut HBM-per-GPU if they assume the model spans multiple racks.
Related Concepts
parallelism | scale-up domain | KV cache | memory bandwidth | latency