The dollar cost of serving a token from a trained model, which is a roofline trade-off between memory bandwidth, compute, and batch size — not a property of the model alone.
Tensions
Lower latency and lower cost-per-token pull in opposite directions: small batches give fast tokens at high amortized cost, large batches give cheap tokens at higher tail latency. Pricing tiers (Fast Mode, Slow Mode) are just different points on this same curve, not different physics.
Related Concepts
batching | memory bandwidth | latency | roofline analysis | cost equalization | KV cache