#kv-cache

Shrink your LLM's memory footprint by 6x, speed up attention by 8x, and lose almost nothing in accuracy — no retraining required.