KV cache

A KV cache is memory that stores the attention keys and values computed for earlier tokens during generation, so a language model does not recompute them for each new token, speeding up inference.

KV cache sits in the Infrastructure part of the Agentik {OS} glossary, which defines the words used to build and run AI agent systems.

Also called Key-value cache.