Speculative decoding

Speculative decoding is an inference acceleration technique in which a small, fast draft model proposes several tokens that a larger model verifies in one step, producing the same outputs faster.

Speculative decoding sits in the Infrastructure part of the Agentik {OS} glossary, which defines the words used to build and run AI agent systems.