Tokens per second

Tokens per second is a measure of how quickly a language model generates output once it starts responding, used to compare the speed of models, hardware and serving setups.

Tokens per second sits in the Infrastructure part of the Agentik {OS} glossary, which defines the words used to build and run AI agent systems.

Also called Output tokens per second, Generation speed.