Batch inference

Batch inference is running a model on a large set of inputs as a background job rather than in real time, typically at lower cost, used when results are not needed immediately.

Batch inference sits in the Infrastructure part of the Agentik {OS} glossary, which defines the words used to build and run AI agent systems.

Also called Batch API, Offline inference.