Knowledge distillation

Knowledge distillation is a technique for training a smaller student model to imitate the outputs of a larger teacher model, producing a faster, cheaper model that retains much of the teacher's ability.

Knowledge distillation sits in the Foundations part of the Agentik {OS} glossary, which defines the words used to build and run AI agent systems.

Also called Distillation, Model distillation.