Continued pretraining

Continued pretraining is the practice of extending a pretrained model's training on additional large unlabeled data, often from a specific domain or language, before any task-specific fine-tuning.

Continued pretraining sits in the Models part of the Agentik {OS} glossary, which defines the words used to build and run AI agent systems.

Also called Continual pretraining, Domain-adaptive pretraining.