Pretraining

Pretraining is the first and most compute-intensive stage of building a foundation model, in which it learns general patterns from very large unlabeled datasets, usually by predicting the next token.

Pretraining sits in the Foundations part of the Agentik {OS} glossary, which defines the words used to build and run AI agent systems.

Also called Pre-training.