Online evaluation

Online evaluation is the measurement of an AI system's quality using live production traffic, such as user feedback, A/B tests or automated scoring of real interactions.

Online evaluation sits in the Evaluation & safety part of the Agentik {OS} glossary, which defines the words used to build and run AI agent systems.