Mixture of experts

Mixture of experts is a model architecture that contains many specialized subnetworks, called experts, and a router that activates only a few of them for each token, increasing capacity without proportional compute.

Mixture of experts sits in the Models part of the Agentik {OS} glossary, which defines the words used to build and run AI agent systems.

Also called MoE, Sparse mixture of experts.