llama.cpp

llama.cpp is an open-source C and C++ project for running large language models efficiently on local hardware, including CPUs and consumer GPUs, and it introduced the widely used GGUF model file format.

llama.cpp sits in the Infrastructure part of the Agentik {OS} glossary, which defines the words used to build and run AI agent systems.