llama.cpp
Reference C/C++ engine for GGUF models; llama-server provides an OpenAI-compatible API and web UI on CPU, CUDA, Metal, Vulkan and ROCm.
- License
- MIT
- Platforms
- Windows, macOS, Linux, Docker, CLI
- Own API keys
- Yes
- Checked
- 2026-09-24
llama.cpp works with OpenAI-compatible APIs. Grab a free key and a model from the catalog: for coding · for agents · providers and base URLs
llama.cpp alternatives
Popular runtime to download and run open LLMs locally via CLI, desktop app and a REST/OpenAI-compatible API.
Desktop app to discover, download and run local LLMs (llama.cpp, MLX) with a chat UI and an OpenAI-compatible local server.
Offline ChatGPT alternative: runs local LLMs, connects to cloud APIs, supports MCP and serves an OpenAI-compatible API on localhost.
High-throughput LLM inference and serving engine with an OpenAI-compatible server for NVIDIA, AMD, Intel, TPU and CPU.
Self-hosted drop-in OpenAI/Anthropic API replacement for LLMs, images, audio, video and embeddings on CPU or GPU.
Single-file llama.cpp-based runner with the KoboldAI Lite UI; OpenAI/Ollama-compatible APIs plus image generation, Whisper and TTS.
FAQ
Is llama.cpp free?
Completely free and open source, with no accounts, API keys or usage limits.
Does llama.cpp work with free models?
Yes. It supports your own keys and OpenAI-compatible APIs, so you can plug in free models from OpenRouter, Groq, NVIDIA Build, or a local Ollama.
What license does llama.cpp use?
MIT. The source code is open.