Free APIs have limits and privacy trade-offs. A model on your own computer has neither: no rate limits, no prompts leaving your machine, no account. In 2026 open-weight models such as Gemma 4, Qwen 3.x, and Nemotron Nano run well on a gaming GPU or an Apple Silicon laptop. This guide compares the four most popular free runners and explains what fits on your hardware.
The four runners
| Ollama | LM Studio | Jan | llama.cpp | |
|---|---|---|---|---|
| License | MIT | Free, closed source | Apache-2.0 | MIT |
| Interface | CLI + desktop app | Desktop app | Desktop app | CLI + built-in web UI |
| Local API | localhost:11434/v1 | localhost:1234/v1 | localhost:1337/v1 | llama-server, port of your choice |
| Best for | Developers, scripts, servers | Beginners, Mac (MLX) | Privacy-first chat | Maximum control and speed |
All four expose an OpenAI-compatible API, so any tool from our free software list — Cline, OpenCode, Open WebUI, Hermes Agent — can use a local model exactly like a cloud one.
Ollama: the default for developers
ollama pull llama3.1:8b
ollama run llama3.1:8bOllama downloads quantized models from its library, keeps them in memory while you use them, and serves them at http://localhost:11434/v1. Local use is unlimited; the optional Ollama Cloud adds hosted models on a separate plan.
LM Studio: the friendliest app
LM Studio has a model browser that shows whether a model fits your memory before you download it, uses Apple's MLX on Macs, and starts a local server with one toggle. It is free for personal and work use.
Jan: offline ChatGPT replacement
Jan is fully open source and designed to work offline. It runs local models, can also connect to cloud APIs with your keys, supports MCP tools, and serves an OpenAI-compatible API on port 1337.
llama.cpp: the engine underneath
Many runners are built on llama.cpp. Using it directly gives you every tuning knob — GPU offload, context size, speculative decoding:
llama-server -m ./model-Q4_K_M.gguf --port 8080 -c 32768What fits on your hardware
A rough rule for 4-bit quantized models: about 0.6 GB of memory per billion parameters, plus room for context.
| Memory (VRAM or unified) | Comfortable model size | Examples |
|---|---|---|
| 8 GB | up to ~8B | Llama 3.1 8B, Qwen 3.5 4B, Gemma 4 E4B |
| 16 GB | 12–14B, small MoE | Gemma 4 12B, Nemotron Nano 9B |
| 24 GB | 27–35B | Qwen3.8 27B, Gemma 4 31B, Qwen 3.6 35B-A3B |
| 64 GB+ (Mac Studio, multi-GPU) | 70B and large MoE | Llama 3.3 70B, gpt-oss-120b |
MoE models are special: a 35B-A3B model needs memory for all 35B parameters but runs almost as fast as a 3B model, which makes them ideal for local use when you have the RAM.
Local or free API?
- Local wins on privacy, latency for autocomplete, and unlimited use.
- Free APIs win on quality: the strongest free models — GLM 5.3 or Kimi K3 — are far too large for consumer hardware.
The practical answer is both: a small local model for autocomplete and private data, and a free API model from our catalog for heavy reasoning.