What for: run an open model locally with a single command — private, free, offline, with no token bills and no data leaving your machine.

What it is and who it's for

Ollama is the simplest way to run open LLMs (Llama, Qwen, Mistral, DeepSeek, Gemma and others) right on your own machine with a single command. For developers and privacy-minded users who want AI without the cloud: data never leaves the machine, there are no token bills, and it works offline. What sets it apart: it hides all the fuss with weights and quantization behind ollama run and spins up a local OpenAI-compatible server.

Key features

Run models with one command; weights download and cache automatically.
A local REST API on localhost:11434, plus an OpenAI-compatible endpoint for integrations.
Modelfile — your own system prompts, parameters and model builds (like a Dockerfile for LLMs).
GPU support (NVIDIA/AMD) and Apple Silicon out of the box; CPU fallback.
Ready-made quantized builds (Q4/Q8) — a balance of quality and memory.
Multimodal and embedding models in the same catalog.

How to start in 5 minutes

Install it (macOS/Windows — the installer from ollama.com; Linux — a script):

curl -fsSL https://ollama.com/install.sh | sh

Run a model and chat right in the terminal:

ollama run llama3.2

Hit the API from your code:

curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"Hello"}'

When to take it, when not to

✅ Take it if: you need privacy/offline use, a local prototype, free experiments, or you're handling sensitive data.
✅ Take it if: you want to plug a local model into Cline/Continue/Goose without any cloud keys.
❌ Don't take it → "LM Studio" is better if you'd rather have a clickable GUI than a terminal, plus easy model picking for your hardware.
❌ Don't take it → "OpenRouter" or "Groq" is better if you need top-tier models on par with Claude/GPT, or speed local hardware can't deliver.

The honest price

Completely free and open source. The only thing you "pay" with is hardware: VRAM/RAM and electricity. 7-8B models run on 16 GB of RAM, 70B ones need tens of GB of VRAM. Exact requirements depend on the model and the quantization.

Gotchas

A local model's quality is noticeably below the frontier cloud — don't expect GPT/Claude level from an 8B.
Big models hit the VRAM wall: without a decent GPU everything is slow or simply doesn't fit.
By default the API listens locally; if you expose it (OLLAMA_HOST), put a firewall in front of it — there's no authentication.
The default context is limited — long documents require tuning num_ctx and memory by hand.

🤖 Prompt accelerator

"I'm on <OS: macOS/Windows/Linux>, with of RAM and a <GPU MODEL or "none"> graphics card. I've installed Ollama. Pick the best open model for the task <DESCRIBE: code / chat in English / summarization> given my memory, give me the exact ollama run command, recommend a quantization (Q4/Q8) and num_ctx settings, and show me how to call it from <Python/Node> through the local API."