Ollama Setup for Coding: Hardware, Models, Real Limits
In short
Ollama runs open-weight models on your own machine and exposes them at http://localhost:11434, including an OpenAI-compatible API at /v1/. For coding that means Cline, Continue, Aider, Zed or anything else that speaks OpenAI can point at a local model and never send your code anywhere.
The setup that actually works: 16 GB of unified memory or VRAM is the floor and buys you gpt-oss:20b; 24–32 GB gets you qwen3-coder:30b or devstral:24b, which are the two models worth using inside an agent. Then raise the context window — Ollama defaults to 4k tokens below 24 GiB of VRAM, and Cline's own docs recommend at least 32k. That single mismatch is why most local agent setups feel broken on day one.
What you give up: speed on long files, tool-calling reliability over multi-step tasks, and roughly 50 points of SWE-Bench versus a frontier cloud model. Local is a privacy and cost decision, not a quality one.
When a local model is actually the right call
Three situations justify it.
Your code cannot leave the building. Client NDAs, regulated data, an employer that blocks external AI endpoints. This is the only reason that is genuinely non-negotiable, and it is a good one.
You want a floor on cost, not a ceiling on quality. Claude Pro is $20/month billed monthly ($17/month on the annual plan) and Max starts at $100/month as of July 2026. A local model costs electricity. If you code intermittently and mostly want autocomplete and small refactors, the subscription is hard to justify.
You are offline or on bad connectivity. Trains, planes, rural internet, air-gapped networks.
Situations that do not justify it: wanting frontier quality for free, wanting to run agents unattended overnight, or expecting local to beat cloud on anything except privacy and marginal cost. If your reason is "cloud models are expensive," check the free vibe-coding stack first — a free or near-free hosted model will outperform anything you run at home.
Hardware you really need
Memory is the only spec that decides what you can run. Everything else affects how fast it is, not whether it works.
Two numbers matter and guides usually conflate them. The download size is the quantized weights on disk. The runtime footprint is those weights plus the KV cache, which grows with your context window. Coding agents want big context windows, so the KV cache is not a rounding error — budget real headroom above the file size.
| Memory (VRAM or Apple unified) | What you can realistically run | What that gets you |
|---|---|---|
| Under 16 GB | 7B-class models | Autocomplete and single-file edits. Do not put this in an agent loop. |
| 16 GB | gpt-oss:20b (14 GB, 128K context) |
Ollama states the 20B runs on systems with as little as 16 GB. Workable, tight on context. |
| 24 GB | devstral:24b (14 GB, 128K context) |
The lightest model with a published agentic score. Room left for context. |
| 32 GB+ | qwen3-coder:30b (19 GB, 256K context) |
30B MoE with 3.3B active params — best quality per gigabyte for local coding. |
| 80 GB | gpt-oss:120b (65 GB) |
Fits a single 80 GB GPU. Workstation territory. |
| 250 GB+ | qwen3-coder:480b (290 GB) |
Ollama documents a 250 GB memory minimum. Not a desk machine. |
On Apple silicon, unified memory is the number — a 32 GB Mac behaves roughly like a 32 GB card for these purposes, with Metal acceleration handled automatically.
On the PC side, check driver compatibility before buying anything. Ollama's GPU documentation requires NVIDIA compute capability 5.0+ with driver 550 or newer, and cards in the 5.0–6.2 range need driver 570+. AMD requires the ROCm v7 stack on both Linux and Windows.
CPU-only inference works and is genuinely too slow for agent use. You will wait tens of seconds for a response an agent then discards on its next step.
Installing and picking a model
Install from the Ollama repo or the site — there are native installers for macOS, Windows and Linux, plus an official Docker image. Then:
ollama pull qwen3-coder:30b # or devstral:24b on 24 GB
ollama run qwen3-coder:30b # sanity check in the terminal
ollama ps # shows what is loaded, and the CONTEXT column
Three models are worth your bandwidth in July 2026:
qwen3-coder:30b— 19 GB, 256K native context, a mixture-of-experts design with 30B total and 3.3B active parameters. The active-parameter count is why it runs fast on modest hardware despite the size. Best default if you have 32 GB.devstral:24b— 14 GB, 128K context, Apache 2.0, built by Mistral AI with All Hands AI specifically for agent loops. It reports 46.8% on SWE-Bench Verified, which is the only hard agentic number you will find on a model card this size. Best if you have 24 GB or want a license with no strings.gpt-oss:20b— 14 GB, 128K context, MXFP4 quantization at 4.25 bits per parameter. The fallback for 16 GB laptops.
Skip the 7B "coder" models for agent work. They handle autocomplete and break down the moment a task spans more than one file.
Wiring it into your editor
Fix the context window first. Per Ollama's context-length docs, the default is chosen by available VRAM: 4k under 24 GiB, 32k between 24 and 48 GiB, 256k at 48 GiB and above. Meanwhile Ollama's own Cline integration guide says coding tools need at least 32k. On a 16 GB laptop you get 4k by default, which is not enough to hold a single medium file plus the agent's system prompt. Raise it explicitly:
OLLAMA_CONTEXT_LENGTH=32768 ollama serve
Then confirm with ollama ps — the CONTEXT column shows what was actually allocated, and the processor column tells you whether the model spilled onto CPU. If it spilled, lower the context or use a smaller model.
Cline (VS Code), for agent work. Settings → API Configuration → provider Ollama, base URL http://localhost:11434, pick your model from the detected list, set the context window to 32K or more.
Anything OpenAI-compatible. Point the base URL at http://localhost:11434/v1/ and supply any string as the API key — Ollama requires the field but ignores the value. /v1/chat/completions, /v1/models and /v1/embeddings all work, which covers Continue, Aider, Zed and most editor plugins.
Claude Code is the exception. Its officially supported providers are Anthropic's API, Amazon Bedrock, Google Vertex AI and Microsoft Foundry — all serving Claude models. There is no supported path to a local Ollama endpoint. If you want a local agent, use Cline, Continue, Aider or OpenHands. The Ollama arsenal card has the short version of this setup.
What local models still lose to cloud on
| Dimension | Local (24–32 GB) | Frontier cloud | Why it matters |
|---|---|---|---|
| Agentic accuracy | Devstral reports 46.8% SWE-Bench Verified | Frontier models score far higher on the same benchmark | Half the tasks fail unattended |
| Effective context | 4k–32k in practice, memory-bound | 200k+ routinely usable | Multi-file refactors do not fit |
| Tool calling | Degrades over long chains | Reliable over dozens of steps | Agents stall or loop |
| Throughput | Seconds per response, slower as context fills | Fast at any context | Agent loops amplify every delay |
| Vision / multimodal | Often stripped (Devstral is text-only) | Standard | No screenshots, no design input |
| Cost shape | Hardware up front, then electricity | $17–$200/month as of July 2026 | Local wins only past heavy sustained use |
The honest framing: a good local model in 2026 is roughly where hosted models were a generation or two ago. That is genuinely useful for autocomplete, single-file edits, test generation, commit messages and rubber-ducking. It is not a replacement for a frontier model driving a multi-hour agent session, and no amount of prompt engineering closes that gap.
The pragmatic setup most people land on is a split one: local model for the constant, low-stakes, privacy-sensitive work, cloud model for the hard tasks. See the rest of the docs for how the cloud half of that split is usually configured.
FAQ
How much RAM do I need to run Ollama for coding?
16 GB of VRAM or Apple unified memory is the practical floor and runs gpt-oss:20b. 24 GB runs devstral:24b comfortably; 32 GB runs qwen3-coder:30b, which is the best local coding model per gigabyte right now. Below 16 GB you are limited to 7B models that are fine for autocomplete and unreliable in an agent.
Which Ollama model is best for coding?
qwen3-coder:30b if you have 32 GB — 256K native context and MoE efficiency. devstral:24b if you have 24 GB or need a permissive Apache 2.0 license; it is the only model in this class with a published SWE-Bench Verified score (46.8%). gpt-oss:20b on a 16 GB machine.
Can I use Ollama with Claude Code?
No. Claude Code's supported providers are the Anthropic API, Amazon Bedrock, Google Vertex AI and Microsoft Foundry, all serving Claude models. For a local-model agent, use Cline, Continue, Aider or OpenHands — all of them speak Ollama's OpenAI-compatible endpoint at http://localhost:11434/v1/.
Why is my local coding agent so bad at multi-file tasks?
Most likely the context window. Ollama defaults to 4k tokens on machines with under 24 GiB of VRAM, and a coding agent burns that on its system prompt alone. Start the server with OLLAMA_CONTEXT_LENGTH=32768 and verify with ollama ps. If the model then spills onto CPU, you need a smaller model rather than a bigger window.
Is running an LLM locally actually cheaper than a subscription?
Only past a certain volume. A 24–32 GB GPU or a high-memory Mac is a real up-front cost, and it depreciates while cloud models improve. Against $17–20/month for a Pro subscription as of July 2026, the payback period is long. Buy local hardware because you need privacy or offline capability, and treat the cost saving as a bonus rather than the reason.