Writing
Running local LLMs for software development
Notes on where local models fit into an AI-assisted engineering workflow.
Local models are not a universal replacement for hosted coding agents, but they are useful in a few specific parts of the workflow.
Good Fits
- Quick summarisation of local notes.
- Private experiments with prompts and workflows.
- Lightweight code search explanations.
- Offline or low-trust drafting where no external service should see the input.
Weak Fits
Local models are less useful when a task needs large context windows, strong tool orchestration, or the best available reasoning model. In those cases, forcing everything local can make the workflow slower and less reliable.
The Useful Middle
The best setup is often hybrid: local tools for private, fast, repeatable tasks; stronger agentic tools for implementation loops where repository context, tests and review matter.