ollama

Local LLM runtime. The only blessed service that defaults to isolation = "shared" (decision 2.46) — multiple projects pinning the same model share one Ollama instance rather than each loading the weights into RAM.

Manifest knobs

[services.ollama]
type = "ollama"
version = "0.23.2"
model = "llama3.2"                # one model to pre-load on healthy
isolation = "shared"               # default for ollama specifically

port is intentionally not surfaced — shared-isolation services share a fixed port across consumers. Healthcheck: GET / returning 200.

On-disk layout

<project>/.unibench-data/<service>/
└── models/                     # ollama's model cache

For shared instances, the cache is per-version-of-Ollama, not per-project — so two projects sharing ollama@0.23.2 see each other's pulled models.

Inspection drawer

  • Connection URL — http://127.0.0.1:<port>.
  • Installed models — every model on disk with its size + family + quantization.
  • Loaded models — currently in-RAM models with their footprint.
  • Server version — what ollama --version reports.
  • External tools — jump to the Ollama CLI or Open WebUI if configured.

$<NAME>_PORT

depends_on = [{ service = "ollama", condition = "model_loaded" }] blocks the dependent until the named model is fully loaded (not just the server being up). Use this for indexers that immediately issue embedding requests.

Common gotchas

  • Models aren't shipped. Pulling llama3.2 is ~2 GB and is your one-time cost on first run. The known-issues catalog covers the pull-not-yet-done case (ollama.model_not_pulled).
  • VRAM. On a 16 GB M1, mistral:7b fits comfortably; llama3.1:70b doesn't. The inspector's "loaded models" surface shows the VRAM footprint so you can tell which model is competing for memory with your dev tools.
  • Shared-isolation semantics. Stopping a project doesn't kill the shared Ollama instance unless that project was the last consumer. The orchestrator tracks the refcount and tears it down at the right time.

Escape hatch

Most teams use Ollama as-is. For commercial APIs (Anthropic / OpenAI), configure the BYOK key and the agent panel routes through the chosen provider directly — no need to model that as a service at all.