What's New in Ollama Herd

0.9.1 · July 2026

An 8th scoring signal, two health checks, and reliability fixes.

• Session affinity, the 8th scoring signal. Multi-turn conversations now stick to the node holding their warm prefix cache, so turn N+1 re-uses the cache instead of re-encoding the whole history. It also keeps a fresh prefill from interfering with other streams on the same node. See the Routing Engine guide.
• Two new health checks. One flags a node whose decode speed has collapsed, the other flags a pin request that cannot physically fit, so both surface before they cost you a slow session.
• Steadier under load. Queue concurrency is now capped at the backend's real admission limit, plus fixes for a node-id shadowing bug that orphaned pins, image requests that could fall back to a text-only model, and a stale dashboard tab that could wedge the router.

No breaking changes and no new dependencies, so upgrading from 0.9.0 is a plain pip install ollama-herd --upgrade.

0.9.0 · July 2026

OpenAI Codex support, image routing, and the first release since 0.7.0.

Nothing between 0.7.0 and this shipped publicly, so upgrading lands 0.8.0, 0.8.1, and 0.8.2 all at once. It contains breaking changes, so read the upgrade notes below before deploying. Soaked for 26 hours across every code change in it: 11,925 requests at 99.85% success, including 302 requests to the new Codex endpoint with zero failures.

• OpenAI Codex runs on your fleet. A native Responses API at /v1/responses, which is the only protocol current Codex speaks after it dropped Chat Completions in February 2026. Agentic coding is verified end to end: Codex read sources, ran pytest, created a module from scratch, patched files, and reached green on its own. Zero config, because model ids auto-route. Works with both the CLI and the macOS desktop app. See the Codex guide.
• No more model map to maintain. The built-in default map is gone. An unmapped claude-* or Codex model id now routes to the best coding model you actually have loaded, instead of a hard-coded name your machine may never have pulled. Explicit pins still win.
• Images route to a model that can see them. On every endpoint. An image-bearing request auto-selects a vision-capable model even when your conversation model is code-tuned. A dropped image used to produce a fluent, specific, wrong answer while every metric reported success.
• Fleet control API. GET /fleet/limits reports serving capacity, and POST /fleet/pin pre-warms a model, optionally blocking until it is actually resident.
• Consistent X-Fleet-* headers on every proxied response, plus per-request strict mode (X-Fleet-No-Fallback) and an optional per-client concurrency cap.
• Reliability. Failed requests are now recorded reliably, so dashboard success rates stop hiding failures. Models are sized by what they actually cost in RAM (weights plus KV cache) rather than on-disk weights, which had under-counted a 30B model by more than 5x.

Upgrading from 0.7.0, six things change: legacy single-server FLEET_NODE_MLX_* env vars are removed (migrate to a one-entry FLEET_NODE_MLX_SERVERS array); X-Fleet-Model is retired for X-Fleet-Served-Model; queue-full returns 429 instead of 503; POST /fleet/pin can now refuse a set that cannot co-reside; image requests fail loudly rather than being answered by a blind model; and backend 4xx errors surface as themselves instead of an opaque 500.

0.7.0 · June 2026

Native text embeddings and a full-fleet dashboard.

• Native text embedding server. nomic-embed-text now runs on a dedicated fastembed server (port 11439) via ONNX Runtime, so embedding requests never queue behind LLM inference. On a real fleet: 573 embed requests over 24 hours, 0.0% error rate, no timeouts.
• "Node Models" dashboard. The dashboard now shows a card for every backend receiving traffic, Ollama, MLX, native fastembed, and vision embedding, each with live per-model stats.
• 4 new health checks (36 total) covering embedding routing and contention.

0.6.2 · May 2026

Reliability hardening for the trace store.

• Trace-store resilience. Dedicated read connections and periodic WAL checkpoints so heavy dashboard use can no longer stall background trace writes. Verified under a 30-write plus 120-poll burst, WAL peak dropped from 103 MB to 410 KB.
• New health check surfaces trace write failures instead of letting them hide.
• Fixed a daily log-rotation race between the router and node processes.

0.6.1 · April 2026

MLX stability and packaging.

• MLX supervisor hardening. Orphaned mlx_lm.server processes are now reaped on startup, and a crash-looping server is quarantined instead of restarted forever.
• brew install ollama-herd now works cleanly.
• Tunable per-model MLX concurrency, and speculative decoding on the dedicated context compactor for faster Claude Code summarization passes.
• Dashboard color semantics fixed so utilization reads as usage, not a warning.

0.6.0 · April 2026

MLX backend and the fix for "Claude Code breaks at 30K tokens."

• MLX backend, first-class alongside Ollama. Run mlx_lm.server as a routing target, with multiple MLX servers per node so a single machine can host a main coding model and a dedicated compactor side by side.
• Three-layer context management. The systematic fix for long agentic sessions on local coding models: mechanical tool-result clearing, an LLM-based compactor, and a pre-inference cap that returns a clean error instead of wedging the model.
• Tool-use reliability. Tool-schema fixup and malformed-JSON tool-call repair so agentic loops hold together on local models.

Earlier releases

• 0.5.x · April 2026. Routing and dashboard refinements on top of the multimodal foundation.
• 0.4.1 · April 2026. Thinking-model support (auto-detects chain-of-thought models and inflates token budgets), image-generation fixes, and streaming reliability improvements.
• 0.1.0 · March 2025. The first release: mDNS auto-discovery and 7-signal scoring across a fleet of Ollama nodes.

Get the latest

Ollama Herd ships often. Upgrade with one command, or watch the repo for release notes:

pip install ollama-herd --upgrade
# or: brew upgrade ollama-herd

Full technical release notes live in the GitHub releases. New to Herd? Start with the Quickstart.