n8n with an Ollama Fleet

Connect n8n to the Ollama Herd router instead of one Ollama process, and every Ollama Chat Model, Ollama Embeddings, or HTTP Request node can use the same local endpoint. Herd then routes each request to an eligible machine in the fleet, which is useful when several workflow executions overlap or when embeddings and chat should run on different hardware.

The essential n8n credential change is:

Ollama Base URL: http://router-ip:11435

This guide goes beyond that one field. It covers Docker networking, agent versus chain nodes, a private classification workflow, a local RAG workflow, concurrency, observability, and failure handling.

Architecture

Triggers, schedules, webhooks, apps
                |
                v
               n8n
     | Chat | Embeddings | HTTP |
                |
                v
        Ollama Herd :11435
          |       |       |
          v       v       v
        node A  node B  node C
        large   fast    embeddings

n8n remains the workflow engine. Herd does not schedule workflows or store business data. It provides one inference endpoint for the local AI steps inside those workflows.

Why use a fleet behind n8n?

A simple n8n flow might make one model call. Real automations often make many:

• an agent calls the model repeatedly while selecting tools;
• several webhook executions overlap;
• a document import creates hundreds of embedding requests;
• one workflow summarizes while another classifies or extracts data;
• retries add bursts after an upstream service recovers.

A single Ollama instance can process parallel work only within its memory and configured concurrency limits. When it is full, requests queue. Herd adds horizontal capacity by routing independent requests across machines that can serve the requested model.

The two controls are complementary:

• Ollama settings decide how much parallel work one node accepts.
• Herd routing decides which node should receive each request.

Prerequisites

You need:

• a self-hosted or reachable n8n instance;
• an Ollama Herd router and at least one online node;
• a chat model available in the fleet;
• an embedding model when building the RAG example;
• network access from n8n to the router on port 11435.

Verify Herd first:

curl -s http://router-ip:11435/api/tags | python3 -m json.tool
curl -s http://router-ip:11435/fleet/status | python3 -m json.tool

Step 1: Create the Ollama credential in n8n

In n8n:

1. Open Credentials.
2. Create an Ollama credential.
3. Set Base URL to:

http://router-ip:11435

4. Leave the API key empty on a trusted private network unless you have placed an authenticated proxy in front of Herd.
5. Test and save the credential.

n8n's official Ollama credential accepts a configurable instance URL and can send a bearer token when an authenticated proxy requires one.

Step 2: Use the correct address from Docker

localhost is the most common source of connection errors.

For Docker Compose on Linux:

services:
  n8n:
    extra_hosts:
      - "host.docker.internal:host-gateway"

Then use:

http://host.docker.internal:11435

A connection test from the n8n container is more useful than guessing:

docker exec -it n8n sh
wget -qO- http://host.docker.internal:11435/api/tags

Step 3: Choose the right n8n Ollama node

n8n has separate model sub-nodes:

• Ollama Chat Model, use with conversational chains and AI Agent workflows. Choose a model that supports the tools or structured behavior your workflow needs.
• Ollama Model, useful for basic LLM chains, but n8n documents that it lacks tool support and should not be used as the model for an AI Agent.
• Embeddings Ollama, use for document indexing and query vectors.

For an agent workflow, a common layout is:

Chat Trigger or Webhook
        |
        v
     AI Agent
        |
        +-- Ollama Chat Model
        +-- approved tools
        +-- memory, if required

Set the Ollama Chat Model credential to the Herd URL. The selected model name must appear in Herd's /api/tags response.

Workflow 1: Private support-ticket classification

This workflow shows a predictable local-AI task that benefits from parallel executions.

Nodes

1. Webhook, receives a ticket with subject and body.
2. Edit Fields, constructs a compact input object.
3. Basic LLM Chain, performs classification.
4. Ollama Chat Model, points to Herd.
5. Structured Output Parser, validates the result.
6. Switch, routes by category.
7. Respond to Webhook, returns the decision.

Prompt

Classify the support ticket into exactly one category:
account, billing, bug, feature_request, or other.

Return JSON only with this shape:
{
  "category": "one category",
  "urgency": "low|normal|high",
  "summary": "one sentence"
}

Subject: {{ $json.subject }}
Body: {{ $json.body }}

Use a low temperature and validate the JSON. Do not let a free-form model response control high-impact actions without a deterministic parser and explicit rules.

Why Herd helps

When ten tickets arrive together, n8n can create overlapping executions. If the same classification model is available on more than one node, Herd can distribute independent calls instead of sending every execution to one queue.

Do not describe this as guaranteed linear scaling. Measure the real model, context, node memory, and network path.

Workflow 2: Local RAG with Qdrant and Ollama

n8n's self-hosted AI ecosystem includes Ollama and Qdrant patterns. A complete RAG system has two workflows.

Ingestion workflow

File trigger or source connector
        |
        v
Document loader -> text splitter
        |
        v
Embeddings Ollama -> Qdrant Vector Store

Configure Embeddings Ollama with:

• credential: the Herd router;
• model: one stable embedding model available in the fleet;
• vector store: a collection dedicated to that embedding model and dimension.

Query workflow

Chat Trigger or Webhook
        |
        v
Embed the question
        |
        v
Qdrant similarity search
        |
        v
Retrieved context -> AI Agent or chain
        |
        v
Ollama Chat Model -> answer with sources

Use the same embedding model for ingestion and query. Changing the model changes the vector space and often the vector dimension, so an existing collection must be rebuilt or migrated deliberately.

The dedicated RAG guide explains batch ingestion, workload placement, and embedding contention in more detail.

Use an HTTP Request node when you need full control

The built-in Ollama nodes are convenient. An HTTP Request node is better when you need Herd-specific request tags, fallback fields, or exact response handling.

Configure:

• Method: POST
• URL: http://router-ip:11435/api/chat
• Send body as JSON
• Response format: JSON

Body:

{
  "model": "llama3.2:3b",
  "messages": [
    {
      "role": "user",
      "content": "Summarize this text in three bullets: {{ $json.text }}"
    }
  ],
  "stream": false,
  "metadata": {
    "tags": ["n8n", "summary-workflow"]
  }
}

Set stream to false for normal n8n HTTP nodes unless you have intentionally built streaming handling. A single JSON response is easier to retry, parse, and pass to later nodes.

Herd request tags make it possible to separate n8n traffic from Open WebUI, coding agents, or other clients in fleet analytics.

Model strategy for automations

Avoid using one large model for every step.

A practical division is:

• small, fast model: classification, extraction, routing, normalization;
• tool-capable model: AI Agent workflows;
• larger reasoning model: difficult synthesis and multi-step analysis;
• embedding model: ingestion and retrieval only;
• fallback model: lower-capacity substitute for noncritical work.

Place frequently used automation models on more than one node when overlapping executions are common. Place specialized large models on machines that can hold them comfortably.

Concurrency and queue design

Three queues can exist at once:

1. n8n waits for an execution slot or worker;
2. Herd waits for an eligible fleet node;
3. Ollama waits for a model slot on that node.

When a workflow is slow, identify the layer instead of increasing every limit.

Measure:

• n8n execution start delay;
• time to first token or first response byte;
• total model latency;
• Herd queue depth and selected node;
• Ollama memory, loaded models, and parallelism;
• retry count and final error rate.

Use n8n's own queue mode for workflow scaling and Herd for inference scaling. They solve different bottlenecks.

Reliability patterns

Make external actions idempotent

A model call or workflow can be retried. Before sending email, changing a record, or calling a payment API, use an idempotency key or a durable “already processed” check.

Separate inference from decisions

Let the model propose a category, summary, or structured plan. Let deterministic n8n nodes validate fields, enforce thresholds, and execute privileged actions.

Set realistic timeouts

A cold large model can take much longer to load than a hot small model. Match n8n's HTTP or node timeout to the expected workload, but do not hide an overloaded design behind extremely long timeouts.

Tag important workflows

Use tags such as:

n8n,ticket-classifier
n8n,rag-ingest
n8n,rag-query
n8n,nightly-batch

This makes latency and error-rate comparisons meaningful.

Security

• Keep the Herd router on a trusted LAN or private VPN.
• Do not expose an unauthenticated inference API directly to the public internet.
• Restrict the Herd dashboard and API to the users or service accounts that need them.
• Treat prompts, retrieved documents, and model responses as potentially sensitive.
• Never place untrusted model output directly into shell commands, SQL, or privileged API calls.
• Use n8n credentials and secret storage rather than embedding secrets in prompts or workflow JSON.

For remote n8n deployments, use the Tailscale guide or an authenticated reverse proxy.

Troubleshooting

ECONNREFUSED ::1:11434

n8n resolved localhost to IPv6 or is still using the default Ollama port. Change the credential to an address reachable from the n8n process, for example:

http://127.0.0.1:11435

or, from Docker:

http://host.docker.internal:11435

The model dropdown is empty

Test the exact base URL from the n8n host:

curl -s http://router-ip:11435/api/tags | python3 -m json.tool

Then confirm the credential is saved and the selected model exists with the exact tag.

The AI Agent cannot call tools

Use Ollama Chat Model, not the basic Ollama Model node, and choose a local model with reliable tool support. Tool support is a model capability, not something Herd can add to an incompatible model.

Several executions become very slow

Check n8n worker saturation, then Herd queues, then per-node Ollama parallelism. Increasing OLLAMA_NUM_PARALLEL may raise throughput but also multiplies context-memory requirements and can worsen individual latency.

Embedding ingestion blocks interactive chat

Place the embedding model on a dedicated node or backend, batch document inputs, and avoid routing ingestion and generation through the same constrained model slot. See the RAG guide.

A workflow receives an incomplete streamed response

Use stream: false with a normal HTTP Request node. Streaming requires an SSE or NDJSON-aware consumer.

Frequently asked questions

Can n8n connect directly to Ollama Herd?

Yes. Use Herd's base URL in the standard n8n Ollama credential. Herd exposes Ollama-compatible chat, generation, embedding, model-list, and pull endpoints.

Do I need a custom n8n community node?

No. The built-in Ollama Chat Model and Embeddings Ollama nodes can use a custom instance URL. Use an HTTP Request node only for fields or headers not exposed by the built-in nodes.

Can n8n use different models for different steps?

Yes. Configure separate model sub-nodes. Herd routes each requested model independently, so a fast classifier and a larger reasoning model can live on different machines.

Does Herd make n8n workflows parallel?

Herd does not change n8n's execution engine. It gives concurrent model calls more local inference capacity by distributing eligible requests across the fleet.

Can I build a completely local n8n RAG system?

Yes. n8n can orchestrate document loading, Ollama embeddings, a local vector store such as Qdrant, retrieval, and local generation. Security and production hardening remain your responsibility.

When should I keep one Ollama server?

Use one server when workflows are low volume, all required models fit, and queueing is acceptable. Add Herd when several machines already exist or when overlapping workflows, model specialization, failover, or workload isolation justify a fleet.

Related Reading

• Install and verify Ollama Herd
• Ollama Herd integrations
• Tune Ollama concurrent requests
• Build RAG and embedding workloads across a fleet
• Secure remote Ollama access with Tailscale
• Ollama Herd API reference
• Deploy Ollama Herd for ongoing operations