What local AI models run well on a 32GB Mac?

On a 32 to 64 GB Apple Silicon Mac, models we verified running include Qwen3-VL 32B (vision), Muse Glimmer 30B (agentic multimodal), Qwen3 32B (general), and Qwen3.6 27B and 35B-A3B (coding) at 4-bit. Plan for extra KV-cache memory at long context.

What is the best local vision model for Mac in 2026?

Qwen3-VL 32B is a strong open vision-language model that fits comfortably on a 32 GB+ Mac, with good OCR and document understanding, and it read images correctly in our test. Muse Glimmer 30B adds agentic multimodal reasoning for loops that read a screen and act.

Why does runtime support matter more than model size on Apple Silicon?

A model only runs locally when a maintained runtime implements its architecture. The newest models often ship weeks ahead of the loaders that can run them, so a model can top a leaderboard and still fail to load on a Mac with plenty of memory. That is why we verify a model actually runs before listing it here.

How do I run these models across several Macs?

Install Ollama Herd (pip install ollama-herd or brew install ollama-herd), start the router on your most powerful Mac with herd, and run herd-node on each other machine. They auto-discover over mDNS. Point any client at one endpoint and requests route across your whole fleet.

Local AI Models That Run on Apple Silicon, Tested 2026

Q: Does DeepSeek-V4-Flash run on Apple Silicon?

Not yet, and it is a runtime-maturity gap rather than a hardware limit. The 284B-A13B DeepSeek-V4-Flash has the leaderboard scores and its memory fits a 512 GB Mac, but the deepseek_v4 architecture is not implemented in mlx-lm, not in the latest release and not on the project's main branch as of August 2026. Support exists only in an unmerged pull request. Until a maintained runtime ships a deepseek_v4 loader, a downloaded quant is just disk usage.

Most "best local model" lists are copied from a leaderboard. This one is not. Every model in the matrix below was pulled, loaded, and run through the herd on Apple Silicon in this test round, and we call out a headline model that does not run locally yet, because a model topping an index is useless if no runtime on your Mac can load it.

Last verified August 2026 · Ollama 0.32.9 · mlx-lm 0.31.3 · routed through Ollama Herd.

Verified-working matrix

These seven models were run through the herd's router on the fleet above and returned correct output. Sizes are the approximate 4-bit on-disk footprint; plan for additional KV-cache memory at long context, which is why the "runs on" column exceeds the file size.

Model	Type	Size (4-bit)	Runs on
Qwen3-VL 32B	Vision + general	~20 GB	32 GB+ Mac
Muse Glimmer 30B	Agentic multimodal (thinking)	~18 GB	32 GB+ Mac
Qwen3.6 35B-A3B	Coding (MoE)	~23 GB	32 GB+ Mac
Qwen3-Coder 30B-A3B	Coding (MoE)	~19 GB	24 GB+ Mac
Qwen3.6 27B	General (thinking)	~17 GB	24 GB+ Mac
Qwen3 32B	General (dense)	~22 GB	32 GB+ Mac
gpt-oss 120B	General + reasoning	~65 GB	96 GB+ Mac

Established models like Gemma 3, DeepSeek-R1, and Llama also run well on Apple Silicon. The matrix is deliberately limited to what we re-verified in this August 2026 round, so "tested" means tested.

Vision and multimodal models for Mac

Qwen3-VL 32B, a strong open vision model

ollama pull qwen3-vl:32b

A capable open vision-language model that fits comfortably on Apple Silicon, with strong OCR, document understanding, and image grounding. In our test it read a generated image and answered correctly. Larger variants exist for big-memory Macs; the 32B is the sweet spot for most fleets. Run it and query it with images directly.

As of v0.9.2 Herd's catalog knows this is a vision model, so an image sent to the fleet routes here on its own. You do not name the model in the request.

Muse Glimmer 30B, Meta's agentic multimodal model

ollama pull muse-glimmer:30b

An Apache-2.0 agentic multimodal model from Meta, with a dedicated perception encoder for screenshots, charts, and documents, a natural fit for agent loops that read a UI and act. It is a thinking model: it reasons before answering, and a tight token budget returns an empty reply because the reasoning consumed it. As of v0.9.2 Herd recognises that and raises the budget for you, which is the difference between this working out of the box and looking broken. Verified on our fleet for both text and vision.

Coding models for local agentic development

Qwen3.6 35B-A3B, fast agentic coder

ollama pull qwen3.6:35b-a3b

A 35B mixture-of-experts model with only ~3B active parameters, so it delivers high throughput at a ~23 GB footprint. Tuned for agentic coding. Pair it with Claude Code or Codex pointed at the herd for local, private coding with no per-token bill.

Qwen3-Coder 30B-A3B, lighter coding MoE

ollama pull qwen3-coder:30b

A ~19 GB coding MoE that runs on a 24 GB+ Mac, a good default when you want a dedicated coder that does not crowd out your general model. Ollama Herd also runs an MLX backend first-class alongside Ollama, so you can serve a coding model through mlx_lm.server behind the herd if you prefer MLX on Apple Silicon; the models here were verified through the Ollama backend.

General and reasoning models

gpt-oss 120B, measured ~69 tokens/sec on M3 Ultra

ollama pull gpt-oss:120b

Our long-running general and reasoning workhorse. Apache 2.0, reliable tool-calling, and it decodes at ~68.6 tok/s on an M3 Ultra, measured over 44,000+ real requests. It is a thinking model, and the herd auto-inflates its token budget so reasoning does not starve the visible answer.

Qwen3.6 27B and Qwen3 32B, dense all-rounders

ollama pull qwen3.6:27b · ollama pull qwen3:32b

Two dependable dense general models with thinking modes, at ~17 GB and ~22 GB. A good balance of quality and memory for a mid-size Mac when you want one model that handles everyday chat, drafting, and light coding.

Embeddings and retrieval

nomic-embed-text runs on a dedicated native fastembed server (ONNX, 130 MB, 768 dims), outside Ollama, so embedding requests never queue behind LLM inference. Ideal for RAG pipelines and dedup loops.
Vision embeddings (DINOv2, SigLIP, CLIP) run via ONNX Runtime for image similarity and frame dedup.

The honest part: what does NOT run locally yet

This is the section no leaderboard will give you.

DeepSeek-V4-Flash, tops the charts, will not load on Apple Silicon yet

The 284B-A13B DeepSeek-V4-Flash is one of the most-recommended open models of mid-2026, and its MLX quant is widely downloaded. We downloaded it and it would not run. Its architecture (deepseek_v4) is not implemented in mlx-lm, not in the latest release and not even on the project's main branch as of August 2026. Support exists only in an unmerged pull request (mlx-lm #1189), and even patched in, an open bug (#1332) leaks Metal memory and crashes at about 11K generated tokens. Cloud providers serve it fine; the gap is a local Apple Silicon runtime. Until upstream ships a working deepseek_v4 loader, a downloaded quant is just disk usage on your Mac. We are tracking it and will update this page when it lands.

The lesson: on Apple Silicon, runtime support gates model availability more than the model's size or benchmark score. A model only runs locally when a maintained runtime implements its architecture, which is exactly what we verify before listing anything here.

We run this as an open-source hobby project on the hardware we can afford, so the matrix above is a starting point, not the last word. If a model here behaves differently on your setup, or one we could not run works fine on yours, we would genuinely like to know. Open an issue or share results on GitHub, and help the next person who searches for it.

How to run any of these on your own fleet

pip install ollama-herd            # or: brew install ollama-herd
herd                               # start the router on your most powerful Mac
herd-node                          # on each other machine, auto-discovers via mDNS

Then point any client at one endpoint. The herd speaks four wire protocols, OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and native Ollama, so Claude Code, Codex, and any existing tool route across your whole fleet, with image requests routed to a vision-capable model on your fleet.

Which of these fit your Mac

Unified memory	Comfortable picks (from the verified set)
16 to 24 GB	Qwen3.6 27B, Qwen3-Coder 30B-A3B
32 to 64 GB	Qwen3-VL 32B, Muse Glimmer 30B, Qwen3.6 35B-A3B, Qwen3 32B
96 to 128 GB	gpt-oss 120B
256 to 512 GB	Multiple large models resident at once, plus the whole fleet's spare Macs

Local AI Models That Actually Run on Apple Silicon