How much RAM do I need to run a 70B model locally on a Mac?

A 70B model at 4-bit quantization is roughly 40 to 42 GB on disk, and you want headroom for the KV cache and macOS, so a 64 GB Mac is the practical floor and 96 GB+ is comfortable for long context. At 4-bit, plan for the model size plus about 20 to 40 percent for context.

Can an M4 Max with 128GB run a 120B model?

Yes. A 120B model at 4-bit is around 65 GB, and a 128 GB Mac has roughly 100 to 110 GB usable after macOS overhead, so a single 120B model runs comfortably with room for context. Two large models resident at once is where 128 GB gets tight.

Does a bigger model always run slower on Apple Silicon?

No. Apple Silicon inference is memory-bandwidth-bound, and for mixture-of-experts (MoE) models throughput scales with active parameters, not total parameters. A 480B MoE with 35B active can run several times faster than a 120B dense model, while needing far more memory to hold its weights. Memory decides what fits; active parameters decide how fast it runs.

What quantization should I use for local models?

Q4_K_M is the default: about 97 percent of full-precision quality at roughly half the memory. Use Q8 (about 99 percent) when you have the memory to spare, and avoid Q2 except as a last resort for very large models, because MoE expert selection degrades badly at 2-bit.

Which Mac should I buy for local AI?

For one 30B-class model, 32 to 64 GB is enough. For a 120B model or several mid-size models at once, get 128 GB. For frontier 400B+ models or a whole resident model set, a 256 to 512 GB M3/M4 Ultra is the tier that unlocks them.

How Much RAM to Run Local AI Models on a Mac (2026)

On a cloud GPU you rent VRAM by the hour. On a Mac, the unified memory you bought is a hard ceiling: a model either fits in RAM or it does not run well at all. So the first question for local AI on Apple Silicon is not "which model is best," it is "which models fit my Mac, and how fast will they run." Those are two different questions with two different answers.

The two numbers that matter: memory and active parameters

Apple Silicon inference is memory-bandwidth-bound, not compute-bound. An M3 Ultra moves memory at about 819 GB/s, and that bandwidth, not raw FLOPs, sets your token rate. This has two consequences that surprise people coming from a 24 GB consumer GPU:

Total parameters decide what fits. The weights have to sit in RAM. A 4-bit model needs roughly half a byte per parameter, so a 70B model is about 40 GB just to load.
Active parameters decide how fast it runs. Modern flagship models are mixture-of-experts (MoE): a router activates only a fraction of the weights per token. A 480B model that activates 35B per token runs at roughly the speed of a 35B model, not a 480B one, while still needing memory for all 480B of weights.

So a large MoE can be simultaneously bigger (more memory) and faster (fewer active params) than a smaller dense model. That is why a 512 GB Mac is interesting: it can hold models that need eight H100s on NVIDIA hardware, and the MoE ones run fast while it does.

The sizing math

To estimate whether a model fits, add three things:

Weights: parameters × bytes-per-param. At 4-bit that is about params × 0.5 GB. (30B ~ 17 to 19 GB, 70B ~ 40 GB, 120B ~ 65 GB, 235B ~ 142 GB, 480B ~ 290 GB.)
KV cache: grows with context length. Budget roughly 20 to 40 percent on top of the weights for a comfortable working context; more for very long sessions. An 8-bit KV cache roughly halves this, at negligible quality cost.
macOS overhead: reserve about 10 to 15 GB, more on the big Ultra machines. Practical usable memory is well below the sticker number: a 128 GB Mac gives you roughly 100 to 110 GB, and a 512 GB Ultra roughly 450 to 470 GB.

Which Mac runs what

Unified memory	Comfortable	Tight but works
16 to 24 GB	7B to 14B models; small coding MoEs (Qwen3-Coder 30B-A3B, ~19 GB)	A single 30B at 4-bit with short context
32 to 64 GB	30B-class comfortably; a 70B at 4-bit (~40 GB) on 64 GB	70B with long context
128 GB (M4 Max)	One 120B model (~65 GB) with room for context; multiple mid-size models	A 235B MoE at a lower quant
256 GB	A 400B-class MoE (~245 GB) or a 480B coder (~290 GB)	A 400B model plus a second hot model
512 GB (M3/M4 Ultra)	A 480B coder plus smaller models resident at once; frontier 671B-class solo	Two frontier models hot simultaneously

The models in the "comfortable" column were verified running on our own fleet; see the tested-models matrix for exact sizes and speeds.

Quantization: how much quality you trade for memory

Quantization shrinks the weights by storing them at lower precision. The quality cost is smaller than most people expect, until you go too far.

Quant	Quality vs full precision	Use when
Q8_0	~99%	You have the memory and want near-lossless. About 1.75× larger than Q4.
Q4_K_M	~97%	The default. Best quality-per-GB tradeoff for almost everyone.
Q4_K_S / IQ4_XS	~95 to 96%	Squeeze a model into slightly less memory when Q4_K_M is just over your ceiling.
Q2_K	~85%	Last resort for running a very large model that otherwise will not fit.

One MoE caveat: mixture-of-experts models are more sensitive to aggressive quantization than dense models, because the router's expert selection degrades before the raw benchmark numbers show it. For any 2026 MoE flagship, stay at Q4_K_M or better; the coherence loss at Q2 is worse than the quality percentage suggests.

A warning about context windows and memory

It is tempting to size for a model's advertised context window. Do not. Advertised context is 2 to 4 times larger than effective context for almost every open-weight model. NVIDIA's RULER benchmark, the standard for measuring real long-context capability, found Llama-3.1-8B's 128K claim holds up to about 32K in practice, and one model's 1M-token claim degraded to a 64K effective window. Allocating 40 GB of KV cache for a 256K context you cannot actually reason over is wasted memory. Size for the context you will really use, lean on your tooling's context management for the rest, and treat the number on the model card as marketing.

The other answer: you do not need it all in one Mac

Everything above assumes one machine. But most people who want to run a 120B model already own two or three Macs that sit idle most of the day. Ollama Herd pools them into one endpoint: the 512 GB Studio holds the big model, the MacBooks handle embeddings and smaller models, and requests route to whichever machine has the memory and the spare capacity. The sizing question becomes "what does my whole fleet add up to," not "what fits in my biggest Mac."

pip install ollama-herd            # or: brew install ollama-herd
herd                               # router on your most powerful Mac
herd-node                          # on each other machine, auto-discovers via mDNS

How Much RAM to Run Local AI Models on a Mac