LocalLLM Rigs

Local LLM rigs: mini PCs, Macs and GPU builds compared

A 70B model at Q4 quantization needs about 40 GB just for its weights. That one number rules out most laptops and every single consumer graphics card, including the 32 GB RTX 5090, before price even comes up. This guide compares mini PCs, Apple Silicon Macs and GPU builds for running a local LLM. It draws on retailer listings, manufacturer specs and independent llama.cpp and MLX benchmarks, and judges every rig on usable memory, memory bandwidth and published tok/s figures.

Buyer’s guide

Best budget mini PC for local AI under $1100

At under $1,100, upgradeable RAM decides how big a local LLM you can run. Here's how the UM890 Pro, UM870 Slim and Mac mini (M6) compare for AI.

Read more →

Top picks

Editor’s Choice
Apple Mac mini (M5 Pro)

307GB/s unified memory for running local LLMs from a five-inch desktop

  • 307GB/s bandwidth: faster LLM token generation
  • GPU Neural Accelerators: quicker prompt processing
  • Five-inch box: compact always-on agent server
  • Fixed memory: choose LLM capacity upfront
  • macOS only: CUDA-based tools won't run
★★★★★ 4.5 $1,549.00
Runner-up
Apple Mac Studio (M5 Max)

Up to 128GB unified memory at 614GB/s for running large LLMs quietly on your desk

  • 128GB unified memory fits large quantized LLMs
  • 614GB/s bandwidth speeds token generation
  • Thunderbolt 5 ports enable multi-Mac clustering
  • 128GB cap limits the largest models
  • No CUDA: relies on Metal/MLX tooling
★★★★★ 4.5 $2,299.99
Editor's Choice
AMD Ryzen AI Halo

128GB unified memory to run up to 200B-parameter models locally on Linux

  • 128GB shared pool: loads 200B models
  • Preloaded ROCm stack: less Linux setup
  • 10GbE LAN: faster model file transfers
  • 256 GB/s limits dense-model token speed
  • Integrated 8060S trails discrete GPU throughput
★★★★★ 4.4 $4,760.00
Best Value
Apple Mac mini (M6)

Up to 32GB unified memory and 170GB/s in a 5x5-inch box for quiet, always-on local LLMs

  • 32GB unified memory fits ~30B quantized models
  • GPU Neural Accelerators speed prompt processing
  • 5x5-inch footprint for always-on agent hosting
  • 170GB/s bandwidth limits tokens per second
  • 32GB cap rules out 70B models
★★★★★ 4.3 $779.00
GEEKOM A9 Max

Ryzen AI 9 470 mini PC with room for 128 GB RAM to run larger local LLMs

  • 128 GB RAM ceiling fits bigger models
  • 2TB SSD stores many model files
  • Dual 2.5GbE for serving models locally
  • Stock 32GB limits model size
  • Shared DDR5 slows tokens vs. dedicated GPUs
★★★★★ 4.2 $1,599.00
MINISFORUM AI X1 Pro-370

128GB RAM ceiling and OCuLink for a GPU upgrade later

  • 128GB RAM: fits 70B quantized models
  • OCuLink: add a desktop eGPU later
  • 65W max draw keeps running costs low
  • 890M iGPU: slow tokens on large models
  • SO-DIMM DDR5 trails unified-memory bandwidth
★★★★★ 4.2 $1,247.00
MINISFORUM UM890 Pro

Up to 96 GB of RAM plus OcuLink, so it can load large models now and take an external GPU later

  • 96 GB RAM: fits 70B quantized models
  • OcuLink: add a desktop GPU later
  • Two PCIe 4.0 slots store models
  • Shared DDR5 slows large-model tokens/sec
  • OcuLink adapter uses up an M.2 slot
★★★★★ 4.2 $1,015.00
MINISFORUM UM870 Slim

Ryzen 7 8745H with a 96GB RAM ceiling for mid-size local models

  • 96GB RAM ceiling fits bigger models
  • 32GB stock handles mid-size quantized models
  • USB4 40 Gbps for eGPU expansion
  • Dual-channel SO-DIMM caps token speed
  • 780M shares RAM, no dedicated VRAM
★★★★★ 4.1 $889.00

Prices as of Nov 12, 2026; they may have changed since.

Top reviews

  1. 1
    Review AMD Ryzen AI Halo
    4.4
  2. 2
    Review Apple Mac mini (M5 Pro)
    4.5
  3. 3
    Review Apple Mac mini (M6)
    4.3
  4. 4
    Review Apple Mac Studio (M5 Max)
    4.5
  5. 5
    Review GEEKOM A9 Max
    4.2

Browse by category

Latest reviews

Guides & articles

Memory decides what you can run, bandwidth decides how fast

Most buyers shop by chip name: Ryzen AI Max vs. M4 Max vs. RTX 4090, or 16 cores vs. 12. For a local LLM, two other numbers matter more. Memory capacity decides which models load at all. Memory bandwidth (GB/s) decides how many tokens per second you get once they do.

  • 256 GB/sStrix Halo (Ryzen AI Max) unified memory bandwidth
  • 546 GB/sM4 Max unified memory bandwidth, about 2× Strix Halo
  • ~1,000 GB/sRTX 4090 VRAM bandwidth, about 4× Strix Halo
  • ~40 GBweights for a 70B model at Q4, before context

Why bandwidth sets the speed

Token generation is memory-bound. To produce each token, the hardware reads nearly the entire set of model weights from memory. Compute units spend much of that time waiting on the memory bus. A simple estimate works well: bandwidth ÷ model size ≈ theoretical tok/s ceiling. Real-world figures land below that ceiling, but they scale with it.

For a 32B model at Q4 (about 20 GB), the ceilings are roughly:

  • Strix Halo: about 13 tok/s
  • M4 Max: about 27 tok/s
  • RTX 4090: about 50 tok/s

The ratio between those numbers explains most of the speed gap between the three platforms. A faster CPU or a newer GPU architecture changes the result much less.

Why capacity sets the model list

Quantization shrinks the weights, but it doesn't make them disappear. The model must fit in memory, with room left for the context window.

Model size Approx. weights at Q4 Realistic memory target
8B ~5 GB 16 GB system, or any 8 GB+ GPU
32B ~20 GB 32 GB unified, or a 24 GB card with short context
70B ~40 GB 64 GB+ unified; no single consumer card fits it

The KV cache grows with the context window. A 32K-token context on a 32B model can add several gigabytes. Budget headroom above the weights, or long-document work will spill out of memory.

Ignore the NPU badge for LLM speed

Ollama, LM Studio and llama.cpp run inference on the iGPU or discrete GPU, not the NPU. A "50 TOPS" sticker tells you almost nothing about tok/s today. Compare GB/s and gigabytes instead.

The unified memory advantage

On Apple Silicon and Ryzen AI Max, the CPU and GPU share one pool of unified memory. The GPU can address most of the system RAM. A 128 GB mini PC or Mac can therefore load a 70B model at Q4 with generous context headroom.

A 24 GB VRAM card can't hold that model at all. It has to offload layers to system RAM, which is far slower, and speed drops sharply.

That is the core trade-off for this page:

  • Unified memory buys capacity: it runs bigger models, at moderate speed.
  • VRAM buys bandwidth: it runs models that fit, and runs them quickly.

Mini PC or GPU tower: the trade-off in one view

Choose the category before you compare products. A unified-memory mini PC (Strix Halo / Ryzen AI Max, or an Apple Silicon Mac) and a discrete-GPU tower fail in opposite ways. The first runs out of speed. The second runs out of room.

Factor Unified-memory mini PC / Mac GPU tower Mini PC + OCuLink eGPU
Memory for the model Up to 128 GB unified memory, most of it usable by the iGPU Capped by VRAM: 24 GB on an RTX 4090, 32 GB on an RTX 5090 Card's VRAM first, then system RAM
Memory bandwidth About 256 GB/s on Strix Halo, up to 546 GB/s on M4 Max About 1,000–1,800 GB/s on high-end cards Full GPU bandwidth only while the model fits in VRAM
Prompt processing Slow on long prompts and large context windows Much faster, because the GPU has far more compute Fast for layers on the card
Token generation Usable on 32B–70B at Q4, but clearly slower Fast, as long as the whole model sits in VRAM Drops sharply once layers spill to system RAM
Largest model per dollar Strongest Weakest. More capacity means more cards Middle
Power and noise Low draw, small and quiet High draw, large PSU, fan noise under load Mini PC plus the card's full power budget
Upgrade path Usually none. Memory is soldered Add or swap GPUs Swap the eGPU card
Software llama.cpp, Ollama, LM Studio. MLX on Macs only llama.cpp and CUDA-first tools llama.cpp, Ollama, LM Studio

Mini PCs and Macs win on capacity per dollar. One quiet box covers what a GPU tower needs two or three 24–32 GB cards to hold in VRAM. The cost is speed. With roughly a quarter to half the bandwidth of a flagship GPU, these machines generate tokens more slowly, and long prompts take noticeably longer to process before the first token appears.

GPU towers win on speed inside a hard ceiling. Prompt processing and generation are both fast, but only for models that fit in VRAM. Going past 24–32 GB means splitting the model across several cards. That adds cost, power draw, PCIe lanes and setup complexity.

OCuLink is not free VRAM

An OCuLink eGPU turns a mini PC into a hybrid. A 14B–32B model that fits on the card runs at GPU speed. A model larger than the card's VRAM still spills its overflow layers into system memory, and generation slows to the speed of that slower memory. The eGPU makes models that fit run faster. It does not let you run bigger models fast.

Software support is part of the decision. MLX runs only on Apple Silicon and is often the fastest way to run models on a Mac. llama.cpp, which Ollama and LM Studio are built on, runs on every platform: CUDA, ROCm, Vulkan and Metal. If you might switch hardware later, standardizing on llama.cpp-based tools keeps your models and workflow portable.

Pick a 64–128 GB mini PC or Mac if you want to run 70B-class models at Q4 on a quiet desk and can accept moderate tok/s. Pick a GPU tower if your models stay at 32B or below, fitting in 24–32 GB of VRAM, and you need fast prompt processing and generation for agents or batch work.