LocalLLM Rigs

About Marcus Lindqvist

user-content-Marcus Lindqvist

Marcus Lindqvist researches and compares the hardware people use to run open-source language models at home: graphics cards, unified-memory Macs, AI mini PCs and the occasional used workstation. He prefers a pragmatic setup. A used 24 GB graphics card in an ordinary desktop is his default pick for chatting and coding help with mid-sized models, and he points people toward large unified-memory machines only when they truly need 70B-class models or a quiet, low-power box. He reads data sheets, price histories and published benchmarks so readers can see what each extra dollar actually buys in tokens per second. The spec he finds most often overrated is the 'TOPS' figure on AI-branded laptops and NPUs. The one he finds most often overlooked is memory bandwidth, which usually decides how fast a model actually feels.

How I choose and rate

I build every rating from Amazon listings and their price history, manufacturer data sheets, and independent reviews and benchmark measurements published on the web. I put all of them through the same criteria:

  • Memory capacity (VRAM or unified memory): 24 GB is my floor for running ~30B-parameter models at 4-bit with room for context. 48 GB or more is where 70B models get comfortable. This decides which models you can run at all.
  • Memory bandwidth: Generation speed tracks bandwidth closely. Around 1 TB/s feels fast, and under 300 GB/s gets sluggish on big models. This decides the trade-off between capacity and speed.
  • Tokens per second per dollar: I divide published generation speeds for a common model size by the current street price. About 10 tokens/s is my bar for comfortable chatting. This often decides new versus used.
  • Prompt processing speed: This matters for long documents and coding agents. Macs and APUs often trail discrete GPUs here even when generation looks fine.
  • Software support: CUDA works with almost everything. ROCm and Metal do well with llama.cpp-based tools but are patchier elsewhere. This decides how much tinkering you sign up for.
  • Power, heat and noise: Idle draw and fan noise matter for a box that runs all day in your home.

Affiliate commissions never change a ranking.