About Marcus Lindqvist
How I choose and rate
I build every rating from Amazon listings and their price history, manufacturer data sheets, and independent reviews and benchmark measurements published on the web. I put all of them through the same criteria:
- Memory capacity (VRAM or unified memory): 24 GB is my floor for running ~30B-parameter models at 4-bit with room for context. 48 GB or more is where 70B models get comfortable. This decides which models you can run at all.
- Memory bandwidth: Generation speed tracks bandwidth closely. Around 1 TB/s feels fast, and under 300 GB/s gets sluggish on big models. This decides the trade-off between capacity and speed.
- Tokens per second per dollar: I divide published generation speeds for a common model size by the current street price. About 10 tokens/s is my bar for comfortable chatting. This often decides new versus used.
- Prompt processing speed: This matters for long documents and coding agents. Macs and APUs often trail discrete GPUs here even when generation looks fine.
- Software support: CUDA works with almost everything. ROCm and Metal do well with llama.cpp-based tools but are patchier elsewhere. This decides how much tinkering you sign up for.
- Power, heat and noise: Idle draw and fan noise matter for a box that runs all day in your home.
Affiliate commissions never change a ranking.