LocalLLM Rigs

What are local LLMs?

Local LLMs are large language models that run entirely on your own computer rather than on a cloud service. You download the model file once, and your prompts and responses stay on your machine. That helps with privacy, offline use and avoiding per-request usage limits. The trade-off is that your hardware sets the limits: memory capacity decides which models fit, and memory bandwidth largely decides how fast they reply. The machines covered on this site start at the Apple Mac mini (M6), with 32 GB of memory and 170 GB/s of bandwidth, enough for quantized models of roughly 30B. They go up to 128 GB on systems such as the Apple Mac Studio (M5 Max) and the AMD Ryzen AI Halo. Software matters too. For example, CUDA-based tools won't run on macOS, where Metal and MLX tooling is used instead.