LocalLLM Rigs

What is a local LLM?

A local LLM is a large language model that runs entirely on hardware you own, such as a mini PC or a Mac, rather than on a remote cloud server. Your prompts stay on your machine, it works offline, and there are no per-request usage limits. In exchange, your hardware sets the limits. Memory capacity decides which models fit: about 30B quantized on the 32 GB Apple Mac mini (M6), and 70B quantized on the 96 GB MINISFORUM UM890 Pro. Memory bandwidth, from 89.6 to 614 GB/s across our catalog, decides speed. Common software includes llama.cpp, Ollama and LM Studio, plus MLX on Apple silicon. CUDA-based tools need an Nvidia GPU, so they won't run on Macs.