What is a model's memory footprint?
On this site, footprint usually means the memory an LLM occupies while running. That includes its weights plus working space such as the KV cache for your context. A rough rule for the weights is parameters × bits per weight ÷ 8. A 70B model quantized to 4 bits needs about 35 GB for weights alone, and roughly 40 GB or more once the cache is added. A 30B model at 4 bits needs roughly 15–20 GB. This is why the 32 GB Apple Mac mini (M6) handles models of about 30B but rules out 70B. The 96 GB MINISFORUM UM890 Pro can fit 70B quantized models. Longer contexts and higher-precision quantization increase the footprint. Leave headroom for the operating system. Occasionally the word means physical desk space instead, so check the context.