What are model weights?
Weights, also called parameters, are the billions of numbers a language model learned during training. Running the model means loading all of them into memory, so the weights file is the main thing that has to fit in your machine's RAM or unified memory. Size depends on parameter count and precision. At 16-bit precision each weight takes 2 bytes. With 4-bit quantization each takes roughly half a byte, plus some overhead. A 4-bit 30B model therefore needs roughly 17–20 GB, and a 4-bit 70B model roughly 40 GB. That is why the Apple Mac mini (M6), capped at 32 GB, suits models around 30B but rules out 70B. Your operating system, apps and the context cache also need memory alongside the weights, so leave headroom beyond the file size.