What is an LLM?
An LLM, or large language model, is a neural network trained on large amounts of text to predict the next token. That simple skill lets it chat, summarize, write code and answer questions. LLMs are usually described by their parameter count. Small models have a few billion parameters (for example 7B or 8B), mid-size models around 30B, and large ones 70B or more. Parameter count largely decides how much memory the model needs. The Apple Mac mini (M6), with 32 GB, handles quantized models of about 30B. The AMD Ryzen AI Halo's 128 GB shared pool can load models as large as 200B. Memory bandwidth then sets how fast the model generates text, measured in tok/s.