What is a context window?
The context window is the largest amount of text, measured in tokens, that a model can take into account in one go. It includes the system prompt, the conversation so far, any pasted documents and the model's own reply. Many current open models support 8K to 128K tokens, and some support more; check each model's card. Locally, the limit you can actually use also depends on memory. The runtime keeps a KV cache that grows with every token of context, and it competes with the model weights for RAM. On a 32 GB machine like the Apple Mac mini (M6), a long context can push a mid-size model out of memory. A 128 GB machine such as the Apple Mac Studio (M5 Max) leaves more headroom. Long prompts also take longer to process before the first word appears.