LocalLLM Rigs

What is an NPU?

An NPU (neural processing unit) is a dedicated block on a processor designed to run AI operations efficiently at low power. Typical uses include background blur, transcription and other small always-on tasks. NPU performance is usually quoted in TOPS. Among this catalog's x86 machines, the GPU/NPU figure ranges from 16 TOPS on the MINISFORUM UM890 Pro to 86 TOPS on the GEEKOM A9 Max. For running large language models, popular tools such as llama.cpp mostly use the CPU or GPU instead, and NPU support remains limited. Token generation is also bound by memory bandwidth, which an NPU does not increase. Treat a high NPU rating as useful for specific supported apps, not as a predictor of LLM speed. Apple's machines here are specified by GPU cores with built-in Neural Accelerators instead.