LocalLLM Rigs

What is an eGPU?

An eGPU (external GPU) is a desktop graphics card in an enclosure or open dock with its own power supply. It connects to a computer by cable, usually via OCuLink or Thunderbolt/USB4. For local LLMs, the appeal is dedicated VRAM, commonly 12–32 GB on desktop cards. That VRAM is far faster than the 89.6 GB/s shared DDR5 in several mini PCs here, so models that fit in it generate tokens much more quickly. OCuLink offers about 64 Gb/s (PCIe 4.0 x4) versus 40 Gb/s for USB4. The link speed mostly affects model loading and any work split with system RAM. The MINISFORUM AI X1 Pro-370 and UM890 Pro support eGPUs via OCuLink. Apple Silicon Macs such as the Mac mini don't support eGPUs.