utAI ON YOUR HARDWARE
Start with your memory.
Choose a VRAM budget to explore downloadable language models. Adjust weight precision and the space you want to leave for running them.
Weights are only part of the memory bill.
Parameters are the numbers that make up a model. Lower-precision weights use fewer bytes per parameter: a theoretical 8-billion-parameter model uses 4 GB at 4-bit, 8 GB at 8-bit, or 16 GB at 16-bit, before any runtime overhead.
Actually running it also takes space for the conversation cache, temporary calculations, quantization metadata, and software. Longer conversations or multiple simultaneous requests can require more memory. Our pages show a starting shortlist, not a tested compatibility list.
VRAM, system RAM, and unified memory
These estimates assume the weights reside in one memory budget. They do not model CPU offloading or several GPUs. On a machine with unified memory, the operating system and other applications share that memory too; leave space for them when choosing your reserve.
What about speed?
Memory capacity alone does not tell you tokens per second. Compare measurements only when the exact hardware, model variant, runtime, prompt size, and output settings are known. Scout does not yet have a verified public hardware benchmark feed.
Before downloading
Open the model’s source to check its license, supported runtime, quantized downloads, and maker-reported requirements. A 4-bit estimate does not mean a compatible 4-bit download exists. For mixture-of-experts models, count total weights, not just the parameters active for one token.
Read the technical references: Hugging Face quantization documentation and cache memory documentation.