YOUR MEMORY BUDGET
Explore models by theoretical weight storage. This shortlist covers available downloadable language models with a known or name-derived parameter count.
The reserve is your assumption, not a measured overhead allowance. The default 2 GB may be too little for your runtime or context length. Results are not confirmed fits and have no verified speed rating. Weight estimates use decimal GB and do not include quantization overhead. For timed results on one 12GB machine, see measured speeds.
BAAI · Downloadable
Generates text, according to its published model architecture (LlamaForCausalLM). No task is listed on Hugging Face, so check the model card for intended use.
BAAI · Downloadable
Generates text for conversations and writing.
Baidu · Downloadable
Understands text and images and responds in text.
DeepSeek · Downloadable
Generates text for conversations and writing. Supports a reasoning mode.
DeepSeek · Downloadable
Generates text for conversations and writing. Supports a reasoning mode.
inclusionAI · Downloadable
Generates text for conversations and writing.
inclusionAI · Downloadable
Generates text for conversations and writing.
inclusionAI · Downloadable
Understands text and images and responds in text.
inclusionAI · Downloadable
Understands text and images and responds in text.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
LightOn · Downloadable
Generates text for conversations and writing.
With 2 GB set aside, 22 GB remains for weights. At 4-bit that allows up to 44 billion parameters in the theoretical calculation. An 8-billion-parameter model alone needs approximately 4 GB at this precision, before its conversation cache and runtime.
Its actual weight format may need more space, the runtime may not support its architecture, or the conversation cache may exceed your reserve. Vision models can also need space for image processing. Check the original model card and measure your exact setup before relying on it.
Read the memory guide and technical sources · Compare two models
ut