Open-Weight Models

10 profiles in this group. Use this hub page to compare practical VRAM floor, expected throughput, and best local-vs-cloud path.

RAM planning is included because Ollama may spill layers to CPU when VRAM is tight, especially with long context.

Model Data VRAM min VRAM optimal System RAM RTX 3090 fit Best local GPU Cloud fallback Detail
GPT-OSS 120B CLOUD Estimated 70GB 78GB 128GB+ Cloud / larger GPU Dual RTX 4090 (model parallel) A100 80GB Open
GPT-OSS 120B FP16 Estimated 80GB 92GB 128GB+ Cloud / larger GPU Cloud-first (no practical single-GPU local) H100/H200 class Open
GPT-OSS 120B Q4 Estimated 70GB 80GB 128GB+ Cloud / larger GPU Dual RTX 4090 (model parallel) A100 80GB Open
GPT-OSS 120B Q5 Estimated 80GB 82GB 128GB+ Cloud / larger GPU Cloud-first (no practical single-GPU local) H100/H200 class Open
GPT-OSS 120B Q8 Estimated 74GB 84GB 128GB+ Cloud / larger GPU Cloud-first (no practical single-GPU local) H100/H200 class Open
GPT-OSS 20B CLOUD Estimated 20GB 28GB 64GB+ Borderline RTX 6000 Ada 48GB A100 80GB Open
GPT-OSS 20B FP16 Estimated 30GB 42GB 64GB+ Cloud / larger GPU RTX 6000 Ada 48GB A100 80GB Open
GPT-OSS 20B Q4 Estimated 16GB 20GB 32GB+ Comfortable RTX 3090 24GB A6000 48GB Open
GPT-OSS 20B Q5 Estimated 20GB 22GB 32GB+ Comfortable RTX 3090 24GB A6000 48GB Open
GPT-OSS 20B Q8 Estimated 24GB 34GB 64GB+ Borderline RTX 6000 Ada 48GB A100 80GB Open
Back to all groups Use VRAM calculator Run large models on RunPod Try Vast.ai fallback

We may earn a commission if you click links on this page.