feat(hardware): model_vram_estimate for model-to-hardware VRAM fit checks #70

Closed

pyr0ball wants to merge 0 commits from feat/64-vram-estimate into main

pyr0ball commented

2026-07-10 17:46:32 -07:00

Owner

Summary

Add circuitforge_core.hardware.model_vram_estimate(hf_model_id, quant_level, ...) -> VramEstimate. Answers can-this-hardware-run-this-model by querying the HF Hub API for parameter count (safetensors.total) and architecture (config.json), applying vram_gb = weights + kv_cache (GQA-aware) + overhead.
Reference algorithm from LLMcalc (https://github.com/Raskoll2/LLMcalc) - unlicensed upstream, algorithm reference only, no code copied, no new dependency.
Supports fp32/fp16/bf16/int8/q8/q8_0/q6_k/q5_k_m/q5_0/int4/q4/q4_k_m/q4_0/q3_k_m/q2_k quant levels.
Raises ModelVramLookupError on HF API failures / missing safetensors metadata; ValueError on unknown quant level.
Docs: appended a section to docs/modules/hardware.md.
Bump to 0.22.0 (branched from main before #58 signal-bus PR merged, so version numbers may need reconciling at merge time).

Closes #64

Test plan

pytest tests/test_hardware -v - 45/45 passed (14 new for vram_estimate)

## Summary - Add circuitforge_core.hardware.model_vram_estimate(hf_model_id, quant_level, ...) -> VramEstimate. Answers can-this-hardware-run-this-model by querying the HF Hub API for parameter count (safetensors.total) and architecture (config.json), applying vram_gb = weights + kv_cache (GQA-aware) + overhead. - Reference algorithm from LLMcalc (https://github.com/Raskoll2/LLMcalc) - unlicensed upstream, algorithm reference only, no code copied, no new dependency. - Supports fp32/fp16/bf16/int8/q8/q8_0/q6_k/q5_k_m/q5_0/int4/q4/q4_k_m/q4_0/q3_k_m/q2_k quant levels. - Raises ModelVramLookupError on HF API failures / missing safetensors metadata; ValueError on unknown quant level. - Docs: appended a section to docs/modules/hardware.md. - Bump to 0.22.0 (branched from main before #58 signal-bus PR merged, so version numbers may need reconciling at merge time). Closes #64 ## Test plan - [x] pytest tests/test_hardware -v - 45/45 passed (14 new for vram_estimate)

pyr0ball added 1 commit 2026-07-10 17:46:33 -07:00

feat(hardware): add model_vram_estimate for model-to-hardware VRAM fit checks

CI / test (pull_request) Has been cancelled

Details

5d5c02ff84

Answers "can this hardware run model X at quantization level Y?" by
querying the HuggingFace Hub API for parameter count (safetensors) and
architecture (config.json), then applying the standard VRAM formula:
weights + KV cache (GQA-aware) + overhead. Reference algorithm from
LLMcalc (no code copied, unlicensed upstream).

Bump to 0.22.0.

Closes: #64

pyr0ball commented

2026-07-10 19:19:34 -07:00

Author

Owner

Merged into main via the freeze/0.22.0 integration branch (fast-forward) and released as v0.22.0: https://git.opensourcesolarpunk.com/Circuit-Forge/circuitforge-core/releases/tag/v0.22.0. Closing since the commits are already on main.