hardware module: VRAM tier ladder + profile generator + auto-detect script #5

New issue

Closed

opened 2026-04-01 08:10:28 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-04-01 08:10:28 -07:00

Owner

Hardware Profile Logic for Self-Hoster Config Generation

Add a hardware module to circuitforge-core that maps a user's hardware spec to an optimized llm.yaml configuration.

What belongs in core (product-agnostic)

VRAM tier ladder: thresholds → recommended models, quantization levels, context window limits
Model catalog: ollama model IDs, VRAM requirements, vision capability flag
Profile generator: takes HardwareSpec → returns LLMConfig (ready to serialize as llm.yaml)
compose.override.yml fragment generator: wires in the right ollama/vllm image + model pull

Auto-detect helper

Small cross-platform script (detect_hardware.py or shell): reads nvidia-smi / rocm-smi / system_profiler (macOS) / /proc/meminfo
Outputs a HardwareSpec JSON blob the UI can ingest
Packaged as a one-liner the account page offers for download/copy-paste

Proposed module layout

circuitforge_core/hardware/
    __init__.py
    models.py       # HardwareSpec, HardwareProfile, LLMConfig dataclasses
    tiers.py        # VRAM tier ladder + model catalog
    generator.py    # HardwareSpec -> LLMConfig + compose fragment
    detect.py       # auto-detect script (cross-platform)

Acceptance criteria

HardwareSpec(vram_gb, ram_gb, gpu_count, gpu_vendor) dataclass
generate_profile(spec: HardwareSpec, products: list[str]) -> LLMConfig
Returns valid llm.yaml-compatible dict
Auto-detect script works on Linux (nvidia + amd) and macOS (Metal/CPU)
80%+ test coverage on tier logic and generator

## Hardware Profile Logic for Self-Hoster Config Generation Add a `hardware` module to `circuitforge-core` that maps a user's hardware spec to an optimized `llm.yaml` configuration. ### What belongs in core (product-agnostic) - VRAM tier ladder: thresholds → recommended models, quantization levels, context window limits - Model catalog: ollama model IDs, VRAM requirements, vision capability flag - Profile generator: takes `HardwareSpec` → returns `LLMConfig` (ready to serialize as `llm.yaml`) - `compose.override.yml` fragment generator: wires in the right ollama/vllm image + model pull ### Auto-detect helper - Small cross-platform script (`detect_hardware.py` or shell): reads `nvidia-smi` / `rocm-smi` / `system_profiler` (macOS) / `/proc/meminfo` - Outputs a `HardwareSpec` JSON blob the UI can ingest - Packaged as a one-liner the account page offers for download/copy-paste ### Proposed module layout ``` circuitforge_core/hardware/ __init__.py models.py # HardwareSpec, HardwareProfile, LLMConfig dataclasses tiers.py # VRAM tier ladder + model catalog generator.py # HardwareSpec -> LLMConfig + compose fragment detect.py # auto-detect script (cross-platform) ``` ### Acceptance criteria - [ ] `HardwareSpec(vram_gb, ram_gb, gpu_count, gpu_vendor)` dataclass - [ ] `generate_profile(spec: HardwareSpec, products: list[str]) -> LLMConfig` - [ ] Returns valid `llm.yaml`-compatible dict - [ ] Auto-detect script works on Linux (nvidia + amd) and macOS (Metal/CPU) - [ ] 80%+ test coverage on tier logic and generator