Hardware profile builder — self-hoster LLM config wizard (account page + cf-core) #26

New issue

Open

opened 2026-04-01 08:10:19 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-04-01 08:10:19 -07:00

Owner

Hardware Profile Builder for Self-Hosters

A guided hardware profile builder that lets self-hosters specify their available hardware and receive an optimised llm.yaml + compose.override.yml config for their CF products.

Status

cf-core backend: complete (v0.3.0)

circuitforge_core.hardware.detect_hardware() — probes nvidia-smi / rocm-smi / Apple silicon / CPU fallback
circuitforge_core.hardware.select_tier(vram_mb) — maps physical VRAM to named tier (2/4/6/8/16/24 GB)
circuitforge_core.hardware.generate_profile(spec) — produces LLMConfig (llm.yaml-compatible)
GPU profile YAMLs for 6/8/16/24 GB profiles shipping with cf-core

Remaining: UI layer

Account/settings page widget (web UI) that calls detect_hardware() and renders recommended config
Download button for generated llm.yaml
Optionally: compose.override.yml generation for Docker resource limits

Problem

Configuring local LLM inference correctly (VRAM tiers, quantisation, model selection, fallback chains) is the #1 setup friction point for non-technical self-hosters. Without guidance, users pick models too large for their VRAM and blame the product.

References

cf-core hardware module: Circuit-Forge/circuitforge-core#5 (closed)
Roadmap #27 (Windows native) — detect_hardware() is also the foundation for Windows process-mode config

## Hardware Profile Builder for Self-Hosters A guided hardware profile builder that lets self-hosters specify their available hardware and receive an optimised `llm.yaml` + `compose.override.yml` config for their CF products. ### Status **cf-core backend: complete (v0.3.0)** - `circuitforge_core.hardware.detect_hardware()` — probes nvidia-smi / rocm-smi / Apple silicon / CPU fallback - `circuitforge_core.hardware.select_tier(vram_mb)` — maps physical VRAM to named tier (2/4/6/8/16/24 GB) - `circuitforge_core.hardware.generate_profile(spec)` — produces `LLMConfig` (llm.yaml-compatible) - GPU profile YAMLs for 6/8/16/24 GB profiles shipping with cf-core **Remaining: UI layer** - Account/settings page widget (web UI) that calls `detect_hardware()` and renders recommended config - Download button for generated `llm.yaml` - Optionally: `compose.override.yml` generation for Docker resource limits ### Problem Configuring local LLM inference correctly (VRAM tiers, quantisation, model selection, fallback chains) is the #1 setup friction point for non-technical self-hosters. Without guidance, users pick models too large for their VRAM and blame the product. ### References - cf-core hardware module: [Circuit-Forge/circuitforge-core#5](https://git.opensourcesolarpunk.com/Circuit-Forge/circuitforge-core/issues/5) (closed) - Roadmap #27 (Windows native) — `detect_hardware()` is also the foundation for Windows process-mode config