Commit graph

107 commits

Author SHA1 Message Date
80b0d5fd34 feat: v0.9.0 — cf-text, pipeline crystallization engine, multimodal pipeline, a11y preferences
Some checks failed
CI / test (push) Waiting to run
Mirror / mirror (push) Has been cancelled
Release — PyPI / release (push) Has been cancelled
Closes #33, #37, #38, #41, #42.

## cf-text (closes #41)
- New module: `circuitforge_core.text` — direct local inference bypassing ollama/vllm
- Backends: llama.cpp (GGUF), transformers (HF), mock
- Auto-detects backend from file extension; CF_TEXT_BACKEND env override
- Optional 4-bit/8-bit quantisation via bitsandbytes (CF_TEXT_4BIT / CF_TEXT_8BIT)
- process-level singleton + per-request `make_backend()` path

## Pipeline crystallization engine (closes #33, #37)
- FPGA→ASIC model: LLM-discovered paths → deterministic workflows after N approvals
- `models.py`: PipelineRun (incl. review_duration_ms + output_modified per #37),
  CrystallizedWorkflow, Step, hash_input()
- `recorder.py`: append-only JSON run log under ~/.config/circuitforge/pipeline/
- `crystallizer.py`: threshold check, majority/most-recent step strategy,
  rubber-stamp warning (review_duration_ms < 5s triggers warnings.warn)
- `registry.py`: exact + fuzzy match, deactivate-without-delete, colon-safe filenames
- `executor.py`: deterministic steps with transparent LLM fallback

## Multimodal chunked pipeline (closes #42)
- `pipeline/multimodal.py`: cf-docuvision pages → cf-text streaming
- `run()` yields PageResult per page (progressive, no full-doc buffer)
- `stream()` yields (page_idx, token) tuples for token-level UI rendering
- `vram_serialise` flag + `swap_fn` hook for 8GB GPU VRAM management
- `prompt_fn` callback for product-specific prompt construction

## Accessibility preferences (closes #38)
- `preferences/accessibility.py`: PREF_REDUCED_MOTION, PREF_HIGH_CONTRAST,
  PREF_FONT_SIZE, PREF_SCREEN_READER with get/set helpers
- Exported from preferences package __init__

## LLM router fix
- cf-orch backends: skip reachability pre-check; allocation starts the service
- Static backends: reachability check remains in place
2026-04-08 23:17:18 -07:00
3075e5d3da feat(tts): add cf-tts module with ChatterboxTurbo backend and FastAPI service
Some checks are pending
CI / test (push) Waiting to run
Mirror / mirror (push) Waiting to run
- TTSBackend Protocol + TTSResult dataclass (audio_bytes, sample_rate, duration_s, format)
- MockTTSBackend: silent WAV clip, no GPU required, Protocol assert at import
- ChatterboxTurboBackend: ResembleAI chatterbox-turbo via chatterbox-tts package
  - from_local() loads model from snapshot dir
  - audio_prompt voice cloning via temp file
  - _encode_audio helper: OGG (default), WAV, MP3 via torchaudio
- circuitforge_core.tts module-level synthesize() singleton (CF_TTS_MODEL / CF_TTS_MOCK)
- FastAPI app: GET /health, POST /synthesize (multipart form, returns audio bytes)
  - default format: ogg (smaller than WAV, no patents)
  - X-Duration-S / X-Model / X-Sample-Rate response headers
  - CLI: --model --port --host --gpu-id --mock
- pyproject.toml: tts-chatterbox + tts-service extras
- Sample rate: 24000 Hz (S3GEN_SR from chatterbox internals)
2026-04-08 23:15:05 -07:00
67493048e2 feat(stt): add cf-stt module — FasterWhisperBackend + managed FastAPI app
Some checks are pending
CI / test (push) Waiting to run
Mirror / mirror (push) Waiting to run
- STTBackend Protocol + STTResult/STTSegment frozen dataclasses (base.py)
- MockSTTBackend for CI/tests (no GPU needed, CF_STT_MOCK=1)
- FasterWhisperBackend: loads model once, thread-safe, VRAM estimate by model size
- app.py: FastAPI service runnable as managed process by cf-orch
  POST /transcribe (multipart audio) → STTTranscribeResponse-compatible JSON
  GET  /health → {status, model, vram_mb}
- __init__.py: process-level singleton + transcribe() convenience fn
- pyproject.toml: stt-faster-whisper + stt-service optional dep groups
2026-04-08 22:14:46 -07:00
5766fa82ab refactor: replace vision stub with cf-vision shim (cf-core#36)
Some checks failed
CI / test (push) Has been cancelled
Mirror / mirror (push) Has been cancelled
circuitforge_core.vision.router now re-exports VisionRouter from the
standalone cf-vision repo. Existing imports unchanged; falls back to
a helpful ImportError stub if cf-vision is not installed.

Closes cf-core#36
2026-04-06 17:59:05 -07:00
48d33a78ef fix: migration runner resilient to partial-failure via retry-with-removal
Some checks are pending
CI / test (push) Waiting to run
Mirror / mirror (push) Waiting to run
Instead of splitting SQL on semicolons (fragile — semicolons appear inside
comments and string literals), use executescript() for correct tokenization.
On 'duplicate column name' error (caused by a prior partial run that
auto-committed some ALTER TABLE statements before crashing), strip the
already-applied ADD COLUMN statement from the script and retry.  Limit
to 20 attempts to prevent infinite loops on genuinely broken SQL.

This replaces the earlier per-statement split approach which broke on
migration 004 comment text containing a semicolon inside a -- comment,
causing the remainder ('one row per...') to be treated as raw SQL.
2026-04-05 22:39:12 -07:00
c9c4828387 fix: make migration runner resilient to partial-failure recovery
Some checks are pending
CI / test (push) Waiting to run
Mirror / mirror (push) Waiting to run
SQLite's executescript() auto-commits each DDL statement individually.
If a migration crashes mid-run, prior ALTER TABLE statements are already
committed but the migration is never recorded as applied.  On restart,
the runner re-runs the same file and hits 'duplicate column name' on
already-applied statements, breaking subsequent startups permanently.

Replace executescript() with per-statement execute() calls.  'Duplicate
column name' OperationalErrors are caught and logged as warnings so the
migration can complete and be marked as done.  All other errors still
propagate normally.
2026-04-05 22:23:29 -07:00
19a26e02a0 Merge pull request 'feat: re-export make_feedback_router from circuitforge_core.api (closes #30)' (#32) from feature/api-exports into main
Some checks are pending
CI / test (push) Waiting to run
Mirror / mirror (push) Waiting to run
2026-04-05 21:37:33 -07:00
e5c26f0e67 Merge pull request 'feat: CF_LICENSE_KEY validation via Heimdall (closes #26)' (#31) from feature/license-validation into main
Some checks are pending
CI / test (push) Waiting to run
Mirror / mirror (push) Waiting to run
2026-04-05 21:33:52 -07:00
3c9c765668 feat: re-export make_feedback_router from circuitforge_core.api (closes #30)
Some checks failed
CI / test (pull_request) Has been cancelled
2026-04-05 21:21:44 -07:00
bb2ed3e992 fix: parameterize bare dict type annotations in license module
Some checks failed
CI / test (pull_request) Has been cancelled
2026-04-05 21:19:10 -07:00
f3bc4ac605 feat: add CF_LICENSE_KEY validation via Heimdall (closes #26)
Introduces circuitforge_core.config.license with validate_license() and
get_license_tier(). Both functions are safe to call when CF_LICENSE_KEY
is absent, returning free tier gracefully. Results are cached 30 min per
(key, product) pair. CF_LICENSE_URL env var overrides the default
Heimdall endpoint. Re-exports added to config.__init__. Existing
test_config.py moved into tests/test_config/ package to co-locate with
new test_license.py (10 tests; 204 total passing).
2026-04-05 21:16:57 -07:00
d98d27be3d chore: remove misplaced cf-orch docker workflow (belongs in circuitforge-orch)
Some checks are pending
CI / test (push) Waiting to run
Mirror / mirror (push) Waiting to run
2026-04-05 20:53:13 -07:00
4d858af4d1 Merge pull request 'ci: Forgejo Actions — CI, PyPI release, mirrors (closes #27)' (#29) from feature/ci-cd into main
Some checks are pending
CI / test (push) Waiting to run
Mirror / mirror (push) Waiting to run
2026-04-05 20:51:31 -07:00
874354f235 fix: continue-on-error for mirror steps; guard duplicate Forgejo release creation
Some checks failed
CI / test (pull_request) Has been cancelled
2026-04-05 20:51:18 -07:00
3050179b2f fix: use jq for safe JSON in release step; remove redundant dev deps in ci 2026-04-05 20:51:18 -07:00
378d125ba6 ci: add Forgejo Actions workflows — CI, PyPI release, mirrors, cliff.toml (closes #27) 2026-04-05 20:51:18 -07:00
1cbea29817 Merge pull request 'feat: shared feedback router factory (closes #23)' (#28) from feature/api-feedback into main 2026-04-05 20:50:24 -07:00
f0a9ec5c37 fix: raise 502 on label creation failure; narrow subprocess exception scope 2026-04-05 17:36:52 -07:00
0a15ad9522 feat: add circuitforge_core.api.feedback — shared feedback router factory (closes #23)
Adds make_feedback_router(repo, product, demo_mode_fn) which returns a
FastAPI APIRouter with GET /status and POST / endpoints. Handles Forgejo
label creation/reuse, issue body assembly (including repro steps for bugs),
demo mode gating, and FORGEJO_API_TOKEN presence checks. 12 tests covering
all status/submit paths, mock Forgejo interaction, and body content assertions.
Also adds fastapi>=0.110 and httpx>=0.27 to [dev] optional deps.
2026-04-05 17:31:02 -07:00
c244260d1c feat!: strip resources/ from MIT core — moves to circuitforge-orch (v0.8.0)
BREAKING CHANGE: circuitforge_core.resources is no longer available.
Import CFOrchClient from circuitforge_orch.client instead.
cf-orch CLI entry point is now in the circuitforge-orch package.
2026-04-04 22:34:27 -07:00
2259382d0b refactor: replace coordinator-aware TaskScheduler with Protocol + LocalScheduler (MIT); update LLMRouter import path 2026-04-04 22:26:06 -07:00
090a86ce1b refactor: update LLMRouter lazy import — circuitforge_core.resources.client → circuitforge_orch.client 2026-04-04 22:16:17 -07:00
c1e825c06a Merge pull request 'feat: affiliates + preferences modules v0.7.0 (closes #21, #22)' (#25) from feature/affiliates-module into main 2026-04-04 19:14:24 -07:00
d16bc569cf chore: bump version to 0.7.0 — affiliates + preferences modules 2026-04-04 18:28:52 -07:00
ccd2a35deb test: affiliates integration tests — full wrap_url round-trip 2026-04-04 18:28:27 -07:00
fe19de3d9a feat: affiliates public API surface (__init__.py) 2026-04-04 18:27:45 -07:00
7837fbcad2 feat: affiliates router — wrap_url() with opt-out, BYOK, and CF env-var resolution 2026-04-04 18:20:21 -07:00
73cec07bd2 feat: affiliates disclosure — per-retailer tooltip copy + first-encounter banner constants 2026-04-04 18:14:58 -07:00
4c3f3a95a5 feat: affiliates programs — AffiliateProgram, registry, eBay EPN + Amazon Associates builders 2026-04-04 18:12:45 -07:00
d719ea2309 feat: preferences public helpers — get_user_preference / set_user_preference (closes #22 self-hosted) 2026-04-04 18:10:24 -07:00
0d9d030320 feat: preferences LocalFileStore — YAML-backed single-user preference store 2026-04-04 18:07:35 -07:00
9ee31a09c1 feat: preferences dot-path utilities (get_path, set_path) 2026-04-04 18:04:44 -07:00
e6cd3a2e96 chore: sync __version__ to 0.6.0 (matches pyproject.toml) 2026-04-03 16:48:11 -07:00
cb51ba72bc feat: cf-orch Docker image + Forgejo CI pipeline
Dockerfile.orch — multi-mode image (coordinator | agent):
- coordinator: runs cf-orch coordinator on $CF_ORCH_PORT (default 7700)
- agent: connects to $CF_COORDINATOR_URL, serves $CF_AGENT_GPU_IDS

.forgejo/workflows/docker.yml — publishes on every vN.N.N tag:
- ghcr.io/circuit-forge/cf-orch:latest
- ghcr.io/circuit-forge/cf-orch:vX.Y.Z
- Layer cache via GHA cache backend

Closes #19. Bumps to v0.6.0.
2026-04-03 09:10:29 -07:00
3deae056de feat: local-first LLM config + hosted coordinator auth
LLMRouter env-var auto-config:
- No llm.yaml required — auto-configures from ANTHROPIC_API_KEY,
  OPENAI_API_KEY, or OLLAMA_HOST on first use
- Bare-metal self-hosters can run any CF product with just env vars
- Falls back to FileNotFoundError with actionable message only when
  no env vars are set either

CFOrchClient auth:
- Reads CF_LICENSE_KEY env var (or explicit api_key param)
- Sends Authorization: Bearer <key> on all allocation/release requests
- Required for the hosted public coordinator; no-op for local deployments

HeimdallAuthMiddleware (new):
- FastAPI middleware for cf-orch coordinator
- Enabled by HEIMDALL_URL env var; self-hosted deployments skip it
- 5-min TTL cache (matching Kiwi cloud session) keeps Heimdall off the
  per-allocation hot path
- /api/health exempt; free-tier keys rejected with 403 + reason
- 13 tests covering cache TTL, tier ranking, and middleware gating
2026-04-03 08:32:15 -07:00
9544f695e6 chore: CHANGELOG for v0.5.0 2026-04-02 23:05:22 -07:00
7397e227e2 Merge pull request 'feat: manage.py cross-platform product manager' (#18) from feature/manage-py into main 2026-04-02 23:04:58 -07:00
8d87ed4c9f feat: manage.py cross-platform product manager (closes #6)
- circuitforge_core.manage module — replaces bash-only manage.sh
  - config.py: ManageConfig from manage.toml (TOML via tomllib/tomli)
    app name, default_url, docker compose_file/project, native services
    Falls back to directory name when no manage.toml present
  - docker_mode.py: DockerManager wrapping 'docker compose' (v2 plugin)
    or 'docker-compose' (v1 fallback); docker_available() probe
    Commands: start, stop, restart, status, logs, build
  - native_mode.py: NativeManager with PID file process management
    platformdirs for platform-appropriate PID/log paths
    Windows-compatible log tailing (polling, no tail -f)
    Cross-platform kill: SIGTERM→SIGKILL on Unix, taskkill /F on Windows
  - cli.py: typer CLI — start/stop/restart/status/logs/build/open/install-shims
    Mode auto-detection: Docker available + compose file → docker; else native
    --mode docker|native|auto override
  - templates/manage.sh: bash shim (conda, venv, python3 detection)
  - templates/manage.ps1: PowerShell shim (same detection, Windows)
  - templates/manage.toml.example: annotated config template
  - __main__.py: python -m circuitforge_core.manage entry point

- pyproject.toml: manage extras group (platformdirs, typer)
  cf-manage console script; version bumped to 0.5.0

- 36 tests: config (6), docker_mode (9), native_mode (21)
2026-04-02 23:04:35 -07:00
6e3474b97b chore: CHANGELOG for v0.4.0 2026-04-02 22:13:01 -07:00
d45d4e1de6 Merge pull request 'feat: agent watchdog + Ollama adopt-if-running' (#17) from feature/agent-watchdog into main 2026-04-02 22:12:32 -07:00
7bb6b76bd5 feat: ollama adopt-if-running + health_path in ProcessSpec (#16)
- ProcessSpec: adopt (bool) and health_path (str, default /health) fields
- ServiceManager: adopt=True probes health_path before spawning; is_running()
  uses health probe for adopt services rather than proc table + socket check
- _probe_health() helper: urllib GET on localhost:port+path, returns bool
- Agent /services/{service}/start: returns adopted=True when service was
  already running; coordinator sets state=running immediately (no probe wait)
- ServiceInstance: health_path field (default /health)
- service_registry.upsert_instance(): health_path kwarg
- Probe loop uses inst.health_path instead of hardcoded /health
- coordinator allocate_service: looks up health_path from profile spec via
  _get_health_path() and stores on ServiceInstance
- All GPU profiles (2/4/6/8/16/24 GB + cpu-16/32): ollama managed block
  with adopt=true, health_path=/api/tags, port 11434
- 11 new tests
2026-04-02 22:09:42 -07:00
a54a530493 feat: agent watchdog — persist known nodes + auto-reconnect after coordinator restart
closes #15

- NodeStore: SQLite persistence for known agent nodes
  (~/.local/share/circuitforge/cf-orch-nodes.db)
  - upsert on every register(); prune_stale() for 30-day cleanup
  - survives coordinator restarts — data readable by next process

- AgentSupervisor.restore_from_store(): reload known nodes on startup,
  mark all offline; heartbeat loop brings back any that respond

- AgentSupervisor.register(): persists to NodeStore on every call

- cli.py coordinator: NodeStore wired in; restore_from_store() called
  before uvicorn starts

- cli.py agent: one-shot registration replaced with persistent reconnect
  loop (daemon thread, 30 s interval) — coordinator restart → nodes
  reappear within one cycle with no manual intervention on agent hosts

- 16 new tests: NodeStore (8) + AgentSupervisor watchdog (8)
2026-04-02 22:01:55 -07:00
a36f469d60 chore: CHANGELOG for v0.3.0 2026-04-02 18:56:49 -07:00
1de5ec767c Merge pull request 'feat: hardware detection, cf-docuvision service, documents ingestion pipeline' (#14) from feature/hardware-docuvision into main 2026-04-02 18:55:50 -07:00
cd9864b5e8 feat: hardware detection, cf-docuvision service, documents ingestion pipeline
Closes #5, #7, #8, #13

## hardware module (closes #5)
- HardwareSpec, LLMBackendConfig, LLMConfig dataclasses
- VramTier ladder (CPU / 2 / 4 / 6 / 8 / 16 / 24 GB) with select_tier()
- generate_profile() maps HardwareSpec → LLMConfig for llm.yaml generation
- detect_hardware() with nvidia-smi / rocm-smi / system_profiler / cpu fallback
- 31 tests across tiers, generator, and detect

## cf-docuvision service (closes #8)
- FastAPI service wrapping ByteDance/Dolphin-v2 (Qwen2.5-VL backbone)
- POST /extract: image_b64 or image_path + hint → ExtractResponse
- Lazy model loading; JSON-structured output with plain-text fallback
- ProcessSpec managed blocks added to all four GPU profiles (6/8/16/24 GB)
- 14 tests

## documents module (closes #7)
- StructuredDocument, Element, ParsedTable dataclasses (frozen, composable)
- DocuvisionClient: thin HTTP client for cf-docuvision POST /extract
- ingest(): primary cf-docuvision path → LLMRouter vision fallback → empty doc
- CF_DOCUVISION_URL env var for URL override
- 22 tests

## coordinator probe loop (closes #13)
- _run_instance_probe_loop: starting → running on 200; starting → stopped on timeout
- 4 async tests with CancelledError-based tick control
2026-04-02 18:53:25 -07:00
482c430cdb docs: add CHANGELOG for v0.1.0 and v0.2.0 2026-04-02 17:25:06 -07:00
749e51ccca Merge pull request 'feat(orch): health probe loop + VRAM pre-flight fix' (#12) from feature/orch-llm-server into main 2026-04-02 17:24:09 -07:00
a7290c1240 feat(orch): background health probe loop — starting → running transition
Coordinator now polls all 'starting' instances every 5 s via GET /health.
On 200: state → running. After 300 s without a healthy response: state →
stopped. Closes #10.
2026-04-02 17:18:16 -07:00
bd132851ec fix(orch): tighten VRAM pre-flight to require full max_mb free (not half)
max_mb // 2 was too loose — Qwen2.5-3B needs ~5.9 GB on an 8 GB card
but the threshold only required 3.25 GB free, allowing Ollama to hold
4.5 GB while a load attempt was still dispatched (causing OOM crash).

- node_selector: can_fit = free_mb >= service_max_mb (was // 2)
- coordinator /start: same threshold fix + updated error message
- tests: two new node_selector tests pin the full-ceiling semantics;
  updated stale docstring in coordinator app test
2026-04-02 16:44:36 -07:00
2d095f0090 fix(llm-server): handle transformers 5.x BatchEncoding; use dtype kwarg
- apply_chat_template() returns BatchEncoding in transformers 5.x (not bare tensor);
  extract .input_ids explicitly with fallback for 4.x compat
- Switch from deprecated torch_dtype= to dtype= in from_pretrained()
2026-04-02 16:36:07 -07:00