pipeline module: FPGA→ASIC crystallization engine (LLM-discovered paths → deterministic workflows) #33

New issue

Closed

opened 2026-04-06 08:24:27 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-04-06 08:24:27 -07:00

Owner

Concept

The pipeline module (currently a stub) should become a crystallization engine — the architectural layer that makes CF products progressively smarter and cheaper to run over time.

Analogy: LLMs are FPGAs (flexible, general-purpose, expensive per-run). After a human approves a discovered solution path, that path crystallizes into a deterministic workflow (ASIC) that runs without LLM involvement until it encounters an edge case.

Full design doc: circuitforge-plans/circuitforge-core/2026-04-06-pipeline-crystallization-design.md

The Missing Step

Current CF pipeline:

Monitor/Discover → AI Assist → Human Approval → Execute → Track

With crystallization:

Monitor/Discover → AI Assist → Human Approval → Execute → Track
                                      ↓
                               Crystallize (N approvals)
                                      ↓
              [Deterministic ASIC path] → Execute
                         ↓ (edge case)
                    AI Assist re-engages

Module Structure

circuitforge_core/pipeline/
    __init__.py
    recorder.py      # writes PipelineRun on each LLM-assisted execution
    crystallizer.py  # evaluates runs, promotes to CrystallizedWorkflow
    registry.py      # matches incoming tasks to crystallized workflows
    executor.py      # runs deterministic workflows + handles LLM fallback
    models.py        # PipelineRun, CrystallizedWorkflow, Step dataclasses

Products call pipeline.run() instead of llm.complete() directly — the module wraps llm.router transparently.

Key Design Properties

Human approval gates crystallization — only approved runs contribute
Input hashing, not raw input — PII never stored in pipeline records
Configurable thresholds — per-product tuning (osprey: threshold=1 after first IVR success; peregrine: fuzzy similarity for cover letters)
Transparent workflows — crystallized paths are human-readable JSON, not black-box ML
Local-first — all records in ~/.config/circuitforge/pipeline/; optional CF account sync

Per-Product Use Cases

Product	Task type	Crystallizes to
`osprey`	`ivr_navigate`	DTMF sequence + timing per agency
`falcon`	`form_fill`	Field ruleset per form schema
`peregrine`	`cover_letter`	Template + scoring rules per niche
`kiwi`	`receipt_parse`	Deterministic parser per merchant
`snipe`	`trust_score`	New signal rule added to aggregator

Acceptance Criteria

PipelineRun and CrystallizedWorkflow dataclasses with full field coverage
recorder writes run records on LLM-assisted executions
crystallizer promotes workflows after configurable approval threshold
registry matches inputs to workflows (exact + fuzzy)
executor runs deterministic path with LLM fallback
pipeline.run() is a drop-in replacement for llm.complete() from product code
Per-product config via pipeline.yaml
80%+ test coverage on crystallizer and registry logic
No PII in pipeline records (input hashing enforced)

## Concept The `pipeline` module (currently a stub) should become a **crystallization engine** — the architectural layer that makes CF products progressively smarter and cheaper to run over time. **Analogy:** LLMs are FPGAs (flexible, general-purpose, expensive per-run). After a human approves a discovered solution path, that path crystallizes into a deterministic workflow (ASIC) that runs without LLM involvement until it encounters an edge case. Full design doc: `circuitforge-plans/circuitforge-core/2026-04-06-pipeline-crystallization-design.md` --- ## The Missing Step Current CF pipeline: ``` Monitor/Discover → AI Assist → Human Approval → Execute → Track ``` With crystallization: ``` Monitor/Discover → AI Assist → Human Approval → Execute → Track ↓ Crystallize (N approvals) ↓ [Deterministic ASIC path] → Execute ↓ (edge case) AI Assist re-engages ``` --- ## Module Structure ``` circuitforge_core/pipeline/ __init__.py recorder.py # writes PipelineRun on each LLM-assisted execution crystallizer.py # evaluates runs, promotes to CrystallizedWorkflow registry.py # matches incoming tasks to crystallized workflows executor.py # runs deterministic workflows + handles LLM fallback models.py # PipelineRun, CrystallizedWorkflow, Step dataclasses ``` Products call `pipeline.run()` instead of `llm.complete()` directly — the module wraps `llm.router` transparently. --- ## Key Design Properties - **Human approval gates crystallization** — only approved runs contribute - **Input hashing, not raw input** — PII never stored in pipeline records - **Configurable thresholds** — per-product tuning (osprey: threshold=1 after first IVR success; peregrine: fuzzy similarity for cover letters) - **Transparent workflows** — crystallized paths are human-readable JSON, not black-box ML - **Local-first** — all records in `~/.config/circuitforge/pipeline/`; optional CF account sync --- ## Per-Product Use Cases | Product | Task type | Crystallizes to | |---------|-----------|----------------| | `osprey` | `ivr_navigate` | DTMF sequence + timing per agency | | `falcon` | `form_fill` | Field ruleset per form schema | | `peregrine` | `cover_letter` | Template + scoring rules per niche | | `kiwi` | `receipt_parse` | Deterministic parser per merchant | | `snipe` | `trust_score` | New signal rule added to aggregator | --- ## Acceptance Criteria - [ ] `PipelineRun` and `CrystallizedWorkflow` dataclasses with full field coverage - [ ] `recorder` writes run records on LLM-assisted executions - [ ] `crystallizer` promotes workflows after configurable approval threshold - [ ] `registry` matches inputs to workflows (exact + fuzzy) - [ ] `executor` runs deterministic path with LLM fallback - [ ] `pipeline.run()` is a drop-in replacement for `llm.complete()` from product code - [ ] Per-product config via `pipeline.yaml` - [ ] 80%+ test coverage on crystallizer and registry logic - [ ] No PII in pipeline records (input hashing enforced)