Integrate task-model assignment routing for imitate inference via cf-orch /api/inference/task #62

New issue

Open

opened 2026-05-13 07:38:23 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-05-13 07:38:23 -07:00

Owner

Background

cf-orch #60 shipped a three-layer task-model assignment system:

Layer 1 (assignments.yaml): product.task → model_id
Layer 2 (model_registry.yaml): model_id → {service_type, vram_mb, alias, ...}
Layer 3: per-node catalogs (existing, unchanged)

Avocet already has the Assignments UI built (shipped alongside #60). This ticket is to wire up Avocet's own inference calls to use the new routing rather than hardcoded model selection.

Spec: circuitforge-plans/circuitforge-orch/superpowers/specs/2026-05-13-task-model-assignments-design.md

What needs to change in Avocet

1. Register tasks in `assignments.yaml`

avocet:
  imitate:
    model_id: ibm-granite--granite-4.1-8b
    description: Style imitation inference (imitate.py)
  classify:
    model_id: ibm-granite--granite-4.1-8b
    description: Email classification inference

2. Migration path for call sites

app/imitate.py — style imitation, currently calls cf-orch with explicit model selection
app/cforch.py — existing cf-orch proxy; add a helper that calls POST /api/inference/task for task-based routing

Replace explicit model selection with {"product": "avocet", "task": "imitate", ...} calls.

3. Self-referential note

Avocet manages the Assignments UI for the whole menagerie — including its own avocet.* entries. This is intentional: Avocet operators can assign the imitate model via the same Assignments tab they use for other products.

4. Backwards compatibility

This is opt-in — existing POST /api/inference calls continue to work.

Acceptance Criteria

assignments.yaml has avocet.imitate entry
app/imitate.py call site migrated to /api/inference/task
Tests updated

cf-orch #60 (task-model assignment layer — already shipped)
Avocet Assignments tab (already shipped — visible in Models view)
circuitforge-plans/circuitforge-orch/superpowers/specs/2026-05-13-task-model-assignments-design.md

## Background cf-orch #60 shipped a three-layer task-model assignment system: - **Layer 1** (`assignments.yaml`): `product.task → model_id` - **Layer 2** (`model_registry.yaml`): `model_id → {service_type, vram_mb, alias, ...}` - **Layer 3**: per-node catalogs (existing, unchanged) Avocet already has the Assignments UI built (shipped alongside #60). This ticket is to wire up Avocet's own inference calls to use the new routing rather than hardcoded model selection. Spec: `circuitforge-plans/circuitforge-orch/superpowers/specs/2026-05-13-task-model-assignments-design.md` ## What needs to change in Avocet ### 1. Register tasks in `assignments.yaml` ```yaml avocet: imitate: model_id: ibm-granite--granite-4.1-8b description: Style imitation inference (imitate.py) classify: model_id: ibm-granite--granite-4.1-8b description: Email classification inference ``` ### 2. Migration path for call sites - `app/imitate.py` — style imitation, currently calls cf-orch with explicit model selection - `app/cforch.py` — existing cf-orch proxy; add a helper that calls `POST /api/inference/task` for task-based routing Replace explicit model selection with `{"product": "avocet", "task": "imitate", ...}` calls. ### 3. Self-referential note Avocet manages the Assignments UI for the whole menagerie — including its own `avocet.*` entries. This is intentional: Avocet operators can assign the imitate model via the same Assignments tab they use for other products. ### 4. Backwards compatibility This is **opt-in** — existing `POST /api/inference` calls continue to work. ## Acceptance Criteria - [ ] `assignments.yaml` has `avocet.imitate` entry - [ ] `app/imitate.py` call site migrated to `/api/inference/task` - [ ] Tests updated ## Related - cf-orch #60 (task-model assignment layer — already shipped) - Avocet Assignments tab (already shipped — visible in Models view) - `circuitforge-plans/circuitforge-orch/superpowers/specs/2026-05-13-task-model-assignments-design.md`