Add assignments.yaml entry to activate cf-orch task routing (already wired in llm.py) #17

New issue

Closed

opened 2026-05-13 10:02:22 -07:00 by pyr0ball · 2 comments

pyr0ball commented

2026-05-13 10:02:22 -07:00

Owner

Background

cf-orch #60 shipped the three-layer task-model assignment system and POST /api/inference/task.

Good news: Turnstone is already wired

app/services/llm.py already calls POST /api/inference/task with {"product": "turnstone", "task": "log_analysis"} and falls back to a direct OpenAI-compat call if the endpoint returns 404 (no assignment configured).

This ticket is just the final step: add the assignment entry so the task routing activates.

What to do

1. Add to `assignments.yaml` (managed via Avocet Assignments UI or direct YAML edit)

turnstone:
  log_analysis:
    model_id: ibm-granite--granite-4.1-8b
    description: Log diagnosis and root cause analysis
  incident_summary:
    model_id: ibm-granite--granite-4.1-8b
    description: Incident pattern summarization

2. Verify the fallback path

After adding the entry and reloading cf-orch (POST /api/nodes/reload-profiles), confirm that:

GET /api/assignments/deployment-status shows turnstone.log_analysis as present
Diagnose UI calls succeed through the task endpoint (no fallback needed)
The direct-model fallback still works when llm_url points to a non-cf-orch instance (Ollama, vllm directly)

3. Optional: wire `incident_summary` call site

If app/services/incidents.py has its own LLM summarization call, migrate it to use task=incident_summary.

Acceptance Criteria

assignments.yaml has turnstone.log_analysis entry
Diagnose feature routes through cf-orch task endpoint when CF_ORCH_URL is set
Fallback path tested (works without assignment or with non-cf-orch URL)

cf-orch #60 (task-model assignment layer)
app/services/llm.py (already has the routing logic)
circuitforge-plans/circuitforge-orch/superpowers/specs/2026-05-13-task-model-assignments-design.md

## Background cf-orch #60 shipped the three-layer task-model assignment system and `POST /api/inference/task`. ## Good news: Turnstone is already wired `app/services/llm.py` already calls `POST /api/inference/task` with `{"product": "turnstone", "task": "log_analysis"}` and falls back to a direct OpenAI-compat call if the endpoint returns 404 (no assignment configured). This ticket is just the final step: **add the assignment entry** so the task routing activates. ## What to do ### 1. Add to `assignments.yaml` (managed via Avocet Assignments UI or direct YAML edit) ```yaml turnstone: log_analysis: model_id: ibm-granite--granite-4.1-8b description: Log diagnosis and root cause analysis incident_summary: model_id: ibm-granite--granite-4.1-8b description: Incident pattern summarization ``` ### 2. Verify the fallback path After adding the entry and reloading cf-orch (`POST /api/nodes/reload-profiles`), confirm that: - `GET /api/assignments/deployment-status` shows `turnstone.log_analysis` as `present` - Diagnose UI calls succeed through the task endpoint (no fallback needed) - The direct-model fallback still works when `llm_url` points to a non-cf-orch instance (Ollama, vllm directly) ### 3. Optional: wire `incident_summary` call site If `app/services/incidents.py` has its own LLM summarization call, migrate it to use `task=incident_summary`. ## Acceptance Criteria - [ ] `assignments.yaml` has `turnstone.log_analysis` entry - [ ] Diagnose feature routes through cf-orch task endpoint when `CF_ORCH_URL` is set - [ ] Fallback path tested (works without assignment or with non-cf-orch URL) ## Related - cf-orch #60 (task-model assignment layer) - `app/services/llm.py` (already has the routing logic) - `circuitforge-plans/circuitforge-orch/superpowers/specs/2026-05-13-task-model-assignments-design.md`

pyr0ball commented

2026-05-20 22:19:32 -07:00

Author

Owner

Status: Blocked on cf-orch #68

What is done:

assignments.yaml already has turnstone.log_analysis → foundation-sec-8b-reasoning-q4_k_m-gguf
turnstone.security_analysis → cybersecurity-baronllm_offensive_security_llm_q6_k_gguf also present
Deployment status shows present on Heimdall for both
Fallback path in llm.py works correctly (catches 500, routes to direct model call)

What is broken:
POST /api/inference/task in the coordinator never proxies the inference — it allocates the service and returns allocation metadata, not a completion. Also has a params: dict[str, str] type mismatch that causes a 500 before allocation even starts.

Filed as cf-orch #68. Once that is fixed, Turnstone will automatically route through foundation-sec-8b-reasoning for diagnose calls with no Turnstone code changes needed.

## Status: Blocked on cf-orch #68 **What is done:** - `assignments.yaml` already has `turnstone.log_analysis` → `foundation-sec-8b-reasoning-q4_k_m-gguf` - `turnstone.security_analysis` → `cybersecurity-baronllm_offensive_security_llm_q6_k_gguf` also present - Deployment status shows `present` on Heimdall for both - Fallback path in `llm.py` works correctly (catches 500, routes to direct model call) **What is broken:** `POST /api/inference/task` in the coordinator never proxies the inference — it allocates the service and returns allocation metadata, not a completion. Also has a `params: dict[str, str]` type mismatch that causes a 500 before allocation even starts. Filed as cf-orch #68. Once that is fixed, Turnstone will automatically route through `foundation-sec-8b-reasoning` for diagnose calls with no Turnstone code changes needed.

pyr0ball commented

2026-05-20 22:48:14 -07:00

Author

Owner

Unblocked by cf-orch#68 (payload type fix merged and deployed).

Status as of 2026-05-20:

app/services/llm.py already tries POST /api/inference/task first, falls back to direct /v1/chat/completions on any non-200
assignments.yaml has turnstone.log_analysis → foundation-sec-8b-reasoning-q4_k_m-gguf and turnstone.security_analysis → cybersecurity-baronllm_offensive_security_llm_q6_k_gguf
Live Turnstone prefs point at coordinator (http://127.0.0.1:7700)
Confirmed task endpoint routes correctly — returns 502 (model not loaded) rather than validation error
Fallback to llama3.1:8b continues to work when security model is not allocated

Task routing will silently activate when foundation-sec-8b gets VRAM headroom. Closing.

Unblocked by cf-orch#68 (payload type fix merged and deployed). Status as of 2026-05-20: - `app/services/llm.py` already tries `POST /api/inference/task` first, falls back to direct `/v1/chat/completions` on any non-200 - `assignments.yaml` has `turnstone.log_analysis → foundation-sec-8b-reasoning-q4_k_m-gguf` and `turnstone.security_analysis → cybersecurity-baronllm_offensive_security_llm_q6_k_gguf` - Live Turnstone prefs point at coordinator (`http://127.0.0.1:7700`) - Confirmed task endpoint routes correctly — returns 502 (model not loaded) rather than validation error - Fallback to `llama3.1:8b` continues to work when security model is not allocated Task routing will silently activate when `foundation-sec-8b` gets VRAM headroom. Closing.