Circuit-Forge/avocet
5
0
Fork 0
Code Issues 40 Pull requests 2 Projects Releases 2 Packages Wiki Activity Actions

Compare commits

base: Circuit-Forge:main
Branches Tags
Circuit-Forge:main
Circuit-Forge:feat/imitate
Circuit-Forge:feat/env-config-sft-import
Circuit-Forge:feat/cforch-benchmark
Circuit-Forge:feat/benchmark-model-picker
Circuit-Forge:feat/hf-model-queue
Circuit-Forge:feat/sft-failure-category
Circuit-Forge:feat/sft-corrections
Circuit-Forge:feat/vue-label-tab
Circuit-Forge:v0.7.0
Circuit-Forge:v0.6.0
Circuit-Forge:v0.5.0
Circuit-Forge:v0.4.0
Circuit-Forge:v0.3.0
..
compare: Circuit-Forge:v0.6.0
Branches Tags
Circuit-Forge:main
Circuit-Forge:feat/imitate
Circuit-Forge:feat/env-config-sft-import
Circuit-Forge:feat/cforch-benchmark
Circuit-Forge:feat/benchmark-model-picker
Circuit-Forge:feat/hf-model-queue
Circuit-Forge:feat/sft-failure-category
Circuit-Forge:feat/sft-corrections
Circuit-Forge:feat/vue-label-tab
Circuit-Forge:v0.7.0
Circuit-Forge:v0.6.0
Circuit-Forge:v0.5.0
Circuit-Forge:v0.4.0
Circuit-Forge:v0.3.0

No commits in common. "main" and "v0.6.0" have entirely different histories.
main ... v0.6.0

45 changed files with 273 additions and 6611 deletions
Show stats Expand all files Collapse all files

3
.gitignore vendored
View file

@ -8,9 +8,6 @@ __pycache__/
config/label_tool.yaml
# Data files (user-generated, not for version control)
data/corpus.db
data/corpus.db-wal
data/corpus.db-shm
data/email_score.jsonl
data/email_label_queue.jsonl
data/email_compare_sample.jsonl

183
README.md
View file

@ -1,120 +1,22 @@
<div align="center">
<img src="docs/avocet-logo.svg" alt="Avocet" height="96" />
# Avocet — Email Classifier Training Tool
# Avocet
> *Part of the CircuitForge LLC internal infrastructure suite.*
**Email classifier training tool — label, benchmark, fine-tune.**
[![Status: Internal Beta](https://img.shields.io/badge/status-internal%20beta-blue)]()
[![Version](https://img.shields.io/badge/version-0.5.0-green)](https://git.opensourcesolarpunk.com/Circuit-Forge/avocet/releases)
[![License: BSL 1.1](https://img.shields.io/badge/license-BSL%201.1-orange)](LICENSE)
[![Stack: Vue 3 + FastAPI](https://img.shields.io/badge/stack-Vue%203%20%2B%20FastAPI-brightgreen)]()
[![CircuitForge](https://img.shields.io/badge/by-CircuitForge-black)](https://circuitforge.tech)
</div>
**Status:** Internal beta — label tool and benchmark harness complete. Used to build training data for Peregrine's email classifier.
---
## What is Avocet?
## What it does
Avocet is the internal data pipeline Circuit Forge uses to build, evaluate, and fine-tune email classifiers. It implements a three-stage workflow: human labelers review emails one at a time in a drag-to-bucket UI and produce a ground-truth dataset; the benchmark harness scores any number of HuggingFace zero-shot models against that dataset and produces a ranked comparison; and the fine-tune harness adapts the best-scoring base model to the labeled distribution. The output feeds directly into Peregrine's email classification layer. No LLM API key required for the label tool or benchmark — all inference runs locally via HuggingFace Transformers.
Avocet is the data pipeline for building and benchmarking email classifiers. It has two layers:
---
**No LLM required.** Avocet uses zero-shot HuggingFace classification models — no API key, no cloud inference, no GPU required for the label tool. The benchmark harness can optionally export LLM-labeled emails from a Peregrine staging DB, but human labeling via the card-stack UI is the primary workflow.
## Quick Start
**Layer 1 — Label tool**
Card-stack UI for building ground-truth classifier benchmark data. Fetch emails from one or more IMAP accounts (with targeted date-range and sender/subject filters), review them card-by-card, and label each with a job-search category. Labeled output feeds the benchmark harness.
```bash
git clone https://git.opensourcesolarpunk.com/Circuit-Forge/avocet.git
cd avocet
# Copy config template and fill in your IMAP credentials
cp config/label_tool.yaml.example config/label_tool.yaml
# Start the label tool (Vue SPA + FastAPI, port 8503)
./manage.sh start
./manage.sh open
```
---
## Features
- **Drag-to-bucket label UI** — ASMR-style card interface; drag emails into labeled buckets or discard without queuing noise into the training set
- **Targeted IMAP fetch** — pull emails by date range, sender, or subject filter across multiple accounts without flooding the queue
- **Email classifier benchmark** — score any HuggingFace zero-shot model against your labeled JSONL; side-by-side comparison on live IMAP emails
- **Planning benchmark** — evaluate LLMs on structured planning tasks; compare models head-to-head with verbose diff output
- **Writing style benchmark** — compare Ollama models on writing style coherence; scan local disk for existing outputs
- **Fine-tune harness** — HuggingFace Transformers fine-tuning from labeled ground truth; classifier adapter interface for swapping backends at runtime
- **Local inference first** — no API key required; GPU optional; designed to run on developer hardware
- **Hot-reload dev mode** — uvicorn `--reload` + Vite HMR (hot module replacement) for fast iteration on both API and UI
---
## CLI Reference
All operations go through `manage.sh`.
### Label Tool
```bash
./manage.sh start # Build Vue SPA and start FastAPI on port 8503
./manage.sh stop # Stop FastAPI server
./manage.sh restart # Stop, rebuild, and restart
./manage.sh status # Show running state and port
./manage.sh logs # Tail the API log
./manage.sh open # Open http://localhost:8503 in browser
./manage.sh dev # Hot-reload: uvicorn --reload + Vite HMR
./manage.sh test # Run pytest suite
```
### Email Classifier Benchmark
```bash
./manage.sh benchmark [args] # Run benchmark_classifier.py
./manage.sh list-models # List available zero-shot models
./manage.sh score # Score models against labeled JSONL
./manage.sh score --include-slow # Include large/slow models
./manage.sh compare --limit 30 # Side-by-side comparison on live IMAP emails
```
### Planning Benchmark
```bash
./manage.sh plans-bench [args] # Run benchmark_plans.py
./manage.sh plans-list # List available models
./manage.sh plans-run <model> [args] # Run a single model (verbose)
./manage.sh plans-compare <m1> <m2> [...] # Compare models side-by-side
```
### Writing Style Benchmark
```bash
./manage.sh style-bench [args] # Run benchmark_style.py
./manage.sh style-list # List available Ollama models
./manage.sh style-run [args] # Run writing style benchmark
./manage.sh style-last # Print most recent benchmark report
```
---
## Data Flow
```
IMAP accounts
→ fetch (targeted or wide)
→ email_label_queue.jsonl
email_label_queue.jsonl
→ label tool drag-to-bucket UI
→ email_score.jsonl (ground truth)
email_score.jsonl
→ benchmark harness
→ model rankings
best model
→ fine-tune harness
→ Peregrine classifier adapter
```
**Layer 2 — Benchmark harness**
Scores HuggingFace zero-shot classification models against the labeled dataset. Supports slow/large model inclusion, visual side-by-side comparison on live emails, and export of LLM-labeled emails from a Peregrine staging DB.
---
@ -136,42 +38,69 @@ best model
## Stack
| Layer | Technology |
|-------|-----------|
| Label UI | Vue 3 SPA (Vite) |
| API | FastAPI + uvicorn (port 8503) |
| Layer | Tech |
|-------|------|
| Label UI | Streamlit (port 8503, auto-increments on collision) |
| Benchmark | Python + HuggingFace Transformers |
| Email fetch | IMAP (multi-account, targeted date/sender/subject filter) |
| Data | JSONL (`data/email_label_queue.jsonl`, `data/email_score.jsonl`) |
| Runtime | SQLite |
| Config | `config/label_tool.yaml` (gitignored — `.example` committed) |
| Config | `config/label_tool.yaml` (gitignored — see `.example`) |
Conda environments:
- `job-seeker` — label tool UI
- `job-seeker-classifiers` — benchmark harness (separate env for heavy deps)
---
## Logo
## Running
The Avocet logo (`avocet_v1_poly.svg`) lives in the shared graphics repo. Copy it to `docs/avocet-logo.svg` to render correctly in this README.
```bash
./manage.sh start # start label tool UI (port collision-safe from 8503)
./manage.sh stop # stop
./manage.sh restart # restart
./manage.sh status # show running state and port
./manage.sh logs # tail label tool log
./manage.sh open # open in browser
```
Benchmark:
```bash
./manage.sh benchmark --list-models # list available zero-shot models
./manage.sh score # score models against labeled JSONL
./manage.sh score --include-slow # include large/slow models
./manage.sh compare --limit 30 # visual comparison on live IMAP emails
```
Dev:
```bash
./manage.sh test # run pytest suite
```
---
## About
## Data flow
Avocet is internal CircuitForge infrastructure, open source as a reference implementation. It is not a user-facing product. The primary consumer is [Peregrine](https://git.opensourcesolarpunk.com/Circuit-Forge/peregrine), CircuitForge's job-search pipeline tool.
```
IMAP accounts → fetch (targeted or wide) → email_label_queue.jsonl
→ label tool card UI → email_score.jsonl
→ benchmark harness → model rankings
→ best model → Peregrine classifier adapter
```
Docs: [docs.circuitforge.tech/avocet](https://docs.circuitforge.tech/avocet)
Targeted fetch: date range + sender/subject filter for pulling historical emails on specific senders or topics without flooding the queue.
## Forgejo-primary
Discard: removes an email from the queue without writing to the score file — for emails that don't belong in the training set.
Avocet is developed and maintained on Forgejo at [git.opensourcesolarpunk.com/Circuit-Forge/avocet](https://git.opensourcesolarpunk.com/Circuit-Forge/avocet). GitHub and Codeberg are read-only mirrors.
---
## Classifier adapters
`app/classifier_adapters.py` provides a common interface for swapping classifier backends. Falls back to the label name when no `LABEL_DESCRIPTIONS` entry is configured for a label (RerankerAdapter).
---
## License
[Business Source License 1.1](LICENSE) — classifier training is an AI feature under the CircuitForge licensing model.
BSL 1.1 — internal tool, not user-facing.
Free for personal non-commercial self-hosting. Commercial use or SaaS re-hosting requires a paid license. Converts to MIT after 4 years.
Humans own design, architecture, code review, testing, and verification. LLMs are part of our development workflow. [Our positions on LLM use →](https://circuitforge.tech/positions)
© 2026 Circuit Forge LLC — Privacy · Safety · Accessibility
© 2026 Circuit Forge LLC

33
app/api.py
View file

@ -40,39 +40,6 @@ app.include_router(plans_bench_router, prefix="/api/plans-bench")
# In-memory last-action store (single user, local tool — in-memory is fine)
_last_action: dict | None = None
# -- Backward-compat shims (ClassifierTab still uses old /api/finetune/* paths)
# Remove once ClassifierTab fine-tune section is migrated to TrainJobsView.
from fastapi import Query
from fastapi.responses import StreamingResponse as _StreamingResponse
@app.get("/api/finetune/run")
def finetune_run_compat(model: str = Query(...), epochs: int = Query(5)) -> _StreamingResponse:
"""Shim: create a classifier train job and immediately stream it."""
from app.train.train import create_job, run_job, CreateJobRequest
job = create_job(CreateJobRequest(type="classifier", model_key=model, config_json={"epochs": epochs}))
return run_job(job["id"])
@app.post("/api/finetune/cancel")
def finetune_cancel_compat() -> dict:
"""Shim: cancel the most recent running classifier job."""
from app.train.train import _db, _init_db, cancel_job
from fastapi import HTTPException
_init_db()
with _db() as conn:
row = conn.execute(
"SELECT id FROM jobs WHERE type='classifier' AND status='running' ORDER BY started_at DESC LIMIT 1"
).fetchone()
if row is None:
return {"status": "nothing_running"}
return cancel_job(row["id"])
from app.data.log_corpus import router as log_corpus_router
app.include_router(log_corpus_router, prefix="/api/corpus")
from app.data.recipe_scan import router as recipe_scan_router
app.include_router(recipe_scan_router, prefix="/api/recipe-scan")
from app.dashboard import router as dashboard_router
app.include_router(dashboard_router, prefix="/api")

132
app/cforch.py
View file

@ -16,18 +16,16 @@ import json
import logging
import os
import re
import select as _select
import subprocess as _subprocess
import tempfile
from pathlib import Path
from typing import Any, Optional
from typing import Any
import urllib.parse
import yaml
from fastapi import APIRouter, HTTPException, Request
from fastapi import APIRouter, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
logger = logging.getLogger(__name__)
@ -313,12 +311,8 @@ def run_benchmark(
"""Spawn cf-orch benchmark.py and stream stdout as SSE progress events."""
global _BENCH_RUNNING, _bench_proc
# Check if the process is actually still alive; reset stale flag if not.
if _BENCH_RUNNING:
if _bench_proc is not None and _bench_proc.poll() is None:
raise HTTPException(409, "A benchmark is already running")
_BENCH_RUNNING = False
_bench_proc = None
cfg = _load_cforch_config()
bench_script = cfg.get("bench_script", "")
@ -442,23 +436,8 @@ def run_benchmark(
env=proc_env,
)
_bench_proc = proc
_IDLE_TIMEOUT_S = 120 # kill if no output for 2 minutes (node crash)
try:
while True:
ready = _select.select([proc.stdout], [], [], _IDLE_TIMEOUT_S)
if not ready[0]:
# No output for IDLE_TIMEOUT_S — node likely crashed
proc.terminate()
try:
proc.wait(timeout=5)
except _subprocess.TimeoutExpired:
proc.kill()
msg = f"Benchmark timed out — no output for {_IDLE_TIMEOUT_S}s (cluster node may have crashed)"
yield f"data: {json.dumps({'type': 'error', 'message': msg})}\n\n"
break
line = proc.stdout.readline()
if not line:
break
for line in proc.stdout:
line = _strip_ansi(line.rstrip())
if line:
yield f"data: {json.dumps({'type': 'progress', 'message': line})}\n\n"
@ -516,7 +495,7 @@ def get_cforch_config() -> dict:
# ── GET /results ───────────────────────────────────────────────────────────────
@router.get("/results")
def get_results() -> dict:
def get_results() -> list:
"""Return the latest benchmark summary.json from results_dir."""
cfg = _load_cforch_config()
results_dir = cfg.get("results_dir", "")
@ -548,106 +527,3 @@ def cancel_benchmark() -> dict:
_BENCH_RUNNING = False
_bench_proc = None
return {"status": "cancelled"}
# ── Coordinator proxy helpers ──────────────────────────────────────────────────
def _coordinator_url() -> str:
"""Return coordinator base URL from config, or raise 503 if not configured."""
url = _load_cforch_config().get("coordinator_url", "").rstrip("/")
if not url:
raise HTTPException(503, "cf-orch coordinator_url not configured")
return url
def _coordinator_get(path: str) -> Any:
"""GET from coordinator, return parsed JSON body. Raises HTTPException on error."""
import httpx as _httpx
try:
resp = _httpx.get(f"{_coordinator_url()}{path}", timeout=10.0)
except Exception as exc:
raise HTTPException(502, f"Coordinator unreachable: {exc}") from exc
if not resp.is_success:
raise HTTPException(resp.status_code, resp.text)
return resp.json()
async def _coordinator_post(path: str, body: dict) -> Any:
import httpx as _httpx
try:
async with _httpx.AsyncClient(timeout=10.0) as client:
resp = await client.post(f"{_coordinator_url()}{path}", json=body)
except Exception as exc:
raise HTTPException(502, f"Coordinator unreachable: {exc}") from exc
if not resp.is_success:
raise HTTPException(resp.status_code, resp.text)
return resp.json()
async def _coordinator_delete(path: str) -> Any:
import httpx as _httpx
try:
async with _httpx.AsyncClient(timeout=10.0) as client:
resp = await client.delete(f"{_coordinator_url()}{path}")
except Exception as exc:
raise HTTPException(502, f"Coordinator unreachable: {exc}") from exc
if not resp.is_success:
raise HTTPException(resp.status_code, resp.text)
return resp.json()
# ── GET /assignments/deployment-status ───────────────────────────────────────
@router.get("/assignments/deployment-status")
def get_deployment_status() -> Any:
return _coordinator_get("/api/assignments/deployment-status")
# ── /assignments ──────────────────────────────────────────────────────────────
@router.get("/assignments")
def list_assignments() -> Any:
return _coordinator_get("/api/assignments")
class AssignmentBody(BaseModel):
product: str
task: str
model_id: str
description: str = ""
@router.post("/assignments")
async def upsert_assignment(body: AssignmentBody) -> Any:
return await _coordinator_post("/api/assignments", body.model_dump())
@router.delete("/assignments/{product}/{task}")
async def delete_assignment(product: str, task: str) -> Any:
return await _coordinator_delete(f"/api/assignments/{urllib.parse.quote(product, safe='')}/{urllib.parse.quote(task, safe='')}")
# ── /model-registry ────────────────────────────────────────────────────────────
@router.get("/model-registry")
def list_model_registry() -> Any:
return _coordinator_get("/api/model-registry")
class ModelRegistryBody(BaseModel):
model_id: str
service_type: str
vram_mb: int
description: str = ""
hf_repo: str = ""
alias: str = ""
@router.post("/model-registry")
async def upsert_model_registry(body: ModelRegistryBody) -> Any:
return await _coordinator_post("/api/model-registry", body.model_dump())
@router.delete("/model-registry/{model_id:path}")
async def delete_model_registry(model_id: str) -> Any:
return await _coordinator_delete(f"/api/model-registry/{urllib.parse.quote(model_id, safe='')}")

109
app/dashboard.py
View file

@ -1,18 +1,17 @@
"""Avocet -- dashboard aggregate API.
GET /api/dashboard returns the current flywheel state:
labeled_since_last_eval -- items labeled after the most recent bench run
labeled_since_last_eval -- items labeled after the most recent eval run
last_eval_timestamp -- ISO timestamp of newest bench_results summary
last_eval_best_score -- best macro_f1 from that summary
active_jobs -- jobs with status queued or running
corrections_pending -- sft_candidates with status=needs_review
corrections_export_ready -- approved sft candidates with non-blank correction
recent_bench_runs -- most-recent timestamp + score per bench type
signals -- computed booleans for UI nudge indicators
Thresholds in label_tool.yaml pipeline: section:
pipeline:
data_eval_threshold: 50 # labeled items since last bench to trigger nudge
data_eval_threshold: 50 # labeled items since last eval to trigger nudge
eval_train_threshold: 0.05 # improvement delta needed before retraining (future)
"""
from __future__ import annotations
@ -78,7 +77,7 @@ def _load_score_records() -> list[dict]:
pass
return records
def _find_latest_classifier_bench(results_dir_override: str = "") -> tuple[str | None, float | None]:
def _find_latest_eval(results_dir_override: str = "") -> tuple[str | None, float | None]:
"""Return (iso_timestamp, best_macro_f1) from the newest bench_results summary.
Checks results_dir from cforch config if set, then falls back to
@ -108,8 +107,6 @@ def _find_latest_classifier_bench(results_dir_override: str = "") -> tuple[str |
if summary.exists():
try:
data = json.loads(summary.read_text(encoding="utf-8"))
if not isinstance(data, dict):
continue # cforch LLM-bench summaries are lists; skip
ts = data.get("timestamp") or subdir.name
score = data.get("best_macro_f1") or data.get("macro_f1")
return ts, (float(score) if isinstance(score, (int, float)) else None)
@ -117,10 +114,6 @@ def _find_latest_classifier_bench(results_dir_override: str = "") -> tuple[str |
logger.warning("Failed to parse summary.json at %s: %s", summary, exc)
return None, None
# Keep old name as alias so existing callers in tests still work.
_find_latest_eval = _find_latest_classifier_bench
def _count_corrections() -> tuple[int, int]:
"""Return (pending_count, export_ready_count)."""
pending = 0
@ -176,106 +169,22 @@ def _count_labeled_since(since_ts: str | None) -> int:
return sum(1 for r in records if r.get("labeled_at", "") > since_ts)
def _get_recent_bench_runs() -> dict:
"""Return most-recent run summary for each bench type.
Each entry: {"timestamp": str|None, "metric": str|None, "score": float|None}
"""
runs: dict[str, dict] = {
"classifier": {"timestamp": None, "metric": "macro_f1", "score": None},
"llm": {"timestamp": None, "metric": None, "score": None},
"style": {"timestamp": None, "metric": None, "score": None},
"plans": {"timestamp": None, "metric": "avg_score", "score": None},
}
# ── Classifier: bench_results/<run>/summary.json ──────────────────────
clf_ts, clf_score = _find_latest_classifier_bench()
if clf_ts:
runs["classifier"]["timestamp"] = clf_ts
runs["classifier"]["score"] = clf_score
# ── LLM bench + Style: benchmark_results/ ─────────────────────────────
f = _config_file()
bench_dir: Path | None = None
if f.exists():
try:
raw = yaml.safe_load(f.read_text(encoding="utf-8")) or {}
rd = (raw.get("cforch", {}) or {}).get("results_dir", "")
if rd:
bench_dir = Path(rd)
except Exception:
pass
if bench_dir is None:
bench_dir = _ROOT / "benchmark_results"
if bench_dir.exists():
llm_files = sorted(
[p for p in bench_dir.glob("*.json") if not p.name.startswith("style_")],
key=lambda p: p.stat().st_mtime, reverse=True,
)
if llm_files:
try:
data = json.loads(llm_files[0].read_text(encoding="utf-8"))
runs["llm"]["timestamp"] = data.get("timestamp") or llm_files[0].stem
except Exception:
pass
style_files = sorted(bench_dir.glob("style_*.json"), reverse=True)
if style_files:
try:
data = json.loads(style_files[0].read_text(encoding="utf-8"))
if isinstance(data, list) and data:
runs["style"]["timestamp"] = data[0].get("timestamp") or style_files[0].stem
except Exception:
pass
# ── Plans bench: data/plans_bench_results/plans_*.json ────────────────
plans_dir = _DATA_DIR / "plans_bench_results"
if plans_dir.exists():
plans_files = sorted(plans_dir.glob("plans_*.json"), reverse=True)
if plans_files:
run_id = plans_files[0].stem
try:
d: dict = json.loads(plans_files[0].read_text(encoding="utf-8"))
all_scores = [
r["total_score"]
for results in d.values()
for r in results
if isinstance(r, dict) and not r.get("error")
]
avg = round(sum(all_scores) / len(all_scores), 3) if all_scores else None
try:
date_part = run_id.removeprefix("plans_")
date, time_part = date_part.split("_")
ts_display = f"{date} {time_part[:2]}:{time_part[2:4]}"
except Exception:
ts_display = run_id
runs["plans"]["timestamp"] = ts_display
runs["plans"]["score"] = avg
except Exception:
pass
return runs
@router.get("/dashboard")
def get_dashboard() -> dict:
data_threshold, _train_threshold = _load_thresholds()
last_ts, last_score = _find_latest_classifier_bench()
labeled_since = _count_labeled_since(last_ts)
data_eval_threshold, eval_train_threshold = _load_thresholds()
last_eval_ts, last_eval_score = _find_latest_eval()
labeled_since = _count_labeled_since(last_eval_ts)
corrections_pending, corrections_export_ready = _count_corrections()
active_jobs = _get_active_jobs()
recent_bench = _get_recent_bench_runs()
return {
"labeled_since_last_eval": labeled_since,
"last_eval_timestamp": last_ts,
"last_eval_best_score": last_score,
"last_eval_timestamp": last_eval_ts,
"last_eval_best_score": last_eval_score,
"active_jobs": active_jobs,
"corrections_pending": corrections_pending,
"corrections_export_ready": corrections_export_ready,
"recent_bench_runs": recent_bench,
"signals": {
"data_to_eval": labeled_since >= data_threshold,
"data_to_eval": labeled_since >= data_eval_threshold,
"eval_to_train": False, # future: implement delta-F1 comparison
"train_to_fleet": False, # future: implement fleet sync signal
},

97
app/data/imitate.py
View file

@ -94,42 +94,6 @@ def _cforch_url() -> str:
return cforch.get("coordinator_url") or "http://localhost:7700"
def _resolve_task_model(cforch_base: str, product: str, task: str) -> dict | None:
"""Return {model_id, service_type} for a product.task assignment, or None if not found.
Calls GET coordinator/api/assignments and filters by product+task.
The model registry entry is fetched separately to get service_type.
Returns None (not raises) — callers emit a 'model_done' error event instead.
"""
try:
asgn_resp = httpx.get(f"{cforch_base}/api/assignments", timeout=5.0)
asgn_resp.raise_for_status()
assignments: list[dict] = asgn_resp.json().get("assignments", []) or []
match = next(
(a for a in assignments if a.get("product") == product and a.get("task") == task),
None,
)
if match is None:
return None
model_id: str = match.get("model_id", "")
if not model_id:
return None
# Look up service_type from model registry
reg_resp = httpx.get(f"{cforch_base}/api/model-registry", timeout=5.0)
service_type = "cf-text" # sensible default
if reg_resp.is_success:
models: list[dict] = reg_resp.json().get("models", []) or []
reg_entry = next((m for m in models if m.get("model_id") == model_id), None)
if reg_entry:
service_type = reg_entry.get("service_type", "cf-text") or "cf-text"
return {"model_id": model_id, "service_type": service_type}
except Exception as exc:
logger.warning("Task resolution failed for %s.%s: %s", product, task, exc)
return None
def _cforch_catalog(cforch_base: str) -> list[dict]:
"""Fetch the live cf-text catalog from cf-orch.
@ -512,19 +476,13 @@ def run_imitate(
prompt: str = "",
model_ids: str = "", # comma-separated ollama model IDs
cf_text_model_ids: str = "", # comma-separated cf-text model IDs (via cf-orch)
task_ids: str = "", # comma-separated "product/task" strings — resolved via assignments
temperature: float = 0.7,
product_id: str = "",
system: str = "", # optional system prompt
image_url: str = "", # optional image URL for vision models
session: "Any" = Depends(_get_imitate_session),
) -> StreamingResponse:
"""Run a prompt through selected models and stream results as SSE.
Models can be selected three ways (combinable):
- model_ids: explicit ollama model IDs
- cf_text_model_ids: explicit cf-text model IDs routed via cf-orch
- task_ids: "product/task" strings resolved via the coordinator assignments table
"""Run a prompt through selected ollama models and stream results as SSE.
If image_url is provided, the image is downloaded once and passed to every
model as a base64-encoded blob — allowing vision-capable local models to
@ -536,37 +494,8 @@ def run_imitate(
ollama_ids = [m.strip() for m in model_ids.split(",") if m.strip()]
cftext_ids = [m.strip() for m in cf_text_model_ids.split(",") if m.strip()]
raw_task_ids = [t.strip() for t in task_ids.split(",") if t.strip()]
# Resolve task assignments to concrete model IDs, routing to the right service.
# Models that fail to resolve emit an error event at run time (non-fatal).
if raw_task_ids:
cforch_base = _cforch_url()
for task_spec in raw_task_ids:
parts = task_spec.split("/", 1)
if len(parts) != 2:
logger.warning("Skipping malformed task_id %r (expected product/task)", task_spec)
continue
product_name, task_name = parts
resolved = _resolve_task_model(cforch_base, product_name, task_name)
if resolved is None:
logger.warning("No assignment found for task %r", task_spec)
# Emit error at stream time via a sentinel in cftext_ids with a special label.
# We instead store the failed task_spec to emit a model_done error.
cftext_ids.append(f"__task_unresolved__:{task_spec}")
continue
mid = resolved["model_id"]
svc = resolved["service_type"]
if svc == "ollama":
if mid not in ollama_ids:
ollama_ids.append(mid)
else:
# cf-text, vllm, and any other cf-orch-managed service
if mid not in cftext_ids:
cftext_ids.append(mid)
if not ollama_ids and not cftext_ids:
raise HTTPException(422, "model_ids, cf_text_model_ids, or task_ids is required")
raise HTTPException(422, "model_ids or cf_text_model_ids is required")
cfg = _load_imitate_config()
ollama_base = _ollama_url(cfg)
@ -610,25 +539,11 @@ def run_imitate(
yield _sse({"type": "model_done", **result})
# cf-text models via cf-orch — fan out in parallel when multiple models selected
# Partition the list: real cf-text IDs vs unresolved-task sentinels.
cftext_real = [m for m in cftext_ids if not m.startswith("__task_unresolved__:")]
cftext_unresolved = [m for m in cftext_ids if m.startswith("__task_unresolved__:")]
for sentinel in cftext_unresolved:
task_spec = sentinel.split(":", 1)[1]
result = {
"model": task_spec,
"response": "",
"elapsed_ms": 0,
"error": f"No assignment configured for task '{task_spec}'",
}
results.append(result)
yield _sse({"type": "model_done", **result})
if cftext_real:
if cftext_ids:
from concurrent.futures import ThreadPoolExecutor, as_completed
# Announce all models upfront so the UI can show loading states immediately
for model_id in cftext_real:
for model_id in cftext_ids:
yield _sse({"type": "model_start", "model": model_id, "service": "cf-text"})
_user_id: str | None = getattr(session, "user_id", None)
@ -636,13 +551,13 @@ def run_imitate(
if _user_id in (None, "local", "local-dev") or (_user_id or "").startswith("anon-"):
_user_id = None
with ThreadPoolExecutor(max_workers=len(cftext_real)) as pool:
with ThreadPoolExecutor(max_workers=len(cftext_ids)) as pool:
future_to_model = {
pool.submit(
_run_cftext, cforch_base, mid, prompt, system_ctx, temperature,
180.0, _user_id,
): mid
for mid in cftext_real
for mid in cftext_ids
}
for future in as_completed(future_to_model):
model_id = future_to_model[future]

462
app/data/log_corpus.py
View file

@ -1,462 +0,0 @@
"""Avocet — Log Corpus receiver and labeling API.
Receives push batches from consented Turnstone nodes, stores entries for labeling,
and exports labeled data as JSONL for the logreading fine-tune pipeline.
DB: data/corpus.db (separate from train_jobs.db — different lifecycle)
Auth: Bearer token validated against corpus_sources table (seeded from label_tool.yaml).
All endpoints registered on `router`. api.py includes this with prefix="/api/corpus".
"""
from __future__ import annotations
import json
import logging
import sqlite3
import uuid
from contextlib import contextmanager
from datetime import datetime, timezone
from pathlib import Path
from typing import Generator
import yaml
from fastapi import APIRouter, Depends, HTTPException
from fastapi.requests import Request
from fastapi.responses import StreamingResponse
logger = logging.getLogger(__name__)
_ROOT = Path(__file__).parent.parent.parent
_CONFIG_DIR: Path | None = None
_DATA_DIR: Path = _ROOT / "data"
router = APIRouter()
_DB_PATH: Path = _ROOT / "data" / "corpus.db"
_PIPELINE_SOURCE_HOST = "pipeline_scrape"
_SCHEMA = """
CREATE TABLE IF NOT EXISTS corpus_sources (
token TEXT PRIMARY KEY,
source_host TEXT NOT NULL,
owner TEXT NOT NULL,
consent_date TEXT NOT NULL,
consent_method TEXT NOT NULL,
active INTEGER NOT NULL DEFAULT 1
);
CREATE TABLE IF NOT EXISTS corpus_batches (
id TEXT PRIMARY KEY,
source_host TEXT NOT NULL,
batch_type TEXT NOT NULL,
received_at TEXT NOT NULL,
entry_count INTEGER NOT NULL,
watermark_from TEXT,
watermark_to TEXT,
raw_json TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS corpus_entries (
id TEXT PRIMARY KEY,
batch_id TEXT NOT NULL REFERENCES corpus_batches(id),
source_host TEXT NOT NULL,
origin_entry_id TEXT,
timestamp_iso TEXT,
severity TEXT,
source_id TEXT,
text TEXT NOT NULL,
matched_patterns TEXT DEFAULT '[]',
label_state TEXT NOT NULL DEFAULT 'unlabeled',
failure_type TEXT,
plain_explanation TEXT,
known_pattern TEXT,
labeled_at TEXT,
labeled_by TEXT DEFAULT 'alan',
pii_flagged INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX IF NOT EXISTS idx_ce_label_state ON corpus_entries(label_state);
CREATE INDEX IF NOT EXISTS idx_ce_source ON corpus_entries(source_host);
CREATE INDEX IF NOT EXISTS idx_ce_severity ON corpus_entries(severity);
CREATE TABLE IF NOT EXISTS ingested_pipeline_files (
filename TEXT PRIMARY KEY,
ingested_at TEXT NOT NULL,
entry_count INTEGER NOT NULL
);
"""
# ── Testability seams ──────────────────────────────────────────────────────────
def set_config_dir(path: Path | None) -> None:
global _CONFIG_DIR
_CONFIG_DIR = path
def set_data_dir(path: Path) -> None:
global _DATA_DIR, _DB_PATH
_DATA_DIR = path
_DB_PATH = path / "corpus.db"
# ── Internal helpers ───────────────────────────────────────────────────────────
def _config_file() -> Path:
if _CONFIG_DIR is not None:
return _CONFIG_DIR / "label_tool.yaml"
return _ROOT / "config" / "label_tool.yaml"
@contextmanager
def _db() -> Generator[sqlite3.Connection, None, None]:
conn = sqlite3.connect(str(_DB_PATH))
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA journal_mode=WAL")
try:
yield conn
conn.commit()
except Exception:
conn.rollback()
raise
finally:
conn.close()
def _init_db() -> None:
with _db() as conn:
conn.executescript(_SCHEMA)
_seed_sources(conn)
def _pipeline_ingest_dir() -> Path | None:
"""Return the configured pipeline log ingest directory, or None if unset."""
f = _config_file()
if not f.exists():
return None
try:
raw = yaml.safe_load(f.read_text(encoding="utf-8")) or {}
except yaml.YAMLError:
return None
val = raw.get("corpus", {}).get("pipeline_ingest_dir", "") or ""
return Path(val) if val else None
def _load_corpus_config() -> list[dict]:
f = _config_file()
if not f.exists():
return []
try:
raw = yaml.safe_load(f.read_text(encoding="utf-8")) or {}
except yaml.YAMLError as exc:
logger.warning("Failed to parse corpus config: %s", exc)
return []
return raw.get("corpus", {}).get("sources", []) or []
def _seed_sources(conn: sqlite3.Connection) -> None:
for src in _load_corpus_config():
conn.execute(
"INSERT OR IGNORE INTO corpus_sources (token, source_host, owner, consent_date, consent_method) "
"VALUES (?, ?, ?, ?, ?)",
(src["token"], src["source_host"], src["owner"],
src["consent_date"], src["consent_method"]),
)
def _validate_token(token: str, conn: sqlite3.Connection) -> str:
"""Return source_host for token, or raise 403."""
row = conn.execute(
"SELECT source_host FROM corpus_sources WHERE token = ? AND active = 1",
(token,),
).fetchone()
if row is None:
raise HTTPException(status_code=403, detail="Unknown or revoked consent token")
return row["source_host"]
def _extract_bearer(request: Request) -> str:
auth = request.headers.get("Authorization", "")
if not auth.startswith("Bearer "):
raise HTTPException(status_code=401, detail="Bearer token required")
return auth.removeprefix("Bearer ").strip()
def _now_iso() -> str:
return datetime.now(timezone.utc).isoformat()
# ── Startup ────────────────────────────────────────────────────────────────────
_init_db()
# ── POST /api/corpus/log-batch ─────────────────────────────────────────────────
@router.post("/log-batch")
def receive_batch(request: Request, payload: dict) -> dict:
"""Accept a push batch from a Turnstone node."""
token = _extract_bearer(request)
batch_type = payload.get("batch_type", "raw_entries")
entries_raw = payload.get("entries", [])
batch_id = payload.get("batch_id") or str(uuid.uuid4())
with _db() as conn:
source_host = _validate_token(token, conn)
conn.execute(
"INSERT INTO corpus_batches (id, source_host, batch_type, received_at, entry_count, "
"watermark_from, watermark_to, raw_json) VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
(batch_id, source_host, batch_type, _now_iso(), len(entries_raw),
str(payload.get("watermark_from", "")),
str(payload.get("watermark_to", "")),
json.dumps(payload)),
)
stored = 0
for entry in entries_raw:
text = entry.get("text", "").strip()
if not text:
continue
conn.execute(
"INSERT OR IGNORE INTO corpus_entries "
"(id, batch_id, source_host, origin_entry_id, timestamp_iso, severity, "
"source_id, text, matched_patterns) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
(str(uuid.uuid4()), batch_id, source_host,
entry.get("entry_id") or entry.get("id"),
entry.get("timestamp_iso"),
entry.get("severity"),
entry.get("source_id"),
text,
json.dumps(entry.get("matched_patterns", []))),
)
stored += 1
logger.info("Received batch %s from %s: %d/%d entries stored",
batch_id, source_host, stored, len(entries_raw))
return {"received": True, "batch_id": batch_id, "entries_stored": stored}
# ── GET /api/corpus/entries ────────────────────────────────────────────────────
@router.get("/entries")
def list_entries(
state: str = "unlabeled",
source_host: str | None = None,
limit: int = 25,
) -> dict:
"""Return entries for labeling. Default: unlabeled entries, oldest first."""
with _db() as conn:
query = "SELECT * FROM corpus_entries WHERE label_state = ?"
params: list = [state]
if source_host:
query += " AND source_host = ?"
params.append(source_host)
query += " ORDER BY rowid LIMIT ?"
params.append(min(limit, 100))
rows = conn.execute(query, params).fetchall()
return {"entries": [dict(r) for r in rows], "count": len(rows)}
# ── POST /api/corpus/entries/{id}/label ───────────────────────────────────────
@router.post("/entries/{entry_id}/label")
def label_entry(entry_id: str, body: dict) -> dict:
"""Submit a label for a corpus entry."""
failure_type = body.get("failure_type")
plain_explanation = body.get("plain_explanation", "").strip()
known_pattern = body.get("known_pattern")
pii_flagged = int(bool(body.get("pii_flagged", False)))
if not failure_type:
raise HTTPException(status_code=422, detail="failure_type is required")
valid_types = {"hardware", "software", "network", "security", "application", "none", "other"}
if failure_type not in valid_types:
raise HTTPException(status_code=422, detail=f"failure_type must be one of {sorted(valid_types)}")
with _db() as conn:
row = conn.execute("SELECT id FROM corpus_entries WHERE id = ?", (entry_id,)).fetchone()
if row is None:
raise HTTPException(status_code=404, detail="Entry not found")
conn.execute(
"UPDATE corpus_entries SET label_state='labeled', failure_type=?, plain_explanation=?, "
"known_pattern=?, labeled_at=?, pii_flagged=? WHERE id=?",
(failure_type, plain_explanation, known_pattern, _now_iso(), pii_flagged, entry_id),
)
return {"labeled": True, "entry_id": entry_id}
# ── POST /api/corpus/entries/{id}/skip ────────────────────────────────────────
@router.post("/entries/{entry_id}/skip")
def skip_entry(entry_id: str) -> dict:
with _db() as conn:
row = conn.execute("SELECT id FROM corpus_entries WHERE id = ?", (entry_id,)).fetchone()
if row is None:
raise HTTPException(status_code=404, detail="Entry not found")
conn.execute(
"UPDATE corpus_entries SET label_state='skipped' WHERE id=?", (entry_id,)
)
return {"skipped": True, "entry_id": entry_id}
# ── GET /api/corpus/stats ──────────────────────────────────────────────────────
@router.get("/stats")
def get_stats() -> dict:
with _db() as conn:
total = conn.execute("SELECT COUNT(*) FROM corpus_entries").fetchone()[0]
by_state = {
r["label_state"]: r["cnt"]
for r in conn.execute(
"SELECT label_state, COUNT(*) AS cnt FROM corpus_entries GROUP BY label_state"
).fetchall()
}
by_source = {
r["source_host"]: r["cnt"]
for r in conn.execute(
"SELECT source_host, COUNT(*) AS cnt FROM corpus_entries GROUP BY source_host"
).fetchall()
}
by_severity = {
r["severity"]: r["cnt"]
for r in conn.execute(
"SELECT severity, COUNT(*) AS cnt FROM corpus_entries "
"WHERE severity IS NOT NULL GROUP BY severity"
).fetchall()
}
batch_count = conn.execute("SELECT COUNT(*) FROM corpus_batches").fetchone()[0]
return {
"total_entries": total,
"batch_count": batch_count,
"by_label_state": by_state,
"by_source": by_source,
"by_severity": by_severity,
}
# ── GET /api/corpus/export ────────────────────────────────────────────────────
@router.get("/export")
def export_labeled() -> StreamingResponse:
"""Stream labeled, non-PII entries as JSONL for SFT harness."""
with _db() as conn:
rows = conn.execute(
"SELECT source_host, source_id, severity, text, failure_type, plain_explanation, known_pattern "
"FROM corpus_entries "
"WHERE label_state = 'labeled' AND pii_flagged = 0 AND plain_explanation != ''"
"ORDER BY rowid"
).fetchall()
def _generate():
for row in rows:
record = {
"input": row["text"],
"output": row["plain_explanation"],
"metadata": {
"failure_type": row["failure_type"],
"source": row["source_host"],
"source_id": row["source_id"],
"severity": row["severity"],
"known_pattern": row["known_pattern"],
},
}
yield json.dumps(record) + "\n"
return StreamingResponse(
_generate(),
media_type="application/x-ndjson",
headers={"Content-Disposition": "attachment; filename=log_corpus_labeled.jsonl"},
)
# ── POST /api/corpus/pipeline-ingest ─────────────────────────────────────────
def _ingest_one_file(conn: sqlite3.Connection, path: Path) -> int:
"""Parse a pipeline JSONL file and insert entries. Returns count stored."""
batch_id = str(uuid.uuid4())
lines = path.read_text(encoding="utf-8").splitlines()
entries_raw: list[dict] = []
for line in lines:
line = line.strip()
if not line:
continue
try:
entries_raw.append(json.loads(line))
except json.JSONDecodeError:
logger.debug("Skipping malformed line in %s", path.name)
conn.execute(
"INSERT INTO corpus_batches (id, source_host, batch_type, received_at, entry_count, raw_json) "
"VALUES (?, ?, ?, ?, ?, ?)",
(batch_id, _PIPELINE_SOURCE_HOST, "pipeline_log", _now_iso(),
len(entries_raw), json.dumps({"file": path.name})),
)
stored = 0
for entry in entries_raw:
text = (entry.get("msg") or "").strip()
if not text:
continue
conn.execute(
"INSERT OR IGNORE INTO corpus_entries "
"(id, batch_id, source_host, timestamp_iso, severity, source_id, text, matched_patterns) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
(str(uuid.uuid4()), batch_id, _PIPELINE_SOURCE_HOST,
entry.get("ts"),
entry.get("level"),
entry.get("logger"),
text,
json.dumps([entry["extra"]] if entry.get("extra") else [])),
)
stored += 1
conn.execute(
"INSERT INTO ingested_pipeline_files (filename, ingested_at, entry_count) VALUES (?, ?, ?)",
(path.name, _now_iso(), stored),
)
return stored
@router.post("/pipeline-ingest")
def pipeline_ingest() -> dict:
"""Walk the configured pipeline log directory and ingest new JSONL files.
Skips files already recorded in ingested_pipeline_files. Safe to call
repeatedly — idempotent by filename.
"""
ingest_dir = _pipeline_ingest_dir()
if ingest_dir is None:
raise HTTPException(404, "pipeline_ingest_dir not configured in label_tool.yaml")
ingested = 0
skipped = 0
total_stored = 0
files_detail: list[dict] = []
with _db() as conn:
already_done: set[str] = {
row[0]
for row in conn.execute("SELECT filename FROM ingested_pipeline_files").fetchall()
}
for path in sorted(ingest_dir.glob("*.jsonl")):
if path.name in already_done:
skipped += 1
continue
stored = _ingest_one_file(conn, path)
ingested += 1
total_stored += stored
files_detail.append({"file": path.name, "entries_stored": stored})
logger.info("Pipeline ingest: %d files ingested, %d skipped, %d entries stored",
ingested, skipped, total_stored)
return {
"ingested_files": ingested,
"skipped_files": skipped,
"entries_stored": total_stored,
"files": files_detail,
}

313
app/data/recipe_scan.py
View file

@ -1,313 +0,0 @@
"""Avocet — Recipe scan labeling API (avocet#65).
Receives recipe scan items from the Kiwi pipeline (scanner/phone image +
docuvision OCR extraction + ground-truth structured recipe), presents them
for human review, and exports approved/edited pairs in the messages chat
format for the vision fine-tune harness.
DB: data/recipe_scan.db (separate from corpus.db — different lifecycle)
No auth required — local admin tool, not a push endpoint.
All endpoints registered on `router`. api.py includes this with
prefix="/api/recipe-scan".
"""
from __future__ import annotations
import json
import logging
import sqlite3
import uuid
from contextlib import contextmanager
from datetime import datetime, timezone
from pathlib import Path
from typing import Generator, Literal
from fastapi import APIRouter, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, field_validator
logger = logging.getLogger(__name__)
_ROOT = Path(__file__).parent.parent.parent
_DB_PATH: Path = _ROOT / "data" / "recipe_scan.db"
_VALID_MODALITIES = {"scanner", "phone", "handwritten"}
_VALID_STATUSES = {"pending", "approved", "edited", "rejected"}
_SCHEMA = """
CREATE TABLE IF NOT EXISTS recipe_scan_items (
id TEXT PRIMARY KEY,
image_path TEXT NOT NULL,
modality TEXT NOT NULL DEFAULT 'scanner',
source TEXT NOT NULL DEFAULT 'purple_carrot',
extracted TEXT NOT NULL,
ground_truth TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'pending',
corrected TEXT,
labeled_at TEXT,
rejected_reason TEXT
);
CREATE INDEX IF NOT EXISTS idx_rsi_status ON recipe_scan_items(status);
CREATE INDEX IF NOT EXISTS idx_rsi_modality ON recipe_scan_items(modality);
"""
router = APIRouter()
# ── Testability seam ──────────────────────────────────────────────────────────
def set_db_path(path: Path) -> None:
global _DB_PATH
_DB_PATH = path
# ── Internal helpers ──────────────────────────────────────────────────────────
@contextmanager
def _db() -> Generator[sqlite3.Connection, None, None]:
conn = sqlite3.connect(str(_DB_PATH))
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA journal_mode=WAL")
try:
yield conn
conn.commit()
except Exception:
conn.rollback()
raise
finally:
conn.close()
def _init_db() -> None:
with _db() as conn:
conn.executescript(_SCHEMA)
def _now_iso() -> str:
return datetime.now(timezone.utc).isoformat()
def _build_training_pair(row: sqlite3.Row) -> dict:
"""Build a messages-format training pair from a labeled row.
user message: correction prompt + the docuvision-extracted JSON draft.
Trains the model to review and correct an existing extraction, which is
more data-efficient than producing from scratch when OCR is usually close.
assistant message: the approved ground truth (or human-corrected JSON).
"""
target_str = row["corrected"] if row["corrected"] else row["ground_truth"]
extracted = json.loads(row["extracted"])
target = json.loads(target_str)
user_content = (
"Review and correct this recipe extraction. "
"Return valid JSON with fields: title, description, ingredients, steps, "
"prep_time, cook_time, servings.\n\n"
f"Extraction to review:\n{json.dumps(extracted, ensure_ascii=False, indent=2)}"
)
return {
"id": row["id"],
"modality": row["modality"],
"source": row["source"],
"image_path": row["image_path"],
"messages": [
{"role": "user", "content": user_content},
{"role": "assistant", "content": json.dumps(target, ensure_ascii=False)},
],
}
_init_db()
# ── POST /import ───────────────────────────────────────────────────────────────
class ImportItem(BaseModel):
id: str = ""
image_path: str
modality: Literal["scanner", "phone", "handwritten"] = "scanner"
source: str = "purple_carrot"
extracted: dict
ground_truth: dict
@field_validator("id", mode="before")
@classmethod
def default_id(cls, v: str) -> str:
return v or str(uuid.uuid4())
class ImportRequest(BaseModel):
items: list[ImportItem]
@router.post("/import")
def import_items(body: ImportRequest) -> dict:
"""Bulk-import scan items from the Kiwi pipeline. Idempotent by item id."""
stored = 0
with _db() as conn:
for item in body.items:
result = conn.execute(
"INSERT OR IGNORE INTO recipe_scan_items "
"(id, image_path, modality, source, extracted, ground_truth) "
"VALUES (?, ?, ?, ?, ?, ?)",
(item.id, item.image_path, item.modality, item.source,
json.dumps(item.extracted), json.dumps(item.ground_truth)),
)
stored += result.rowcount
return {"imported": stored, "total_submitted": len(body.items)}
# ── GET /next ─────────────────────────────────────────────────────────────────
@router.get("/next")
def get_next() -> dict:
"""Return the next pending item for review, oldest-first."""
with _db() as conn:
row = conn.execute(
"SELECT * FROM recipe_scan_items WHERE status = 'pending' ORDER BY rowid LIMIT 1"
).fetchone()
if row is None:
raise HTTPException(404, "No pending items in queue")
return {
**dict(row),
"extracted": json.loads(row["extracted"]),
"ground_truth": json.loads(row["ground_truth"]),
}
# ── POST /items/{id}/approve ──────────────────────────────────────────────────
@router.post("/items/{item_id}/approve")
def approve_item(item_id: str) -> dict:
"""Mark item as approved — extracted JSON is close enough to ground truth."""
with _db() as conn:
row = conn.execute("SELECT id FROM recipe_scan_items WHERE id = ?", (item_id,)).fetchone()
if row is None:
raise HTTPException(404, "Item not found")
conn.execute(
"UPDATE recipe_scan_items SET status='approved', labeled_at=? WHERE id=?",
(_now_iso(), item_id),
)
return {"status": "approved", "id": item_id}
# ── POST /items/{id}/edit ─────────────────────────────────────────────────────
class EditBody(BaseModel):
corrected: dict
@router.post("/items/{item_id}/edit")
def edit_item(item_id: str, body: EditBody) -> dict:
"""Approve with a human-corrected JSON. corrected overrides extracted in export."""
with _db() as conn:
row = conn.execute("SELECT id FROM recipe_scan_items WHERE id = ?", (item_id,)).fetchone()
if row is None:
raise HTTPException(404, "Item not found")
conn.execute(
"UPDATE recipe_scan_items SET status='edited', corrected=?, labeled_at=? WHERE id=?",
(json.dumps(body.corrected), _now_iso(), item_id),
)
return {"status": "edited", "id": item_id}
# ── POST /items/{id}/reject ───────────────────────────────────────────────────
class RejectBody(BaseModel):
reason: str = ""
@router.post("/items/{item_id}/reject")
def reject_item(item_id: str, body: RejectBody = RejectBody()) -> dict:
"""Reject item — extraction too broken to use for training."""
with _db() as conn:
row = conn.execute("SELECT id FROM recipe_scan_items WHERE id = ?", (item_id,)).fetchone()
if row is None:
raise HTTPException(404, "Item not found")
conn.execute(
"UPDATE recipe_scan_items SET status='rejected', rejected_reason=?, labeled_at=? WHERE id=?",
(body.reason or None, _now_iso(), item_id),
)
return {"status": "rejected", "id": item_id}
# ── GET /stats ────────────────────────────────────────────────────────────────
@router.get("/stats")
def get_stats() -> dict:
with _db() as conn:
total = conn.execute("SELECT COUNT(*) FROM recipe_scan_items").fetchone()[0]
by_status = {
r["status"]: r["cnt"]
for r in conn.execute(
"SELECT status, COUNT(*) AS cnt FROM recipe_scan_items GROUP BY status"
).fetchall()
}
by_modality = {
r["modality"]: r["cnt"]
for r in conn.execute(
"SELECT modality, COUNT(*) AS cnt FROM recipe_scan_items GROUP BY modality"
).fetchall()
}
export_ready = conn.execute(
"SELECT COUNT(*) FROM recipe_scan_items WHERE status IN ('approved', 'edited')"
).fetchone()[0]
return {
"total": total,
"by_status": by_status,
"by_modality": by_modality,
"export_ready": export_ready,
}
# ── GET /export ───────────────────────────────────────────────────────────────
@router.get("/export")
def export_pairs() -> StreamingResponse:
"""Stream approved/edited items as JSONL training pairs (messages format)."""
with _db() as conn:
rows = conn.execute(
"SELECT * FROM recipe_scan_items WHERE status IN ('approved', 'edited') ORDER BY rowid"
).fetchall()
def _generate():
for row in rows:
yield json.dumps(_build_training_pair(row), ensure_ascii=False) + "\n"
return StreamingResponse(
_generate(),
media_type="application/x-ndjson",
headers={"Content-Disposition": "attachment; filename=recipe_scan_pairs.jsonl"},
)
# ── GET /image ────────────────────────────────────────────────────────────────
_IMAGE_ROOT = Path("/Library/Assets/kiwi")
@router.get("/image")
def serve_image(path: str) -> StreamingResponse:
"""Serve a scan image from /Library/Assets/kiwi/.
path must resolve within /Library/Assets/kiwi/ — rejects traversal attempts.
"""
try:
resolved = Path(path).resolve()
_IMAGE_ROOT.resolve() # ensure root itself is valid
resolved.relative_to(_IMAGE_ROOT.resolve())
except (ValueError, OSError):
raise HTTPException(403, "Path outside allowed image directory")
if not resolved.exists():
raise HTTPException(404, "Image not found")
suffix = resolved.suffix.lower()
media_types = {".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".png": "image/png", ".webp": "image/webp"}
media_type = media_types.get(suffix, "application/octet-stream")
return StreamingResponse(
open(resolved, "rb"),
media_type=media_type,
headers={"Cache-Control": "public, max-age=86400"},
)

8
app/eval/cforch.py
View file

@ -12,33 +12,27 @@ Route prefixes when mounted at /api in api.py:
"""
from __future__ import annotations
from pathlib import Path
from fastapi import APIRouter
from app.cforch import router as _cforch_router
from app.style import router as _style_router
from app.voice import router as _voice_router
from app.plans_bench import router as _plans_router
from app.eval.embed_bench import router as _embed_router
router = APIRouter()
router.include_router(_cforch_router, prefix="/cforch")
router.include_router(_style_router, prefix="/style")
router.include_router(_voice_router, prefix="/voice")
router.include_router(_plans_router, prefix="/plans-bench")
router.include_router(_embed_router, prefix="/embed-bench")
def set_config_dir(path: Path | None) -> None:
def set_config_dir(path) -> None:
"""Propagate config dir override to all sub-modules -- used by tests."""
import app.cforch as _cforch_mod
import app.style as _style_mod
import app.voice as _voice_mod
import app.plans_bench as _plans_mod
import app.eval.embed_bench as _embed_mod
_cforch_mod.set_config_dir(path)
_style_mod.set_config_dir(path)
_voice_mod.set_config_dir(path)
_plans_mod.set_config_dir(path)
_embed_mod.set_config_dir(path)

293
app/eval/embed_bench.py
View file

@ -1,293 +0,0 @@
"""Avocet — embedding model comparison harness.
Exposes FastAPI routes under /api/embed-bench (mounted via app/eval/cforch.py).
All computation is local: no LLM inference, Ollama only. MIT tier throughout.
"""
from __future__ import annotations
import csv
import io
import json
import logging
import math
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
import httpx
import yaml
from fastapi import APIRouter, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, field_validator
logger = logging.getLogger(__name__)
_ROOT = Path(__file__).parent.parent.parent
_CONFIG_DIR: Path | None = None # override via set_config_dir() in tests
_RUN_ACTIVE: bool = False
_RATINGS_FILE = _ROOT / "data" / "embed_bench_ratings.jsonl"
router = APIRouter()
# ── Testability seam ──────────────────────────────────────────────────────────
def set_config_dir(path: Path | None) -> None:
global _CONFIG_DIR
_CONFIG_DIR = path
# ── Internal helpers ──────────────────────────────────────────────────────────
def _config_file() -> Path:
if _CONFIG_DIR is not None:
return _CONFIG_DIR / "label_tool.yaml"
return _ROOT / "config" / "label_tool.yaml"
def _load_config() -> dict[str, Any]:
f = _config_file()
if not f.exists():
return {}
try:
return yaml.safe_load(f.read_text(encoding="utf-8")) or {}
except yaml.YAMLError as exc:
logger.warning("Failed to parse embed_bench config %s: %s", f, exc)
return {}
def _ollama_url() -> str:
cfg = _load_config()
embed_cfg = cfg.get("embed_bench", {}) or {}
cforch_cfg = cfg.get("cforch", {}) or {}
return (
embed_cfg.get("ollama_url")
or cforch_cfg.get("ollama_url", "http://localhost:11434")
)
def _ratings_path() -> Path:
if _CONFIG_DIR is not None:
return _CONFIG_DIR / "embed_bench_ratings.jsonl"
return _RATINGS_FILE
def _cosine(a: list[float], b: list[float]) -> float:
if len(a) != len(b):
raise ValueError(
f"Embedding dimension mismatch: {len(a)} vs {len(b)}"
)
dot = sum(x * y for x, y in zip(a, b))
mag_a = math.sqrt(sum(x * x for x in a))
mag_b = math.sqrt(sum(x * x for x in b))
if mag_a == 0.0 or mag_b == 0.0:
return 0.0
return dot / (mag_a * mag_b)
# ── GET /models ───────────────────────────────────────────────────────────────
@router.get("/models")
def get_models() -> dict:
"""Return Ollama embedding models available on the configured instance."""
ollama = _ollama_url()
models: list[dict] = []
try:
resp = httpx.get(f"{ollama}/api/tags", timeout=5.0)
resp.raise_for_status()
for entry in resp.json().get("models", []):
models.append({
"name": entry.get("name", ""),
"size": entry.get("size", 0),
})
except httpx.HTTPStatusError as exc:
logger.warning("Ollama /api/tags returned HTTP %s: %s", exc.response.status_code, exc)
except httpx.RequestError as exc:
logger.warning("Failed to reach Ollama for model list: %s", exc)
return {"models": models, "ollama_url": ollama}
# ── POST /run ─────────────────────────────────────────────────────────────────
class RunRequest(BaseModel):
corpus: list[str]
queries: list[str]
models: list[str]
top_k: int = 5
ollama_url: str = ""
@field_validator("corpus")
@classmethod
def corpus_nonempty(cls, v: list[str]) -> list[str]:
if not v:
raise ValueError("corpus must not be empty")
return v
@field_validator("queries")
@classmethod
def queries_nonempty(cls, v: list[str]) -> list[str]:
if not v:
raise ValueError("queries must not be empty")
return v
@field_validator("models")
@classmethod
def models_nonempty(cls, v: list[str]) -> list[str]:
if not v:
raise ValueError("models must contain at least one model name")
return v
def _embed_texts(ollama: str, model: str, texts: list[str]) -> list[list[float]]:
"""Batch-embed texts via Ollama /v1/embeddings. Returns one vector per text."""
resp = httpx.post(
f"{ollama}/v1/embeddings",
json={"model": model, "input": texts},
timeout=120.0,
)
resp.raise_for_status()
data = resp.json().get("data", [])
return [item["embedding"] for item in data]
def _sse(event: dict) -> str:
return f"data: {json.dumps(event)}\n\n"
@router.post("/run")
def run_embed_bench(req: RunRequest) -> StreamingResponse:
"""Embed corpus + queries with each model; stream SSE results."""
global _RUN_ACTIVE
if _RUN_ACTIVE:
raise HTTPException(409, "An embedding benchmark run is already active")
ollama = req.ollama_url or _ollama_url()
def _generate():
global _RUN_ACTIVE
_RUN_ACTIVE = True
try:
for model_idx, model in enumerate(req.models, start=1):
yield _sse({
"type": "progress",
"msg": f"Indexing corpus with {model} ({model_idx}/{len(req.models)})...",
})
try:
corpus_vecs = _embed_texts(ollama, model, req.corpus)
except Exception as exc:
yield _sse({"type": "error", "msg": f"Ollama error for {model}: {exc}"})
continue
yield _sse({
"type": "progress",
"msg": f"Running queries with {model}...",
})
for q_idx, query in enumerate(req.queries):
try:
q_vecs = _embed_texts(ollama, model, [query])
except Exception as exc:
yield _sse({"type": "error", "msg": f"Query embed error ({model}): {exc}"})
continue
q_vec = q_vecs[0]
scored = sorted(
[
{"chunk_idx": i, "text": chunk, "score": round(_cosine(q_vec, cv), 4)}
for i, (chunk, cv) in enumerate(zip(req.corpus, corpus_vecs))
],
key=lambda h: h["score"],
reverse=True,
)[: req.top_k]
yield _sse({
"type": "result",
"query_idx": q_idx,
"query": query,
"model": model,
"hits": scored,
})
yield _sse({"type": "done"})
finally:
_RUN_ACTIVE = False
return StreamingResponse(_generate(), media_type="text/event-stream")
# ── POST /rate ────────────────────────────────────────────────────────────────
_VALID_RATINGS = {"relevant", "not_relevant"}
class RatingRequest(BaseModel):
query: str
model: str
chunk_text: str
chunk_idx: int
rating: str
@field_validator("rating")
@classmethod
def rating_valid(cls, v: str) -> str:
if v not in _VALID_RATINGS:
raise ValueError(f"rating must be one of {_VALID_RATINGS}")
return v
@router.post("/rate")
def rate_result(req: RatingRequest) -> dict:
"""Append one rating to the JSONL ratings file."""
entry = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"query": req.query,
"model": req.model,
"chunk_idx": req.chunk_idx,
"chunk_text": req.chunk_text,
"rating": req.rating,
}
path = _ratings_path()
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(entry) + "\n")
return {"ok": True}
# ── GET /export ───────────────────────────────────────────────────────────────
_CSV_FIELDS = ["timestamp", "query", "model", "chunk_idx", "chunk_text", "rating"]
@router.get("/export")
def export_ratings(format: str = "csv") -> Any:
"""Download ratings as CSV or JSON."""
path = _ratings_path()
rows: list[dict] = []
if path.exists():
for raw in path.read_text(encoding="utf-8").splitlines():
raw = raw.strip()
if raw:
try:
rows.append(json.loads(raw))
except json.JSONDecodeError:
pass
date_str = datetime.now(timezone.utc).strftime("%Y-%m-%d")
if format == "json":
content = json.dumps(rows, ensure_ascii=False, indent=2)
return StreamingResponse(
iter([content]),
media_type="application/json",
headers={"Content-Disposition": f'attachment; filename="embed_comparison_{date_str}.json"'},
)
# Default: CSV
buf = io.StringIO()
writer = csv.DictWriter(buf, fieldnames=_CSV_FIELDS, extrasaction="ignore")
writer.writeheader()
writer.writerows(rows)
return StreamingResponse(
iter([buf.getvalue()]),
media_type="text/csv",
headers={"Content-Disposition": f'attachment; filename="embed_comparison_{date_str}.csv"'},
)

26
app/models.py
View file

@ -38,17 +38,13 @@ except ImportError: # pragma: no cover
logger = logging.getLogger(__name__)
_ROOT = Path(__file__).parent.parent
_MODELS_DIR: Path = Path(
os.environ.get("AVOCET_MODELS_DIR", str(_ROOT / "models"))
)
_MODELS_DIR: Path = _ROOT / "models"
_QUEUE_DIR: Path = _ROOT / "data"
# Service-specific model destinations.
# cf-text models land on the NFS-mounted shared asset store so every cluster
# node can reach them without a separate download. Avocet classifiers default
# to a local path but can be redirected via AVOCET_MODELS_DIR — set this to
# /Library/Assets/LLM/avocet/models on NFS-connected nodes to keep all model
# weights out of the repo directory.
# node can reach them without a separate download. Avocet classifiers stay local
# because they are fine-tuned in-place and are only consumed by avocet itself.
# Override via CF_TEXT_MODELS_DIR env var (useful for dev / non-NFS setups).
_CF_TEXT_MODELS_DIR: Path = Path(
os.environ.get("CF_TEXT_MODELS_DIR", "/Library/Assets/LLM/cf-text/models")
@ -124,12 +120,11 @@ _TAG_TO_INFO: dict[str, _TagInfo] = {
"image-classification": {"adapter": None, "role": "vision", "service": "cf-vision"},
"zero-shot-image-classification": {"adapter": None, "role": "vision", "service": "cf-vision"},
"image-feature-extraction": {"adapter": None, "role": "embedding", "service": "cf-vision"},
# Generative VLMs (image+text → text) — GGUF quants run via llama.cpp (cf-text).
# cf-vision is a classifier/embedder service; generative VLMs like Qwen2-VL
# and LLaVA accept image inputs but are textgen at the backend level.
# Full-precision HF-format VLMs would use vllm, but our fleet uses GGUF quants.
"image-text-to-text": {"adapter": None, "role": "vlm", "service": "cf-text"},
"visual-question-answering": {"adapter": None, "role": "vlm", "service": "cf-text"},
# Generative VLMs (image+text → text) — run under vllm, not cf-vision.
# cf-vision is a classifier/embedder service; generative VLMs like Qwen-VL,
# LLaVA, and InternVL are textgen models that happen to accept image inputs.
"image-text-to-text": {"adapter": None, "role": "vlm", "service": "vllm"},
"visual-question-answering": {"adapter": None, "role": "vlm", "service": "vllm"},
# Image generation — cf-image (text → image; distinct from cf-vision image understanding)
"text-to-image": {"adapter": None, "role": "image-gen", "service": "cf-image"},
# Embedding — cf-core shared embedding layer
@ -144,11 +139,6 @@ def set_models_dir(path: Path) -> None:
_MODELS_DIR = path
def set_cf_text_models_dir(path: Path) -> None:
global _CF_TEXT_MODELS_DIR
_CF_TEXT_MODELS_DIR = path
def set_queue_dir(path: Path) -> None:
global _QUEUE_DIR
_QUEUE_DIR = path

180
app/nodes.py
View file

@ -120,7 +120,7 @@ def list_nodes() -> list:
try:
r = httpx.get(f"{coordinator_url}/api/nodes", timeout=5.0)
r.raise_for_status()
coord_nodes: list[dict] = r.json().get("nodes", [])
coord_nodes: list[dict] = r.json()
except httpx.HTTPError as exc:
logger.warning("Coordinator unreachable: %s", exc)
return []
@ -128,7 +128,7 @@ def list_nodes() -> list:
try:
sr = httpx.get(f"{coordinator_url}/api/services", timeout=5.0)
sr.raise_for_status()
services_data: list[dict] = sr.json().get("services", [])
services_data: list[dict] = sr.json()
except httpx.HTTPError:
logger.warning("Services API unreachable for %s, skipping", coordinator_url)
services_data = []
@ -294,99 +294,6 @@ def update_gpu_services(node_id: str, gpu_id: int, body: UpdateServicesRequest)
return {"ok": True, "reloaded": reloaded, "warnings": []}
# ── Profile save / generate ────────────────────────────────────────────────────
class SaveProfileRequest(BaseModel):
profile: dict
@router.put("/nodes/{node_id}/profile", status_code=200)
def save_profile(node_id: str, body: SaveProfileRequest) -> dict:
"""Write a full profile dict to disk as YAML, then trigger coordinator reload."""
p = _profile_path(node_id)
if p is None:
raise HTTPException(500, "profiles_dir not configured in label_tool.yaml")
p.parent.mkdir(parents=True, exist_ok=True)
tmp = Path(str(p) + ".tmp")
tmp.write_text(
yaml.dump(body.profile, default_flow_style=False, allow_unicode=True, sort_keys=False),
encoding="utf-8",
)
os.replace(tmp, p)
cfg = _load_config()
coordinator_url = cfg.get("coordinator_url", "") or ""
reloaded = False
if coordinator_url:
try:
import httpx
rr = httpx.post(f"{coordinator_url}/api/nodes/{node_id}/reload-profile", timeout=5.0)
reloaded = rr.status_code < 300
except Exception as exc:
logger.warning("Coordinator reload failed for %s: %s", node_id, exc)
return {"ok": True, "reloaded": reloaded}
@router.post("/nodes/{node_id}/profile/generate")
def generate_profile(node_id: str) -> dict:
"""Return a profile skeleton seeded from coordinator GPU data.
If a profile already exists, preserves its services section and only
refreshes the nodes hardware section. Never writes to disk — the caller
must call PUT /profile to persist.
"""
import httpx
cfg = _load_config()
coordinator_url = cfg.get("coordinator_url", "") or ""
if not coordinator_url:
raise HTTPException(503, "coordinator_url not configured")
try:
r = httpx.get(f"{coordinator_url}/api/nodes", timeout=5.0)
r.raise_for_status()
coord_nodes: list[dict] = r.json().get("nodes", [])
except httpx.HTTPError as exc:
raise HTTPException(502, f"Coordinator unreachable: {exc}")
node = next((n for n in coord_nodes if n.get("node_id") == node_id), None)
if node is None:
raise HTTPException(404, f"Node {node_id!r} not found in coordinator")
gpus = [
{
"id": g.get("gpu_id", i),
"vram_mb": g.get("vram_total_mb", 0),
"compute_cap": g.get("compute_cap", 0.0),
"card": g.get("card", g.get("name", "")),
"role": "inference",
"services": [],
}
for i, g in enumerate(node.get("gpus", []))
]
vram_total = max((g["vram_mb"] for g in gpus), default=0)
existing = _load_profile(node_id) or {}
return {
"schema_version": existing.get("schema_version", 1),
"name": existing.get("name", f"node-{node_id}"),
"vram_total_mb": vram_total,
"eviction_timeout_s": existing.get("eviction_timeout_s", 10.0),
"services": existing.get("services", {}),
"nodes": {
node_id: {
"local_model_root": (
(existing.get("nodes", {}) or {})
.get(node_id, {})
.get("local_model_root", "")
),
"gpus": gpus,
}
},
"model_size_hints": existing.get("model_size_hints", {}),
}
# ── Ollama model management ────────────────────────────────────────────────────
class PullRequest(BaseModel):
@ -450,86 +357,3 @@ def delete_ollama_model(node_id: str, name: str) -> dict:
raise
except Exception as exc:
raise HTTPException(502, f"Ollama unreachable: {exc}")
# ── Model deploy (add catalog entry) ──────────────────────────────────────────
class DeployModelRequest(BaseModel):
model_id: str
service_type: str
vram_mb: int
description: str = ""
hf_repo: str = ""
path: str = "" # explicit path; if empty, constructed from model_base_path + hf_repo slug
@router.post("/nodes/{node_id}/models/deploy", status_code=200)
def deploy_model(node_id: str, body: DeployModelRequest) -> dict:
"""Register a model in the node's service catalog.
Adds (or updates) the catalog entry for body.model_id under the given
service_type in the node's profile YAML, then triggers a coordinator reload.
Does not download the model — that is the user's responsibility.
Returns the resolved path so the caller can see where the model should land.
"""
p = _profile_path(node_id)
if p is None or not p.exists():
raise HTTPException(404, f"No profile found for node {node_id!r}")
try:
profile = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
except yaml.YAMLError as exc:
raise HTTPException(500, f"Malformed profile YAML: {exc}")
services_def = profile.get("services", {}) or {}
svc = services_def.get(body.service_type)
if svc is None:
raise HTTPException(
422,
f"Service '{body.service_type}' not defined in node '{node_id}' profile; "
"add it first via the profile editor",
)
# Resolve path: explicit > model_base_path + hf slug > model_id slug
model_path = body.path.strip()
if not model_path:
base = (svc.get("model_base_path", "") or "").rstrip("/")
if not base:
raise HTTPException(
422,
f"Service '{body.service_type}' has no model_base_path; supply an explicit path",
)
slug_src = body.hf_repo.strip() if body.hf_repo.strip() else body.model_id
hf_slug = slug_src.replace("/", "--")
model_path = f"{base}/{hf_slug}"
# Immutable catalog update — spread, never mutate
entry: dict = {"path": model_path, "vram_mb": body.vram_mb}
if body.description:
entry["description"] = body.description
new_catalog = {**(svc.get("catalog") or {}), body.model_id: entry}
new_svc = {**svc, "catalog": new_catalog}
new_services = {**services_def, body.service_type: new_svc}
new_profile = {**profile, "services": new_services}
# Atomic write
tmp = Path(str(p) + ".tmp")
tmp.write_text(
yaml.dump(new_profile, default_flow_style=False, allow_unicode=True, sort_keys=False),
encoding="utf-8",
)
os.replace(tmp, p)
# Trigger coordinator reload
cfg = _load_config()
coordinator_url = cfg.get("coordinator_url", "") or ""
reloaded = False
if coordinator_url:
try:
import httpx
rr = httpx.post(f"{coordinator_url}/api/nodes/{node_id}/reload-profile", timeout=5.0)
reloaded = rr.status_code < 300
except Exception as exc:
logger.warning("Coordinator reload failed for %s: %s", node_id, exc)
return {"ok": True, "reloaded": reloaded, "path": model_path}

14
app/plans_bench.py
View file

@ -38,15 +38,11 @@ router = APIRouter()
# Kept here so the UI can list them without importing the script.
MODEL_REGISTRY: dict[str, str] = {
"deepseek-r1-1.5b": "DeepSeek R1 1.5B distill (cf-orch catalog key)",
"deepseek-r1-7b-4bit": "DeepSeek R1 7B distill, 4-bit (cf-orch catalog key)",
"deepseek-r1-0528-qwen3-8b-gguf": "DeepSeek R1 0528 Qwen3 8B GGUF (4 nodes)",
"deepseek-coder-6.7b-4bit": "DeepSeek Coder 6.7B instruct, 4-bit (cf-orch catalog key)",
"granite-4.1-8b": "IBM Granite 4.1 8B, 4-bit (cf-orch catalog key)",
"qwen2.5-3b": "Qwen 2.5 3B Q4 GGUF (cf-orch catalog key)",
"qwen2.5-7b": "Qwen 2.5 7B Q4 GGUF (cf-orch catalog key)",
"capybarahermes-2.5-mistral-7b-gguf": "CapybaraHermes 2.5 Mistral 7B GGUF (4 nodes)",
"darwin-9b-opus-gguf": "Darwin 9B Opus GGUF -- long-form writing (3 nodes)",
"llama3.2-3b": "Llama 3.2 3B Instruct (local via cf-text)",
"llama3.2-1b": "Llama 3.2 1B Instruct (local via cf-text)",
"mistral-7b": "Mistral 7B Instruct (local via cf-text)",
"phi3-mini": "Phi-3 Mini 3.8B (local via cf-text)",
"qwen2.5-3b": "Qwen 2.5 3B Instruct (local via cf-text)",
}
RUBRIC_LABELS: dict[str, str] = {

32
config/label_tool.yaml.example
View file

@ -110,35 +110,3 @@ imitate:
sample_endpoint: /api/listings
text_fields: [title, description, seller_info]
prompt_template: "Evaluate the trustworthiness of this listing and flag any red flags:\n\n{text}"
- id: pagepiper
name: Pagepiper
icon: "📄"
description: "PDF/rulebook RAG tool: page-level text chunks"
base_url: http://localhost:8511
health_path: /api/health
sample_endpoint: /api/library
chunk_endpoint: /api/library/sample-chunks?limit=50 # requires pagepiper#6
text_fields: [title]
prompt_template: "Summarize the key rules described in this passage:\n\n{text}"
# ── Log corpus (Turnstone training data) ──────────────────────────────────────
corpus:
# Directory containing pipeline JSONL log files to ingest (pull-side).
# Files named <script>_<ts>.jsonl; one structured record per line.
# POST /api/corpus/pipeline-ingest walks this dir and imports new files.
# NFS-mounted on both Heimdall and Sif at /Library/Assets/
pipeline_ingest_dir: /Library/Assets/logs/pipeline/
# Turnstone push sources (consent-gated, token-authenticated).
# sources:
# - token: "your-bearer-token"
# source_host: "node.local"
# owner: YourName
# consent_date: "2026-05-17"
# consent_method: signal_chat
# ── Embedding model comparison harness ────────────────────────────────────────
embed_bench:
# ollama_url: http://localhost:11434 # optional; falls back to cforch.ollama_url
# top_k: 5 # default hits per model per query

3
pytest.ini
View file

@ -3,6 +3,3 @@ testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
markers =
gpu: requires an idle GPU; excluded from default runs
slow: long-running test; excluded from default CI runs

29
scripts/benchmark_plans.py
View file

@ -23,16 +23,16 @@ Usage
python scripts/benchmark_plans.py --list-models
# Run all held-out prompts against a single model, print report
python scripts/benchmark_plans.py --model granite-4.1-8b
python scripts/benchmark_plans.py --model llama3.2-3b
# Compare two models side-by-side
python scripts/benchmark_plans.py --compare granite-4.1-8b deepseek-r1-7b-4bit
python scripts/benchmark_plans.py --compare llama3.2-3b mistral-7b
# Run with a custom API base (cf-text default: http://localhost:8080/v1)
python scripts/benchmark_plans.py --model granite-4.1-8b --api-base http://localhost:8080/v1
python scripts/benchmark_plans.py --model llama3.2-3b --api-base http://localhost:8080/v1
# Export detailed results JSON
python scripts/benchmark_plans.py --model granite-4.1-8b --output data/bench_results.json
python scripts/benchmark_plans.py --model llama3.2-3b --output data/bench_results.json
"""
from __future__ import annotations
@ -290,11 +290,6 @@ MODEL_REGISTRY: dict[str, dict[str, str]] = {
"model": "deepseek-r1-7b-4bit",
"description": "DeepSeek R1 7B distill, 4-bit (cf-orch catalog key)",
},
"deepseek-r1-0528-qwen3-8b-gguf": {
"api_base": CF_TEXT_BASE,
"model": "deepseek-r1-0528-qwen3-8b-gguf",
"description": "DeepSeek R1 0528 Qwen3 8B GGUF -- current reasoning model (4 nodes)",
},
"deepseek-coder-6.7b-4bit": {
"api_base": CF_TEXT_BASE,
"model": "deepseek-coder-6.7b-4bit",
@ -303,27 +298,17 @@ MODEL_REGISTRY: dict[str, dict[str, str]] = {
"granite-4.1-8b": {
"api_base": CF_TEXT_BASE,
"model": "granite-4.1-8b",
"description": "IBM Granite 4.1 8B, 4-bit -- safety-trained (cf-orch catalog key)",
},
"capybarahermes-2.5-mistral-7b-gguf": {
"api_base": CF_TEXT_BASE,
"model": "capybarahermes-2.5-mistral-7b-gguf",
"description": "CapybaraHermes 2.5 Mistral 7B GGUF -- conversational/creative (4 nodes)",
},
"darwin-9b-opus-gguf": {
"api_base": CF_TEXT_BASE,
"model": "darwin-9b-opus-gguf",
"description": "Darwin 9B Opus GGUF -- high-quality long-form writing (3 nodes)",
"description": "IBM Granite 4.1 8B, 4-bit (cf-orch catalog key)",
},
"qwen2.5-3b": {
"api_base": CF_TEXT_BASE,
"model": "qwen2.5-3b",
"description": "Qwen 2.5 3B Q4 GGUF (cf-orch catalog key)",
"description": "Qwen 2.5 3B Q4 GGUF (cf-orch catalog key, navi only)",
},
"qwen2.5-7b": {
"api_base": CF_TEXT_BASE,
"model": "qwen2.5-7b",
"description": "Qwen 2.5 7B Q4 GGUF (cf-orch catalog key)",
"description": "Qwen 2.5 7B Q4 GGUF (cf-orch catalog key, navi only)",
},
}

27
tests/test_cforch.py
View file

@ -176,14 +176,9 @@ def test_models_merges_installed_generators(client, config_dir, tmp_path):
# ── GET /run ───────────────────────────────────────────────────────────────────
def test_run_returns_409_when_already_running(client):
"""If a benchmark subprocess is actively running, GET /run returns 409."""
from unittest.mock import MagicMock
"""If _BENCH_RUNNING is True, GET /run returns 409."""
from app import cforch as cforch_module
mock_proc = MagicMock()
mock_proc.poll.return_value = None # process still alive
cforch_module._BENCH_RUNNING = True
cforch_module._bench_proc = mock_proc
r = client.get("/api/cforch/run")
assert r.status_code == 409
@ -217,15 +212,16 @@ def test_run_streams_progress_events(client, config_dir, tmp_path):
"python_bin": "/usr/bin/python3",
})
mock_stdout = MagicMock()
mock_stdout.readline.side_effect = ["Running task 1\n", "Running task 2\n", ""]
mock_proc = MagicMock()
mock_proc.stdout = mock_stdout
mock_proc.stdout = iter(["Running task 1\n", "Running task 2\n"])
mock_proc.returncode = 1 # non-zero so we don't need summary.json
mock_proc.wait = MagicMock()
with patch("app.cforch._subprocess.Popen", return_value=mock_proc), \
patch("app.cforch._select.select", return_value=([mock_stdout], [], [])):
def mock_wait():
pass
mock_proc.wait = mock_wait
with patch("app.cforch._subprocess.Popen", return_value=mock_proc):
r = client.get("/api/cforch/run")
assert r.status_code == 200
@ -258,15 +254,12 @@ def test_run_emits_result_on_success(client, config_dir, tmp_path):
"python_bin": "/usr/bin/python3",
})
mock_stdout = MagicMock()
mock_stdout.readline.side_effect = [""] # no output lines, immediate EOF
mock_proc = MagicMock()
mock_proc.stdout = mock_stdout
mock_proc.stdout = iter([])
mock_proc.returncode = 0
mock_proc.wait = MagicMock()
with patch("app.cforch._subprocess.Popen", return_value=mock_proc), \
patch("app.cforch._select.select", return_value=([mock_stdout], [], [])):
with patch("app.cforch._subprocess.Popen", return_value=mock_proc):
r = client.get("/api/cforch/run")
assert r.status_code == 200

234
tests/test_embed_bench.py
View file

@ -1,234 +0,0 @@
"""Tests for app/eval/embed_bench.py."""
from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from fastapi.testclient import TestClient
# ── Fixtures ──────────────────────────────────────────────────────────────────
@pytest.fixture(autouse=True)
def reset_embed_bench_globals(tmp_path):
"""Redirect config dir to tmp_path and reset running flag."""
from app.eval import embed_bench as mod
prev_config_dir = mod._CONFIG_DIR
prev_running = mod._RUN_ACTIVE
mod.set_config_dir(tmp_path)
mod._RUN_ACTIVE = False
yield tmp_path
mod.set_config_dir(prev_config_dir)
mod._RUN_ACTIVE = prev_running
@pytest.fixture
def client():
from app.api import app
return TestClient(app)
# ── cosine helper ──────────────────────────────────────────────────────────────
def test_cosine_identical():
from app.eval.embed_bench import _cosine
assert _cosine([1.0, 0.0], [1.0, 0.0]) == pytest.approx(1.0)
def test_cosine_orthogonal():
from app.eval.embed_bench import _cosine
assert _cosine([1.0, 0.0], [0.0, 1.0]) == pytest.approx(0.0)
def test_cosine_opposite():
from app.eval.embed_bench import _cosine
assert _cosine([1.0, 0.0], [-1.0, 0.0]) == pytest.approx(-1.0)
def test_cosine_zero_vector_returns_zero():
from app.eval.embed_bench import _cosine
assert _cosine([0.0, 0.0], [1.0, 0.0]) == pytest.approx(0.0)
# ── models endpoint ────────────────────────────────────────────────────────────
def test_models_returns_list_with_mock(client, tmp_path):
"""GET /api/embed-bench/models returns list from Ollama tags endpoint."""
import yaml
cfg = {"cforch": {"ollama_url": "http://localhost:11434"}}
(tmp_path / "label_tool.yaml").write_text(yaml.dump(cfg))
mock_resp = MagicMock()
mock_resp.status_code = 200
mock_resp.json.return_value = {
"models": [
{"name": "nomic-embed-text", "size": 274302480},
{"name": "mxbai-embed-large", "size": 669000000},
]
}
mock_resp.raise_for_status = MagicMock()
with patch("app.eval.embed_bench.httpx.get", return_value=mock_resp):
r = client.get("/api/embed-bench/models")
assert r.status_code == 200
data = r.json()
assert isinstance(data["models"], list)
assert any(m["name"] == "nomic-embed-text" for m in data["models"])
def test_models_returns_empty_on_ollama_error(client, tmp_path):
"""GET /api/embed-bench/models returns empty list if Ollama unreachable."""
import httpx
with patch("app.eval.embed_bench.httpx.get", side_effect=httpx.ConnectError("refused")):
r = client.get("/api/embed-bench/models")
assert r.status_code == 200
assert r.json()["models"] == []
# ── run endpoint ───────────────────────────────────────────────────────────────
def test_run_empty_corpus_returns_422(client):
r = client.post("/api/embed-bench/run", json={
"corpus": [], "queries": ["test"], "models": ["nomic-embed-text"], "top_k": 3
})
assert r.status_code == 422
def test_run_empty_queries_returns_422(client):
r = client.post("/api/embed-bench/run", json={
"corpus": ["chunk 1"], "queries": [], "models": ["nomic-embed-text"], "top_k": 3
})
assert r.status_code == 422
def test_run_empty_models_returns_422(client):
r = client.post("/api/embed-bench/run", json={
"corpus": ["chunk 1"], "queries": ["test"], "models": [], "top_k": 3
})
assert r.status_code == 422
def _fake_embed_response(texts: list[str]) -> MagicMock:
"""Build a mock httpx.post response returning unit vectors for each text."""
resp = MagicMock()
resp.raise_for_status = MagicMock()
resp.json.return_value = {
"data": [{"embedding": [1.0, 0.0, 0.0] if i % 2 == 0 else [0.0, 1.0, 0.0]}
for i, _ in enumerate(texts)]
}
return resp
def _collect_sse(raw: bytes) -> list[dict]:
"""Parse SSE stream bytes into a list of decoded event dicts."""
events = []
for line in raw.decode().splitlines():
if line.startswith("data: "):
events.append(json.loads(line[6:]))
return events
def test_run_single_model_returns_result_and_done(client, tmp_path):
import yaml
(tmp_path / "label_tool.yaml").write_text(yaml.dump({"cforch": {"ollama_url": "http://localhost:11434"}}))
with patch("app.eval.embed_bench.httpx.post", return_value=_fake_embed_response(["chunk 1", "chunk 2"])):
r = client.post("/api/embed-bench/run", json={
"corpus": ["chunk 1", "chunk 2"],
"queries": ["what is chunk one?"],
"models": ["nomic-embed-text"],
"top_k": 2,
})
assert r.status_code == 200
events = _collect_sse(r.content)
types = [e["type"] for e in events]
assert "result" in types
assert types[-1] == "done"
result_events = [e for e in events if e["type"] == "result"]
assert result_events[0]["model"] == "nomic-embed-text"
assert result_events[0]["query_idx"] == 0
assert len(result_events[0]["hits"]) <= 2
def test_run_two_models_returns_two_result_events_per_query(client, tmp_path):
import yaml
(tmp_path / "label_tool.yaml").write_text(yaml.dump({"cforch": {"ollama_url": "http://localhost:11434"}}))
with patch("app.eval.embed_bench.httpx.post", return_value=_fake_embed_response(["chunk A", "chunk B"])):
r = client.post("/api/embed-bench/run", json={
"corpus": ["chunk A", "chunk B"],
"queries": ["find it"],
"models": ["nomic-embed-text", "mxbai-embed-large"],
"top_k": 2,
})
events = _collect_sse(r.content)
result_events = [e for e in events if e["type"] == "result"]
models_seen = {e["model"] for e in result_events}
assert "nomic-embed-text" in models_seen
assert "mxbai-embed-large" in models_seen
# ── rate + export ──────────────────────────────────────────────────────────────
def test_rate_appends_jsonl_line(client, tmp_path):
r = client.post("/api/embed-bench/rate", json={
"query": "test query",
"model": "nomic-embed-text",
"chunk_text": "some text",
"chunk_idx": 2,
"rating": "relevant",
})
assert r.status_code == 200
assert r.json() == {"ok": True}
ratings_file = tmp_path / "embed_bench_ratings.jsonl"
assert ratings_file.exists()
line = json.loads(ratings_file.read_text().strip())
assert line["query"] == "test query"
assert line["rating"] == "relevant"
assert line["chunk_idx"] == 2
assert "timestamp" in line
def test_export_csv_two_rows(client, tmp_path):
for i in range(2):
client.post("/api/embed-bench/rate", json={
"query": f"q{i}", "model": "nomic-embed-text",
"chunk_text": f"chunk {i}", "chunk_idx": i, "rating": "relevant",
})
r = client.get("/api/embed-bench/export?format=csv")
assert r.status_code == 200
assert "text/csv" in r.headers["content-type"]
lines = r.text.strip().splitlines()
assert len(lines) == 3 # header + 2 rows
assert "query" in lines[0]
def test_export_json_two_entries(client, tmp_path):
for i in range(2):
client.post("/api/embed-bench/rate", json={
"query": f"q{i}", "model": "nomic-embed-text",
"chunk_text": f"chunk {i}", "chunk_idx": i, "rating": "not_relevant",
})
r = client.get("/api/embed-bench/export?format=json")
assert r.status_code == 200
data = r.json()
assert isinstance(data, list)
assert len(data) == 2
assert data[0]["rating"] == "not_relevant"
def test_export_empty_returns_csv_header_only(client):
r = client.get("/api/embed-bench/export?format=csv")
assert r.status_code == 200
lines = r.text.strip().splitlines()
assert len(lines) == 1 # header only
assert "query" in lines[0]

1
tests/test_finetune.py
View file

@ -321,7 +321,6 @@ def test_load_and_prepare_data_single_path_still_works(tmp_path):
# ---- Integration test ----
@pytest.mark.gpu
def test_integration_finetune_on_example_data(tmp_path):
"""Fine-tune deberta-small on example data for 1 epoch.

454
tests/test_log_corpus.py
View file

@ -1,454 +0,0 @@
"""Tests for app/data/log_corpus.py — corpus receiver and labeling endpoints."""
from __future__ import annotations
import json
import uuid
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from app.data import log_corpus as lc
VALID_TOKEN = str(uuid.uuid4())
VALID_HOST = "testnode.local"
@pytest.fixture(autouse=True)
def isolated_db(tmp_path, monkeypatch):
"""Each test gets its own fresh corpus DB and config dir."""
monkeypatch.setattr(lc, "_DATA_DIR", tmp_path)
monkeypatch.setattr(lc, "_DB_PATH", tmp_path / "corpus.db")
# Config dir pointing to a temp yaml with one test source
config_dir = tmp_path / "config"
config_dir.mkdir()
(config_dir / "label_tool.yaml").write_text(
f"corpus:\n sources:\n"
f" - token: \"{VALID_TOKEN}\"\n"
f" source_host: \"{VALID_HOST}\"\n"
f" owner: TestOwner\n"
f" consent_date: \"2026-05-11\"\n"
f" consent_method: signal_chat\n"
)
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
lc._init_db()
@pytest.fixture()
def client():
from fastapi import FastAPI
app = FastAPI()
app.include_router(lc.router, prefix="/api/corpus")
return TestClient(app)
def _batch(batch_type="raw_entries", entries=None, source_host=VALID_HOST):
return {
"batch_version": 1,
"batch_id": str(uuid.uuid4()),
"pushed_at": "2026-05-11T10:00:00Z",
"source_host": source_host,
"batch_type": batch_type,
"watermark_from": 0,
"watermark_to": 5,
"entries": entries or [
{
"entry_id": str(uuid.uuid4()),
"source_id": "sonarr",
"timestamp_iso": "2026-05-11T09:58:00Z",
"severity": "ERROR",
"text": "Connection refused to indexer",
"matched_patterns": [],
}
],
}
# ── Receive endpoint ───────────────────────────────────────────────────────────
def test_receive_missing_auth(client):
resp = client.post("/api/corpus/log-batch", json=_batch())
assert resp.status_code == 401
def test_receive_invalid_token(client):
resp = client.post(
"/api/corpus/log-batch",
json=_batch(),
headers={"Authorization": "Bearer bad-token"},
)
assert resp.status_code == 403
def test_receive_valid_batch(client):
resp = client.post(
"/api/corpus/log-batch",
json=_batch(),
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
assert resp.status_code == 200
data = resp.json()
assert data["received"] is True
assert data["entries_stored"] == 1
def test_receive_stores_source_host_from_token_not_payload(client):
"""source_host is always taken from the DB lookup, not the payload."""
payload = _batch(source_host="attacker-injected-host")
resp = client.post(
"/api/corpus/log-batch",
json=payload,
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
assert resp.status_code == 200
entries_resp = client.get("/api/corpus/entries")
entry = entries_resp.json()["entries"][0]
assert entry["source_host"] == VALID_HOST
def test_receive_skips_empty_text_entries(client):
payload = _batch(entries=[
{"entry_id": "e1", "source_id": "svc", "severity": "ERROR", "text": ""},
{"entry_id": "e2", "source_id": "svc", "severity": "ERROR", "text": " "},
{"entry_id": "e3", "source_id": "svc", "severity": "ERROR", "text": "real error"},
])
resp = client.post(
"/api/corpus/log-batch",
json=payload,
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
assert resp.json()["entries_stored"] == 1
def test_receive_incident_bundle(client):
payload = _batch(batch_type="incident_bundles", entries=[
{"id": "inc-1", "label": "plex crash", "issue_type": "plex",
"started_at": "2026-05-11T09:00:00", "ended_at": "2026-05-11T09:30:00",
"notes": "audio dropped", "created_at": "2026-05-11T09:35:00",
"severity": "high", "text": "plex crash"},
])
resp = client.post(
"/api/corpus/log-batch",
json=payload,
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
assert resp.status_code == 200
assert resp.json()["entries_stored"] == 1
# ── Labeling endpoints ─────────────────────────────────────────────────────────
def test_label_entry(client):
client.post(
"/api/corpus/log-batch",
json=_batch(),
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
resp = client.post(f"/api/corpus/entries/{entry_id}/label", json={
"failure_type": "software",
"plain_explanation": "Sonarr lost connection to its indexer — restart the service.",
"known_pattern": "y",
})
assert resp.status_code == 200
assert resp.json()["labeled"] is True
entries = client.get("/api/corpus/entries", params={"state": "labeled"}).json()["entries"]
assert len(entries) == 1
assert entries[0]["failure_type"] == "software"
def test_label_entry_invalid_failure_type(client):
client.post(
"/api/corpus/log-batch",
json=_batch(),
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
resp = client.post(f"/api/corpus/entries/{entry_id}/label", json={"failure_type": "aliens"})
assert resp.status_code == 422
def test_label_entry_missing_failure_type(client):
client.post(
"/api/corpus/log-batch",
json=_batch(),
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
resp = client.post(f"/api/corpus/entries/{entry_id}/label", json={})
assert resp.status_code == 422
def test_label_entry_not_found(client):
resp = client.post("/api/corpus/entries/nonexistent/label", json={"failure_type": "software"})
assert resp.status_code == 404
def test_skip_entry(client):
client.post(
"/api/corpus/log-batch",
json=_batch(),
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
resp = client.post(f"/api/corpus/entries/{entry_id}/skip")
assert resp.status_code == 200
unlabeled = client.get("/api/corpus/entries").json()["entries"]
assert len(unlabeled) == 0
# ── Stats ──────────────────────────────────────────────────────────────────────
def test_stats_empty(client):
stats = client.get("/api/corpus/stats").json()
assert stats["total_entries"] == 0
assert stats["batch_count"] == 0
def test_stats_after_receive(client):
client.post(
"/api/corpus/log-batch",
json=_batch(),
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
stats = client.get("/api/corpus/stats").json()
assert stats["total_entries"] == 1
assert stats["batch_count"] == 1
assert stats["by_label_state"].get("unlabeled", 0) == 1
# ── Export ─────────────────────────────────────────────────────────────────────
def test_export_excludes_unlabeled(client):
client.post(
"/api/corpus/log-batch",
json=_batch(),
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
resp = client.get("/api/corpus/export")
assert resp.status_code == 200
assert resp.text.strip() == ""
def test_export_includes_labeled(client):
client.post(
"/api/corpus/log-batch",
json=_batch(),
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
client.post(f"/api/corpus/entries/{entry_id}/label", json={
"failure_type": "software",
"plain_explanation": "Sonarr lost connection to indexer.",
})
resp = client.get("/api/corpus/export")
assert resp.status_code == 200
lines = [l for l in resp.text.strip().splitlines() if l]
assert len(lines) == 1
record = json.loads(lines[0])
assert record["output"] == "Sonarr lost connection to indexer."
assert record["metadata"]["failure_type"] == "software"
def test_export_excludes_pii_flagged(client):
client.post(
"/api/corpus/log-batch",
json=_batch(),
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
)
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
client.post(f"/api/corpus/entries/{entry_id}/label", json={
"failure_type": "software",
"plain_explanation": "Contains username — should not export.",
"pii_flagged": True,
})
resp = client.get("/api/corpus/export")
assert resp.text.strip() == ""
# ── Pipeline ingest endpoint ───────────────────────────────────────────────────
def _make_pipeline_file(directory: Path, name: str, lines: list[dict]) -> Path:
"""Write a JSONL pipeline log file to directory."""
p = directory / name
p.write_text("\n".join(json.dumps(l) for l in lines), encoding="utf-8")
return p
_PIPELINE_LINE = {
"ts": "2026-05-17T10:00:00Z",
"level": "INFO",
"logger": "scripts.pipeline.purple_carrot_scraper",
"msg": "Fetched recipe page",
"extra": {"url": "https://example.com/recipe/1", "status": 200},
}
def test_pipeline_ingest_returns_404_when_dir_not_configured(client, tmp_path):
"""No pipeline_ingest_dir in config — endpoint returns 404."""
resp = client.post("/api/corpus/pipeline-ingest")
assert resp.status_code == 404
def test_pipeline_ingest_empty_dir(client, tmp_path, monkeypatch):
"""Configured dir exists but is empty — returns zeros, no error."""
ingest_dir = tmp_path / "pipeline_logs"
ingest_dir.mkdir()
config_dir = tmp_path / "config"
config_dir.mkdir(exist_ok=True)
(config_dir / "label_tool.yaml").write_text(
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
)
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
resp = client.post("/api/corpus/pipeline-ingest")
assert resp.status_code == 200
data = resp.json()
assert data["ingested_files"] == 0
assert data["skipped_files"] == 0
assert data["entries_stored"] == 0
def test_pipeline_ingest_ingests_valid_file(client, tmp_path, monkeypatch):
"""Valid JSONL file is ingested; entries appear in corpus."""
ingest_dir = tmp_path / "pipeline_logs"
ingest_dir.mkdir()
_make_pipeline_file(ingest_dir, "scraper_20260517.jsonl", [
_PIPELINE_LINE,
{**_PIPELINE_LINE, "msg": "Saved 3 recipes", "level": "INFO"},
])
config_dir = tmp_path / "config"
config_dir.mkdir(exist_ok=True)
(config_dir / "label_tool.yaml").write_text(
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
)
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
resp = client.post("/api/corpus/pipeline-ingest")
assert resp.status_code == 200
data = resp.json()
assert data["ingested_files"] == 1
assert data["entries_stored"] == 2
entries = client.get("/api/corpus/entries", params={"limit": 10}).json()["entries"]
assert len(entries) == 2
assert all(e["source_host"] == "pipeline_scrape" for e in entries)
def test_pipeline_ingest_source_id_from_logger(client, tmp_path, monkeypatch):
"""source_id is populated from the 'logger' field of each log line."""
ingest_dir = tmp_path / "pipeline_logs"
ingest_dir.mkdir()
_make_pipeline_file(ingest_dir, "run_20260517.jsonl", [_PIPELINE_LINE])
config_dir = tmp_path / "config"
config_dir.mkdir(exist_ok=True)
(config_dir / "label_tool.yaml").write_text(
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
)
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
client.post("/api/corpus/pipeline-ingest")
entries = client.get("/api/corpus/entries", params={"limit": 10}).json()["entries"]
assert entries[0]["source_id"] == "scripts.pipeline.purple_carrot_scraper"
def test_pipeline_ingest_idempotent(client, tmp_path, monkeypatch):
"""Calling the endpoint twice does not re-ingest already-processed files."""
ingest_dir = tmp_path / "pipeline_logs"
ingest_dir.mkdir()
_make_pipeline_file(ingest_dir, "scraper_20260517.jsonl", [_PIPELINE_LINE])
config_dir = tmp_path / "config"
config_dir.mkdir(exist_ok=True)
(config_dir / "label_tool.yaml").write_text(
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
)
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
client.post("/api/corpus/pipeline-ingest")
resp2 = client.post("/api/corpus/pipeline-ingest")
data = resp2.json()
assert data["ingested_files"] == 0
assert data["skipped_files"] == 1
assert data["entries_stored"] == 0
entries = client.get("/api/corpus/entries", params={"limit": 10}).json()["entries"]
assert len(entries) == 1 # still just the one from the first ingest
def test_pipeline_ingest_skips_non_jsonl(client, tmp_path, monkeypatch):
"""Non-.jsonl files in the dir are silently ignored."""
ingest_dir = tmp_path / "pipeline_logs"
ingest_dir.mkdir()
(ingest_dir / "notes.txt").write_text("this is not a log file")
(ingest_dir / "run.csv").write_text("a,b,c\n1,2,3")
_make_pipeline_file(ingest_dir, "valid_20260517.jsonl", [_PIPELINE_LINE])
config_dir = tmp_path / "config"
config_dir.mkdir(exist_ok=True)
(config_dir / "label_tool.yaml").write_text(
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
)
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
resp = client.post("/api/corpus/pipeline-ingest")
assert resp.json()["ingested_files"] == 1
def test_pipeline_ingest_skips_malformed_lines(client, tmp_path, monkeypatch):
"""Lines that are not valid JSON are skipped; valid lines in the same file still land."""
ingest_dir = tmp_path / "pipeline_logs"
ingest_dir.mkdir()
p = ingest_dir / "mixed_20260517.jsonl"
p.write_text(
json.dumps(_PIPELINE_LINE) + "\n"
"this is not json\n"
+ json.dumps({**_PIPELINE_LINE, "msg": "another valid line"}),
encoding="utf-8",
)
config_dir = tmp_path / "config"
config_dir.mkdir(exist_ok=True)
(config_dir / "label_tool.yaml").write_text(
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
)
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
resp = client.post("/api/corpus/pipeline-ingest")
assert resp.status_code == 200
assert resp.json()["entries_stored"] == 2 # 2 valid lines, 1 skipped
def test_pipeline_ingest_new_file_after_first_run(client, tmp_path, monkeypatch):
"""A new file added after the first ingest is picked up on the next call."""
ingest_dir = tmp_path / "pipeline_logs"
ingest_dir.mkdir()
_make_pipeline_file(ingest_dir, "run_a.jsonl", [_PIPELINE_LINE])
config_dir = tmp_path / "config"
config_dir.mkdir(exist_ok=True)
(config_dir / "label_tool.yaml").write_text(
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
)
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
client.post("/api/corpus/pipeline-ingest") # ingest run_a.jsonl
_make_pipeline_file(ingest_dir, "run_b.jsonl", [
{**_PIPELINE_LINE, "msg": "Second run line"},
])
resp2 = client.post("/api/corpus/pipeline-ingest")
data = resp2.json()
assert data["ingested_files"] == 1
assert data["skipped_files"] == 1
assert data["entries_stored"] == 1

3
tests/test_models.py
View file

@ -17,7 +17,6 @@ def reset_models_globals(tmp_path):
from app import models as models_module
prev_models = models_module._MODELS_DIR
prev_cf_text = models_module._CF_TEXT_MODELS_DIR
prev_queue = models_module._QUEUE_DIR
prev_progress = dict(models_module._download_progress)
@ -27,14 +26,12 @@ def reset_models_globals(tmp_path):
queue_dir.mkdir()
models_module.set_models_dir(models_dir)
models_module.set_cf_text_models_dir(tmp_path / "cf-text-models")
models_module.set_queue_dir(queue_dir)
models_module._download_progress = {}
yield
models_module.set_models_dir(prev_models)
models_module.set_cf_text_models_dir(prev_cf_text)
models_module.set_queue_dir(prev_queue)
models_module._download_progress = prev_progress

108
tests/test_nodes.py
View file

@ -55,11 +55,11 @@ def _fake_nodes_response(nodes_json: list, services_json: list | None = None):
"""Build side_effect list for two httpx.get calls: nodes then services."""
mock_nodes = MagicMock()
mock_nodes.raise_for_status = MagicMock()
mock_nodes.json.return_value = {"nodes": nodes_json}
mock_nodes.json.return_value = nodes_json
mock_services = MagicMock()
mock_services.raise_for_status = MagicMock()
mock_services.json.return_value = {"services": services_json or []}
mock_services.json.return_value = services_json or []
return [mock_nodes, mock_services]
@ -469,107 +469,3 @@ def test_delete_ollama_model_404_when_not_found(client, tmp_path):
r = client.delete("/api/nodes-mgmt/nodes/heimdall/models/ollama/missing-model")
assert r.status_code == 404
# ── Deploy model endpoint ──────────────────────────────────────────────────────
_DEPLOY_PROFILE = {
"services": {
"cf-text": {
"max_mb": 20000,
"min_compute_cap": 7.0,
"model_base_path": "/devl/Assets/LLM/cf-text/models",
"catalog": {},
},
},
"nodes": {
"heimdall": {
"gpus": [],
"agent_url": "http://10.1.10.71:7701",
}
}
}
def test_deploy_model_adds_catalog_entry(client, tmp_path):
"""Deploy endpoint should add the model to the service catalog."""
profiles_dir = tmp_path / "profiles"
_write_config(tmp_path, {
"coordinator_url": "http://fake-coord:7700",
"profiles_dir": str(profiles_dir),
})
_write_profile(profiles_dir, "heimdall", _DEPLOY_PROFILE)
mock_reload = MagicMock()
mock_reload.status_code = 200
with patch("httpx.post", return_value=mock_reload):
r = client.post(
"/api/nodes-mgmt/nodes/heimdall/models/deploy",
json={
"model_id": "fdtn-ai--Foundation-Sec-8B-Q4",
"service_type": "cf-text",
"vram_mb": 5180,
"hf_repo": "fdtn-ai/Foundation-Sec-8B-Q4_K_M-GGUF",
},
)
assert r.status_code == 200
data = r.json()
assert data["ok"] is True
assert data["reloaded"] is True
assert "fdtn-ai--Foundation-Sec-8B-Q4_K_M-GGUF" in data["path"]
saved = yaml.safe_load((profiles_dir / "heimdall.yaml").read_text())
catalog = saved["services"]["cf-text"]["catalog"]
assert "fdtn-ai--Foundation-Sec-8B-Q4" in catalog
entry = catalog["fdtn-ai--Foundation-Sec-8B-Q4"]
assert entry["vram_mb"] == 5180
assert entry["path"].endswith("fdtn-ai--Foundation-Sec-8B-Q4_K_M-GGUF")
def test_deploy_model_explicit_path_overrides_base(client, tmp_path):
"""An explicit path in the request body takes precedence over model_base_path."""
profiles_dir = tmp_path / "profiles"
_write_config(tmp_path, {
"coordinator_url": "http://fake-coord:7700",
"profiles_dir": str(profiles_dir),
})
_write_profile(profiles_dir, "heimdall", _DEPLOY_PROFILE)
with patch("httpx.post", return_value=MagicMock(status_code=200)):
r = client.post(
"/api/nodes-mgmt/nodes/heimdall/models/deploy",
json={
"model_id": "my-model",
"service_type": "cf-text",
"vram_mb": 8000,
"path": "/custom/path/to/model",
},
)
assert r.status_code == 200
assert r.json()["path"] == "/custom/path/to/model"
def test_deploy_model_unknown_service_returns_422(client, tmp_path):
"""Service type not in profile → 422."""
profiles_dir = tmp_path / "profiles"
_write_config(tmp_path, {"profiles_dir": str(profiles_dir)})
_write_profile(profiles_dir, "heimdall", _DEPLOY_PROFILE)
r = client.post(
"/api/nodes-mgmt/nodes/heimdall/models/deploy",
json={"model_id": "x", "service_type": "vllm", "vram_mb": 8000},
)
assert r.status_code == 422
assert "vllm" in r.json()["detail"]
def test_deploy_model_missing_profile_returns_404(client, tmp_path):
_write_config(tmp_path, {"profiles_dir": str(tmp_path / "profiles")})
r = client.post(
"/api/nodes-mgmt/nodes/nonexistent/models/deploy",
json={"model_id": "x", "service_type": "cf-text", "vram_mb": 100},
)
assert r.status_code == 404

227
tests/test_recipe_scan.py
View file

@ -1,227 +0,0 @@
"""Tests for app/data/recipe_scan.py — recipe scan labeling endpoints."""
from __future__ import annotations
import json
import uuid
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from app.data import recipe_scan as rs
EXTRACTED = {"title": "Shepherd's Pie", "ingredients": ["lamb", "potato"], "steps": ["brown meat", "mash potato"]}
GROUND_TRUTH = {"title": "Shepherd's Pie", "ingredients": ["ground lamb", "mashed potato", "peas"], "steps": ["brown meat", "add veg", "mash potato", "bake"]}
@pytest.fixture(autouse=True)
def isolated_db(tmp_path, monkeypatch):
monkeypatch.setattr(rs, "_DB_PATH", tmp_path / "recipe_scan.db")
rs._init_db()
@pytest.fixture()
def client():
from fastapi import FastAPI
app = FastAPI()
app.include_router(rs.router, prefix="/api/recipe-scan")
return TestClient(app)
def _item(**kwargs) -> dict:
return {
"id": str(uuid.uuid4()),
"image_path": "/Library/Assets/kiwi/scans/pc_test.jpg",
"modality": kwargs.get("modality", "scanner"),
"source": kwargs.get("source", "purple_carrot"),
"extracted": kwargs.get("extracted", EXTRACTED),
"ground_truth": kwargs.get("ground_truth", GROUND_TRUTH),
}
def _import(client, items: list[dict]) -> None:
resp = client.post("/api/recipe-scan/import", json={"items": items})
assert resp.status_code == 200
# ── Import ─────────────────────────────────────────────────────────────────────
def test_import_stores_items(client):
_import(client, [_item()])
stats = client.get("/api/recipe-scan/stats").json()
assert stats["total"] == 1
assert stats["by_status"]["pending"] == 1
def test_import_rejects_unknown_modality(client):
bad = _item()
bad["modality"] = "telepathy"
resp = client.post("/api/recipe-scan/import", json={"items": [bad]})
assert resp.status_code == 422
def test_import_is_idempotent(client):
item = _item()
_import(client, [item])
_import(client, [item]) # same id — should not duplicate
stats = client.get("/api/recipe-scan/stats").json()
assert stats["total"] == 1
def test_import_multiple_items(client):
_import(client, [_item(), _item(), _item()])
assert client.get("/api/recipe-scan/stats").json()["total"] == 3
# ── Next ───────────────────────────────────────────────────────────────────────
def test_next_returns_404_when_queue_empty(client):
resp = client.get("/api/recipe-scan/next")
assert resp.status_code == 404
def test_next_returns_pending_item(client):
item = _item()
_import(client, [item])
resp = client.get("/api/recipe-scan/next")
assert resp.status_code == 200
data = resp.json()
assert data["id"] == item["id"]
assert data["status"] == "pending"
assert "extracted" in data
assert "ground_truth" in data
def test_next_skips_non_pending(client):
item = _item()
_import(client, [item])
client.post(f"/api/recipe-scan/items/{item['id']}/reject")
resp = client.get("/api/recipe-scan/next")
assert resp.status_code == 404
# ── Approve ────────────────────────────────────────────────────────────────────
def test_approve_marks_item_approved(client):
item = _item()
_import(client, [item])
resp = client.post(f"/api/recipe-scan/items/{item['id']}/approve")
assert resp.status_code == 200
assert resp.json()["status"] == "approved"
stats = client.get("/api/recipe-scan/stats").json()
assert stats["by_status"]["approved"] == 1
def test_approve_returns_404_for_unknown_id(client):
resp = client.post("/api/recipe-scan/items/no-such-id/approve")
assert resp.status_code == 404
# ── Edit ───────────────────────────────────────────────────────────────────────
def test_edit_stores_corrected_json(client):
item = _item()
_import(client, [item])
corrected = {**GROUND_TRUTH, "servings": 4}
resp = client.post(
f"/api/recipe-scan/items/{item['id']}/edit",
json={"corrected": corrected},
)
assert resp.status_code == 200
assert resp.json()["status"] == "edited"
stats = client.get("/api/recipe-scan/stats").json()
assert stats["by_status"]["edited"] == 1
def test_edit_requires_corrected_field(client):
item = _item()
_import(client, [item])
resp = client.post(f"/api/recipe-scan/items/{item['id']}/edit", json={})
assert resp.status_code == 422
# ── Reject ─────────────────────────────────────────────────────────────────────
def test_reject_marks_item_rejected(client):
item = _item()
_import(client, [item])
resp = client.post(
f"/api/recipe-scan/items/{item['id']}/reject",
json={"reason": "OCR completely unreadable"},
)
assert resp.status_code == 200
assert resp.json()["status"] == "rejected"
def test_reject_without_reason_is_valid(client):
item = _item()
_import(client, [item])
resp = client.post(f"/api/recipe-scan/items/{item['id']}/reject")
assert resp.status_code == 200
# ── Export ─────────────────────────────────────────────────────────────────────
def test_export_empty_when_nothing_approved(client):
item = _item()
_import(client, [item])
resp = client.get("/api/recipe-scan/export")
assert resp.status_code == 200
assert resp.text.strip() == ""
def test_export_includes_approved_item(client):
item = _item()
_import(client, [item])
client.post(f"/api/recipe-scan/items/{item['id']}/approve")
resp = client.get("/api/recipe-scan/export")
lines = [l for l in resp.text.strip().splitlines() if l]
assert len(lines) == 1
pair = json.loads(lines[0])
assert pair["id"] == item["id"]
assert pair["modality"] == "scanner"
assert "messages" in pair
assert len(pair["messages"]) == 2
assert pair["messages"][0]["role"] == "user"
assert pair["messages"][1]["role"] == "assistant"
def test_export_includes_edited_item_with_correction(client):
item = _item()
_import(client, [item])
corrected = {**GROUND_TRUTH, "servings": 4}
client.post(
f"/api/recipe-scan/items/{item['id']}/edit",
json={"corrected": corrected},
)
resp = client.get("/api/recipe-scan/export")
lines = [l for l in resp.text.strip().splitlines() if l]
pair = json.loads(lines[0])
assistant_content = json.loads(pair["messages"][1]["content"])
assert assistant_content["servings"] == 4
def test_export_excludes_rejected_items(client):
item = _item()
_import(client, [item])
client.post(f"/api/recipe-scan/items/{item['id']}/reject")
resp = client.get("/api/recipe-scan/export")
assert resp.text.strip() == ""
# ── Stats ──────────────────────────────────────────────────────────────────────
def test_stats_counts_all_statuses(client):
items = [_item(), _item(), _item(), _item()]
_import(client, items)
client.post(f"/api/recipe-scan/items/{items[0]['id']}/approve")
client.post(f"/api/recipe-scan/items/{items[1]['id']}/edit", json={"corrected": GROUND_TRUTH})
client.post(f"/api/recipe-scan/items/{items[2]['id']}/reject")
stats = client.get("/api/recipe-scan/stats").json()
assert stats["total"] == 4
assert stats["by_status"]["pending"] == 1
assert stats["by_status"]["approved"] == 1
assert stats["by_status"]["edited"] == 1
assert stats["by_status"]["rejected"] == 1
assert stats["export_ready"] == 2 # approved + edited

42
web/package-lock.json generated
View file

@ -2676,9 +2676,9 @@
}
},
"node_modules/brace-expansion": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.0.tgz",
"integrity": "sha512-TN1kCZAgdgweJhWWpgKYrQaMNHcDULHkWwQIspdtjV4Y5aurRdZpjAqn6yX3FPqTA9ngHCc4hJxMAMgGfve85w==",
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz",
"integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==",
"dev": true,
"license": "MIT",
"dependencies": {
@ -2890,9 +2890,9 @@
"license": "MIT"
},
"node_modules/defu": {
"version": "6.1.7",
"resolved": "https://registry.npmjs.org/defu/-/defu-6.1.7.tgz",
"integrity": "sha512-7z22QmUWiQ/2d0KkdYmANbRUVABpZ9SNYyH5vx6PZ+nE5bcC0l7uFvEfHlyld/HcGBFTL536ClDt3DEcSlEJAQ==",
"version": "6.1.4",
"resolved": "https://registry.npmjs.org/defu/-/defu-6.1.4.tgz",
"integrity": "sha512-mEQCMmwJu317oSz8CwdIOdwf3xMif1ttiM8LTufzc3g6kR+9Pe236twL8j3IYT1F7GfRgGcW6MWxzZjLIkuHIg==",
"dev": true,
"license": "MIT"
},
@ -3725,9 +3725,9 @@
"license": "ISC"
},
"node_modules/picomatch": {
"version": "4.0.4",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.4.tgz",
"integrity": "sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A==",
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"license": "MIT",
"engines": {
"node": ">=12"
@ -3769,9 +3769,9 @@
}
},
"node_modules/postcss": {
"version": "8.5.14",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.14.tgz",
"integrity": "sha512-SoSL4+OSEtR99LHFZQiJLkT59C5B1amGO1NzTwj7TT1qCUgUO6hxOvzkOYxD+vMrXBM3XJIKzokoERdqQq/Zmg==",
"version": "8.5.8",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.8.tgz",
"integrity": "sha512-OW/rX8O/jXnm82Ey1k44pObPtdblfiuWnrd8X7GJ7emImCOstunGbXUpp7HdBrFQX6rJzn3sPT397Wp5aCwCHg==",
"funding": [
{
"type": "opencollective",
@ -4325,9 +4325,9 @@
}
},
"node_modules/undici": {
"version": "7.25.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.25.0.tgz",
"integrity": "sha512-xXnp4kTyor2Zq+J1FfPI6Eq3ew5h6Vl0F/8d9XU5zZQf1tX9s2Su1/3PiMmUANFULpmksxkClamIZcaUqryHsQ==",
"version": "7.22.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.22.0.tgz",
"integrity": "sha512-RqslV2Us5BrllB+JeiZnK4peryVTndy9Dnqq62S3yYRRTj0tFQCwEniUy2167skdGOy3vqRzEvl1Dm4sV2ReDg==",
"dev": true,
"license": "MIT",
"engines": {
@ -4422,9 +4422,9 @@
}
},
"node_modules/vite": {
"version": "7.3.2",
"resolved": "https://registry.npmjs.org/vite/-/vite-7.3.2.tgz",
"integrity": "sha512-Bby3NOsna2jsjfLVOHKes8sGwgl4TT0E6vvpYgnAYDIF/tie7MRaFthmKuHx1NSXjiTueXH3do80FMQgvEktRg==",
"version": "7.3.1",
"resolved": "https://registry.npmjs.org/vite/-/vite-7.3.1.tgz",
"integrity": "sha512-w+N7Hifpc3gRjZ63vYBXA56dvvRlNWRczTdmCBBa+CotUzAPf5b7YMdMR/8CQoeYE5LX3W4wj6RYTgonm1b9DA==",
"dev": true,
"license": "MIT",
"dependencies": {
@ -4921,9 +4921,9 @@
"license": "MIT"
},
"node_modules/yaml": {
"version": "2.8.4",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.8.4.tgz",
"integrity": "sha512-ml/JPOj9fOQK8RNnWojA67GbZ0ApXAUlN2UQclwv2eVgTgn7O9gg9o7paZWKMp4g0H3nTLtS9LVzhkpOFIKzog==",
"version": "2.8.2",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.8.2.tgz",
"integrity": "sha512-mplynKqc1C2hTVYxd0PU2xQAc22TI1vShAYGksCCfxbn/dFwnHTNi1bvYsBTkhdUNtGIf5xNOg938rrSSYvS9A==",
"license": "ISC",
"bin": {
"yaml": "bin.mjs"

2
web/src/components/AppSidebar.vue
View file

@ -220,13 +220,11 @@ const dataItems: NavItem[] = [
{ path: '/data/fetch', icon: '📬', label: 'Fetch' },
{ path: '/data/corrections', icon: '✏️', label: 'Corrections' },
{ path: '/data/imitate', icon: '🪞', label: 'Imitate' },
{ path: '/data/recipe-scan', icon: '📷', label: 'Recipe Scan' },
]
const evalItems: NavItem[] = [
{ path: '/eval/benchmark', icon: '📊', label: 'Benchmark' },
{ path: '/eval/compare', icon: '🔍', label: 'Compare' },
{ path: '/eval/embed-compare', icon: '🧮', label: 'Embed Compare' },
]
const trainItems: NavItem[] = [

170
web/src/components/nodes/CatalogEntryFormModal.vue
View file

@ -1,170 +0,0 @@
<script setup lang="ts">
import { ref, watch } from 'vue'
import type { CatalogEntryFull } from '../../types/nodes'
const props = defineProps<{
svcName: string
modelName?: string
entry?: CatalogEntryFull
}>()
const emit = defineEmits<{
save: [svcName: string, modelName: string, entry: CatalogEntryFull]
cancel: []
}>()
const name = ref(props.modelName ?? '')
const path = ref(props.entry?.path ?? '')
const vramMb = ref(props.entry?.vram_mb ?? 0)
const description = ref(props.entry?.description ?? '')
const multiGpu = ref(props.entry?.multi_gpu ?? false)
const envPairs = ref<{ k: string; v: string }[]>(
Object.entries(props.entry?.env ?? {}).map(([k, v]) => ({ k, v }))
)
const formError = ref('')
watch(() => props.entry, (e) => {
name.value = props.modelName ?? ''
path.value = e?.path ?? ''
vramMb.value = e?.vram_mb ?? 0
description.value = e?.description ?? ''
multiGpu.value = e?.multi_gpu ?? false
envPairs.value = Object.entries(e?.env ?? {}).map(([k, v]) => ({ k, v }))
})
function addEnvPair() {
envPairs.value = [...envPairs.value, { k: '', v: '' }]
}
function removeEnvPair(i: number) {
envPairs.value = envPairs.value.filter((_, idx) => idx !== i)
}
function submit() {
formError.value = ''
if (!name.value.trim()) { formError.value = 'Model name is required.'; return }
if (!path.value.trim()) { formError.value = 'Path is required.'; return }
if (!vramMb.value || vramMb.value < 0) { formError.value = 'vram_mb must be a positive number.'; return }
const envObj: Record<string, string> = {}
for (const { k, v } of envPairs.value) {
if (k.trim()) envObj[k.trim()] = v
}
const entry: CatalogEntryFull = { path: path.value.trim(), vram_mb: vramMb.value }
if (description.value.trim()) entry.description = description.value.trim()
if (multiGpu.value) entry.multi_gpu = true
if (Object.keys(envObj).length) entry.env = envObj
emit('save', props.svcName, name.value.trim(), entry)
}
</script>
<template>
<div class="modal-backdrop" role="dialog" aria-modal="true" :aria-label="`${modelName ? 'Edit' : 'Add'} catalog entry`">
<div class="modal-box">
<h3 class="modal-title">{{ modelName ? 'Edit' : 'Add' }} Catalog Entry — {{ svcName }}</h3>
<div class="field-row">
<label class="field-label" for="ce-name">Model name</label>
<input id="ce-name" v-model="name" class="field-input" :readonly="!!modelName" placeholder="deepseek-r1-7b" />
</div>
<div class="field-row">
<label class="field-label" for="ce-path">Path</label>
<input id="ce-path" v-model="path" class="field-input" placeholder="/devl/Assets/LLM/cf-text/models/..." />
</div>
<div class="field-row">
<label class="field-label" for="ce-vram">VRAM (MB)</label>
<input id="ce-vram" v-model.number="vramMb" type="number" min="0" class="field-input field-input--sm" />
</div>
<div class="field-row">
<label class="field-label" for="ce-desc">Description</label>
<input id="ce-desc" v-model="description" class="field-input" placeholder="Short description" />
</div>
<div class="field-row field-row--check">
<input id="ce-mgpu" v-model="multiGpu" type="checkbox" />
<label for="ce-mgpu">Multi-GPU span</label>
</div>
<div class="env-section">
<div class="env-header">
<span class="field-label">Env vars</span>
<button type="button" class="btn-link" @click="addEnvPair">+ Add</button>
</div>
<div v-for="(pair, i) in envPairs" :key="i" class="env-row">
<input v-model="pair.k" class="field-input field-input--sm" placeholder="CF_TEXT_4BIT" />
<span>=</span>
<input v-model="pair.v" class="field-input field-input--sm" placeholder="1" />
<button type="button" class="btn-icon" @click="removeEnvPair(i)" aria-label="Remove">✕</button>
</div>
</div>
<div v-if="formError" class="form-error" role="alert">{{ formError }}</div>
<div class="modal-actions">
<button class="btn-secondary" @click="emit('cancel')">Cancel</button>
<button class="btn-primary" @click="submit">Save</button>
</div>
</div>
</div>
</template>
<style scoped>
.modal-backdrop {
position: fixed; inset: 0;
background: rgba(0,0,0,0.5);
display: flex; align-items: center; justify-content: center;
z-index: 200;
}
.modal-box {
background: var(--color-surface-raised);
border: 1px solid var(--color-border);
border-radius: 8px;
padding: 1.5rem;
width: 100%; max-width: 500px;
max-height: 90vh; overflow-y: auto;
display: flex; flex-direction: column; gap: 0.75rem;
color: var(--color-text);
}
.modal-title { margin: 0 0 0.25rem; font-size: 1rem; font-weight: 600; color: var(--color-text); }
.field-row { display: flex; align-items: center; gap: 0.5rem; }
.field-row--check { gap: 0.4rem; color: var(--color-text); }
.field-label { min-width: 8rem; font-size: 0.85rem; color: var(--color-text-muted); }
.field-input {
flex: 1;
background: var(--color-surface-alt);
border: 1px solid var(--color-border);
border-radius: 4px;
padding: 0.3rem 0.5rem;
color: var(--color-text);
font-size: 0.85rem;
}
.field-input--sm { flex: 0 0 8rem; }
.env-section { display: flex; flex-direction: column; gap: 0.35rem; }
.env-header { display: flex; align-items: center; justify-content: space-between; }
.env-row { display: flex; align-items: center; gap: 0.4rem; }
.btn-link { background: none; border: none; color: var(--app-primary); cursor: pointer; font-size: 0.8rem; padding: 0; }
.btn-link:hover { color: var(--app-primary-hover); }
.btn-icon { background: none; border: none; color: var(--color-text-muted); cursor: pointer; padding: 0 0.2rem; font-size: 0.85rem; }
.btn-icon:hover { color: var(--color-error); }
.form-error { color: var(--color-error); font-size: 0.8rem; }
.modal-actions { display: flex; justify-content: flex-end; gap: 0.5rem; margin-top: 0.25rem; }
.btn-primary {
background: var(--app-primary);
color: var(--color-text-inverse);
border: none;
border-radius: 4px;
padding: 0.4rem 1rem;
cursor: pointer;
font-size: 0.875rem;
}
.btn-primary:hover { background: var(--app-primary-hover); }
.btn-secondary {
background: transparent;
border: 1px solid var(--color-border);
color: var(--color-text);
border-radius: 4px;
padding: 0.4rem 0.75rem;
cursor: pointer;
font-size: 0.875rem;
}
.btn-secondary:hover { background: var(--color-surface-alt); }
</style>

14
web/src/components/nodes/GpuRow.vue
View file

@ -106,24 +106,24 @@ async function toggleService(svcName: string) {
.gpu-row {
padding: 0.5rem 0.75rem;
border-radius: 4px;
background: var(--color-surface-alt);
background: var(--bg-secondary, #111);
display: flex;
flex-direction: column;
gap: 0.4rem;
}
.gpu-info { display: flex; gap: 0.75rem; align-items: center; flex-wrap: wrap; font-size: 0.875rem; }
.gpu-label { font-weight: 500; color: var(--color-text); }
.gpu-meta { color: var(--color-text-muted); font-size: 0.8rem; }
.gpu-label { font-weight: 500; }
.gpu-meta { color: var(--text-secondary, #888); font-size: 0.8rem; }
.vram-wrap { display: flex; align-items: center; gap: 0.5rem; }
.vram-bar {
flex: 1;
height: 8px;
background: var(--color-border);
background: var(--bg-bar, #2a2a2a);
border-radius: 4px;
overflow: hidden;
}
.vram-fill { height: 100%; background: var(--app-primary); transition: width 0.3s; }
.vram-text { font-size: 0.75rem; color: var(--color-text-muted); white-space: nowrap; }
.vram-fill { height: 100%; background: var(--color-primary, #4080ff); transition: width 0.3s; }
.vram-text { font-size: 0.75rem; color: var(--text-secondary, #888); white-space: nowrap; }
.services-row { display: flex; flex-wrap: wrap; gap: 0.4rem; }
.save-msg { color: var(--color-warning); font-size: 0.8rem; }
.save-msg { color: var(--color-warning, #ed8936); font-size: 0.8rem; }
</style>

24
web/src/components/nodes/HfNodeModelPanel.vue
View file

@ -99,21 +99,19 @@ onUnmounted(() => { fetchAbort?.abort() })
.hf-panel {
margin-top: 0.75rem;
padding: 0.75rem;
border: 1px solid var(--color-border);
border: 1px solid var(--border, #333);
border-radius: 6px;
color: var(--color-text);
}
.panel-title { margin: 0 0 0.5rem; font-size: 0.9rem; color: var(--color-text); }
.hf-hint { font-size: 0.8rem; color: var(--color-text-muted); margin: 0 0 0.75rem; }
.hf-link { color: var(--app-primary); }
.hf-link:hover { color: var(--app-primary-hover); }
.panel-title { margin: 0 0 0.5rem; font-size: 0.9rem; }
.hf-hint { font-size: 0.8rem; color: var(--text-secondary, #888); margin: 0 0 0.75rem; }
.hf-link { color: var(--color-primary, #4080ff); }
.svc-section { margin-bottom: 0.75rem; }
.svc-name {
margin: 0 0 0.25rem;
font-size: 0.75rem;
text-transform: uppercase;
letter-spacing: 0.05em;
color: var(--color-text-muted);
color: var(--text-secondary, #888);
}
.catalog-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.2rem; }
.catalog-item {
@ -121,14 +119,14 @@ onUnmounted(() => { fetchAbort?.abort() })
align-items: center;
gap: 0.5rem;
padding: 0.25rem 0.5rem;
background: var(--color-surface-alt);
background: var(--bg-secondary, #111);
border-radius: 4px;
font-size: 0.8rem;
}
.catalog-model { font-family: var(--font-mono, monospace); flex: 1; }
.catalog-vram { color: var(--color-text-muted); white-space: nowrap; }
.catalog-desc { color: var(--color-text-muted); font-size: 0.75rem; flex: 2; }
.catalog-empty, .panel-empty { color: var(--color-text-muted); font-size: 0.875rem; }
.catalog-model { font-family: monospace; flex: 1; }
.catalog-vram { color: var(--text-secondary, #888); white-space: nowrap; }
.catalog-desc { color: var(--text-secondary, #888); font-size: 0.75rem; flex: 2; }
.catalog-empty, .panel-empty { color: var(--text-secondary, #888); font-size: 0.875rem; }
.sr-announce { min-height: 1.2em; }
.panel-error { color: var(--color-error); font-size: 0.8rem; }
.panel-error { color: var(--color-error, #fc8181); font-size: 0.8rem; }
</style>

90
web/src/components/nodes/NodeCard.vue
View file

@ -2,43 +2,14 @@
import { ref } from 'vue'
import GpuRow from './GpuRow.vue'
import OllamaModelPanel from './OllamaModelPanel.vue'
import ProfileEditorPanel from './ProfileEditorPanel.vue'
import type { NodeSummary, FullProfile } from '../../types/nodes'
import HfNodeModelPanel from './HfNodeModelPanel.vue'
import type { NodeSummary } from '../../types/nodes'
const props = defineProps<{ node: NodeSummary }>()
const emit = defineEmits<{ updated: [] }>()
const showOllama = ref(false)
const showEditor = ref(false)
const loadedProfile = ref<FullProfile | null>(null)
const profileLoading = ref(false)
const profileError = ref('')
async function openEditor() {
if (showEditor.value) { showEditor.value = false; return }
profileLoading.value = true
profileError.value = ''
try {
const r = await fetch(`/api/nodes-mgmt/nodes/${props.node.node_id}/profile`)
if (r.status === 404) {
loadedProfile.value = null
} else if (!r.ok) {
throw new Error(`HTTP ${r.status}`)
} else {
loadedProfile.value = await r.json() as FullProfile
}
showEditor.value = true
} catch (e) {
profileError.value = e instanceof Error ? e.message : 'Failed to load profile'
} finally {
profileLoading.value = false
}
}
function onProfileSaved() {
showEditor.value = false
emit('updated')
}
const showHf = ref(false)
</script>
<template>
@ -54,20 +25,12 @@ function onProfileSaved() {
<h2 class="node-name">{{ node.node_id }}</h2>
<span class="node-agent">{{ node.agent_url }}</span>
</div>
<div class="node-actions">
<button
v-if="node.profile_loaded"
class="btn-secondary btn-sm"
@click="showOllama = !showOllama"
>
<div v-if="node.profile_loaded" class="node-actions">
<button class="btn-secondary btn-sm" @click="showOllama = !showOllama">
{{ showOllama ? 'Hide Ollama' : 'Ollama' }}
</button>
<button
class="btn-secondary btn-sm"
:disabled="profileLoading"
@click="openEditor"
>
{{ profileLoading ? 'Loading…' : node.profile_loaded ? (showEditor ? 'Close Editor' : 'Edit Profile') : 'Create Profile' }}
<button class="btn-secondary btn-sm" @click="showHf = !showHf">
{{ showHf ? 'Hide Catalog' : 'Catalog' }}
</button>
</div>
</header>
@ -89,24 +52,16 @@ function onProfileSaved() {
</div>
<OllamaModelPanel v-if="showOllama" :node-id="node.node_id" />
<div v-if="profileError" class="profile-load-error" role="alert">{{ profileError }}</div>
<ProfileEditorPanel
v-if="showEditor"
:node-id="node.node_id"
:initial-profile="loadedProfile"
@saved="onProfileSaved"
@close="showEditor = false"
/>
<HfNodeModelPanel v-if="showHf" :node-id="node.node_id" />
</section>
</template>
<style scoped>
.node-card {
border: 1px solid var(--color-border);
border: 1px solid var(--border, #333);
border-radius: 8px;
padding: 1rem;
background: var(--color-surface-raised);
color: var(--color-text);
background: var(--bg-card, #1a1a1a);
}
.node-card.offline { opacity: 0.65; }
.node-card-header {
@ -117,32 +72,19 @@ function onProfileSaved() {
margin-bottom: 0.75rem;
}
.node-identity { display: flex; align-items: center; gap: 0.5rem; flex-wrap: wrap; }
.node-name { margin: 0; font-size: 1rem; font-weight: 600; color: var(--color-text); }
.node-agent { color: var(--color-text-muted); font-size: 0.8rem; font-family: var(--font-mono, monospace); }
.node-name { margin: 0; font-size: 1rem; font-weight: 600; }
.node-agent { color: var(--text-secondary, #888); font-size: 0.8rem; font-family: monospace; }
.status-dot { width: 10px; height: 10px; border-radius: 50%; flex-shrink: 0; }
.status-dot.online { background: var(--color-success); }
.status-dot.offline { background: var(--color-warning); }
.status-dot.online { background: var(--color-success, #48bb78); }
.status-dot.offline { background: var(--color-warning, #ed8936); }
.node-actions { display: flex; gap: 0.5rem; flex-shrink: 0; }
.btn-secondary {
background: transparent;
border: 1px solid var(--color-border);
color: var(--color-text);
border-radius: 4px;
padding: 0.3rem 0.65rem;
cursor: pointer;
font-size: 0.8rem;
}
.btn-secondary:hover { background: var(--color-surface-alt); }
.btn-secondary:disabled { opacity: 0.5; cursor: not-allowed; }
.btn-sm { font-size: 0.8rem; padding: 0.25rem 0.6rem; }
.no-profile {
padding: 0.6rem 0.75rem;
background: var(--color-surface-alt);
background: var(--bg-notice, #1e1e1e);
border-radius: 4px;
color: var(--color-text-muted);
color: var(--text-secondary, #888);
font-size: 0.875rem;
margin-bottom: 0.5rem;
}
.gpu-list { display: flex; flex-direction: column; gap: 0.5rem; }
.profile-load-error { color: var(--color-error); font-size: 0.8rem; margin-top: 0.5rem; }
</style>

29
web/src/components/nodes/OllamaModelPanel.vue
View file

@ -198,45 +198,44 @@ onUnmounted(() => {
.ollama-panel {
margin-top: 0.75rem;
padding: 0.75rem;
border: 1px solid var(--color-border);
border: 1px solid var(--border, #333);
border-radius: 6px;
color: var(--color-text);
}
.panel-title { margin: 0 0 0.75rem; font-size: 0.9rem; color: var(--color-text); }
.panel-title { margin: 0 0 0.75rem; font-size: 0.9rem; }
.pull-form { display: flex; gap: 0.5rem; margin-bottom: 0.5rem; }
.pull-input {
flex: 1;
padding: 0.3rem 0.5rem;
background: var(--color-surface-alt);
border: 1px solid var(--color-border);
background: var(--bg-input, #111);
border: 1px solid var(--border, #333);
border-radius: 4px;
color: var(--color-text);
color: inherit;
font-size: 0.875rem;
}
.pull-progress { margin-bottom: 0.5rem; }
.progress-bar {
height: 8px;
background: var(--color-border);
background: var(--bg-bar, #2a2a2a);
border-radius: 4px;
overflow: hidden;
margin-bottom: 0.25rem;
}
.progress-fill { height: 100%; background: var(--app-primary); transition: width 0.2s; }
.progress-label { font-size: 0.75rem; color: var(--color-text-muted); }
.pull-error, .panel-error { color: var(--color-error); font-size: 0.8rem; margin-bottom: 0.5rem; }
.progress-fill { height: 100%; background: var(--color-primary, #4080ff); transition: width 0.2s; }
.progress-label { font-size: 0.75rem; color: var(--text-secondary, #888); }
.pull-error, .panel-error { color: var(--color-error, #fc8181); font-size: 0.8rem; margin-bottom: 0.5rem; }
.sr-announce { min-height: 1.2em; }
.panel-loading { color: var(--color-text-muted); font-size: 0.875rem; }
.panel-loading { color: var(--text-secondary, #888); font-size: 0.875rem; }
.model-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.3rem; }
.model-item {
display: flex;
align-items: center;
gap: 0.5rem;
padding: 0.3rem 0.5rem;
background: var(--color-surface-alt);
background: var(--bg-secondary, #111);
border-radius: 4px;
font-size: 0.875rem;
}
.model-name { flex: 1; font-family: var(--font-mono, monospace); }
.model-size { color: var(--color-text-muted); font-size: 0.8rem; }
.model-empty { color: var(--color-text-muted); font-size: 0.875rem; padding: 0.25rem 0; }
.model-name { flex: 1; font-family: monospace; }
.model-size { color: var(--text-secondary, #888); font-size: 0.8rem; }
.model-empty { color: var(--text-secondary, #888); font-size: 0.875rem; padding: 0.25rem 0; }
</style>

597
web/src/components/nodes/ProfileEditorPanel.vue
View file

@ -1,597 +0,0 @@
<script setup lang="ts">
import { ref, onMounted } from 'vue'
import type { FullProfile, ServiceDefinition, CatalogEntryFull } from '../../types/nodes'
import ServiceFormModal from './ServiceFormModal.vue'
import CatalogEntryFormModal from './CatalogEntryFormModal.vue'
const props = defineProps<{
nodeId: string
initialProfile: FullProfile | null
}>()
const emit = defineEmits<{ saved: []; close: [] }>()
// Deep-clone initial profile so edits don't mutate the parent's data
const profile = ref<FullProfile>(
props.initialProfile
? JSON.parse(JSON.stringify(props.initialProfile))
: { services: {}, nodes: {} }
)
const saving = ref(false)
const generating = ref(false)
const opError = ref('')
const expandedSvcs = ref<Set<string>>(new Set())
// Service modal
const showSvcModal = ref(false)
const editingSvcName = ref<string | undefined>()
const editingSvcDef = ref<ServiceDefinition | undefined>()
// Catalog modal
const showCatalogModal = ref(false)
const catalogTargetSvc = ref('')
const editingModelName = ref<string | undefined>()
const editingEntry = ref<CatalogEntryFull | undefined>()
// ── Generate nodes section from coordinator ────────────────────────────────────
async function generate() {
generating.value = true
opError.value = ''
try {
const r = await fetch(`/api/nodes-mgmt/nodes/${props.nodeId}/profile/generate`, { method: 'POST' })
if (!r.ok) { const d = await r.json().catch(() => ({})); throw new Error((d as {detail?: string}).detail ?? `HTTP ${r.status}`) }
const generated = await r.json() as FullProfile
// Merge: keep current services edits, replace nodes section
profile.value = { ...generated, services: profile.value.services }
} catch (e) {
opError.value = e instanceof Error ? e.message : 'Generate failed'
} finally {
generating.value = false
}
}
// ── Save full profile ──────────────────────────────────────────────────────────
async function save() {
saving.value = true
opError.value = ''
try {
const r = await fetch(`/api/nodes-mgmt/nodes/${props.nodeId}/profile`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ profile: profile.value }),
})
if (!r.ok) { const d = await r.json().catch(() => ({})); throw new Error((d as {detail?: string}).detail ?? `HTTP ${r.status}`) }
emit('saved')
} catch (e) {
opError.value = e instanceof Error ? e.message : 'Save failed'
} finally {
saving.value = false
}
}
// ── Service CRUD ───────────────────────────────────────────────────────────────
function openAddService() {
editingSvcName.value = undefined
editingSvcDef.value = undefined
showSvcModal.value = true
}
function openEditService(name: string) {
editingSvcName.value = name
editingSvcDef.value = JSON.parse(JSON.stringify(profile.value.services[name]))
showSvcModal.value = true
}
function onServiceSaved(name: string, def: ServiceDefinition) {
profile.value = { ...profile.value, services: { ...profile.value.services, [name]: def } }
expandedSvcs.value = new Set([...expandedSvcs.value, name])
showSvcModal.value = false
}
function deleteService(name: string) {
if (!confirm(`Remove service "${name}" from this profile?`)) return
const svcs = { ...profile.value.services }
delete svcs[name]
profile.value = { ...profile.value, services: svcs }
expandedSvcs.value = new Set([...expandedSvcs.value].filter(s => s !== name))
}
function toggleSvc(name: string) {
const s = new Set(expandedSvcs.value)
s.has(name) ? s.delete(name) : s.add(name)
expandedSvcs.value = s
}
// ── Catalog CRUD ───────────────────────────────────────────────────────────────
function openAddCatalogEntry(svcName: string) {
catalogTargetSvc.value = svcName
editingModelName.value = undefined
editingEntry.value = undefined
showCatalogModal.value = true
}
function openEditCatalogEntry(svcName: string, modelName: string) {
catalogTargetSvc.value = svcName
editingModelName.value = modelName
editingEntry.value = JSON.parse(JSON.stringify(profile.value.services[svcName].catalog![modelName]))
showCatalogModal.value = true
}
function onCatalogEntrySaved(svcName: string, modelName: string, entry: CatalogEntryFull) {
const svcs = { ...profile.value.services }
const svc = { ...svcs[svcName], catalog: { ...(svcs[svcName].catalog ?? {}), [modelName]: entry } }
svcs[svcName] = svc
profile.value = { ...profile.value, services: svcs }
showCatalogModal.value = false
}
function deleteCatalogEntry(svcName: string, modelName: string) {
if (!confirm(`Remove model "${modelName}" from ${svcName} catalog?`)) return
const svcs = { ...profile.value.services }
const catalog = { ...(svcs[svcName].catalog ?? {}) }
delete catalog[modelName]
svcs[svcName] = { ...svcs[svcName], catalog }
profile.value = { ...profile.value, services: svcs }
}
// ── Helpers ────────────────────────────────────────────────────────────────────
function gpuList() {
return (profile.value.nodes[props.nodeId]?.gpus ?? [])
}
function serviceCount() {
return Object.keys(profile.value.services).length
}
// ── Ollama model suggestions ───────────────────────────────────────────────────
interface OllamaModel { name: string; size: number }
const ollamaModels = ref<OllamaModel[]>([])
const ollamaLoading = ref(false)
onMounted(async () => {
ollamaLoading.value = true
try {
const r = await fetch(`/api/nodes-mgmt/nodes/${props.nodeId}/models/ollama`)
if (r.ok) {
const d = await r.json() as { models?: OllamaModel[] }
ollamaModels.value = d.models ?? []
}
} catch { /* Ollama offline — silently skip */ }
finally { ollamaLoading.value = false }
})
function ollamaNotInCatalog(svcName: string): OllamaModel[] {
const catalog = profile.value.services[svcName]?.catalog ?? {}
return ollamaModels.value.filter(m => !(m.name in catalog))
}
function openAddFromOllama(svcName: string, modelName: string) {
catalogTargetSvc.value = svcName
editingModelName.value = modelName
editingEntry.value = {
path: profile.value.services[svcName]?.model_base_path
? `${profile.value.services[svcName].model_base_path}/${modelName}`
: '',
vram_mb: 0,
}
showCatalogModal.value = true
}
function formatMb(bytes: number): string {
return bytes >= 1_000_000_000
? `${(bytes / 1_073_741_824).toFixed(1)} GB`
: `${Math.round(bytes / 1_048_576)} MB`
}
// ── Pull model onto node ───────────────────────────────────────────────────────
const pullName = ref('')
const pulling = ref(false)
const pullStatus = ref('')
const pullPct = ref(0)
const pullError = ref('')
let pullAbort: AbortController | null = null
async function doPull() {
const name = pullName.value.trim()
if (!name || pulling.value) return
pulling.value = true
pullStatus.value = 'Starting…'
pullError.value = ''
pullPct.value = 0
pullAbort?.abort()
pullAbort = new AbortController()
try {
const resp = await fetch(`/api/nodes-mgmt/nodes/${props.nodeId}/models/ollama/pull`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name }),
signal: pullAbort.signal,
})
if (!resp.ok || !resp.body) {
pullError.value = `HTTP ${resp.status}`
return
}
const reader = resp.body.getReader()
const dec = new TextDecoder()
let buf = ''
while (true) {
const { done, value } = await reader.read()
if (done) break
buf += dec.decode(value, { stream: true })
const lines = buf.split('\n')
buf = lines.pop() ?? ''
for (const line of lines) {
if (!line.startsWith('data:')) continue
try {
const d = JSON.parse(line.slice(5)) as {
status?: string; completed?: number; total?: number; error?: string; done?: boolean
}
if (d.error) { pullError.value = d.error; return }
pullStatus.value = d.status ?? ''
if (d.total && d.total > 0) pullPct.value = Math.round((d.completed ?? 0) / d.total * 100)
if (d.done) {
pullName.value = ''
pullPct.value = 100
// Refresh Ollama model list so new model appears in suggest chips
const r = await fetch(`/api/nodes-mgmt/nodes/${props.nodeId}/models/ollama`)
if (r.ok) { const d2 = await r.json() as { models?: OllamaModel[] }; ollamaModels.value = d2.models ?? [] }
}
} catch { /* skip malformed SSE line */ }
}
}
} catch (e) {
if (e instanceof Error && e.name !== 'AbortError') pullError.value = e.message
} finally {
pulling.value = false
if (pullPct.value === 100) setTimeout(() => { pullStatus.value = ''; pullPct.value = 0 }, 2000)
}
}
</script>
<template>
<section class="pep" aria-label="Profile editor">
<!-- Header -->
<div class="pep-header">
<div class="pep-title-row">
<h3 class="pep-title">Profile — {{ nodeId }}</h3>
<span class="pep-svc-count">{{ serviceCount() }} service{{ serviceCount() === 1 ? '' : 's' }}</span>
</div>
<div class="pep-actions">
<button class="btn-secondary btn-sm" :disabled="generating" @click="generate">
{{ generating ? 'Refreshing…' : 'Refresh Hardware' }}
</button>
<button class="btn-primary btn-sm" :disabled="saving" @click="save">
{{ saving ? 'Saving…' : 'Save Profile' }}
</button>
<button class="btn-icon-lg" aria-label="Close editor" @click="emit('close')">✕</button>
</div>
</div>
<div v-if="opError" class="pep-error" role="alert">{{ opError }}</div>
<!-- Meta fields -->
<div class="pep-meta">
<label class="meta-label" for="pep-vram">vram_total_mb</label>
<input id="pep-vram" v-model.number="profile.vram_total_mb" type="number" min="0" class="meta-input" />
<label class="meta-label" for="pep-evict">eviction_timeout_s</label>
<input id="pep-evict" v-model.number="profile.eviction_timeout_s" type="number" min="0" step="0.5" class="meta-input" />
</div>
<!-- Hardware summary -->
<div v-if="gpuList().length" class="hw-section">
<span class="hw-label">Hardware</span>
<span v-for="g in gpuList()" :key="g.id" class="hw-gpu">
GPU {{ g.id }}: {{ g.card || 'unknown' }} · {{ g.vram_mb }} MB · sm{{ g.compute_cap ?? '?' }}
</span>
<span v-if="!gpuList().length" class="hw-none">No hardware data — click Refresh Hardware.</span>
</div>
<div v-else class="hw-section">
<span class="hw-none">No hardware data — click Refresh Hardware to seed from coordinator.</span>
</div>
<!-- Services -->
<div class="svcs-header">
<span class="svcs-title">Services</span>
<button class="btn-secondary btn-sm" @click="openAddService">+ Add Service</button>
</div>
<div v-if="serviceCount() === 0" class="svcs-empty">
No services defined. Add a service to configure what can run on this node.
</div>
<ul class="svcs-list" role="list">
<li
v-for="(def, svcName) in profile.services"
:key="String(svcName)"
class="svc-item"
>
<!-- Service row header -->
<div class="svc-row">
<button
class="svc-toggle"
:aria-expanded="expandedSvcs.has(String(svcName))"
@click="toggleSvc(String(svcName))"
>
<span class="svc-arrow">{{ expandedSvcs.has(String(svcName)) ? '▾' : '▸' }}</span>
<span class="svc-name">{{ svcName }}</span>
</button>
<span class="svc-badges">
<span class="badge">{{ def.max_mb }} MB</span>
<span class="badge">p{{ def.priority }}</span>
<span v-if="def.shared" class="badge badge--blue">shared</span>
<span v-if="def.managed" class="badge badge--dim">managed</span>
<span v-if="def.catalog" class="badge badge--dim">{{ Object.keys(def.catalog).length }} models</span>
</span>
<div class="svc-btns">
<button class="btn-secondary btn-xs" @click="openEditService(String(svcName))">Edit</button>
<button class="btn-danger btn-xs" @click="deleteService(String(svcName))">Delete</button>
</div>
</div>
<!-- Expanded catalog -->
<div v-if="expandedSvcs.has(String(svcName))" class="svc-detail">
<div class="svc-detail-meta">
<span v-if="def.min_compute_cap">min sm{{ def.min_compute_cap }}</span>
<span v-if="def.max_concurrent">max_concurrent: {{ def.max_concurrent }}</span>
<span v-if="def.idle_stop_after_s">idle_stop: {{ def.idle_stop_after_s }}s</span>
<span v-if="def.always_on" class="badge badge--blue">always_on</span>
</div>
<!-- Ollama model suggestions + pull -->
<div class="ollama-suggest">
<div class="suggest-row">
<span class="suggest-label">On node (Ollama):</span>
<span v-if="ollamaLoading" class="suggest-loading">loading…</span>
<template v-else-if="ollamaNotInCatalog(String(svcName)).length">
<button
v-for="m in ollamaNotInCatalog(String(svcName))"
:key="m.name"
class="suggest-chip"
:title="`Add ${m.name} (${formatMb(m.size)}) to this service catalog`"
@click="openAddFromOllama(String(svcName), m.name)"
>
+ {{ m.name }} <span class="chip-size">{{ formatMb(m.size) }}</span>
</button>
</template>
<span v-else-if="!ollamaLoading" class="suggest-none">All Ollama models already in catalog.</span>
</div>
<!-- Pull model onto this node -->
<div class="pull-row">
<input
v-model="pullName"
class="pull-input"
placeholder="Pull model on node (e.g. llama3:8b)"
:disabled="pulling"
@keyup.enter="doPull"
/>
<button class="btn-pull" :disabled="pulling || !pullName.trim()" @click="doPull">
{{ pulling ? 'Pulling…' : 'Pull' }}
</button>
</div>
<div v-if="pulling || pullPct > 0" class="pull-progress">
<div class="pull-bar"><div class="pull-fill" :style="{ width: pullPct + '%' }" /></div>
<span class="pull-status">{{ pullStatus }}</span>
</div>
<div v-if="pullError" class="pull-err" role="alert">{{ pullError }}</div>
</div>
<div class="catalog-header">
<span class="catalog-title">Catalog</span>
<button class="btn-link" @click="openAddCatalogEntry(String(svcName))">+ Add Model</button>
</div>
<div v-if="!def.catalog || !Object.keys(def.catalog).length" class="catalog-empty">
No catalog entries. Only services like cf-text need a catalog.
</div>
<ul v-else class="catalog-list" role="list">
<li
v-for="(entry, modelName) in def.catalog"
:key="String(modelName)"
class="catalog-item"
>
<span class="catalog-model">{{ modelName }}</span>
<span class="catalog-vram">{{ entry.vram_mb }} MB</span>
<span v-if="entry.multi_gpu" class="badge badge--dim">multi-gpu</span>
<span v-if="entry.description" class="catalog-desc">{{ entry.description }}</span>
<div class="catalog-btns">
<button class="btn-secondary btn-xs" @click="openEditCatalogEntry(String(svcName), String(modelName))">Edit</button>
<button class="btn-danger btn-xs" @click="deleteCatalogEntry(String(svcName), String(modelName))">✕</button>
</div>
</li>
</ul>
</div>
</li>
</ul>
</section>
<!-- Service form modal -->
<ServiceFormModal
v-if="showSvcModal"
:service-name="editingSvcName"
:definition="editingSvcDef"
@save="onServiceSaved"
@cancel="showSvcModal = false"
/>
<!-- Catalog entry form modal -->
<CatalogEntryFormModal
v-if="showCatalogModal"
:svc-name="catalogTargetSvc"
:model-name="editingModelName"
:entry="editingEntry"
@save="onCatalogEntrySaved"
@cancel="showCatalogModal = false"
/>
</template>
<style scoped>
.pep {
margin-top: 0.75rem;
padding: 1rem;
border: 1px solid var(--color-primary);
border-radius: 6px;
background: var(--color-surface-raised);
color: var(--color-text);
}
.pep-header {
display: flex; align-items: center; justify-content: space-between; gap: 0.5rem;
margin-bottom: 0.75rem; flex-wrap: wrap;
}
.pep-title-row { display: flex; align-items: baseline; gap: 0.5rem; }
.pep-title { margin: 0; font-size: 0.95rem; font-weight: 600; color: var(--color-text); }
.pep-svc-count { font-size: 0.75rem; color: var(--color-text-muted); }
.pep-actions { display: flex; align-items: center; gap: 0.4rem; flex-wrap: wrap; }
.pep-error { color: var(--color-error); font-size: 0.8rem; margin-bottom: 0.5rem; }
.pep-meta {
display: flex; align-items: center; gap: 0.5rem; flex-wrap: wrap;
padding: 0.5rem; background: var(--color-surface-alt); border-radius: 4px; margin-bottom: 0.75rem;
}
.meta-label { font-size: 0.8rem; color: var(--color-text-muted); }
.meta-input {
width: 7rem; background: var(--color-surface); border: 1px solid var(--color-border);
border-radius: 4px; padding: 0.2rem 0.4rem; color: var(--color-text); font-size: 0.8rem;
}
.hw-section {
display: flex; flex-wrap: wrap; align-items: center; gap: 0.5rem;
font-size: 0.8rem; color: var(--color-text-muted);
padding: 0.4rem 0.5rem; border-radius: 4px; background: var(--color-surface-alt);
margin-bottom: 0.75rem;
}
.hw-label { font-weight: 600; color: var(--color-text); }
.hw-gpu { font-family: monospace; color: var(--color-text); }
.hw-none { font-style: italic; }
.svcs-header {
display: flex; align-items: center; justify-content: space-between;
margin-bottom: 0.5rem;
}
.svcs-title { font-size: 0.85rem; font-weight: 600; color: var(--color-text); }
.svcs-empty { color: var(--color-text-muted); font-size: 0.85rem; padding: 0.5rem 0; }
.svcs-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.4rem; }
.svc-item { border: 1px solid var(--color-border); border-radius: 4px; overflow: hidden; }
.svc-row {
display: flex; align-items: center; gap: 0.5rem; padding: 0.4rem 0.5rem;
background: var(--color-surface-alt); flex-wrap: wrap;
}
.svc-toggle {
display: flex; align-items: center; gap: 0.35rem;
background: none; border: none; cursor: pointer; color: var(--color-text); padding: 0; flex: 1; min-width: 0;
}
.svc-arrow { font-size: 0.7rem; color: var(--color-text-muted); }
.svc-name { font-size: 0.875rem; font-weight: 500; font-family: monospace; }
.svc-badges { display: flex; gap: 0.3rem; flex-wrap: wrap; }
.svc-btns { display: flex; gap: 0.3rem; margin-left: auto; }
.svc-detail { padding: 0.5rem 0.75rem; display: flex; flex-direction: column; gap: 0.5rem; background: var(--color-surface-raised); }
.svc-detail-meta {
display: flex; gap: 0.5rem; flex-wrap: wrap;
font-size: 0.78rem; color: var(--color-text-muted);
}
.ollama-suggest {
display: flex; flex-direction: column; gap: 0.35rem;
padding: 0.4rem 0.5rem;
background: var(--color-primary-light);
border: 1px solid var(--color-border-light);
border-radius: 4px;
font-size: 0.78rem;
}
.suggest-row { display: flex; flex-wrap: wrap; align-items: center; gap: 0.35rem; }
.suggest-label { color: var(--color-text-muted); font-weight: 500; white-space: nowrap; }
.suggest-loading { color: var(--color-text-muted); font-style: italic; }
.suggest-none { color: var(--color-text-muted); font-style: italic; }
.suggest-chip {
display: inline-flex; align-items: center; gap: 0.25rem;
padding: 0.15rem 0.45rem;
background: var(--color-surface-raised);
border: 1px solid var(--color-border);
border-radius: 3px;
color: var(--color-text);
cursor: pointer;
font-size: 0.78rem;
transition: border-color 0.15s, background 0.15s;
}
.suggest-chip:hover { border-color: var(--app-primary); background: var(--color-surface-alt); }
.chip-size { color: var(--color-text-muted); font-size: 0.72rem; }
.pull-row { display: flex; gap: 0.4rem; align-items: center; }
.pull-input {
flex: 1;
padding: 0.25rem 0.5rem;
background: var(--color-surface-raised);
border: 1px solid var(--color-border);
border-radius: 4px;
color: var(--color-text);
font-size: 0.78rem;
font-family: var(--font-mono, monospace);
}
.pull-input:disabled { opacity: 0.5; }
.btn-pull {
padding: 0.25rem 0.6rem;
background: var(--app-primary);
color: var(--color-text-inverse);
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 0.78rem;
white-space: nowrap;
}
.btn-pull:hover:not(:disabled) { background: var(--app-primary-hover); }
.btn-pull:disabled { opacity: 0.5; cursor: not-allowed; }
.pull-progress { display: flex; align-items: center; gap: 0.4rem; }
.pull-bar {
flex: 1; height: 6px;
background: var(--color-border);
border-radius: 3px; overflow: hidden;
}
.pull-fill { height: 100%; background: var(--app-primary); transition: width 0.2s; }
.pull-status { color: var(--color-text-muted); font-size: 0.72rem; white-space: nowrap; max-width: 14rem; overflow: hidden; text-overflow: ellipsis; }
.pull-err { color: var(--color-error); font-size: 0.75rem; }
.catalog-header { display: flex; align-items: center; justify-content: space-between; }
.catalog-title { font-size: 0.8rem; font-weight: 600; color: var(--color-text-muted); text-transform: uppercase; letter-spacing: 0.05em; }
.catalog-empty { font-size: 0.8rem; color: var(--color-text-muted); font-style: italic; }
.catalog-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.25rem; }
.catalog-item {
display: flex; align-items: center; gap: 0.4rem; flex-wrap: wrap;
padding: 0.25rem 0.5rem; background: var(--color-surface-alt); border-radius: 3px; font-size: 0.8rem;
color: var(--color-text);
}
.catalog-model { font-family: monospace; flex: 1; min-width: 12rem; }
.catalog-vram { color: var(--color-text-muted); white-space: nowrap; }
.catalog-desc { color: var(--color-text-muted); flex: 2; font-size: 0.75rem; }
.catalog-btns { display: flex; gap: 0.25rem; margin-left: auto; }
.badge {
padding: 0.1rem 0.4rem; border-radius: 3px; font-size: 0.72rem;
background: var(--color-surface); border: 1px solid var(--color-border); color: var(--color-text);
}
.badge--blue { border-color: var(--color-primary); color: var(--color-primary); background: var(--color-primary-light); }
.badge--dim { opacity: 0.75; }
.btn-link { background: none; border: none; color: var(--color-accent); cursor: pointer; font-size: 0.8rem; padding: 0; }
.btn-link:hover { color: var(--color-accent-hover); }
.btn-primary {
background: var(--color-primary); color: var(--color-text-inverse); border: none;
border-radius: 4px; cursor: pointer; font-size: 0.8rem;
}
.btn-primary:hover { background: var(--color-primary-hover); }
.btn-primary:disabled { opacity: 0.6; cursor: not-allowed; }
.btn-secondary {
background: transparent; border: 1px solid var(--color-border); color: var(--color-text);
border-radius: 4px; cursor: pointer; font-size: 0.8rem;
}
.btn-secondary:hover { background: var(--color-surface-alt); }
.btn-secondary:disabled { opacity: 0.6; cursor: not-allowed; }
.btn-danger {
background: transparent; border: 1px solid var(--color-error); color: var(--color-error);
border-radius: 4px; cursor: pointer; font-size: 0.8rem;
}
.btn-danger:hover { background: var(--color-surface-alt); }
.btn-sm { padding: 0.3rem 0.6rem; }
.btn-xs { padding: 0.15rem 0.4rem; }
.btn-icon-lg { background: none; border: none; color: var(--color-text-muted); cursor: pointer; font-size: 1rem; padding: 0.2rem 0.3rem; }
.btn-icon-lg:hover { color: var(--color-text); }
</style>

17
web/src/components/nodes/ServiceBadge.vue
View file

@ -64,19 +64,18 @@ function handleToggle() {
gap: 0.3rem;
padding: 0.2rem 0.5rem;
border-radius: 4px;
border: 1px solid var(--color-border);
background: var(--color-surface);
color: var(--color-text);
border: 1px solid var(--border, #333);
background: var(--bg-badge, #1e1e1e);
font-size: 0.75rem;
cursor: pointer;
transition: opacity 0.1s, border-color 0.1s;
}
.service-badge:hover:not(.is-disabled) { opacity: 0.8; }
.service-badge.is-disabled { cursor: not-allowed; opacity: 0.5; }
.service-badge.state-running { border-color: var(--color-success); }
.service-badge.state-stopped { border-color: var(--color-warning); }
.service-badge.state-assigned-only { border-color: var(--color-info); }
.service-badge.state-incompatible { border-color: var(--color-error); }
.service-badge.state-vram-tight { border-color: var(--color-warning); }
.badge-state { color: var(--color-text-muted); }
.service-badge.state-running { border-color: var(--color-success, #48bb78); }
.service-badge.state-stopped { border-color: var(--color-warning, #ed8936); }
.service-badge.state-assigned-only { border-color: var(--color-info, #4299e1); }
.service-badge.state-incompatible { border-color: var(--color-error, #fc8181); }
.service-badge.state-vram-tight { border-color: var(--color-warning, #ed8936); }
.badge-state { color: var(--text-secondary, #888); }
</style>

231
web/src/components/nodes/ServiceFormModal.vue
View file

@ -1,231 +0,0 @@
<script setup lang="ts">
import { ref, watch, computed } from 'vue'
import type { ServiceDefinition } from '../../types/nodes'
const props = defineProps<{
serviceName?: string
definition?: ServiceDefinition
}>()
const emit = defineEmits<{
save: [name: string, def: ServiceDefinition]
cancel: []
}>()
const name = ref(props.serviceName ?? '')
const maxMb = ref(props.definition?.max_mb ?? 0)
const priority = ref(props.definition?.priority ?? 1)
const minCap = ref(props.definition?.min_compute_cap ?? 0)
const prefCap = ref<number | ''>(props.definition?.preferred_compute_cap ?? '')
const shared = ref(props.definition?.shared ?? false)
const maxConcurrent = ref<number | ''>(props.definition?.max_concurrent ?? '')
const idleStop = ref<number | ''>(props.definition?.idle_stop_after_s ?? '')
const alwaysOn = ref(props.definition?.always_on ?? false)
const modelBasePath = ref(props.definition?.model_base_path ?? '')
const hasManaged = ref(!!props.definition?.managed)
const managedJson = ref(
props.definition?.managed ? JSON.stringify(props.definition.managed, null, 2) : ''
)
const formError = ref('')
watch(() => props.definition, (d) => {
name.value = props.serviceName ?? ''
maxMb.value = d?.max_mb ?? 0
priority.value = d?.priority ?? 1
minCap.value = d?.min_compute_cap ?? 0
prefCap.value = d?.preferred_compute_cap ?? ''
shared.value = d?.shared ?? false
maxConcurrent.value = d?.max_concurrent ?? ''
idleStop.value = d?.idle_stop_after_s ?? ''
alwaysOn.value = d?.always_on ?? false
modelBasePath.value = d?.model_base_path ?? ''
hasManaged.value = !!d?.managed
managedJson.value = d?.managed ? JSON.stringify(d.managed, null, 2) : ''
})
const managedJsonError = computed(() => {
if (!hasManaged.value || !managedJson.value.trim()) return ''
try { JSON.parse(managedJson.value); return '' }
catch { return 'Invalid JSON' }
})
function submit() {
formError.value = ''
if (!name.value.trim()) { formError.value = 'Service name is required.'; return }
if (!maxMb.value || maxMb.value <= 0) { formError.value = 'max_mb must be > 0.'; return }
if (managedJsonError.value) { formError.value = 'Fix the managed JSON before saving.'; return }
const def: ServiceDefinition = { max_mb: maxMb.value, priority: priority.value }
if (minCap.value) def.min_compute_cap = minCap.value
if (prefCap.value !== '') def.preferred_compute_cap = Number(prefCap.value)
if (shared.value) def.shared = true
if (maxConcurrent.value !== '') def.max_concurrent = Number(maxConcurrent.value)
if (idleStop.value !== '') def.idle_stop_after_s = Number(idleStop.value)
if (alwaysOn.value) def.always_on = true
if (modelBasePath.value.trim()) def.model_base_path = modelBasePath.value.trim()
if (hasManaged.value && managedJson.value.trim()) {
def.managed = JSON.parse(managedJson.value)
}
// Preserve existing catalog when editing
if (props.definition?.catalog) def.catalog = props.definition.catalog
emit('save', name.value.trim(), def)
}
</script>
<template>
<div class="modal-backdrop" role="dialog" aria-modal="true" :aria-label="`${serviceName ? 'Edit' : 'Add'} service`">
<div class="modal-box">
<h3 class="modal-title">{{ serviceName ? 'Edit' : 'Add' }} Service</h3>
<div class="field-row">
<label class="field-label" for="sf-name">Service name</label>
<input id="sf-name" v-model="name" class="field-input" :readonly="!!serviceName" placeholder="cf-text" />
</div>
<div class="field-row">
<label class="field-label" for="sf-maxmb">max_mb</label>
<input id="sf-maxmb" v-model.number="maxMb" type="number" min="0" class="field-input field-input--sm" />
<span class="field-hint">VRAM ceiling</span>
</div>
<div class="field-row">
<label class="field-label" for="sf-prio">priority</label>
<input id="sf-prio" v-model.number="priority" type="number" min="1" max="10" class="field-input field-input--sm" />
<span class="field-hint">1 = highest</span>
</div>
<div class="field-row">
<label class="field-label" for="sf-mincap">min_compute_cap</label>
<input id="sf-mincap" v-model.number="minCap" type="number" step="0.1" min="0" class="field-input field-input--sm" placeholder="0.0" />
</div>
<div class="field-row">
<label class="field-label" for="sf-prefcap">preferred_cap</label>
<input id="sf-prefcap" v-model="prefCap" type="number" step="0.1" min="0" class="field-input field-input--sm" placeholder="optional" />
</div>
<div class="field-row field-row--check">
<input id="sf-shared" v-model="shared" type="checkbox" />
<label for="sf-shared">shared (multiple concurrent users)</label>
</div>
<div class="field-row">
<label class="field-label" for="sf-maxcon">max_concurrent</label>
<input id="sf-maxcon" v-model="maxConcurrent" type="number" min="1" class="field-input field-input--sm" placeholder="optional" />
</div>
<div class="field-row">
<label class="field-label" for="sf-idle">idle_stop_after_s</label>
<input id="sf-idle" v-model="idleStop" type="number" min="0" class="field-input field-input--sm" placeholder="optional" />
<span class="field-hint">seconds</span>
</div>
<div class="field-row field-row--check">
<input id="sf-always" v-model="alwaysOn" type="checkbox" />
<label for="sf-always">always_on (never evict)</label>
</div>
<div class="field-row">
<label class="field-label" for="sf-base">model_base_path</label>
<input id="sf-base" v-model="modelBasePath" class="field-input" placeholder="/devl/Assets/LLM/cf-text/models (optional)" />
</div>
<div class="managed-section">
<div class="field-row field-row--check">
<input id="sf-has-managed" v-model="hasManaged" type="checkbox" />
<label for="sf-has-managed">Has managed process config</label>
</div>
<div v-if="hasManaged" class="managed-body">
<label class="field-label" for="sf-managed">managed (JSON)</label>
<textarea
id="sf-managed"
v-model="managedJson"
class="field-textarea"
rows="6"
spellcheck="false"
placeholder='{"type": "process", "exec_path": "...", "args_template": "...", "port": 8008, "host_port": 8008}'
/>
<span v-if="managedJsonError" class="json-error" role="alert">{{ managedJsonError }}</span>
</div>
</div>
<div v-if="formError" class="form-error" role="alert">{{ formError }}</div>
<div class="modal-actions">
<button class="btn-secondary" @click="emit('cancel')">Cancel</button>
<button class="btn-primary" @click="submit">Save</button>
</div>
</div>
</div>
</template>
<style scoped>
.modal-backdrop {
position: fixed; inset: 0;
background: rgba(0,0,0,0.5);
display: flex; align-items: center; justify-content: center;
z-index: 200;
}
.modal-box {
background: var(--color-surface-raised);
border: 1px solid var(--color-border);
border-radius: 8px;
padding: 1.5rem;
width: 100%; max-width: 540px;
max-height: 90vh; overflow-y: auto;
display: flex; flex-direction: column; gap: 0.65rem;
color: var(--color-text);
}
.modal-title { margin: 0 0 0.25rem; font-size: 1rem; font-weight: 600; color: var(--color-text); }
.field-row { display: flex; align-items: center; gap: 0.5rem; }
.field-row--check { gap: 0.4rem; font-size: 0.875rem; color: var(--color-text); }
.field-label { min-width: 9rem; font-size: 0.85rem; color: var(--color-text-muted); flex-shrink: 0; }
.field-hint { font-size: 0.75rem; color: var(--color-text-muted); }
.field-input {
flex: 1;
background: var(--color-surface-alt);
border: 1px solid var(--color-border);
border-radius: 4px;
padding: 0.3rem 0.5rem;
color: var(--color-text);
font-size: 0.85rem;
}
.field-input--sm { flex: 0 0 8rem; }
.managed-section { display: flex; flex-direction: column; gap: 0.4rem; border-top: 1px solid var(--color-border); padding-top: 0.5rem; }
.managed-body { display: flex; flex-direction: column; gap: 0.3rem; }
.field-textarea {
width: 100%;
background: var(--color-surface-alt);
border: 1px solid var(--color-border);
border-radius: 4px;
padding: 0.4rem 0.5rem;
color: var(--color-text);
font-size: 0.8rem;
font-family: var(--font-mono, monospace);
resize: vertical;
box-sizing: border-box;
}
.json-error { color: var(--color-error); font-size: 0.78rem; }
.form-error { color: var(--color-error); font-size: 0.8rem; }
.modal-actions { display: flex; justify-content: flex-end; gap: 0.5rem; margin-top: 0.25rem; }
.btn-primary {
background: var(--app-primary);
color: var(--color-text-inverse);
border: none;
border-radius: 4px;
padding: 0.4rem 1rem;
cursor: pointer;
font-size: 0.875rem;
}
.btn-primary:hover { background: var(--app-primary-hover); }
.btn-secondary {
background: transparent;
border: 1px solid var(--color-border);
color: var(--color-text);
border-radius: 4px;
padding: 0.4rem 0.75rem;
cursor: pointer;
font-size: 0.875rem;
}
.btn-secondary:hover { background: var(--color-surface-alt); }
</style>

2
web/src/router/index.ts
View file

@ -26,12 +26,10 @@ export const routes = [
{ path: '/data/fetch', component: FetchView, meta: { title: 'Fetch' } },
{ path: '/data/corrections', component: CorrectionsView, meta: { title: 'Corrections' } },
{ path: '/data/imitate', component: ImitateView, meta: { title: 'Imitate' } },
{ path: '/data/recipe-scan', component: () => import('../views/RecipeScanView.vue'), meta: { title: 'Recipe Scan' } },
// ── Eval domain ──────────────────────────────────────────
{ path: '/eval/benchmark', component: BenchmarkView, meta: { title: 'Benchmark' } },
{ path: '/eval/compare', component: CompareView, meta: { title: 'Compare' } },
{ path: '/eval/embed-compare', component: () => import('../views/EmbedCompareView.vue'), meta: { title: 'Embed Compare' } },
// ── Train domain ─────────────────────────────────────────
{ path: '/train/jobs', component: TrainJobsView, meta: { title: 'Training Jobs' } },

62
web/src/types/nodes.ts
View file

@ -25,65 +25,3 @@ export interface NodeSummary {
profile_loaded: boolean
services_catalog: Record<string, ServiceInfo>
}
// ── Full profile types (for profile editor) ────────────────────────────────────
export interface ServiceManaged {
type: string
exec_path?: string
args_template?: string
port?: number
host_port?: number
base_port?: number
health_path?: string
cwd?: string
adopt?: boolean
[key: string]: unknown
}
export interface CatalogEntryFull {
path: string
vram_mb: number
description?: string
multi_gpu?: boolean
env?: Record<string, string>
}
export interface ServiceDefinition {
max_mb: number
priority: number
min_compute_cap?: number
preferred_compute_cap?: number
shared?: boolean
max_concurrent?: number
idle_stop_after_s?: number
always_on?: boolean
model_base_path?: string
managed?: ServiceManaged
catalog?: Record<string, CatalogEntryFull>
}
export interface NodeHardwareGpu {
id: number
vram_mb: number
compute_cap?: number
card?: string
role?: string
services?: string[]
}
export interface NodeHardwareEntry {
local_model_root?: string
agent_url?: string
gpus: NodeHardwareGpu[]
}
export interface FullProfile {
schema_version?: number
name?: string
vram_total_mb?: number
eviction_timeout_s?: number
services: Record<string, ServiceDefinition>
nodes: Record<string, NodeHardwareEntry>
model_size_hints?: Record<string, string>
}

987
web/src/views/AssignmentsTab.vue
View file

@ -1,987 +0,0 @@
<template>
<div class="assignments-tab">
<!-- ── Toast ───────────────────────────────────────────── -->
<div v-if="toast" class="toast" :class="toast.type" role="status" aria-live="polite">
{{ toast.message }}
</div>
<!-- ── Assignments section ─────────────────────────────── -->
<div class="section-header">
<h2 class="section-title">Task Assignments</h2>
<button class="btn-primary btn-sm" @click="openNewAssignment">+ New Assignment</button>
</div>
<div class="filter-row">
<label for="product-filter" class="filter-label">Product</label>
<select id="product-filter" v-model="productFilter" class="filter-select">
<option value="">All products</option>
<option v-for="p in allProducts" :key="p" :value="p">{{ p }}</option>
</select>
</div>
<div v-if="assignmentsLoading" class="empty-state">Loading assignments…</div>
<div v-else-if="assignmentsError" class="error-notice" role="alert">{{ assignmentsError }}</div>
<div v-else-if="filteredGroups.length === 0" class="empty-state">No assignments yet. Add one above.</div>
<div v-else class="product-groups">
<div v-for="group in filteredGroups" :key="group.product" class="product-group">
<h3 class="product-name">{{ group.product.toUpperCase() }}</h3>
<div class="assignment-list">
<div v-for="a in group.assignments" :key="`${a.product}/${a.task}`" class="assignment-row">
<div class="assignment-main">
<span class="task-id">{{ a.task }}</span>
<span
class="model-name"
:title="a.model_id"
>{{ displayModelId(a) }}</span>
<span v-if="a.vram_mb" class="chip chip-vram">{{ formatVram(a.vram_mb) }}</span>
<span v-if="a.service_type" class="chip" :class="serviceChipClass(a.service_type)">{{ a.service_type }}</span>
</div>
<!-- Node deployment status -->
<div v-if="deploymentMap[`${a.product}/${a.task}`]" class="node-statuses">
<span
v-for="ns in deploymentMap[`${a.product}/${a.task}`]"
:key="ns.node_id"
class="node-badge-wrap"
>
<span
class="node-badge"
:class="ns.status"
:title="`${ns.node_id}: ${ns.status}`"
>
<span class="node-icon">{{ nodeIcon(ns.status) }}</span>
{{ ns.node_id }}
</span>
<button
v-if="ns.status === 'absent'"
class="btn-deploy"
:disabled="deploying.has(`${a.product}/${a.task}/${ns.node_id}`)"
:title="`Register ${a.model_id} in ${ns.node_id} catalog`"
@click="deployModel(a, ns.node_id)"
>{{ deploying.has(`${a.product}/${a.task}/${ns.node_id}`) ? '…' : 'Register' }}</button>
</span>
</div>
<div class="assignment-actions">
<button
v-if="editingKey !== `${a.product}/${a.task}`"
class="btn-ghost btn-sm"
@click="startEdit(a)"
>Edit</button>
<button
class="btn-ghost btn-sm btn-danger"
@click="deleteAssignment(a.product, a.task)"
>Delete</button>
</div>
<!-- Inline edit form -->
<div v-if="editingKey === `${a.product}/${a.task}`" class="inline-edit">
<select v-model="editDraft.model_id" class="edit-select" aria-label="Model">
<option value="" disabled>Select model…</option>
<option v-for="m in registryModels" :key="m.model_id" :value="m.model_id">
{{ m.alias || truncate(m.model_id, 40) }}
</option>
</select>
<input
v-model="editDraft.description"
type="text"
class="edit-input"
placeholder="Description (optional)"
/>
<div class="inline-edit-btns">
<button class="btn-primary btn-sm" :disabled="!editDraft.model_id" @click="saveEdit(a)">Save</button>
<button class="btn-ghost btn-sm" @click="editingKey = null">Cancel</button>
</div>
</div>
</div>
</div>
</div>
</div>
<!-- ── Model Registry section ───────────────────────────── -->
<div class="section-header section-header-mt">
<h2 class="section-title">Model Registry</h2>
<button class="btn-primary btn-sm" @click="showRegisterModal = true">Register Model</button>
</div>
<div v-if="registryLoading" class="empty-state">Loading model registry…</div>
<div v-else-if="registryError" class="error-notice" role="alert">{{ registryError }}</div>
<div v-else-if="registryModels.length === 0" class="empty-state">No models registered yet.</div>
<div v-else class="registry-table-wrap">
<table class="registry-table">
<thead>
<tr>
<th>Alias</th>
<th>Model ID</th>
<th>VRAM</th>
<th>Service</th>
<th class="col-hf">HF Repo</th>
<th></th>
</tr>
</thead>
<tbody>
<tr v-for="m in registryModels" :key="m.model_id">
<td>{{ m.alias || '—' }}</td>
<td>
<span class="truncated" :title="m.model_id">{{ truncate(m.model_id, 36) }}</span>
</td>
<td>{{ formatVram(m.vram_mb) }}</td>
<td><span class="chip" :class="serviceChipClass(m.service_type)">{{ m.service_type }}</span></td>
<td class="col-hf">
<a
v-if="m.hf_repo"
:href="`https://huggingface.co/${m.hf_repo}`"
target="_blank"
rel="noopener noreferrer"
class="hf-link"
>{{ truncate(m.hf_repo, 30) }}</a>
<span v-else class="text-muted">—</span>
</td>
<td>
<button class="btn-ghost btn-sm btn-danger" @click="deleteModel(m.model_id)">Delete</button>
</td>
</tr>
</tbody>
</table>
</div>
<!-- ── New Assignment modal ─────────────────────────────── -->
<div v-if="showNewAssignmentModal" class="modal-backdrop" @click.self="showNewAssignmentModal = false">
<div class="modal" role="dialog" aria-modal="true" aria-labelledby="modal-new-assignment-title">
<h3 id="modal-new-assignment-title" class="modal-title">New Assignment</h3>
<label class="form-label">Product</label>
<input
v-model="newAssignment.product"
list="product-list"
class="form-input"
placeholder="e.g. peregrine"
autocomplete="off"
/>
<datalist id="product-list">
<option v-for="p in allProducts" :key="p" :value="p" />
</datalist>
<label class="form-label">Task ID</label>
<input
v-model="newAssignment.task"
type="text"
class="form-input"
placeholder="e.g. cover_letter"
/>
<label class="form-label">Model</label>
<select v-model="newAssignment.model_id" class="form-select">
<option value="" disabled>Select from registry…</option>
<option v-for="m in registryModels" :key="m.model_id" :value="m.model_id">
{{ m.alias || truncate(m.model_id, 50) }}
</option>
</select>
<label class="form-label">Description <span class="optional">(optional)</span></label>
<input
v-model="newAssignment.description"
type="text"
class="form-input"
placeholder="Human-readable note for operators"
/>
<div class="modal-actions">
<button
class="btn-primary"
:disabled="!newAssignment.product || !newAssignment.task || !newAssignment.model_id || saving"
@click="saveNewAssignment"
>{{ saving ? 'Saving…' : 'Save' }}</button>
<button class="btn-ghost" @click="showNewAssignmentModal = false">Cancel</button>
</div>
</div>
</div>
<!-- ── Register Model modal ─────────────────────────────── -->
<div v-if="showRegisterModal" class="modal-backdrop" @click.self="showRegisterModal = false">
<div class="modal" role="dialog" aria-modal="true" aria-labelledby="modal-register-title">
<h3 id="modal-register-title" class="modal-title">Register Model</h3>
<label class="form-label">Model ID <span class="hint">(HuggingFace slug, e.g. ibm-granite/granite-4.1-8b)</span></label>
<input v-model="newModel.model_id" type="text" class="form-input" placeholder="org/model-name" />
<label class="form-label">Alias <span class="optional">(optional, short name for assignments)</span></label>
<input v-model="newModel.alias" type="text" class="form-input" placeholder="e.g. granite-8b" />
<label class="form-label">Service type</label>
<select v-model="newModel.service_type" class="form-select">
<option value="" disabled>Select service…</option>
<option value="cf-text">cf-text — Language Models</option>
<option value="cf-stt">cf-stt — Speech Recognition</option>
<option value="cf-tts">cf-tts — Text to Speech</option>
<option value="cf-vision">cf-vision — Vision / VLM</option>
<option value="cf-image">cf-image — Image Generation</option>
<option value="cf-voice">cf-voice — Audio Classification</option>
<option value="vllm">vllm — vLLM inference</option>
<option value="ollama">ollama — Ollama inference</option>
</select>
<label class="form-label">VRAM required (MB)</label>
<input v-model.number="newModel.vram_mb" type="number" min="0" class="form-input" placeholder="e.g. 16384" />
<label class="form-label">HF Repo <span class="optional">(optional)</span></label>
<input v-model="newModel.hf_repo" type="text" class="form-input" placeholder="org/repo-name" />
<label class="form-label">Description <span class="optional">(optional)</span></label>
<input v-model="newModel.description" type="text" class="form-input" placeholder="Human-readable note" />
<div class="modal-actions">
<button
class="btn-primary"
:disabled="!newModel.model_id || !newModel.service_type || !newModel.vram_mb || saving"
@click="saveNewModel"
>{{ saving ? 'Saving…' : 'Register' }}</button>
<button class="btn-ghost" @click="showRegisterModal = false">Cancel</button>
</div>
</div>
</div>
</div>
</template>
<script setup lang="ts">
import { ref, computed, onMounted } from 'vue'
// ── Types ──────────────────────────────────────────────
interface AssignmentNode {
node_id: string
status: 'present' | 'absent' | 'vram_tight'
}
interface DeployingKey {
nodeId: string
assignmentKey: string
}
interface Assignment {
product: string
task: string
model_id: string
description: string
alias?: string
service_type?: string
vram_mb?: number
nodes?: AssignmentNode[]
}
interface RegistryModel {
model_id: string
alias: string
service_type: string
vram_mb: number
hf_repo: string
description: string
}
interface ProductGroup {
product: string
assignments: Assignment[]
}
interface Toast {
message: string
type: 'success' | 'error'
}
// ── State ──────────────────────────────────────────────
const assignments = ref<Assignment[]>([])
const assignmentsLoading = ref(false)
const assignmentsError = ref<string | null>(null)
const registryModels = ref<RegistryModel[]>([])
const registryLoading = ref(false)
const registryError = ref<string | null>(null)
const productFilter = ref('')
const editingKey = ref<string | null>(null)
const editDraft = ref({ model_id: '', description: '' })
const showNewAssignmentModal = ref(false)
const newAssignment = ref({ product: '', task: '', model_id: '', description: '' })
const showRegisterModal = ref(false)
const newModel = ref({ model_id: '', alias: '', service_type: '', vram_mb: 0, hf_repo: '', description: '' })
const saving = ref(false)
const toast = ref<Toast | null>(null)
let toastTimer: ReturnType<typeof setTimeout> | null = null
const deploying = ref<Set<string>>(new Set())
// ── Derived ────────────────────────────────────────────
const allProducts = computed(() => {
const seen = new Set<string>()
for (const a of assignments.value) seen.add(a.product)
return [...seen].sort()
})
const deploymentMap = computed(() => {
const map: Record<string, AssignmentNode[]> = {}
for (const a of assignments.value) {
if (a.nodes) map[`${a.product}/${a.task}`] = a.nodes
}
return map
})
const filteredGroups = computed((): ProductGroup[] => {
const filtered = productFilter.value
? assignments.value.filter(a => a.product === productFilter.value)
: assignments.value
const byProduct: Record<string, Assignment[]> = {}
for (const a of filtered) {
if (!byProduct[a.product]) byProduct[a.product] = []
byProduct[a.product].push(a)
}
return Object.keys(byProduct)
.sort()
.map(product => ({ product, assignments: byProduct[product] }))
})
// ── Helpers ────────────────────────────────────────────
function truncate(s: string, max: number): string {
return s.length > max ? s.slice(0, max - 1) + '…' : s
}
function displayModelId(a: Assignment): string {
if (a.alias) return a.alias
const id = a.model_id
// Show only the model name part (after /) and truncate long slugs
const short = id.includes('/') ? id.split('/').slice(1).join('/') : id
return truncate(short, 36)
}
function formatVram(mb: number | undefined): string {
if (!mb) return ''
if (mb >= 1024) return `${(mb / 1024).toFixed(1)} GB`
return `${mb} MB`
}
function serviceChipClass(service: string): string {
return `chip-service-${service.replace(/[^a-z0-9]/g, '-')}`
}
function nodeIcon(status: string): string {
if (status === 'present') return '✓'
if (status === 'vram_tight') return '~'
return '✗'
}
function showToast(message: string, type: 'success' | 'error' = 'success') {
if (toastTimer) clearTimeout(toastTimer)
toast.value = { message, type }
toastTimer = setTimeout(() => { toast.value = null }, 3500)
}
function openNewAssignment() {
newAssignment.value = { product: '', task: '', model_id: '', description: '' }
showNewAssignmentModal.value = true
}
function startEdit(a: Assignment) {
editingKey.value = `${a.product}/${a.task}`
editDraft.value = { model_id: a.model_id, description: a.description }
}
// ── API ────────────────────────────────────────────────
async function loadAssignments() {
assignmentsLoading.value = true
assignmentsError.value = null
try {
// Fetch both list and deployment status in parallel
const [listRes, statusRes] = await Promise.all([
fetch('/api/cforch/assignments'),
fetch('/api/cforch/assignments/deployment-status'),
])
if (!listRes.ok) throw new Error(`HTTP ${listRes.status}`)
const list: Assignment[] = (await listRes.json()).assignments ?? []
// Merge deployment status into assignments if available
if (statusRes.ok) {
const statusList: Assignment[] = (await statusRes.json()).deployment_status ?? []
const statusMap: Record<string, AssignmentNode[]> = {}
for (const s of statusList) {
statusMap[`${s.product}/${s.task}`] = s.nodes ?? []
}
for (const a of list) {
a.nodes = statusMap[`${a.product}/${a.task}`] ?? []
// Enrich with service_type/vram_mb from status payload
const s = statusList.find(x => x.product === a.product && x.task === a.task)
if (s) {
a.service_type = s.service_type
a.vram_mb = s.vram_mb
a.alias = s.alias
}
}
}
assignments.value = list
} catch (e) {
assignmentsError.value = `Could not load assignments: ${e}`
} finally {
assignmentsLoading.value = false
}
}
async function loadRegistry() {
registryLoading.value = true
registryError.value = null
try {
const res = await fetch('/api/cforch/model-registry')
if (!res.ok) throw new Error(`HTTP ${res.status}`)
registryModels.value = (await res.json()).models ?? []
} catch (e) {
registryError.value = `Could not load model registry: ${e}`
} finally {
registryLoading.value = false
}
}
async function saveNewAssignment() {
saving.value = true
try {
const res = await fetch('/api/cforch/assignments', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(newAssignment.value),
})
if (!res.ok) throw new Error(await res.text())
showNewAssignmentModal.value = false
showToast('Assignment saved')
await loadAssignments()
} catch (e) {
showToast(`Save failed: ${e}`, 'error')
} finally {
saving.value = false
}
}
async function saveEdit(a: Assignment) {
saving.value = true
try {
const res = await fetch('/api/cforch/assignments', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
product: a.product,
task: a.task,
model_id: editDraft.value.model_id,
description: editDraft.value.description,
}),
})
if (!res.ok) throw new Error(await res.text())
editingKey.value = null
showToast('Assignment updated')
await loadAssignments()
} catch (e) {
showToast(`Update failed: ${e}`, 'error')
} finally {
saving.value = false
}
}
async function deleteAssignment(product: string, task: string) {
if (!confirm(`Delete assignment ${product}.${task}?`)) return
try {
const res = await fetch(
`/api/cforch/assignments/${encodeURIComponent(product)}/${encodeURIComponent(task)}`,
{ method: 'DELETE' },
)
if (!res.ok) throw new Error(await res.text())
showToast('Assignment deleted')
await loadAssignments()
} catch (e) {
showToast(`Delete failed: ${e}`, 'error')
}
}
async function saveNewModel() {
saving.value = true
try {
const res = await fetch('/api/cforch/model-registry', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(newModel.value),
})
if (!res.ok) throw new Error(await res.text())
showRegisterModal.value = false
showToast('Model registered')
await loadRegistry()
} catch (e) {
showToast(`Register failed: ${e}`, 'error')
} finally {
saving.value = false
}
}
async function deleteModel(model_id: string) {
if (!confirm(`Remove ${model_id} from the registry?`)) return
try {
const res = await fetch(
`/api/cforch/model-registry/${encodeURIComponent(model_id)}`,
{ method: 'DELETE' },
)
if (!res.ok) throw new Error(await res.text())
showToast('Model removed')
await loadRegistry()
} catch (e) {
showToast(`Delete failed: ${e}`, 'error')
}
}
async function deployModel(a: Assignment, nodeId: string) {
const key = `${a.product}/${a.task}/${nodeId}`
if (deploying.value.has(key)) return
// Look up hf_repo from registry for cleaner path construction
const regEntry = registryModels.value.find(m => m.model_id === a.model_id)
const hf_repo = regEntry?.hf_repo ?? ''
const service_type = a.service_type ?? regEntry?.service_type ?? ''
const vram_mb = a.vram_mb ?? regEntry?.vram_mb ?? 0
const description = regEntry?.alias ? `${regEntry.alias} (via assignments)` : ''
if (!service_type) {
showToast(`No service type for model ${a.model_id}`, 'error')
return
}
deploying.value = new Set([...deploying.value, key])
try {
const res = await fetch(`/api/nodes-mgmt/nodes/${encodeURIComponent(nodeId)}/models/deploy`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ model_id: a.model_id, service_type, vram_mb, hf_repo, description }),
})
if (!res.ok) throw new Error(await res.text())
const data = await res.json()
showToast(`Registered ${a.model_id} on ${nodeId} at ${data.path}`)
// Optimistic update: flip node to 'present' immediately so the Register button
// disappears before the coordinator reload confirms. loadAssignments() reconciles
// with real server state on the next round-trip.
assignments.value = assignments.value.map(asgn => {
if (asgn.product !== a.product || asgn.task !== a.task) return asgn
return {
...asgn,
nodes: (asgn.nodes ?? []).map(ns =>
ns.node_id === nodeId ? { ...ns, status: 'present' as const } : ns
),
}
})
await loadAssignments()
} catch (e) {
showToast(`Deploy failed: ${e}`, 'error')
} finally {
deploying.value = new Set([...deploying.value].filter(k => k !== key))
}
}
onMounted(() => {
loadAssignments()
loadRegistry()
})
</script>
<style scoped>
.assignments-tab {
display: flex;
flex-direction: column;
gap: 1.25rem;
}
/* ── Toast ── */
.toast {
position: fixed;
bottom: 1.5rem;
right: 1.5rem;
padding: 0.65rem 1.1rem;
border-radius: 0.5rem;
font-size: 0.88rem;
font-weight: 500;
z-index: 200;
box-shadow: 0 2px 8px rgba(0,0,0,0.15);
}
.toast.success {
background: var(--color-success, #2a8050);
color: #fff;
}
.toast.error {
background: var(--color-danger, #b03030);
color: #fff;
}
/* ── Section headers ── */
.section-header {
display: flex;
align-items: center;
justify-content: space-between;
gap: 1rem;
}
.section-header-mt {
margin-top: 1.5rem;
}
.section-title {
font-size: 1rem;
font-weight: 600;
color: var(--app-primary, #2A6080);
margin: 0;
}
/* ── Filter row ── */
.filter-row {
display: flex;
align-items: center;
gap: 0.6rem;
}
.filter-label {
font-size: 0.85rem;
color: var(--color-text-muted, #6b7a99);
}
.filter-select {
padding: 0.3rem 0.6rem;
font-size: 0.85rem;
border: 1px solid var(--color-border, #d0d7e8);
border-radius: 0.4rem;
background: var(--color-surface, #fff);
color: var(--color-text, #1a2030);
}
/* ── Product groups ── */
.product-groups {
display: flex;
flex-direction: column;
gap: 1rem;
}
.product-group {}
.product-name {
font-size: 0.75rem;
font-weight: 700;
letter-spacing: 0.08em;
color: var(--color-text-muted, #6b7a99);
text-transform: uppercase;
margin: 0 0 0.4rem;
}
.assignment-list {
display: flex;
flex-direction: column;
gap: 0.4rem;
}
/* ── Assignment rows ── */
.assignment-row {
background: var(--color-surface-raised, #f0f4fa);
border: 1px solid var(--color-border, #d0d7e8);
border-radius: 0.5rem;
padding: 0.65rem 0.85rem;
display: flex;
flex-direction: column;
gap: 0.4rem;
}
.assignment-main {
display: flex;
align-items: center;
gap: 0.5rem;
flex-wrap: wrap;
}
.task-id {
font-family: var(--font-mono, monospace);
font-size: 0.88rem;
font-weight: 600;
color: var(--color-text, #1a2030);
min-width: 0;
}
.model-name {
font-size: 0.85rem;
color: var(--color-text-muted, #6b7a99);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
max-width: 280px;
cursor: default;
}
.assignment-actions {
display: flex;
gap: 0.4rem;
flex-wrap: wrap;
}
/* ── Node status badges ── */
.node-statuses {
display: flex;
gap: 0.35rem;
flex-wrap: wrap;
}
.node-badge-wrap {
display: inline-flex;
align-items: center;
gap: 0.2rem;
}
.node-badge {
display: inline-flex;
align-items: center;
gap: 0.2rem;
font-size: 0.78rem;
padding: 0.15rem 0.5rem;
border-radius: 0.35rem;
font-weight: 500;
}
.node-badge.present {
background: color-mix(in srgb, var(--color-success, #2a8050) 15%, transparent);
color: var(--color-success, #2a8050);
border: 1px solid color-mix(in srgb, var(--color-success, #2a8050) 30%, transparent);
}
.node-badge.absent {
background: color-mix(in srgb, var(--color-danger, #b03030) 12%, transparent);
color: var(--color-danger, #b03030);
border: 1px solid color-mix(in srgb, var(--color-danger, #b03030) 25%, transparent);
}
.node-badge.vram_tight {
background: color-mix(in srgb, #c08030 15%, transparent);
color: #8a5500;
border: 1px solid color-mix(in srgb, #c08030 30%, transparent);
}
.node-icon {
font-size: 0.85em;
}
.btn-deploy {
padding: 0.1rem 0.4rem;
font-size: 0.72rem;
font-weight: 600;
background: color-mix(in srgb, var(--app-primary, #2A6080) 12%, transparent);
color: var(--app-primary, #2A6080);
border: 1px solid color-mix(in srgb, var(--app-primary, #2A6080) 30%, transparent);
border-radius: 0.3rem;
cursor: pointer;
white-space: nowrap;
transition: background 0.15s;
}
.btn-deploy:hover:not(:disabled) {
background: color-mix(in srgb, var(--app-primary, #2A6080) 22%, transparent);
}
.btn-deploy:disabled { opacity: 0.5; cursor: default; }
/* ── Inline edit ── */
.inline-edit {
display: flex;
flex-wrap: wrap;
gap: 0.4rem;
padding-top: 0.35rem;
border-top: 1px solid var(--color-border, #d0d7e8);
}
.edit-select,
.edit-input {
flex: 1;
min-width: 160px;
padding: 0.35rem 0.55rem;
font-size: 0.85rem;
border: 1px solid var(--color-border, #d0d7e8);
border-radius: 0.4rem;
background: var(--color-surface, #fff);
color: var(--color-text, #1a2030);
}
.inline-edit-btns {
display: flex;
gap: 0.35rem;
align-items: center;
}
/* ── Registry table ── */
.registry-table-wrap {
overflow-x: auto;
border-radius: 0.5rem;
border: 1px solid var(--color-border, #d0d7e8);
}
.registry-table {
width: 100%;
border-collapse: collapse;
font-size: 0.85rem;
}
.registry-table th {
text-align: left;
padding: 0.5rem 0.75rem;
font-size: 0.78rem;
font-weight: 600;
color: var(--color-text-muted, #6b7a99);
background: var(--color-surface-raised, #f0f4fa);
border-bottom: 1px solid var(--color-border, #d0d7e8);
white-space: nowrap;
}
.registry-table td {
padding: 0.5rem 0.75rem;
border-bottom: 1px solid var(--color-border, #d0d7e8);
vertical-align: middle;
}
.registry-table tbody tr:last-child td {
border-bottom: none;
}
.truncated {
display: inline-block;
max-width: 220px;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
vertical-align: bottom;
cursor: default;
}
.hf-link {
color: var(--app-primary, #2A6080);
text-decoration: none;
font-size: 0.82rem;
}
.hf-link:hover { text-decoration: underline; }
.text-muted { color: var(--color-text-muted, #6b7a99); }
/* ── Chips ── */
.chip {
display: inline-block;
padding: 0.15rem 0.5rem;
border-radius: 0.35rem;
font-size: 0.75rem;
font-weight: 600;
white-space: nowrap;
}
.chip-vram {
background: color-mix(in srgb, var(--app-primary, #2A6080) 12%, transparent);
color: var(--app-primary, #2A6080);
border: 1px solid color-mix(in srgb, var(--app-primary, #2A6080) 25%, transparent);
}
/* service chips — match ModelsView convention */
.chip-service-cf-text { background: #e8f0fe; color: #1a5276; border: 1px solid #a9c4e8; }
.chip-service-cf-stt { background: #eaf6ea; color: #1e6b3a; border: 1px solid #a2d9b1; }
.chip-service-cf-tts { background: #fdf3e3; color: #7d4e00; border: 1px solid #e8c98a; }
.chip-service-cf-vision { background: #f3e8fd; color: #5b2d8e; border: 1px solid #c8a0e8; }
.chip-service-cf-image { background: #fce8f0; color: #8e1a4f; border: 1px solid #e8a0c0; }
.chip-service-cf-voice { background: #e8f8fc; color: #0a5c6e; border: 1px solid #88d0e0; }
.chip-service-vllm { background: #f5ece0; color: #7a3800; border: 1px solid #d4a87a; }
.chip-service-ollama { background: #eeeeee; color: #444; border: 1px solid #ccc; }
/* ── Buttons ── */
.btn-primary {
padding: 0.45rem 1rem;
background: var(--app-primary, #2A6080);
color: #fff;
border: none;
border-radius: 0.4rem;
font-size: 0.85rem;
font-weight: 600;
cursor: pointer;
transition: opacity 0.15s;
}
.btn-primary:disabled { opacity: 0.5; cursor: default; }
.btn-primary:not(:disabled):hover { opacity: 0.88; }
.btn-ghost {
padding: 0.35rem 0.75rem;
background: transparent;
border: 1px solid var(--color-border, #d0d7e8);
border-radius: 0.4rem;
font-size: 0.82rem;
color: var(--color-text-muted, #6b7a99);
cursor: pointer;
transition: background 0.15s;
}
.btn-ghost:hover { background: var(--color-surface-raised, #e4ebf5); }
.btn-ghost.btn-danger { color: var(--color-danger, #b03030); border-color: color-mix(in srgb, var(--color-danger, #b03030) 30%, transparent); }
.btn-ghost.btn-danger:hover { background: color-mix(in srgb, var(--color-danger, #b03030) 10%, transparent); }
.btn-sm { padding: 0.3rem 0.65rem; font-size: 0.8rem; }
/* ── Empty / error states ── */
.empty-state {
padding: 1.5rem;
text-align: center;
color: var(--color-text-muted, #6b7a99);
font-size: 0.9rem;
background: var(--color-surface-raised, #f0f4fa);
border: 1px dashed var(--color-border, #d0d7e8);
border-radius: 0.5rem;
}
.error-notice {
padding: 0.75rem 1rem;
background: color-mix(in srgb, var(--color-danger, #b03030) 10%, transparent);
color: var(--color-danger, #b03030);
border: 1px solid color-mix(in srgb, var(--color-danger, #b03030) 25%, transparent);
border-radius: 0.4rem;
font-size: 0.87rem;
}
/* ── Modal ── */
.modal-backdrop {
position: fixed;
inset: 0;
background: rgba(0,0,0,0.35);
display: flex;
align-items: center;
justify-content: center;
z-index: 100;
padding: 1rem;
}
.modal {
background: var(--color-surface, #fff);
border-radius: 0.65rem;
padding: 1.5rem;
width: 100%;
max-width: 480px;
display: flex;
flex-direction: column;
gap: 0.65rem;
box-shadow: 0 8px 32px rgba(0,0,0,0.18);
max-height: 90vh;
overflow-y: auto;
}
.modal-title {
font-size: 1rem;
font-weight: 700;
color: var(--app-primary, #2A6080);
margin: 0 0 0.25rem;
}
.form-label {
font-size: 0.82rem;
font-weight: 600;
color: var(--color-text-muted, #6b7a99);
}
.form-input,
.form-select {
padding: 0.4rem 0.65rem;
font-size: 0.88rem;
border: 1px solid var(--color-border, #d0d7e8);
border-radius: 0.4rem;
background: var(--color-surface, #fff);
color: var(--color-text, #1a2030);
width: 100%;
box-sizing: border-box;
}
.form-input:focus, .form-select:focus {
outline: 2px solid var(--app-primary, #2A6080);
outline-offset: 1px;
}
.modal-actions {
display: flex;
gap: 0.5rem;
justify-content: flex-end;
margin-top: 0.25rem;
}
.optional, .hint {
font-weight: 400;
color: var(--color-text-muted, #6b7a99);
font-size: 0.78rem;
}
/* ── Responsive ── */
@media (max-width: 600px) {
.assignment-main { flex-direction: column; align-items: flex-start; }
.col-hf { display: none; }
.model-name { max-width: 100%; }
.modal { padding: 1rem; }
}
</style>

16
web/src/views/ClassifierTab.vue
View file

@ -325,7 +325,7 @@ function toggleCategory(models: AvailableModel[], checked: boolean) {
async function loadModelCategories() {
modelsLoading.value = true
const { data } = await useApiFetch<ModelCategoriesResponse>('/api/cforch/models')
const { data } = await useApiFetch<ModelCategoriesResponse>('/api/benchmark/models')
modelsLoading.value = false
if (data?.categories) {
modelCategories.value = data.categories
@ -342,7 +342,7 @@ const modelCount = computed(() => modelNames.value.length)
const labelNames = computed(() => {
const canonical = Object.keys(LABEL_META)
const inResults = new Set(
modelNames.value.flatMap(n => Object.keys(results.value?.models[n]?.per_label ?? {}))
modelNames.value.flatMap(n => Object.keys(results.value!.models[n].per_label))
)
return [...canonical.filter(l => inResults.has(l)), ...[...inResults].filter(l => !canonical.includes(l))]
})
@ -401,16 +401,16 @@ function formatDate(iso: string | null): string {
// ── Data loading ─────────────────────────────────────────────────────────────
async function loadResults() {
loading.value = true
const { data } = await useApiFetch<BenchResults>('/api/cforch/results')
const { data } = await useApiFetch<BenchResults>('/api/benchmark/results')
loading.value = false
if (data?.models && Object.keys(data.models).length > 0) {
if (data && Object.keys(data.models).length > 0) {
results.value = data
}
}
async function loadFineTunedModels() {
const { data } = await useApiFetch<{ results: FineTunedModel[] }>('/api/train/results')
if (Array.isArray(data?.results)) fineTunedModels.value = data.results
const { data } = await useApiFetch<FineTunedModel[]>('/api/finetune/status')
if (Array.isArray(data)) fineTunedModels.value = data
}
// ── Benchmark run ────────────────────────────────────────────────────────────
@ -428,7 +428,7 @@ function startBenchmark() {
params.set('model_names', [...selectedModels.value].join(','))
}
const qs = params.toString()
const url = `/api/cforch/run${qs ? `?${qs}` : ''}`
const url = `/api/benchmark/run${qs ? `?${qs}` : ''}`
useApiSSE(
url,
async (event) => {
@ -457,7 +457,7 @@ function startBenchmark() {
}
async function cancelBenchmark() {
await fetch('/api/cforch/cancel', { method: 'POST' }).catch(() => {})
await fetch('/api/benchmark/cancel', { method: 'POST' }).catch(() => {})
}
// ── Fine-tune ─────────────────────────────────────────────────────────────────

40
web/src/views/CompareTab.vue
View file

@ -71,37 +71,34 @@
rows="6"
/>
<!-- LLM model picker (ollama + vllm + cf-text) -->
<!-- Ollama model picker -->
<details class="model-picker" open>
<summary class="picker-summary">
<span class="picker-title">🤖 LLM Models</span>
<span class="picker-badge">{{ cmpSelectedModels.size }} / {{ llmSelectableModels.length }}</span>
<span class="picker-title">🤖 Ollama Models</span>
<span class="picker-badge">{{ cmpSelectedModels.size }} / {{ ollamaLlmModels.length }}</span>
</summary>
<div class="picker-body">
<label class="picker-cat-header">
<input
type="checkbox"
:checked="cmpSelectedModels.size === llmSelectableModels.length"
:indeterminate="cmpSelectedModels.size > 0 && cmpSelectedModels.size < llmSelectableModels.length"
:checked="cmpSelectedModels.size === ollamaLlmModels.length"
:indeterminate="cmpSelectedModels.size > 0 && cmpSelectedModels.size < ollamaLlmModels.length"
@change="toggleAllCmpModels(($event.target as HTMLInputElement).checked)"
/>
<span class="picker-cat-name">All LLM models</span>
<span class="picker-cat-name">All ollama models</span>
</label>
<div v-for="(models, service) in llmModelsByService" :key="service" class="picker-category">
<span class="picker-cat-section">{{ service }}</span>
<div class="picker-model-list">
<label v-for="m in models" :key="m.id" class="picker-model-row">
<label v-for="m in ollamaLlmModels" :key="m.id" class="picker-model-row">
<input
type="checkbox"
:checked="cmpSelectedModels.has(m.id)"
@change="toggleCmpModel(m.id, ($event.target as HTMLInputElement).checked)"
/>
<span class="picker-model-name">{{ m.name }}</span>
<span class="picker-adapter-type">{{ m.tags.slice(0, 2).join(', ') }}</span>
<span class="picker-adapter-type">{{ m.tags.slice(0, 3).join(', ') }}</span>
</label>
</div>
</div>
</div>
</details>
<!-- Run controls -->
@ -235,22 +232,10 @@ const cmpResults = ref<CmpResult[]>([])
const cmpEventSource = ref<EventSource | null>(null)
// ── Computed ────────────────────────────────────────────────────────────────
const LLM_SERVICES = new Set(['ollama', 'vllm', 'cf-text'])
const llmSelectableModels = computed(() =>
llmModels.value.filter(m => LLM_SERVICES.has(m.service))
const ollamaLlmModels = computed(() =>
llmModels.value.filter(m => m.service === 'ollama')
)
/** Group selectable models by service for the picker UI */
const llmModelsByService = computed((): Record<string, CfOrchModel[]> => {
const groups: Record<string, CfOrchModel[]> = {}
for (const m of llmSelectableModels.value) {
if (!groups[m.service]) groups[m.service] = []
groups[m.service].push(m)
}
return groups
})
const llmTasksByType = computed((): Record<string, CfOrchTask[]> => {
const groups: Record<string, CfOrchTask[]> = {}
for (const t of llmTasks.value) {
@ -285,7 +270,7 @@ function toggleCmpModel(id: string, checked: boolean) {
function toggleAllCmpModels(checked: boolean) {
cmpSelectedModels.value = checked
? new Set(llmSelectableModels.value.map(m => m.id))
? new Set(ollamaLlmModels.value.map(m => m.id))
: new Set()
}
@ -303,8 +288,9 @@ async function loadLlmModels() {
const { data } = await useApiFetch<{ models: CfOrchModel[] }>('/api/cforch/models')
if (data?.models) {
llmModels.value = data.models
// Pre-select all ollama models
cmpSelectedModels.value = new Set(
data.models.filter(m => LLM_SERVICES.has(m.service)).map(m => m.id)
data.models.filter(m => m.service === 'ollama').map(m => m.id)
)
}
}

99
web/src/views/DashboardView.vue
View file

@ -28,6 +28,9 @@
<span class="metric-label"> labeled since last eval</span>
</p>
</div>
<div v-if="data.signals.data_to_eval" class="card-cta">
<RouterLink to="/eval/benchmark" class="cta-btn">Run Eval</RouterLink>
</div>
</div>
<!-- ② Eval card -->
@ -37,28 +40,18 @@
<h2 class="card-title">Eval</h2>
</div>
<div class="card-body">
<div class="bench-run-table">
<div
v-for="(run, type) in data.recent_bench_runs"
:key="type"
class="bench-run-row"
>
<span class="bench-type-label">{{ BENCH_LABELS[type as BenchType] ?? type }}</span>
<span class="bench-run-time" :class="{ 'metric-muted': !run.timestamp }">
{{ run.timestamp ? formatBenchTs(run.timestamp) : '—' }}
</span>
<span v-if="run.score != null" class="bench-run-score">
{{ formatScore(run.score) }}
</span>
</div>
</div>
<p class="card-metric">
<span class="metric-label">Last run: </span>
<strong class="metric-value">{{ formattedEvalTime }}</strong>
</p>
<p v-if="data.last_eval_best_score != null" class="card-metric">
<span class="metric-label">Best score: </span>
<strong class="metric-value">{{ formatScore(data.last_eval_best_score) }}</strong>
</p>
</div>
<div v-if="data.signals.eval_to_train" class="card-cta">
<RouterLink to="/train/jobs" class="cta-btn">Queue Finetune</RouterLink>
</div>
<div v-if="data.signals.data_to_eval" class="card-cta">
<RouterLink to="/eval/benchmark" class="cta-btn">Run Eval</RouterLink>
</div>
</div>
<!-- ③ Train card -->
@ -111,49 +104,33 @@ interface DashboardSignals {
train_to_fleet: boolean
}
interface BenchRun {
timestamp: string | null
metric: string | null
score: number | null
}
type BenchType = 'classifier' | 'llm' | 'style' | 'plans'
interface DashboardData {
labeled_since_last_eval: number
last_eval_timestamp: string | null
last_eval_best_score: number | null
active_jobs: ActiveJob[]
corrections_export_ready: number
recent_bench_runs: Record<BenchType, BenchRun>
signals: DashboardSignals
}
const BENCH_LABELS: Record<BenchType, string> = {
classifier: 'Classifier',
llm: 'LLM Eval',
style: 'Style',
plans: 'Planning',
}
const data = ref<DashboardData | null>(null)
const loading = ref(false)
const error = ref<string | null>(null)
function formatBenchTs(ts: string): string {
const date = new Date(ts)
if (!isNaN(date.getTime())) {
const diff = Date.now() - date.getTime()
const formattedEvalTime = computed(() => {
if (!data.value?.last_eval_timestamp) return 'Never'
const date = new Date(data.value.last_eval_timestamp)
if (isNaN(date.getTime())) return 'Unknown'
const now = Date.now()
const diff = now - date.getTime()
const mins = Math.floor(diff / 60000)
if (mins < 1) return 'just now'
if (mins < 60) return `${mins}m ago`
const hrs = Math.floor(mins / 60)
if (hrs < 24) return `${hrs}h ago`
return `${Math.floor(hrs / 24)}d ago`
}
// Non-ISO: show as-is (plans bench uses "YYYY-MM-DD HH:MM")
return ts.length > 16 ? ts.slice(0, 16) : ts
}
const days = Math.floor(hrs / 24)
return `${days}d ago`
})
function formatScore(score: number): string {
return `${(score * 100).toFixed(1)}%`
@ -308,42 +285,6 @@ onMounted(() => load())
.cta-btn:hover { background: color-mix(in srgb, var(--app-primary, #2A6080) 85%, black); }
/* ── Bench run table ── */
.bench-run-table {
display: flex;
flex-direction: column;
gap: 0.3rem;
}
.bench-run-row {
display: grid;
grid-template-columns: 6rem 1fr auto;
align-items: center;
gap: 0.5rem;
font-size: 0.82rem;
}
.bench-type-label {
font-weight: 600;
color: var(--color-text, #1a2338);
font-size: 0.78rem;
}
.bench-run-time {
color: var(--color-text-secondary, #6b7a99);
font-size: 0.78rem;
}
.bench-run-score {
font-family: var(--font-mono, monospace);
font-size: 0.75rem;
font-weight: 600;
color: var(--app-primary, #2A6080);
background: color-mix(in srgb, var(--app-primary, #2A6080) 10%, transparent);
padding: 0.1rem 0.35rem;
border-radius: 0.25rem;
}
/* ── Job pills ── */
.job-row {
display: flex;

705
web/src/views/EmbedCompareTab.vue
View file

@ -1,705 +0,0 @@
<template>
<div class="embed-compare-page">
<!-- Step indicator (non-interactive) -->
<ol class="step-indicator" aria-label="Setup progress">
<li :class="{ complete: corpus.length > 0 }">Corpus</li>
<li :class="{ complete: queries.length > 0 }">Queries</li>
<li :class="{ complete: selectedModels.length > 0 }">Models</li>
<li :class="{ complete: hasResults }">Run &amp; Rate</li>
</ol>
<!-- Persistent aria-live region — always in DOM, never v-if -->
<div
ref="liveRegion"
class="sr-live"
aria-live="polite"
aria-atomic="true"
v-text="liveMessage"
/>
<!-- ① Corpus section -->
<section class="card" aria-labelledby="corpus-heading">
<h2 id="corpus-heading">① Corpus</h2>
<div class="corpus-controls">
<div class="field">
<label for="corpus-paste">Paste chunks (one per line)</label>
<textarea
id="corpus-paste"
v-model="rawCorpus"
rows="6"
placeholder="Paste one chunk per line, or use Import below..."
@change="onCorpusPaste"
/>
</div>
<div class="import-row">
<label for="imitate-product-select">Import from product</label>
<select id="imitate-product-select" v-model="selectedProduct">
<option value="">-- select product --</option>
<option
v-for="p in imitateProducts"
:key="p.id"
:value="p.id"
>{{ p.name }}</option>
</select>
<button
class="btn-secondary"
:disabled="!selectedProduct || importing"
@click="importCorpus"
>
{{ importing ? 'Importing…' : 'Import' }}
</button>
<span v-if="importError" class="error-text" role="alert">{{ importError }}</span>
</div>
<p v-if="corpus.length > 0" class="corpus-count">
{{ corpus.length }} chunk{{ corpus.length === 1 ? '' : 's' }} loaded.
</p>
</div>
</section>
<!-- ② Queries section -->
<section class="card" aria-labelledby="queries-heading">
<h2 id="queries-heading">② Queries</h2>
<div class="field">
<label for="query-input">Enter queries (one per line)</label>
<textarea
id="query-input"
v-model="rawQueries"
rows="4"
placeholder="One query per line..."
@change="onQueriesChange"
/>
</div>
<p v-if="queries.length > 0" class="query-count">
{{ queries.length }} quer{{ queries.length === 1 ? 'y' : 'ies' }}.
</p>
</section>
<!-- ③ Model selection -->
<section class="card" aria-labelledby="models-heading">
<h2 id="models-heading">③ Models</h2>
<p v-if="loadingModels" class="muted">Loading models from Ollama…</p>
<p v-else-if="modelsError" class="error-text" role="alert">{{ modelsError }}</p>
<ul v-else class="model-list" role="list">
<li v-for="m in availableModels" :key="m.name">
<label class="model-checkbox">
<input
type="checkbox"
:value="m.name"
v-model="selectedModels"
/>
{{ m.name }}
<span class="model-size muted" aria-label="model size">
{{ formatBytes(m.size) }}
</span>
</label>
</li>
</ul>
<p v-if="availableModels.length === 0 && !loadingModels && !modelsError" class="muted">
No Ollama models found. Pull an embedding model first.
</p>
</section>
<!-- ④ Run controls -->
<section class="card run-controls" aria-labelledby="run-heading">
<h2 id="run-heading">④ Run</h2>
<div class="run-row">
<div class="field-inline">
<label for="top-k-input">Results per query</label>
<input
id="top-k-input"
type="number"
v-model.number="topK"
min="1"
max="20"
style="width: 5rem"
/>
</div>
<button
class="btn-primary"
:disabled="!canRun || running"
@click="startRun"
>
{{ running ? 'Running…' : 'Run' }}
</button>
<button
v-if="running"
class="btn-danger"
aria-label="Cancel embedding run"
@click="cancelRun"
>
Cancel
</button>
</div>
<p v-if="!canRun && !running" class="muted">
Fill corpus, at least one query, and select at least one model to run.
</p>
</section>
<!-- Results -->
<section
v-if="hasResults"
class="card results-section"
aria-labelledby="results-heading"
>
<h2 id="results-heading">Results</h2>
<!-- Query pagination -->
<div class="query-nav" role="navigation" aria-label="Query navigation">
<button
class="btn-secondary"
aria-label="Previous query"
:disabled="currentQueryIdx === 0"
@click="currentQueryIdx--"
>‹</button>
<span class="query-counter">
Query {{ currentQueryIdx + 1 }} of {{ uniqueQueries.length }}:
<em>{{ uniqueQueries[currentQueryIdx] }}</em>
</span>
<button
class="btn-secondary"
aria-label="Next query"
:disabled="currentQueryIdx >= uniqueQueries.length - 1"
@click="currentQueryIdx++"
>›</button>
</div>
<!-- Results table: one column per model -->
<div class="table-wrap">
<table class="results-table">
<thead>
<tr>
<th scope="col" class="rank-col">#</th>
<th
v-for="model in selectedModels"
:key="model"
scope="col"
>{{ model }}</th>
</tr>
</thead>
<tbody>
<tr v-for="rank in topK" :key="rank">
<td class="rank-col muted">{{ rank }}</td>
<td
v-for="model in selectedModels"
:key="model"
class="hit-cell"
>
<template v-if="getHit(currentQueryIdx, model, rank - 1) as hit">
<div class="hit-text">{{ hit.text }}</div>
<!-- Visual score bar: decorative only -->
<div class="score-row">
<div class="score-bar-wrap" aria-hidden="true">
<div class="score-bar" :style="{ width: `${hit.score * 100}%` }" />
</div>
<span class="score-label">{{ hit.score.toFixed(3) }}</span>
</div>
<!-- Rating buttons -->
<div class="rating-row">
<button
class="rate-btn"
:class="{ active: getRating(currentQueryIdx, model, hit.chunk_idx) === 'relevant' }"
:aria-pressed="getRating(currentQueryIdx, model, hit.chunk_idx) === 'relevant'"
aria-label="Mark as relevant"
@click="rate(currentQueryIdx, model, hit, 'relevant')"
>
👍 Relevant
</button>
<button
class="rate-btn rate-btn-neg"
:class="{ active: getRating(currentQueryIdx, model, hit.chunk_idx) === 'not_relevant' }"
:aria-pressed="getRating(currentQueryIdx, model, hit.chunk_idx) === 'not_relevant'"
aria-label="Mark as not relevant"
@click="rate(currentQueryIdx, model, hit, 'not_relevant')"
>
👎 Not relevant
</button>
</div>
</template>
<span v-else class="muted">—</span>
</td>
</tr>
</tbody>
</table>
</div>
</section>
<!-- Export -->
<section
v-if="hasResults"
class="card export-section"
aria-labelledby="export-heading"
>
<h2 id="export-heading">Export Ratings</h2>
<div class="export-row">
<fieldset class="export-format-group">
<legend>Format</legend>
<label><input type="radio" v-model="exportFormat" value="csv" /> CSV</label>
<label><input type="radio" v-model="exportFormat" value="json" /> JSON</label>
</fieldset>
<button class="btn-secondary" @click="exportRatings">Export</button>
</div>
</section>
</div>
</template>
<script setup lang="ts">
import { ref, computed, onMounted } from 'vue'
// ── Types ─────────────────────────────────────────────────────────────────────
interface OllamaModel { name: string; size: number }
interface ImitateProduct { id: string; name: string }
interface HitResult { chunk_idx: number; text: string; score: number }
interface ResultEvent {
type: 'result'
query_idx: number
query: string
model: string
hits: HitResult[]
}
// ── State ─────────────────────────────────────────────────────────────────────
const rawCorpus = ref('')
const corpus = ref<string[]>([])
const rawQueries = ref('')
const queries = ref<string[]>([])
const selectedModels = ref<string[]>([])
const topK = ref(5)
const availableModels = ref<OllamaModel[]>([])
const loadingModels = ref(false)
const modelsError = ref('')
const imitateProducts = ref<ImitateProduct[]>([])
const selectedProduct = ref('')
const importing = ref(false)
const importError = ref('')
const running = ref(false)
const liveMessage = ref('')
const resultEvents = ref<ResultEvent[]>([])
const runController = ref<AbortController | null>(null)
const currentQueryIdx = ref(0)
const exportFormat = ref<'csv' | 'json'>('csv')
type RatingMap = Record<string, Record<string, Record<number, 'relevant' | 'not_relevant'>>>
const ratings = ref<RatingMap>({})
const uniqueQueries = computed(() => {
const seen = new Set<string>()
const out: string[] = []
for (const e of resultEvents.value) {
if (!seen.has(e.query)) { seen.add(e.query); out.push(e.query) }
}
return out
})
const hasResults = computed(() => resultEvents.value.length > 0)
const canRun = computed(
() => corpus.value.length > 0 && queries.value.length > 0 && selectedModels.value.length > 0
)
// ── Corpus helpers ────────────────────────────────────────────────────────────
function onCorpusPaste() {
const chunks = rawCorpus.value.split('\n').map(l => l.trim()).filter(Boolean)
corpus.value = chunks
if (chunks.length > 0) {
liveMessage.value = `${chunks.length} chunk${chunks.length === 1 ? '' : 's'} loaded.`
}
}
function onQueriesChange() {
queries.value = rawQueries.value.split('\n').map(l => l.trim()).filter(Boolean)
}
async function importCorpus() {
if (!selectedProduct.value) return
importing.value = true
importError.value = ''
try {
const r = await fetch(`/api/imitate/products/${selectedProduct.value}/sample-chunks`)
if (!r.ok) {
const text = await r.text()
throw new Error(text || `HTTP ${r.status}`)
}
const data = await r.json() as { chunks?: string[] }
const chunks = data.chunks ?? []
corpus.value = chunks
rawCorpus.value = chunks.join('\n')
liveMessage.value = `${chunks.length} chunk${chunks.length === 1 ? '' : 's'} loaded from import.`
} catch (err) {
importError.value = String(err)
} finally {
importing.value = false
}
}
// ── Model loading ─────────────────────────────────────────────────────────────
async function loadModels() {
loadingModels.value = true
modelsError.value = ''
try {
const r = await fetch('/api/embed-bench/models')
if (!r.ok) throw new Error(`HTTP ${r.status}`)
const data = await r.json() as { models: OllamaModel[] }
availableModels.value = data.models
} catch (err) {
modelsError.value = `Failed to load models: ${err}`
} finally {
loadingModels.value = false
}
}
// ── Run ───────────────────────────────────────────────────────────────────────
async function startRun() {
if (!canRun.value) return
running.value = true
resultEvents.value = []
liveMessage.value = 'Starting embedding run…'
runController.value = new AbortController()
try {
const resp = await fetch('/api/embed-bench/run', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
corpus: corpus.value,
queries: queries.value,
models: selectedModels.value,
top_k: topK.value,
}),
signal: runController.value.signal,
})
const reader = resp.body!.getReader()
const decoder = new TextDecoder()
let buf = ''
while (true) {
const { done, value } = await reader.read()
if (done) break
buf += decoder.decode(value, { stream: true })
const lines = buf.split('\n')
buf = lines.pop() ?? ''
for (const line of lines) {
if (!line.startsWith('data: ')) continue
const event = JSON.parse(line.slice(6))
if (event.type === 'progress') {
liveMessage.value = event.msg
} else if (event.type === 'result') {
resultEvents.value.push(event as ResultEvent)
} else if (event.type === 'done') {
liveMessage.value = 'Run complete.'
} else if (event.type === 'error') {
liveMessage.value = `Error: ${event.msg}`
}
}
}
} catch (err) {
if ((err as Error).name !== 'AbortError') {
liveMessage.value = `Run failed: ${err}`
}
} finally {
running.value = false
runController.value = null
}
}
function cancelRun() {
runController.value?.abort()
liveMessage.value = 'Run cancelled.'
}
// ── Utilities ─────────────────────────────────────────────────────────────────
function formatBytes(bytes: number): string {
if (bytes < 1_000_000) return `${(bytes / 1000).toFixed(0)} KB`
if (bytes < 1_000_000_000) return `${(bytes / 1_000_000).toFixed(0)} MB`
return `${(bytes / 1_000_000_000).toFixed(1)} GB`
}
function getHit(queryIdx: number, model: string, rank: number): HitResult | null {
const query = uniqueQueries.value[queryIdx]
if (!query) return null
const ev = resultEvents.value.find(e => e.query === query && e.model === model)
return ev?.hits[rank] ?? null
}
function getRating(queryIdx: number, model: string, chunkIdx: number): string | undefined {
const query = uniqueQueries.value[queryIdx]
return ratings.value[query]?.[model]?.[chunkIdx]
}
async function rate(
queryIdx: number,
model: string,
hit: HitResult,
rating: 'relevant' | 'not_relevant',
) {
const query = uniqueQueries.value[queryIdx]
// Optimistic update
if (!ratings.value[query]) ratings.value[query] = {}
if (!ratings.value[query][model]) ratings.value[query][model] = {}
ratings.value[query][model][hit.chunk_idx] = rating
try {
await fetch('/api/embed-bench/rate', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
query,
model,
chunk_text: hit.text,
chunk_idx: hit.chunk_idx,
rating,
}),
})
liveMessage.value = `Rated chunk ${hit.chunk_idx + 1} as ${rating}.`
} catch (err) {
liveMessage.value = `Rating failed: ${err}`
}
}
async function exportRatings() {
const r = await fetch(`/api/embed-bench/export?format=${exportFormat.value}`)
if (!r.ok) {
liveMessage.value = `Export failed: HTTP ${r.status}`
return
}
const blob = await r.blob()
const disposition = r.headers.get('Content-Disposition') ?? ''
const filenameMatch = disposition.match(/filename="([^"]+)"/)
const filename = filenameMatch ? filenameMatch[1] : `embed_comparison.${exportFormat.value}`
const url = URL.createObjectURL(blob)
const a = document.createElement('a')
a.href = url
a.download = filename
a.click()
URL.revokeObjectURL(url)
liveMessage.value = `Exported ${filename}.`
}
// ── Lifecycle ─────────────────────────────────────────────────────────────────
onMounted(() => {
loadModels()
})
</script>
<style scoped>
.embed-compare-page {
padding: var(--space-4, 1.5rem);
max-width: 1100px;
}
/* Step indicator */
.step-indicator {
display: flex;
gap: 0;
list-style: none;
margin: 0 0 var(--space-4, 1.5rem);
padding: 0;
border-bottom: 2px solid var(--color-border, #d0d7e8);
}
.step-indicator li {
padding: 0.4rem 1rem;
font-size: 0.8rem;
font-weight: 600;
color: var(--color-text-muted, #4a5c7a);
text-transform: uppercase;
letter-spacing: 0.05em;
border-bottom: 2px solid transparent;
margin-bottom: -2px;
}
.step-indicator li.complete {
color: var(--app-primary, #2A6080);
border-bottom-color: var(--app-primary, #2A6080);
}
/* Accessibility: screen-reader live region — visually hidden but always present */
.sr-live {
position: absolute;
width: 1px; height: 1px;
overflow: hidden;
clip: rect(0 0 0 0);
white-space: nowrap;
}
/* Cards */
.card {
background: var(--color-surface-raised, #e4ebf5);
border: 1px solid var(--color-border, #d0d7e8);
border-radius: var(--radius-md, 0.5rem);
padding: var(--space-4, 1.5rem);
margin-bottom: var(--space-4, 1.5rem);
}
.card h2 {
font-size: 1rem;
font-weight: 700;
margin: 0 0 var(--space-3, 1rem);
color: var(--color-text, #1a2338);
}
.field { display: flex; flex-direction: column; gap: 0.3rem; margin-bottom: 0.75rem; }
.field label { font-size: 0.85rem; font-weight: 600; }
textarea, input[type="number"] {
border: 1px solid var(--color-border, #d0d7e8);
border-radius: var(--radius-sm, 0.25rem);
padding: 0.5rem;
font-size: 0.875rem;
background: var(--color-surface, #f0f4fb);
color: var(--color-text, #1a2338);
resize: vertical;
}
.corpus-controls { display: flex; flex-direction: column; gap: 0.5rem; }
.import-row {
display: flex; flex-wrap: wrap; gap: 0.5rem; align-items: center;
}
.import-row label { font-size: 0.85rem; font-weight: 600; }
.corpus-count, .query-count { font-size: 0.875rem; color: var(--app-primary, #2A6080); margin: 0; }
.model-list { list-style: none; padding: 0; margin: 0; display: flex; flex-wrap: wrap; gap: 0.5rem; }
.model-checkbox {
display: flex; align-items: center; gap: 0.4rem;
font-size: 0.875rem; cursor: pointer;
padding: 0.3rem 0.6rem;
border: 1px solid var(--color-border, #d0d7e8);
border-radius: var(--radius-sm, 0.25rem);
background: var(--color-surface, #f0f4fb);
}
.model-size { font-size: 0.75rem; }
.run-row { display: flex; flex-wrap: wrap; gap: 0.75rem; align-items: flex-end; }
.field-inline { display: flex; align-items: center; gap: 0.4rem; }
.field-inline label { font-size: 0.85rem; font-weight: 600; white-space: nowrap; }
.btn-primary, .btn-secondary, .btn-danger {
padding: 0.4rem 1rem;
border-radius: var(--radius-sm, 0.25rem);
border: 1px solid transparent;
font-size: 0.875rem;
font-weight: 600;
cursor: pointer;
transition: background 0.15s;
}
.btn-primary { background: var(--app-primary, #2A6080); color: #fff; }
.btn-primary:hover:not(:disabled) { filter: brightness(1.1); }
.btn-primary:disabled { opacity: 0.5; cursor: not-allowed; }
.btn-secondary { background: var(--color-surface, #f0f4fb); color: var(--color-text, #1a2338); border-color: var(--color-border, #d0d7e8); }
.btn-secondary:hover:not(:disabled) { background: var(--color-border, #d0d7e8); }
.btn-secondary:disabled { opacity: 0.5; cursor: not-allowed; }
.btn-danger { background: var(--color-error, #c0392b); color: #fff; }
.muted { color: var(--color-text-muted, #4a5c7a); font-size: 0.875rem; }
.error-text { color: var(--color-error, #c0392b); font-size: 0.875rem; }
@media (max-width: 768px) {
.import-row { flex-direction: column; align-items: flex-start; }
.run-row { flex-direction: column; }
.model-list { flex-direction: column; }
}
/* Results table */
.table-wrap { overflow-x: auto; }
.results-table {
width: 100%;
border-collapse: collapse;
font-size: 0.875rem;
}
.results-table thead th {
position: sticky;
top: 0;
background: var(--color-surface-raised, #e4ebf5);
border-bottom: 2px solid var(--color-border, #d0d7e8);
padding: 0.5rem 0.75rem;
text-align: left;
font-weight: 700;
white-space: nowrap;
z-index: 1;
}
.results-table td {
padding: 0.5rem 0.75rem;
vertical-align: top;
border-bottom: 1px solid var(--color-border, #d0d7e8);
}
.rank-col { width: 2rem; text-align: center; }
.hit-text { margin-bottom: 0.25rem; line-height: 1.4; }
.score-row { display: flex; align-items: center; gap: 0.4rem; margin-bottom: 0.25rem; }
.score-bar-wrap {
flex: 1;
height: 6px;
background: var(--color-border, #d0d7e8);
border-radius: 3px;
overflow: hidden;
}
.score-bar {
height: 100%;
background: var(--app-primary, #2A6080);
border-radius: 3px;
transition: width 0.3s ease;
}
.score-label { font-size: 0.75rem; color: var(--color-text-muted, #4a5c7a); min-width: 3rem; text-align: right; }
.rating-row { display: flex; gap: 0.4rem; flex-wrap: wrap; }
.rate-btn {
padding: 0.2rem 0.5rem;
border: 1px solid var(--color-border, #d0d7e8);
border-radius: var(--radius-sm, 0.25rem);
background: var(--color-surface, #f0f4fb);
color: var(--color-text, #1a2338);
font-size: 0.75rem;
cursor: pointer;
transition: background 0.15s, border-color 0.15s;
}
.rate-btn.active {
background: color-mix(in srgb, var(--app-primary, #2A6080) 20%, transparent);
border-color: var(--app-primary, #2A6080);
font-weight: 700;
}
.rate-btn-neg.active {
background: color-mix(in srgb, var(--color-error, #c0392b) 15%, transparent);
border-color: var(--color-error, #c0392b);
}
/* Query nav */
.query-nav {
display: flex;
align-items: center;
gap: 0.5rem;
margin-bottom: 0.75rem;
flex-wrap: wrap;
}
.query-counter { font-size: 0.875rem; flex: 1; }
/* Export */
.export-row { display: flex; gap: 1rem; align-items: center; flex-wrap: wrap; }
.export-format-group {
border: none;
padding: 0;
display: flex;
gap: 0.75rem;
}
.export-format-group legend {
font-size: 0.85rem;
font-weight: 600;
margin-bottom: 0.25rem;
float: left;
margin-right: 0.5rem;
}
.export-format-group label { font-size: 0.875rem; display: flex; align-items: center; gap: 0.3rem; }
@media (max-width: 768px) {
.results-table thead th,
.results-table td { padding: 0.35rem 0.4rem; font-size: 0.8rem; }
.query-nav { flex-direction: column; align-items: flex-start; }
}
@media (prefers-reduced-motion: reduce) {
.score-bar { transition: none; }
}
</style>

7
web/src/views/EmbedCompareView.vue
View file

@ -1,7 +0,0 @@
<template>
<EmbedCompareTab />
</template>
<script setup lang="ts">
import EmbedCompareTab from './EmbedCompareTab.vue'
</script>

4
web/src/views/LlmEvalTab.vue
View file

@ -302,7 +302,7 @@ const llmModelBadge = computed(() => {
const llmTaskTypeCols = computed(() => {
const types = new Set<string>()
for (const r of llmResults.value) {
for (const k of Object.keys(r.quality_by_task_type ?? {})) types.add(k)
for (const k of Object.keys(r.quality_by_task_type)) types.add(k)
}
return [...types].sort()
})
@ -338,7 +338,7 @@ const llmBestByCol = computed((): Record<string, string> => {
for (const col of llmTaskTypeCols.value) {
bestId = ''; bestVal = -Infinity
for (const r of llmResults.value) {
const v = r.quality_by_task_type?.[col]
const v = r.quality_by_task_type[col]
if (v != null && v > bestVal) { bestVal = v; bestId = r.model_id }
}
best[col] = bestId

57
web/src/views/ModelsView.vue
View file

@ -2,24 +2,6 @@
<div class="models-view">
<h1 class="page-title">🤗 Models</h1>
<!-- ── Fleet tab bar ─────────────────────────────── -->
<div class="mode-toggle" role="group" aria-label="Fleet view">
<button
class="mode-btn"
:class="{ active: fleetTab === 'models' }"
@click="fleetTab = 'models'"
>Models</button>
<button
class="mode-btn"
:class="{ active: fleetTab === 'assignments' }"
@click="fleetTab = 'assignments'"
>Assignments</button>
</div>
<AssignmentsTab v-if="fleetTab === 'assignments'" />
<template v-if="fleetTab === 'models'">
<!-- ── 1. HF Lookup ───────────────────────────────── -->
<section class="section">
<h2 class="section-title">HuggingFace Lookup</h2>
@ -315,17 +297,11 @@
</div>
</template>
</section>
</template><!-- end fleetTab === 'models' -->
</div>
</template>
<script setup lang="ts">
import { ref, computed, onMounted, onUnmounted } from 'vue'
import AssignmentsTab from './AssignmentsTab.vue'
type FleetTab = 'models' | 'assignments'
const fleetTab = ref<FleetTab>('models')
// ── Type definitions ──────────────────────────────────
@ -762,39 +738,6 @@ onUnmounted(() => {
color: var(--color-primary, #2d5a27);
}
/* ── Fleet tab bar (mode-toggle pattern from BenchmarkView) ── */
.mode-toggle {
display: inline-flex;
border: 1px solid var(--color-border, #d0d7e8);
border-radius: 0.5rem;
overflow: hidden;
align-self: flex-start;
}
.mode-btn {
padding: 0.4rem 1.1rem;
font-size: 0.85rem;
font-family: var(--font-body, sans-serif);
font-weight: 500;
border: none;
background: var(--color-surface, #fff);
color: var(--color-text-secondary, #6b7a99);
cursor: pointer;
transition: background 0.15s, color 0.15s;
}
.mode-btn:not(:last-child) {
border-right: 1px solid var(--color-border, #d0d7e8);
}
.mode-btn.active {
background: var(--app-primary, #2A6080);
color: #fff;
}
.mode-btn:not(.active):hover {
background: var(--color-surface-raised, #e4ebf5);
}
@media (max-width: 600px) {
.mode-btn { padding: 0.4rem 0.65rem; font-size: 0.78rem; }
}
/* ── Sections ── */
.section {
display: flex;

120
web/src/views/NodeManagementView.vue
View file

@ -1,12 +1,8 @@
<script setup lang="ts">
import { ref, onMounted } from 'vue'
import NodeCard from '../components/nodes/NodeCard.vue'
import AssignmentsTab from './AssignmentsTab.vue'
import type { NodeSummary } from '../types/nodes'
type Tab = 'nodes' | 'assignments'
const activeTab = ref<Tab>('nodes')
const nodes = ref<NodeSummary[]>([])
const loading = ref(true)
const error = ref('')
@ -29,39 +25,12 @@ onMounted(fetchNodes)
</script>
<template>
<main class="fleet-page">
<header class="fleet-header">
<h1 class="fleet-title">Fleet</h1>
<main class="nodes-page">
<header class="nodes-header">
<h1>Nodes</h1>
<button class="btn-secondary" @click="fetchNodes" :disabled="loading">Refresh</button>
</header>
<!-- Tab bar -->
<nav class="tab-bar" role="tablist" aria-label="Fleet sections">
<button
id="tab-nodes"
role="tab"
:aria-selected="activeTab === 'nodes'"
:class="['tab', { active: activeTab === 'nodes' }]"
@click="activeTab = 'nodes'"
>Nodes</button>
<button
id="tab-assignments"
role="tab"
:aria-selected="activeTab === 'assignments'"
:class="['tab', { active: activeTab === 'assignments' }]"
@click="activeTab = 'assignments'"
>Assignments</button>
</nav>
<!-- Nodes tab -->
<section
v-if="activeTab === 'nodes'"
role="tabpanel"
aria-labelledby="tab-nodes"
class="tab-panel"
>
<div class="nodes-toolbar">
<button class="btn-secondary btn-sm" @click="fetchNodes" :disabled="loading">Refresh</button>
</div>
<div aria-live="polite" aria-atomic="true" class="sr-announce">
<span v-if="loading">Loading nodes...</span>
</div>
@ -77,89 +46,24 @@ onMounted(fetchNodes)
@updated="fetchNodes"
/>
</div>
</section>
<!-- Assignments tab -->
<section
v-else-if="activeTab === 'assignments'"
role="tabpanel"
aria-labelledby="tab-assignments"
class="tab-panel"
>
<AssignmentsTab />
</section>
</main>
</template>
<style scoped>
.fleet-page { padding: 1.5rem; }
.fleet-header {
margin-bottom: 1rem;
}
.fleet-title {
margin: 0;
font-size: 1.5rem;
color: var(--color-text);
}
/* ── Tab bar ── */
.tab-bar {
.nodes-page { padding: 1.5rem; }
.nodes-header {
display: flex;
gap: 0;
border-bottom: 2px solid var(--color-border);
margin-bottom: 1.25rem;
align-items: center;
justify-content: space-between;
margin-bottom: 1.5rem;
}
.tab {
padding: 0.55rem 1.1rem;
font-size: 0.88rem;
font-weight: 600;
background: none;
border: none;
border-bottom: 2px solid transparent;
margin-bottom: -2px;
cursor: pointer;
color: var(--color-text-muted);
transition: color 0.15s, border-color 0.15s;
}
.tab:hover { color: var(--color-text); }
.tab.active {
color: var(--app-primary);
border-bottom-color: var(--app-primary);
}
/* ── Tab panel ── */
.tab-panel { min-height: 200px; }
/* ── Nodes toolbar ── */
.nodes-toolbar {
display: flex;
justify-content: flex-end;
margin-bottom: 1rem;
}
/* ── Nodes grid / status ── */
.nodes-header h1 { margin: 0; font-size: 1.5rem; }
.nodes-grid { display: flex; flex-direction: column; gap: 1.5rem; }
.nodes-status {
color: var(--color-text-muted);
color: var(--text-secondary, #888);
padding: 2rem;
text-align: center;
}
.nodes-error { color: var(--color-error); }
.nodes-error { color: var(--color-error, #fc8181); }
.sr-announce { min-height: 1.2em; }
/* ── Shared button ── */
.btn-secondary {
padding: 0.4rem 0.9rem;
background: var(--color-surface-alt);
border: 1px solid var(--color-border);
border-radius: 0.4rem;
font-size: 0.85rem;
color: var(--color-text);
cursor: pointer;
transition: background 0.15s;
}
.btn-secondary:hover:not(:disabled) { background: var(--color-surface-raised); }
.btn-secondary:disabled { opacity: 0.5; cursor: default; }
.btn-sm { padding: 0.3rem 0.65rem; font-size: 0.8rem; }
</style>

536
web/src/views/RecipeScanView.vue
View file

@ -1,536 +0,0 @@
<template>
<div class="rsv">
<!-- Header -->
<header class="rsv-header">
<h1 class="rsv-title">Recipe Scan Review</h1>
<div class="rsv-stats" v-if="stats">
<span class="stat-chip">{{ stats.by_status?.pending ?? 0 }} pending</span>
<span class="stat-chip stat-chip--ok">{{ stats.by_status?.approved ?? 0 }} approved</span>
<span class="stat-chip stat-chip--edited">{{ stats.by_status?.edited ?? 0 }} edited</span>
<span class="stat-chip stat-chip--bad">{{ stats.by_status?.rejected ?? 0 }} rejected</span>
<a
v-if="(stats.export_ready ?? 0) > 0"
:href="`${apiBase}/api/recipe-scan/export`"
download
class="btn-export"
>
⬇ Export {{ stats.export_ready }} pairs
</a>
</div>
</header>
<!-- Loading -->
<div v-if="loading" class="rsv-state" aria-label="Loading">
<div class="skeleton-block" />
</div>
<!-- Error -->
<div v-else-if="apiError" class="rsv-state rsv-error" role="alert">
<p>{{ apiError }}</p>
<button class="btn-action" @click="fetchNext">Retry</button>
</div>
<!-- Queue empty -->
<div v-else-if="!item" class="rsv-state rsv-empty">
<p>Queue is empty — all items reviewed.</p>
<p class="rsv-hint">Import items from the Kiwi pipeline to continue.</p>
</div>
<!-- Review panel -->
<div v-else class="rsv-workspace">
<!-- Left: image -->
<section class="rsv-image-panel" aria-label="Scan image">
<div class="rsv-panel-label">
<span class="modality-badge">{{ item.modality }}</span>
<span class="source-badge">{{ item.source }}</span>
</div>
<div class="rsv-image-wrap">
<img
v-if="imageUrl"
:src="imageUrl"
:alt="`Recipe scan — ${item.source}`"
class="rsv-image"
/>
<div v-else class="rsv-image-placeholder">
<span>Image not available</span>
<code class="rsv-path">{{ item.image_path }}</code>
</div>
</div>
</section>
<!-- Right: JSON comparison -->
<section class="rsv-json-panel" aria-label="Extraction review">
<!-- Ground truth (read-only reference) -->
<div class="rsv-json-block">
<h2 class="rsv-json-label">Ground truth <span class="label-tag">reference</span></h2>
<pre class="rsv-json rsv-json--ground-truth" tabindex="0" aria-label="Ground truth JSON">{{ prettyJson(item.ground_truth) }}</pre>
</div>
<!-- Extracted / editable -->
<div class="rsv-json-block">
<h2 class="rsv-json-label">
Extracted
<span class="label-tag label-tag--edit">edit before approving</span>
</h2>
<textarea
v-model="draftJson"
class="rsv-json rsv-json--edit"
spellcheck="false"
aria-label="Extracted JSON — edit to correct"
:class="{ 'rsv-json--invalid': jsonError }"
/>
<p v-if="jsonError" class="rsv-json-error" role="alert">{{ jsonError }}</p>
</div>
<!-- Actions -->
<div class="rsv-actions" role="group" aria-label="Review actions">
<button
class="btn-approve"
:disabled="acting"
@click="handleApprove"
title="Extracted JSON is accurate — approve as-is (A)"
>
✓ Approve
</button>
<button
class="btn-edit"
:disabled="acting || !!jsonError"
@click="handleEdit"
title="Approve the edited JSON in the text area (E)"
>
✎ Approve edited
</button>
<button
class="btn-reject"
:disabled="acting"
@click="handleReject"
title="Extraction too broken to use — reject (R)"
>
✕ Reject
</button>
</div>
</section>
</div>
<!-- Feedback toast -->
<Transition name="toast">
<div v-if="toast" class="rsv-toast" role="status" aria-live="polite">
{{ toast }}
</div>
</Transition>
</div>
</template>
<script setup lang="ts">
import { ref, computed, watch, onMounted, onUnmounted } from 'vue'
const apiBase = window.location.origin
interface RecipeScanItem {
id: string
image_path: string
modality: string
source: string
extracted: Record<string, unknown>
ground_truth: Record<string, unknown>
status: string
}
interface Stats {
total: number
by_status: Record<string, number>
by_modality: Record<string, number>
export_ready: number
}
const item = ref<RecipeScanItem | null>(null)
const stats = ref<Stats | null>(null)
const loading = ref(true)
const acting = ref(false)
const apiError = ref('')
const draftJson = ref('')
const toast = ref('')
let toastTimer: ReturnType<typeof setTimeout> | null = null
const jsonError = computed(() => {
if (!draftJson.value.trim()) return ''
try {
JSON.parse(draftJson.value)
return ''
} catch (e) {
return 'Invalid JSON — fix before approving'
}
})
const imageUrl = computed(() => {
if (!item.value) return ''
const encoded = encodeURIComponent(item.value.image_path)
return `${apiBase}/api/recipe-scan/image?path=${encoded}`
})
function prettyJson(obj: unknown): string {
return JSON.stringify(obj, null, 2)
}
function showToast(msg: string) {
toast.value = msg
if (toastTimer) clearTimeout(toastTimer)
toastTimer = setTimeout(() => { toast.value = '' }, 2500)
}
async function fetchNext() {
loading.value = true
apiError.value = ''
try {
const r = await fetch(`${apiBase}/api/recipe-scan/next`)
if (r.status === 404) {
item.value = null
} else if (!r.ok) {
throw new Error(`API error ${r.status}`)
} else {
item.value = await r.json()
draftJson.value = prettyJson(item.value!.extracted)
}
} catch (e) {
apiError.value = e instanceof Error ? e.message : 'Could not reach API'
} finally {
loading.value = false
}
}
async function fetchStats() {
try {
const r = await fetch(`${apiBase}/api/recipe-scan/stats`)
if (r.ok) stats.value = await r.json()
} catch { /* non-critical */ }
}
async function act(endpoint: string, body?: unknown) {
if (!item.value || acting.value) return
acting.value = true
try {
const r = await fetch(`${apiBase}/api/recipe-scan/items/${item.value.id}/${endpoint}`, {
method: 'POST',
headers: body ? { 'Content-Type': 'application/json' } : {},
body: body ? JSON.stringify(body) : undefined,
})
if (!r.ok) throw new Error(`API error ${r.status}`)
} catch (e) {
showToast(e instanceof Error ? e.message : 'Action failed')
acting.value = false
return
}
acting.value = false
await Promise.all([fetchNext(), fetchStats()])
}
async function handleApprove() {
showToast('Approved')
await act('approve')
}
async function handleEdit() {
if (jsonError.value) return
let corrected: unknown
try {
corrected = JSON.parse(draftJson.value)
} catch {
return
}
showToast('Saved edit')
await act('edit', { corrected })
}
async function handleReject() {
showToast('Rejected')
await act('reject')
}
// Keyboard shortcuts: A = approve, E = edit+approve, R = reject
function handleKey(e: KeyboardEvent) {
const tag = (e.target as HTMLElement)?.tagName?.toLowerCase()
if (tag === 'textarea' || tag === 'input') return
if (e.key === 'a' || e.key === 'A') handleApprove()
if (e.key === 'e' || e.key === 'E') handleEdit()
if (e.key === 'r' || e.key === 'R') handleReject()
}
watch(item, (newItem) => {
if (newItem) draftJson.value = prettyJson(newItem.extracted)
})
onMounted(() => {
fetchNext()
fetchStats()
window.addEventListener('keydown', handleKey)
})
onUnmounted(() => {
window.removeEventListener('keydown', handleKey)
if (toastTimer) clearTimeout(toastTimer)
})
</script>
<style scoped>
.rsv {
display: flex;
flex-direction: column;
height: 100%;
padding: var(--space-md, 1rem);
gap: var(--space-md, 1rem);
box-sizing: border-box;
overflow: hidden;
}
/* Header */
.rsv-header {
display: flex;
align-items: center;
gap: var(--space-md, 1rem);
flex-wrap: wrap;
}
.rsv-title {
font-size: 1.1rem;
font-weight: 600;
margin: 0;
color: var(--color-text, #fff);
}
.rsv-stats {
display: flex;
align-items: center;
gap: 0.5rem;
flex-wrap: wrap;
}
.stat-chip {
font-size: 0.75rem;
padding: 2px 8px;
border-radius: 12px;
background: var(--color-surface-alt, #2a2a2a);
color: var(--color-text-muted, #aaa);
}
.stat-chip--ok { background: #1a3a1a; color: #6fcf97; }
.stat-chip--edited { background: #2a2a00; color: #f2c94c; }
.stat-chip--bad { background: #3a1a1a; color: #eb5757; }
.btn-export {
font-size: 0.8rem;
padding: 4px 12px;
border-radius: 6px;
background: var(--color-accent, #4a9eff);
color: #fff;
text-decoration: none;
}
/* State panels */
.rsv-state {
flex: 1;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 0.5rem;
color: var(--color-text-muted, #aaa);
}
.rsv-error { color: var(--color-danger, #eb5757); }
.rsv-empty { font-size: 1rem; }
.rsv-hint { font-size: 0.85rem; opacity: 0.7; margin: 0; }
.skeleton-block {
width: 100%; height: 300px;
border-radius: 8px;
background: var(--color-surface-alt, #2a2a2a);
animation: pulse 1.5s ease-in-out infinite;
}
@keyframes pulse { 0%, 100% { opacity: 1; } 50% { opacity: 0.5; } }
/* Workspace: two-column layout */
.rsv-workspace {
flex: 1;
display: grid;
grid-template-columns: 1fr 1fr;
gap: var(--space-md, 1rem);
min-height: 0;
overflow: hidden;
}
@media (max-width: 900px) {
.rsv-workspace {
grid-template-columns: 1fr;
overflow-y: auto;
}
}
/* Image panel */
.rsv-image-panel {
display: flex;
flex-direction: column;
gap: 0.5rem;
min-height: 0;
}
.rsv-panel-label {
display: flex;
gap: 0.5rem;
}
.modality-badge, .source-badge {
font-size: 0.72rem;
padding: 2px 8px;
border-radius: 10px;
background: var(--color-surface-alt, #2a2a2a);
color: var(--color-text-muted, #aaa);
text-transform: uppercase;
letter-spacing: 0.04em;
}
.rsv-image-wrap {
flex: 1;
display: flex;
align-items: center;
justify-content: center;
background: var(--color-surface-alt, #111);
border-radius: 8px;
overflow: hidden;
min-height: 200px;
}
.rsv-image {
max-width: 100%;
max-height: 100%;
object-fit: contain;
}
.rsv-image-placeholder {
display: flex;
flex-direction: column;
align-items: center;
gap: 0.5rem;
color: var(--color-text-muted, #666);
font-size: 0.85rem;
padding: 1rem;
text-align: center;
}
.rsv-path {
font-size: 0.7rem;
word-break: break-all;
opacity: 0.6;
}
/* JSON panel */
.rsv-json-panel {
display: flex;
flex-direction: column;
gap: 0.5rem;
min-height: 0;
overflow-y: auto;
}
.rsv-json-block {
display: flex;
flex-direction: column;
gap: 0.25rem;
flex: 1;
min-height: 0;
}
.rsv-json-label {
font-size: 0.8rem;
font-weight: 600;
color: var(--color-text-muted, #aaa);
margin: 0;
display: flex;
align-items: center;
gap: 0.5rem;
}
.label-tag {
font-size: 0.68rem;
font-weight: 400;
padding: 1px 6px;
border-radius: 8px;
background: var(--color-surface-alt, #2a2a2a);
color: var(--color-text-muted, #888);
}
.label-tag--edit {
background: #2a2a00;
color: #f2c94c;
}
.rsv-json {
font-family: var(--font-mono, monospace);
font-size: 0.75rem;
line-height: 1.5;
padding: 0.75rem;
border-radius: 6px;
min-height: 120px;
flex: 1;
overflow-y: auto;
resize: vertical;
white-space: pre;
}
.rsv-json--ground-truth {
background: var(--color-surface-alt, #111);
color: var(--color-text, #ccc);
border: 1px solid var(--color-border, #333);
}
.rsv-json--edit {
background: var(--color-surface, #1a1a1a);
color: var(--color-text, #e0e0e0);
border: 1px solid var(--color-border, #444);
caret-color: var(--color-accent, #4a9eff);
outline: none;
transition: border-color 0.15s;
}
.rsv-json--edit:focus {
border-color: var(--color-accent, #4a9eff);
}
.rsv-json--invalid {
border-color: var(--color-danger, #eb5757) !important;
}
.rsv-json-error {
font-size: 0.75rem;
color: var(--color-danger, #eb5757);
margin: 0;
}
/* Action buttons */
.rsv-actions {
display: flex;
gap: 0.5rem;
padding-top: 0.25rem;
flex-wrap: wrap;
}
.btn-approve, .btn-edit, .btn-reject {
flex: 1;
min-width: 80px;
padding: 0.5rem 0.75rem;
border: none;
border-radius: 6px;
font-size: 0.85rem;
font-weight: 600;
cursor: pointer;
transition: opacity 0.15s;
}
.btn-approve, .btn-edit, .btn-reject {
opacity: 1;
}
.btn-approve:disabled, .btn-edit:disabled, .btn-reject:disabled {
opacity: 0.4;
cursor: default;
}
.btn-approve { background: #1e6e1e; color: #6fcf97; }
.btn-approve:hover:not(:disabled) { background: #256325; }
.btn-edit { background: #4a4a00; color: #f2c94c; }
.btn-edit:hover:not(:disabled) { background: #606000; }
.btn-reject { background: #6e1e1e; color: #eb8f8f; }
.btn-reject:hover:not(:disabled) { background: #7a2222; }
/* Toast */
.rsv-toast {
position: fixed;
bottom: 1.5rem;
left: 50%;
transform: translateX(-50%);
background: var(--color-surface, #222);
color: var(--color-text, #fff);
border: 1px solid var(--color-border, #444);
border-radius: 8px;
padding: 0.5rem 1.25rem;
font-size: 0.85rem;
box-shadow: 0 4px 20px rgba(0,0,0,0.4);
pointer-events: none;
z-index: 100;
}
.toast-enter-active, .toast-leave-active { transition: opacity 0.2s, transform 0.2s; }
.toast-enter-from, .toast-leave-to { opacity: 0; transform: translateX(-50%) translateY(8px); }
</style>