Compare commits
21 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c123492a1c | |||
| 391ebb3cd1 | |||
| 9bb88b168f | |||
| 13ca082a43 | |||
| d416ef8aa4 | |||
| 79b9ccbd3d | |||
| e93afec271 | |||
| cac91dd8a2 | |||
| 2b990a603a | |||
| 9fdaeeb3d6 | |||
| 71bf88d09b | |||
| bc4ca1095c | |||
| b6aed3dd1b | |||
| 1ad7ba322a | |||
| 32e3b2a0dd | |||
| 12117ad0c6 | |||
| 5939c67b9f | |||
| 5ea77da97d | |||
| 276bdadb92 | |||
| 6f9aad126e | |||
| 258bbdc0af |
45 changed files with 6612 additions and 274 deletions
3
.gitignore
vendored
3
.gitignore
vendored
|
|
@ -8,6 +8,9 @@ __pycache__/
|
||||||
config/label_tool.yaml
|
config/label_tool.yaml
|
||||||
|
|
||||||
# Data files (user-generated, not for version control)
|
# Data files (user-generated, not for version control)
|
||||||
|
data/corpus.db
|
||||||
|
data/corpus.db-wal
|
||||||
|
data/corpus.db-shm
|
||||||
data/email_score.jsonl
|
data/email_score.jsonl
|
||||||
data/email_label_queue.jsonl
|
data/email_label_queue.jsonl
|
||||||
data/email_compare_sample.jsonl
|
data/email_compare_sample.jsonl
|
||||||
|
|
|
||||||
183
README.md
183
README.md
|
|
@ -1,22 +1,120 @@
|
||||||
# Avocet — Email Classifier Training Tool
|
<div align="center">
|
||||||
|
<img src="docs/avocet-logo.svg" alt="Avocet" height="96" />
|
||||||
|
|
||||||
> *Part of the CircuitForge LLC internal infrastructure suite.*
|
# Avocet
|
||||||
|
|
||||||
**Status:** Internal beta — label tool and benchmark harness complete. Used to build training data for Peregrine's email classifier.
|
**Email classifier training tool — label, benchmark, fine-tune.**
|
||||||
|
|
||||||
|
[]()
|
||||||
|
[](https://git.opensourcesolarpunk.com/Circuit-Forge/avocet/releases)
|
||||||
|
[](LICENSE)
|
||||||
|
[]()
|
||||||
|
[](https://circuitforge.tech)
|
||||||
|
</div>
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## What it does
|
## What is Avocet?
|
||||||
|
|
||||||
Avocet is the data pipeline for building and benchmarking email classifiers. It has two layers:
|
Avocet is the internal data pipeline Circuit Forge uses to build, evaluate, and fine-tune email classifiers. It implements a three-stage workflow: human labelers review emails one at a time in a drag-to-bucket UI and produce a ground-truth dataset; the benchmark harness scores any number of HuggingFace zero-shot models against that dataset and produces a ranked comparison; and the fine-tune harness adapts the best-scoring base model to the labeled distribution. The output feeds directly into Peregrine's email classification layer. No LLM API key required for the label tool or benchmark — all inference runs locally via HuggingFace Transformers.
|
||||||
|
|
||||||
**No LLM required.** Avocet uses zero-shot HuggingFace classification models — no API key, no cloud inference, no GPU required for the label tool. The benchmark harness can optionally export LLM-labeled emails from a Peregrine staging DB, but human labeling via the card-stack UI is the primary workflow.
|
---
|
||||||
|
|
||||||
**Layer 1 — Label tool**
|
## Quick Start
|
||||||
Card-stack UI for building ground-truth classifier benchmark data. Fetch emails from one or more IMAP accounts (with targeted date-range and sender/subject filters), review them card-by-card, and label each with a job-search category. Labeled output feeds the benchmark harness.
|
|
||||||
|
|
||||||
**Layer 2 — Benchmark harness**
|
```bash
|
||||||
Scores HuggingFace zero-shot classification models against the labeled dataset. Supports slow/large model inclusion, visual side-by-side comparison on live emails, and export of LLM-labeled emails from a Peregrine staging DB.
|
git clone https://git.opensourcesolarpunk.com/Circuit-Forge/avocet.git
|
||||||
|
cd avocet
|
||||||
|
|
||||||
|
# Copy config template and fill in your IMAP credentials
|
||||||
|
cp config/label_tool.yaml.example config/label_tool.yaml
|
||||||
|
|
||||||
|
# Start the label tool (Vue SPA + FastAPI, port 8503)
|
||||||
|
./manage.sh start
|
||||||
|
./manage.sh open
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- **Drag-to-bucket label UI** — ASMR-style card interface; drag emails into labeled buckets or discard without queuing noise into the training set
|
||||||
|
- **Targeted IMAP fetch** — pull emails by date range, sender, or subject filter across multiple accounts without flooding the queue
|
||||||
|
- **Email classifier benchmark** — score any HuggingFace zero-shot model against your labeled JSONL; side-by-side comparison on live IMAP emails
|
||||||
|
- **Planning benchmark** — evaluate LLMs on structured planning tasks; compare models head-to-head with verbose diff output
|
||||||
|
- **Writing style benchmark** — compare Ollama models on writing style coherence; scan local disk for existing outputs
|
||||||
|
- **Fine-tune harness** — HuggingFace Transformers fine-tuning from labeled ground truth; classifier adapter interface for swapping backends at runtime
|
||||||
|
- **Local inference first** — no API key required; GPU optional; designed to run on developer hardware
|
||||||
|
- **Hot-reload dev mode** — uvicorn `--reload` + Vite HMR (hot module replacement) for fast iteration on both API and UI
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## CLI Reference
|
||||||
|
|
||||||
|
All operations go through `manage.sh`.
|
||||||
|
|
||||||
|
### Label Tool
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./manage.sh start # Build Vue SPA and start FastAPI on port 8503
|
||||||
|
./manage.sh stop # Stop FastAPI server
|
||||||
|
./manage.sh restart # Stop, rebuild, and restart
|
||||||
|
./manage.sh status # Show running state and port
|
||||||
|
./manage.sh logs # Tail the API log
|
||||||
|
./manage.sh open # Open http://localhost:8503 in browser
|
||||||
|
./manage.sh dev # Hot-reload: uvicorn --reload + Vite HMR
|
||||||
|
./manage.sh test # Run pytest suite
|
||||||
|
```
|
||||||
|
|
||||||
|
### Email Classifier Benchmark
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./manage.sh benchmark [args] # Run benchmark_classifier.py
|
||||||
|
./manage.sh list-models # List available zero-shot models
|
||||||
|
./manage.sh score # Score models against labeled JSONL
|
||||||
|
./manage.sh score --include-slow # Include large/slow models
|
||||||
|
./manage.sh compare --limit 30 # Side-by-side comparison on live IMAP emails
|
||||||
|
```
|
||||||
|
|
||||||
|
### Planning Benchmark
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./manage.sh plans-bench [args] # Run benchmark_plans.py
|
||||||
|
./manage.sh plans-list # List available models
|
||||||
|
./manage.sh plans-run <model> [args] # Run a single model (verbose)
|
||||||
|
./manage.sh plans-compare <m1> <m2> [...] # Compare models side-by-side
|
||||||
|
```
|
||||||
|
|
||||||
|
### Writing Style Benchmark
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./manage.sh style-bench [args] # Run benchmark_style.py
|
||||||
|
./manage.sh style-list # List available Ollama models
|
||||||
|
./manage.sh style-run [args] # Run writing style benchmark
|
||||||
|
./manage.sh style-last # Print most recent benchmark report
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
IMAP accounts
|
||||||
|
→ fetch (targeted or wide)
|
||||||
|
→ email_label_queue.jsonl
|
||||||
|
|
||||||
|
email_label_queue.jsonl
|
||||||
|
→ label tool drag-to-bucket UI
|
||||||
|
→ email_score.jsonl (ground truth)
|
||||||
|
|
||||||
|
email_score.jsonl
|
||||||
|
→ benchmark harness
|
||||||
|
→ model rankings
|
||||||
|
|
||||||
|
best model
|
||||||
|
→ fine-tune harness
|
||||||
|
→ Peregrine classifier adapter
|
||||||
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -38,69 +136,42 @@ Scores HuggingFace zero-shot classification models against the labeled dataset.
|
||||||
|
|
||||||
## Stack
|
## Stack
|
||||||
|
|
||||||
| Layer | Tech |
|
| Layer | Technology |
|
||||||
|-------|------|
|
|-------|-----------|
|
||||||
| Label UI | Streamlit (port 8503, auto-increments on collision) |
|
| Label UI | Vue 3 SPA (Vite) |
|
||||||
|
| API | FastAPI + uvicorn (port 8503) |
|
||||||
| Benchmark | Python + HuggingFace Transformers |
|
| Benchmark | Python + HuggingFace Transformers |
|
||||||
| Email fetch | IMAP (multi-account, targeted date/sender/subject filter) |
|
| Email fetch | IMAP (multi-account, targeted date/sender/subject filter) |
|
||||||
| Data | JSONL (`data/email_label_queue.jsonl`, `data/email_score.jsonl`) |
|
| Data | JSONL (`data/email_label_queue.jsonl`, `data/email_score.jsonl`) |
|
||||||
| Config | `config/label_tool.yaml` (gitignored — see `.example`) |
|
| Runtime | SQLite |
|
||||||
|
| Config | `config/label_tool.yaml` (gitignored — `.example` committed) |
|
||||||
Conda environments:
|
|
||||||
- `job-seeker` — label tool UI
|
|
||||||
- `job-seeker-classifiers` — benchmark harness (separate env for heavy deps)
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Running
|
## Logo
|
||||||
|
|
||||||
```bash
|
The Avocet logo (`avocet_v1_poly.svg`) lives in the shared graphics repo. Copy it to `docs/avocet-logo.svg` to render correctly in this README.
|
||||||
./manage.sh start # start label tool UI (port collision-safe from 8503)
|
|
||||||
./manage.sh stop # stop
|
|
||||||
./manage.sh restart # restart
|
|
||||||
./manage.sh status # show running state and port
|
|
||||||
./manage.sh logs # tail label tool log
|
|
||||||
./manage.sh open # open in browser
|
|
||||||
```
|
|
||||||
|
|
||||||
Benchmark:
|
|
||||||
```bash
|
|
||||||
./manage.sh benchmark --list-models # list available zero-shot models
|
|
||||||
./manage.sh score # score models against labeled JSONL
|
|
||||||
./manage.sh score --include-slow # include large/slow models
|
|
||||||
./manage.sh compare --limit 30 # visual comparison on live IMAP emails
|
|
||||||
```
|
|
||||||
|
|
||||||
Dev:
|
|
||||||
```bash
|
|
||||||
./manage.sh test # run pytest suite
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Data flow
|
## About
|
||||||
|
|
||||||
```
|
Avocet is internal CircuitForge infrastructure, open source as a reference implementation. It is not a user-facing product. The primary consumer is [Peregrine](https://git.opensourcesolarpunk.com/Circuit-Forge/peregrine), CircuitForge's job-search pipeline tool.
|
||||||
IMAP accounts → fetch (targeted or wide) → email_label_queue.jsonl
|
|
||||||
→ label tool card UI → email_score.jsonl
|
|
||||||
→ benchmark harness → model rankings
|
|
||||||
→ best model → Peregrine classifier adapter
|
|
||||||
```
|
|
||||||
|
|
||||||
Targeted fetch: date range + sender/subject filter for pulling historical emails on specific senders or topics without flooding the queue.
|
Docs: [docs.circuitforge.tech/avocet](https://docs.circuitforge.tech/avocet)
|
||||||
|
|
||||||
Discard: removes an email from the queue without writing to the score file — for emails that don't belong in the training set.
|
## Forgejo-primary
|
||||||
|
|
||||||
---
|
Avocet is developed and maintained on Forgejo at [git.opensourcesolarpunk.com/Circuit-Forge/avocet](https://git.opensourcesolarpunk.com/Circuit-Forge/avocet). GitHub and Codeberg are read-only mirrors.
|
||||||
|
|
||||||
## Classifier adapters
|
|
||||||
|
|
||||||
`app/classifier_adapters.py` provides a common interface for swapping classifier backends. Falls back to the label name when no `LABEL_DESCRIPTIONS` entry is configured for a label (RerankerAdapter).
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
BSL 1.1 — internal tool, not user-facing.
|
[Business Source License 1.1](LICENSE) — classifier training is an AI feature under the CircuitForge licensing model.
|
||||||
|
|
||||||
© 2026 Circuit Forge LLC
|
Free for personal non-commercial self-hosting. Commercial use or SaaS re-hosting requires a paid license. Converts to MIT after 4 years.
|
||||||
|
|
||||||
|
Humans own design, architecture, code review, testing, and verification. LLMs are part of our development workflow. [Our positions on LLM use →](https://circuitforge.tech/positions)
|
||||||
|
|
||||||
|
© 2026 Circuit Forge LLC — Privacy · Safety · Accessibility
|
||||||
|
|
|
||||||
33
app/api.py
33
app/api.py
|
|
@ -40,6 +40,39 @@ app.include_router(plans_bench_router, prefix="/api/plans-bench")
|
||||||
# In-memory last-action store (single user, local tool — in-memory is fine)
|
# In-memory last-action store (single user, local tool — in-memory is fine)
|
||||||
_last_action: dict | None = None
|
_last_action: dict | None = None
|
||||||
|
|
||||||
|
# -- Backward-compat shims (ClassifierTab still uses old /api/finetune/* paths)
|
||||||
|
# Remove once ClassifierTab fine-tune section is migrated to TrainJobsView.
|
||||||
|
|
||||||
|
from fastapi import Query
|
||||||
|
from fastapi.responses import StreamingResponse as _StreamingResponse
|
||||||
|
|
||||||
|
@app.get("/api/finetune/run")
|
||||||
|
def finetune_run_compat(model: str = Query(...), epochs: int = Query(5)) -> _StreamingResponse:
|
||||||
|
"""Shim: create a classifier train job and immediately stream it."""
|
||||||
|
from app.train.train import create_job, run_job, CreateJobRequest
|
||||||
|
job = create_job(CreateJobRequest(type="classifier", model_key=model, config_json={"epochs": epochs}))
|
||||||
|
return run_job(job["id"])
|
||||||
|
|
||||||
|
@app.post("/api/finetune/cancel")
|
||||||
|
def finetune_cancel_compat() -> dict:
|
||||||
|
"""Shim: cancel the most recent running classifier job."""
|
||||||
|
from app.train.train import _db, _init_db, cancel_job
|
||||||
|
from fastapi import HTTPException
|
||||||
|
_init_db()
|
||||||
|
with _db() as conn:
|
||||||
|
row = conn.execute(
|
||||||
|
"SELECT id FROM jobs WHERE type='classifier' AND status='running' ORDER BY started_at DESC LIMIT 1"
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
return {"status": "nothing_running"}
|
||||||
|
return cancel_job(row["id"])
|
||||||
|
|
||||||
|
from app.data.log_corpus import router as log_corpus_router
|
||||||
|
app.include_router(log_corpus_router, prefix="/api/corpus")
|
||||||
|
|
||||||
|
from app.data.recipe_scan import router as recipe_scan_router
|
||||||
|
app.include_router(recipe_scan_router, prefix="/api/recipe-scan")
|
||||||
|
|
||||||
from app.dashboard import router as dashboard_router
|
from app.dashboard import router as dashboard_router
|
||||||
app.include_router(dashboard_router, prefix="/api")
|
app.include_router(dashboard_router, prefix="/api")
|
||||||
|
|
||||||
|
|
|
||||||
134
app/cforch.py
134
app/cforch.py
|
|
@ -16,16 +16,18 @@ import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
|
import select as _select
|
||||||
import subprocess as _subprocess
|
import subprocess as _subprocess
|
||||||
import tempfile
|
import tempfile
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any, Optional
|
||||||
|
|
||||||
import urllib.parse
|
import urllib.parse
|
||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
from fastapi import APIRouter, HTTPException
|
from fastapi import APIRouter, HTTPException, Request
|
||||||
from fastapi.responses import StreamingResponse
|
from fastapi.responses import StreamingResponse
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
@ -311,8 +313,12 @@ def run_benchmark(
|
||||||
"""Spawn cf-orch benchmark.py and stream stdout as SSE progress events."""
|
"""Spawn cf-orch benchmark.py and stream stdout as SSE progress events."""
|
||||||
global _BENCH_RUNNING, _bench_proc
|
global _BENCH_RUNNING, _bench_proc
|
||||||
|
|
||||||
|
# Check if the process is actually still alive; reset stale flag if not.
|
||||||
if _BENCH_RUNNING:
|
if _BENCH_RUNNING:
|
||||||
raise HTTPException(409, "A benchmark is already running")
|
if _bench_proc is not None and _bench_proc.poll() is None:
|
||||||
|
raise HTTPException(409, "A benchmark is already running")
|
||||||
|
_BENCH_RUNNING = False
|
||||||
|
_bench_proc = None
|
||||||
|
|
||||||
cfg = _load_cforch_config()
|
cfg = _load_cforch_config()
|
||||||
bench_script = cfg.get("bench_script", "")
|
bench_script = cfg.get("bench_script", "")
|
||||||
|
|
@ -436,8 +442,23 @@ def run_benchmark(
|
||||||
env=proc_env,
|
env=proc_env,
|
||||||
)
|
)
|
||||||
_bench_proc = proc
|
_bench_proc = proc
|
||||||
|
_IDLE_TIMEOUT_S = 120 # kill if no output for 2 minutes (node crash)
|
||||||
try:
|
try:
|
||||||
for line in proc.stdout:
|
while True:
|
||||||
|
ready = _select.select([proc.stdout], [], [], _IDLE_TIMEOUT_S)
|
||||||
|
if not ready[0]:
|
||||||
|
# No output for IDLE_TIMEOUT_S — node likely crashed
|
||||||
|
proc.terminate()
|
||||||
|
try:
|
||||||
|
proc.wait(timeout=5)
|
||||||
|
except _subprocess.TimeoutExpired:
|
||||||
|
proc.kill()
|
||||||
|
msg = f"Benchmark timed out — no output for {_IDLE_TIMEOUT_S}s (cluster node may have crashed)"
|
||||||
|
yield f"data: {json.dumps({'type': 'error', 'message': msg})}\n\n"
|
||||||
|
break
|
||||||
|
line = proc.stdout.readline()
|
||||||
|
if not line:
|
||||||
|
break
|
||||||
line = _strip_ansi(line.rstrip())
|
line = _strip_ansi(line.rstrip())
|
||||||
if line:
|
if line:
|
||||||
yield f"data: {json.dumps({'type': 'progress', 'message': line})}\n\n"
|
yield f"data: {json.dumps({'type': 'progress', 'message': line})}\n\n"
|
||||||
|
|
@ -495,7 +516,7 @@ def get_cforch_config() -> dict:
|
||||||
# ── GET /results ───────────────────────────────────────────────────────────────
|
# ── GET /results ───────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
@router.get("/results")
|
@router.get("/results")
|
||||||
def get_results() -> list:
|
def get_results() -> dict:
|
||||||
"""Return the latest benchmark summary.json from results_dir."""
|
"""Return the latest benchmark summary.json from results_dir."""
|
||||||
cfg = _load_cforch_config()
|
cfg = _load_cforch_config()
|
||||||
results_dir = cfg.get("results_dir", "")
|
results_dir = cfg.get("results_dir", "")
|
||||||
|
|
@ -527,3 +548,106 @@ def cancel_benchmark() -> dict:
|
||||||
_BENCH_RUNNING = False
|
_BENCH_RUNNING = False
|
||||||
_bench_proc = None
|
_bench_proc = None
|
||||||
return {"status": "cancelled"}
|
return {"status": "cancelled"}
|
||||||
|
|
||||||
|
|
||||||
|
# ── Coordinator proxy helpers ──────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _coordinator_url() -> str:
|
||||||
|
"""Return coordinator base URL from config, or raise 503 if not configured."""
|
||||||
|
url = _load_cforch_config().get("coordinator_url", "").rstrip("/")
|
||||||
|
if not url:
|
||||||
|
raise HTTPException(503, "cf-orch coordinator_url not configured")
|
||||||
|
return url
|
||||||
|
|
||||||
|
|
||||||
|
def _coordinator_get(path: str) -> Any:
|
||||||
|
"""GET from coordinator, return parsed JSON body. Raises HTTPException on error."""
|
||||||
|
import httpx as _httpx
|
||||||
|
try:
|
||||||
|
resp = _httpx.get(f"{_coordinator_url()}{path}", timeout=10.0)
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(502, f"Coordinator unreachable: {exc}") from exc
|
||||||
|
if not resp.is_success:
|
||||||
|
raise HTTPException(resp.status_code, resp.text)
|
||||||
|
return resp.json()
|
||||||
|
|
||||||
|
|
||||||
|
async def _coordinator_post(path: str, body: dict) -> Any:
|
||||||
|
import httpx as _httpx
|
||||||
|
try:
|
||||||
|
async with _httpx.AsyncClient(timeout=10.0) as client:
|
||||||
|
resp = await client.post(f"{_coordinator_url()}{path}", json=body)
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(502, f"Coordinator unreachable: {exc}") from exc
|
||||||
|
if not resp.is_success:
|
||||||
|
raise HTTPException(resp.status_code, resp.text)
|
||||||
|
return resp.json()
|
||||||
|
|
||||||
|
|
||||||
|
async def _coordinator_delete(path: str) -> Any:
|
||||||
|
import httpx as _httpx
|
||||||
|
try:
|
||||||
|
async with _httpx.AsyncClient(timeout=10.0) as client:
|
||||||
|
resp = await client.delete(f"{_coordinator_url()}{path}")
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(502, f"Coordinator unreachable: {exc}") from exc
|
||||||
|
if not resp.is_success:
|
||||||
|
raise HTTPException(resp.status_code, resp.text)
|
||||||
|
return resp.json()
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET /assignments/deployment-status ───────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/assignments/deployment-status")
|
||||||
|
def get_deployment_status() -> Any:
|
||||||
|
return _coordinator_get("/api/assignments/deployment-status")
|
||||||
|
|
||||||
|
|
||||||
|
# ── /assignments ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/assignments")
|
||||||
|
def list_assignments() -> Any:
|
||||||
|
return _coordinator_get("/api/assignments")
|
||||||
|
|
||||||
|
|
||||||
|
class AssignmentBody(BaseModel):
|
||||||
|
product: str
|
||||||
|
task: str
|
||||||
|
model_id: str
|
||||||
|
description: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/assignments")
|
||||||
|
async def upsert_assignment(body: AssignmentBody) -> Any:
|
||||||
|
return await _coordinator_post("/api/assignments", body.model_dump())
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/assignments/{product}/{task}")
|
||||||
|
async def delete_assignment(product: str, task: str) -> Any:
|
||||||
|
return await _coordinator_delete(f"/api/assignments/{urllib.parse.quote(product, safe='')}/{urllib.parse.quote(task, safe='')}")
|
||||||
|
|
||||||
|
|
||||||
|
# ── /model-registry ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/model-registry")
|
||||||
|
def list_model_registry() -> Any:
|
||||||
|
return _coordinator_get("/api/model-registry")
|
||||||
|
|
||||||
|
|
||||||
|
class ModelRegistryBody(BaseModel):
|
||||||
|
model_id: str
|
||||||
|
service_type: str
|
||||||
|
vram_mb: int
|
||||||
|
description: str = ""
|
||||||
|
hf_repo: str = ""
|
||||||
|
alias: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/model-registry")
|
||||||
|
async def upsert_model_registry(body: ModelRegistryBody) -> Any:
|
||||||
|
return await _coordinator_post("/api/model-registry", body.model_dump())
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/model-registry/{model_id:path}")
|
||||||
|
async def delete_model_registry(model_id: str) -> Any:
|
||||||
|
return await _coordinator_delete(f"/api/model-registry/{urllib.parse.quote(model_id, safe='')}")
|
||||||
|
|
|
||||||
109
app/dashboard.py
109
app/dashboard.py
|
|
@ -1,17 +1,18 @@
|
||||||
"""Avocet -- dashboard aggregate API.
|
"""Avocet -- dashboard aggregate API.
|
||||||
|
|
||||||
GET /api/dashboard returns the current flywheel state:
|
GET /api/dashboard returns the current flywheel state:
|
||||||
labeled_since_last_eval -- items labeled after the most recent eval run
|
labeled_since_last_eval -- items labeled after the most recent bench run
|
||||||
last_eval_timestamp -- ISO timestamp of newest bench_results summary
|
last_eval_timestamp -- ISO timestamp of newest bench_results summary
|
||||||
last_eval_best_score -- best macro_f1 from that summary
|
last_eval_best_score -- best macro_f1 from that summary
|
||||||
active_jobs -- jobs with status queued or running
|
active_jobs -- jobs with status queued or running
|
||||||
corrections_pending -- sft_candidates with status=needs_review
|
corrections_pending -- sft_candidates with status=needs_review
|
||||||
corrections_export_ready -- approved sft candidates with non-blank correction
|
corrections_export_ready -- approved sft candidates with non-blank correction
|
||||||
|
recent_bench_runs -- most-recent timestamp + score per bench type
|
||||||
signals -- computed booleans for UI nudge indicators
|
signals -- computed booleans for UI nudge indicators
|
||||||
|
|
||||||
Thresholds in label_tool.yaml pipeline: section:
|
Thresholds in label_tool.yaml pipeline: section:
|
||||||
pipeline:
|
pipeline:
|
||||||
data_eval_threshold: 50 # labeled items since last eval to trigger nudge
|
data_eval_threshold: 50 # labeled items since last bench to trigger nudge
|
||||||
eval_train_threshold: 0.05 # improvement delta needed before retraining (future)
|
eval_train_threshold: 0.05 # improvement delta needed before retraining (future)
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
@ -77,7 +78,7 @@ def _load_score_records() -> list[dict]:
|
||||||
pass
|
pass
|
||||||
return records
|
return records
|
||||||
|
|
||||||
def _find_latest_eval(results_dir_override: str = "") -> tuple[str | None, float | None]:
|
def _find_latest_classifier_bench(results_dir_override: str = "") -> tuple[str | None, float | None]:
|
||||||
"""Return (iso_timestamp, best_macro_f1) from the newest bench_results summary.
|
"""Return (iso_timestamp, best_macro_f1) from the newest bench_results summary.
|
||||||
|
|
||||||
Checks results_dir from cforch config if set, then falls back to
|
Checks results_dir from cforch config if set, then falls back to
|
||||||
|
|
@ -107,6 +108,8 @@ def _find_latest_eval(results_dir_override: str = "") -> tuple[str | None, float
|
||||||
if summary.exists():
|
if summary.exists():
|
||||||
try:
|
try:
|
||||||
data = json.loads(summary.read_text(encoding="utf-8"))
|
data = json.loads(summary.read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
continue # cforch LLM-bench summaries are lists; skip
|
||||||
ts = data.get("timestamp") or subdir.name
|
ts = data.get("timestamp") or subdir.name
|
||||||
score = data.get("best_macro_f1") or data.get("macro_f1")
|
score = data.get("best_macro_f1") or data.get("macro_f1")
|
||||||
return ts, (float(score) if isinstance(score, (int, float)) else None)
|
return ts, (float(score) if isinstance(score, (int, float)) else None)
|
||||||
|
|
@ -114,6 +117,10 @@ def _find_latest_eval(results_dir_override: str = "") -> tuple[str | None, float
|
||||||
logger.warning("Failed to parse summary.json at %s: %s", summary, exc)
|
logger.warning("Failed to parse summary.json at %s: %s", summary, exc)
|
||||||
return None, None
|
return None, None
|
||||||
|
|
||||||
|
# Keep old name as alias so existing callers in tests still work.
|
||||||
|
_find_latest_eval = _find_latest_classifier_bench
|
||||||
|
|
||||||
|
|
||||||
def _count_corrections() -> tuple[int, int]:
|
def _count_corrections() -> tuple[int, int]:
|
||||||
"""Return (pending_count, export_ready_count)."""
|
"""Return (pending_count, export_ready_count)."""
|
||||||
pending = 0
|
pending = 0
|
||||||
|
|
@ -169,22 +176,106 @@ def _count_labeled_since(since_ts: str | None) -> int:
|
||||||
return sum(1 for r in records if r.get("labeled_at", "") > since_ts)
|
return sum(1 for r in records if r.get("labeled_at", "") > since_ts)
|
||||||
|
|
||||||
|
|
||||||
|
def _get_recent_bench_runs() -> dict:
|
||||||
|
"""Return most-recent run summary for each bench type.
|
||||||
|
|
||||||
|
Each entry: {"timestamp": str|None, "metric": str|None, "score": float|None}
|
||||||
|
"""
|
||||||
|
runs: dict[str, dict] = {
|
||||||
|
"classifier": {"timestamp": None, "metric": "macro_f1", "score": None},
|
||||||
|
"llm": {"timestamp": None, "metric": None, "score": None},
|
||||||
|
"style": {"timestamp": None, "metric": None, "score": None},
|
||||||
|
"plans": {"timestamp": None, "metric": "avg_score", "score": None},
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── Classifier: bench_results/<run>/summary.json ──────────────────────
|
||||||
|
clf_ts, clf_score = _find_latest_classifier_bench()
|
||||||
|
if clf_ts:
|
||||||
|
runs["classifier"]["timestamp"] = clf_ts
|
||||||
|
runs["classifier"]["score"] = clf_score
|
||||||
|
|
||||||
|
# ── LLM bench + Style: benchmark_results/ ─────────────────────────────
|
||||||
|
f = _config_file()
|
||||||
|
bench_dir: Path | None = None
|
||||||
|
if f.exists():
|
||||||
|
try:
|
||||||
|
raw = yaml.safe_load(f.read_text(encoding="utf-8")) or {}
|
||||||
|
rd = (raw.get("cforch", {}) or {}).get("results_dir", "")
|
||||||
|
if rd:
|
||||||
|
bench_dir = Path(rd)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
if bench_dir is None:
|
||||||
|
bench_dir = _ROOT / "benchmark_results"
|
||||||
|
|
||||||
|
if bench_dir.exists():
|
||||||
|
llm_files = sorted(
|
||||||
|
[p for p in bench_dir.glob("*.json") if not p.name.startswith("style_")],
|
||||||
|
key=lambda p: p.stat().st_mtime, reverse=True,
|
||||||
|
)
|
||||||
|
if llm_files:
|
||||||
|
try:
|
||||||
|
data = json.loads(llm_files[0].read_text(encoding="utf-8"))
|
||||||
|
runs["llm"]["timestamp"] = data.get("timestamp") or llm_files[0].stem
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
style_files = sorted(bench_dir.glob("style_*.json"), reverse=True)
|
||||||
|
if style_files:
|
||||||
|
try:
|
||||||
|
data = json.loads(style_files[0].read_text(encoding="utf-8"))
|
||||||
|
if isinstance(data, list) and data:
|
||||||
|
runs["style"]["timestamp"] = data[0].get("timestamp") or style_files[0].stem
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# ── Plans bench: data/plans_bench_results/plans_*.json ────────────────
|
||||||
|
plans_dir = _DATA_DIR / "plans_bench_results"
|
||||||
|
if plans_dir.exists():
|
||||||
|
plans_files = sorted(plans_dir.glob("plans_*.json"), reverse=True)
|
||||||
|
if plans_files:
|
||||||
|
run_id = plans_files[0].stem
|
||||||
|
try:
|
||||||
|
d: dict = json.loads(plans_files[0].read_text(encoding="utf-8"))
|
||||||
|
all_scores = [
|
||||||
|
r["total_score"]
|
||||||
|
for results in d.values()
|
||||||
|
for r in results
|
||||||
|
if isinstance(r, dict) and not r.get("error")
|
||||||
|
]
|
||||||
|
avg = round(sum(all_scores) / len(all_scores), 3) if all_scores else None
|
||||||
|
try:
|
||||||
|
date_part = run_id.removeprefix("plans_")
|
||||||
|
date, time_part = date_part.split("_")
|
||||||
|
ts_display = f"{date} {time_part[:2]}:{time_part[2:4]}"
|
||||||
|
except Exception:
|
||||||
|
ts_display = run_id
|
||||||
|
runs["plans"]["timestamp"] = ts_display
|
||||||
|
runs["plans"]["score"] = avg
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
return runs
|
||||||
|
|
||||||
|
|
||||||
@router.get("/dashboard")
|
@router.get("/dashboard")
|
||||||
def get_dashboard() -> dict:
|
def get_dashboard() -> dict:
|
||||||
data_eval_threshold, eval_train_threshold = _load_thresholds()
|
data_threshold, _train_threshold = _load_thresholds()
|
||||||
last_eval_ts, last_eval_score = _find_latest_eval()
|
last_ts, last_score = _find_latest_classifier_bench()
|
||||||
labeled_since = _count_labeled_since(last_eval_ts)
|
labeled_since = _count_labeled_since(last_ts)
|
||||||
corrections_pending, corrections_export_ready = _count_corrections()
|
corrections_pending, corrections_export_ready = _count_corrections()
|
||||||
active_jobs = _get_active_jobs()
|
active_jobs = _get_active_jobs()
|
||||||
|
recent_bench = _get_recent_bench_runs()
|
||||||
return {
|
return {
|
||||||
"labeled_since_last_eval": labeled_since,
|
"labeled_since_last_eval": labeled_since,
|
||||||
"last_eval_timestamp": last_eval_ts,
|
"last_eval_timestamp": last_ts,
|
||||||
"last_eval_best_score": last_eval_score,
|
"last_eval_best_score": last_score,
|
||||||
"active_jobs": active_jobs,
|
"active_jobs": active_jobs,
|
||||||
"corrections_pending": corrections_pending,
|
"corrections_pending": corrections_pending,
|
||||||
"corrections_export_ready": corrections_export_ready,
|
"corrections_export_ready": corrections_export_ready,
|
||||||
|
"recent_bench_runs": recent_bench,
|
||||||
"signals": {
|
"signals": {
|
||||||
"data_to_eval": labeled_since >= data_eval_threshold,
|
"data_to_eval": labeled_since >= data_threshold,
|
||||||
"eval_to_train": False, # future: implement delta-F1 comparison
|
"eval_to_train": False, # future: implement delta-F1 comparison
|
||||||
"train_to_fleet": False, # future: implement fleet sync signal
|
"train_to_fleet": False, # future: implement fleet sync signal
|
||||||
},
|
},
|
||||||
|
|
|
||||||
|
|
@ -94,6 +94,42 @@ def _cforch_url() -> str:
|
||||||
return cforch.get("coordinator_url") or "http://localhost:7700"
|
return cforch.get("coordinator_url") or "http://localhost:7700"
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_task_model(cforch_base: str, product: str, task: str) -> dict | None:
|
||||||
|
"""Return {model_id, service_type} for a product.task assignment, or None if not found.
|
||||||
|
|
||||||
|
Calls GET coordinator/api/assignments and filters by product+task.
|
||||||
|
The model registry entry is fetched separately to get service_type.
|
||||||
|
Returns None (not raises) — callers emit a 'model_done' error event instead.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
asgn_resp = httpx.get(f"{cforch_base}/api/assignments", timeout=5.0)
|
||||||
|
asgn_resp.raise_for_status()
|
||||||
|
assignments: list[dict] = asgn_resp.json().get("assignments", []) or []
|
||||||
|
match = next(
|
||||||
|
(a for a in assignments if a.get("product") == product and a.get("task") == task),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
if match is None:
|
||||||
|
return None
|
||||||
|
model_id: str = match.get("model_id", "")
|
||||||
|
if not model_id:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Look up service_type from model registry
|
||||||
|
reg_resp = httpx.get(f"{cforch_base}/api/model-registry", timeout=5.0)
|
||||||
|
service_type = "cf-text" # sensible default
|
||||||
|
if reg_resp.is_success:
|
||||||
|
models: list[dict] = reg_resp.json().get("models", []) or []
|
||||||
|
reg_entry = next((m for m in models if m.get("model_id") == model_id), None)
|
||||||
|
if reg_entry:
|
||||||
|
service_type = reg_entry.get("service_type", "cf-text") or "cf-text"
|
||||||
|
|
||||||
|
return {"model_id": model_id, "service_type": service_type}
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("Task resolution failed for %s.%s: %s", product, task, exc)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def _cforch_catalog(cforch_base: str) -> list[dict]:
|
def _cforch_catalog(cforch_base: str) -> list[dict]:
|
||||||
"""Fetch the live cf-text catalog from cf-orch.
|
"""Fetch the live cf-text catalog from cf-orch.
|
||||||
|
|
||||||
|
|
@ -476,13 +512,19 @@ def run_imitate(
|
||||||
prompt: str = "",
|
prompt: str = "",
|
||||||
model_ids: str = "", # comma-separated ollama model IDs
|
model_ids: str = "", # comma-separated ollama model IDs
|
||||||
cf_text_model_ids: str = "", # comma-separated cf-text model IDs (via cf-orch)
|
cf_text_model_ids: str = "", # comma-separated cf-text model IDs (via cf-orch)
|
||||||
|
task_ids: str = "", # comma-separated "product/task" strings — resolved via assignments
|
||||||
temperature: float = 0.7,
|
temperature: float = 0.7,
|
||||||
product_id: str = "",
|
product_id: str = "",
|
||||||
system: str = "", # optional system prompt
|
system: str = "", # optional system prompt
|
||||||
image_url: str = "", # optional image URL for vision models
|
image_url: str = "", # optional image URL for vision models
|
||||||
session: "Any" = Depends(_get_imitate_session),
|
session: "Any" = Depends(_get_imitate_session),
|
||||||
) -> StreamingResponse:
|
) -> StreamingResponse:
|
||||||
"""Run a prompt through selected ollama models and stream results as SSE.
|
"""Run a prompt through selected models and stream results as SSE.
|
||||||
|
|
||||||
|
Models can be selected three ways (combinable):
|
||||||
|
- model_ids: explicit ollama model IDs
|
||||||
|
- cf_text_model_ids: explicit cf-text model IDs routed via cf-orch
|
||||||
|
- task_ids: "product/task" strings resolved via the coordinator assignments table
|
||||||
|
|
||||||
If image_url is provided, the image is downloaded once and passed to every
|
If image_url is provided, the image is downloaded once and passed to every
|
||||||
model as a base64-encoded blob — allowing vision-capable local models to
|
model as a base64-encoded blob — allowing vision-capable local models to
|
||||||
|
|
@ -494,8 +536,37 @@ def run_imitate(
|
||||||
|
|
||||||
ollama_ids = [m.strip() for m in model_ids.split(",") if m.strip()]
|
ollama_ids = [m.strip() for m in model_ids.split(",") if m.strip()]
|
||||||
cftext_ids = [m.strip() for m in cf_text_model_ids.split(",") if m.strip()]
|
cftext_ids = [m.strip() for m in cf_text_model_ids.split(",") if m.strip()]
|
||||||
|
raw_task_ids = [t.strip() for t in task_ids.split(",") if t.strip()]
|
||||||
|
|
||||||
|
# Resolve task assignments to concrete model IDs, routing to the right service.
|
||||||
|
# Models that fail to resolve emit an error event at run time (non-fatal).
|
||||||
|
if raw_task_ids:
|
||||||
|
cforch_base = _cforch_url()
|
||||||
|
for task_spec in raw_task_ids:
|
||||||
|
parts = task_spec.split("/", 1)
|
||||||
|
if len(parts) != 2:
|
||||||
|
logger.warning("Skipping malformed task_id %r (expected product/task)", task_spec)
|
||||||
|
continue
|
||||||
|
product_name, task_name = parts
|
||||||
|
resolved = _resolve_task_model(cforch_base, product_name, task_name)
|
||||||
|
if resolved is None:
|
||||||
|
logger.warning("No assignment found for task %r", task_spec)
|
||||||
|
# Emit error at stream time via a sentinel in cftext_ids with a special label.
|
||||||
|
# We instead store the failed task_spec to emit a model_done error.
|
||||||
|
cftext_ids.append(f"__task_unresolved__:{task_spec}")
|
||||||
|
continue
|
||||||
|
mid = resolved["model_id"]
|
||||||
|
svc = resolved["service_type"]
|
||||||
|
if svc == "ollama":
|
||||||
|
if mid not in ollama_ids:
|
||||||
|
ollama_ids.append(mid)
|
||||||
|
else:
|
||||||
|
# cf-text, vllm, and any other cf-orch-managed service
|
||||||
|
if mid not in cftext_ids:
|
||||||
|
cftext_ids.append(mid)
|
||||||
|
|
||||||
if not ollama_ids and not cftext_ids:
|
if not ollama_ids and not cftext_ids:
|
||||||
raise HTTPException(422, "model_ids or cf_text_model_ids is required")
|
raise HTTPException(422, "model_ids, cf_text_model_ids, or task_ids is required")
|
||||||
|
|
||||||
cfg = _load_imitate_config()
|
cfg = _load_imitate_config()
|
||||||
ollama_base = _ollama_url(cfg)
|
ollama_base = _ollama_url(cfg)
|
||||||
|
|
@ -539,11 +610,25 @@ def run_imitate(
|
||||||
yield _sse({"type": "model_done", **result})
|
yield _sse({"type": "model_done", **result})
|
||||||
|
|
||||||
# cf-text models via cf-orch — fan out in parallel when multiple models selected
|
# cf-text models via cf-orch — fan out in parallel when multiple models selected
|
||||||
if cftext_ids:
|
# Partition the list: real cf-text IDs vs unresolved-task sentinels.
|
||||||
|
cftext_real = [m for m in cftext_ids if not m.startswith("__task_unresolved__:")]
|
||||||
|
cftext_unresolved = [m for m in cftext_ids if m.startswith("__task_unresolved__:")]
|
||||||
|
for sentinel in cftext_unresolved:
|
||||||
|
task_spec = sentinel.split(":", 1)[1]
|
||||||
|
result = {
|
||||||
|
"model": task_spec,
|
||||||
|
"response": "",
|
||||||
|
"elapsed_ms": 0,
|
||||||
|
"error": f"No assignment configured for task '{task_spec}'",
|
||||||
|
}
|
||||||
|
results.append(result)
|
||||||
|
yield _sse({"type": "model_done", **result})
|
||||||
|
|
||||||
|
if cftext_real:
|
||||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||||
|
|
||||||
# Announce all models upfront so the UI can show loading states immediately
|
# Announce all models upfront so the UI can show loading states immediately
|
||||||
for model_id in cftext_ids:
|
for model_id in cftext_real:
|
||||||
yield _sse({"type": "model_start", "model": model_id, "service": "cf-text"})
|
yield _sse({"type": "model_start", "model": model_id, "service": "cf-text"})
|
||||||
|
|
||||||
_user_id: str | None = getattr(session, "user_id", None)
|
_user_id: str | None = getattr(session, "user_id", None)
|
||||||
|
|
@ -551,13 +636,13 @@ def run_imitate(
|
||||||
if _user_id in (None, "local", "local-dev") or (_user_id or "").startswith("anon-"):
|
if _user_id in (None, "local", "local-dev") or (_user_id or "").startswith("anon-"):
|
||||||
_user_id = None
|
_user_id = None
|
||||||
|
|
||||||
with ThreadPoolExecutor(max_workers=len(cftext_ids)) as pool:
|
with ThreadPoolExecutor(max_workers=len(cftext_real)) as pool:
|
||||||
future_to_model = {
|
future_to_model = {
|
||||||
pool.submit(
|
pool.submit(
|
||||||
_run_cftext, cforch_base, mid, prompt, system_ctx, temperature,
|
_run_cftext, cforch_base, mid, prompt, system_ctx, temperature,
|
||||||
180.0, _user_id,
|
180.0, _user_id,
|
||||||
): mid
|
): mid
|
||||||
for mid in cftext_ids
|
for mid in cftext_real
|
||||||
}
|
}
|
||||||
for future in as_completed(future_to_model):
|
for future in as_completed(future_to_model):
|
||||||
model_id = future_to_model[future]
|
model_id = future_to_model[future]
|
||||||
|
|
|
||||||
462
app/data/log_corpus.py
Normal file
462
app/data/log_corpus.py
Normal file
|
|
@ -0,0 +1,462 @@
|
||||||
|
"""Avocet — Log Corpus receiver and labeling API.
|
||||||
|
|
||||||
|
Receives push batches from consented Turnstone nodes, stores entries for labeling,
|
||||||
|
and exports labeled data as JSONL for the logreading fine-tune pipeline.
|
||||||
|
|
||||||
|
DB: data/corpus.db (separate from train_jobs.db — different lifecycle)
|
||||||
|
Auth: Bearer token validated against corpus_sources table (seeded from label_tool.yaml).
|
||||||
|
|
||||||
|
All endpoints registered on `router`. api.py includes this with prefix="/api/corpus".
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import sqlite3
|
||||||
|
import uuid
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Generator
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
|
from fastapi.requests import Request
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_ROOT = Path(__file__).parent.parent.parent
|
||||||
|
_CONFIG_DIR: Path | None = None
|
||||||
|
_DATA_DIR: Path = _ROOT / "data"
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
_DB_PATH: Path = _ROOT / "data" / "corpus.db"
|
||||||
|
|
||||||
|
_PIPELINE_SOURCE_HOST = "pipeline_scrape"
|
||||||
|
|
||||||
|
_SCHEMA = """
|
||||||
|
CREATE TABLE IF NOT EXISTS corpus_sources (
|
||||||
|
token TEXT PRIMARY KEY,
|
||||||
|
source_host TEXT NOT NULL,
|
||||||
|
owner TEXT NOT NULL,
|
||||||
|
consent_date TEXT NOT NULL,
|
||||||
|
consent_method TEXT NOT NULL,
|
||||||
|
active INTEGER NOT NULL DEFAULT 1
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS corpus_batches (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
source_host TEXT NOT NULL,
|
||||||
|
batch_type TEXT NOT NULL,
|
||||||
|
received_at TEXT NOT NULL,
|
||||||
|
entry_count INTEGER NOT NULL,
|
||||||
|
watermark_from TEXT,
|
||||||
|
watermark_to TEXT,
|
||||||
|
raw_json TEXT NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS corpus_entries (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
batch_id TEXT NOT NULL REFERENCES corpus_batches(id),
|
||||||
|
source_host TEXT NOT NULL,
|
||||||
|
origin_entry_id TEXT,
|
||||||
|
timestamp_iso TEXT,
|
||||||
|
severity TEXT,
|
||||||
|
source_id TEXT,
|
||||||
|
text TEXT NOT NULL,
|
||||||
|
matched_patterns TEXT DEFAULT '[]',
|
||||||
|
label_state TEXT NOT NULL DEFAULT 'unlabeled',
|
||||||
|
failure_type TEXT,
|
||||||
|
plain_explanation TEXT,
|
||||||
|
known_pattern TEXT,
|
||||||
|
labeled_at TEXT,
|
||||||
|
labeled_by TEXT DEFAULT 'alan',
|
||||||
|
pii_flagged INTEGER NOT NULL DEFAULT 0
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_ce_label_state ON corpus_entries(label_state);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_ce_source ON corpus_entries(source_host);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_ce_severity ON corpus_entries(severity);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS ingested_pipeline_files (
|
||||||
|
filename TEXT PRIMARY KEY,
|
||||||
|
ingested_at TEXT NOT NULL,
|
||||||
|
entry_count INTEGER NOT NULL
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
# ── Testability seams ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def set_config_dir(path: Path | None) -> None:
|
||||||
|
global _CONFIG_DIR
|
||||||
|
_CONFIG_DIR = path
|
||||||
|
|
||||||
|
|
||||||
|
def set_data_dir(path: Path) -> None:
|
||||||
|
global _DATA_DIR, _DB_PATH
|
||||||
|
_DATA_DIR = path
|
||||||
|
_DB_PATH = path / "corpus.db"
|
||||||
|
|
||||||
|
|
||||||
|
# ── Internal helpers ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _config_file() -> Path:
|
||||||
|
if _CONFIG_DIR is not None:
|
||||||
|
return _CONFIG_DIR / "label_tool.yaml"
|
||||||
|
return _ROOT / "config" / "label_tool.yaml"
|
||||||
|
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def _db() -> Generator[sqlite3.Connection, None, None]:
|
||||||
|
conn = sqlite3.connect(str(_DB_PATH))
|
||||||
|
conn.row_factory = sqlite3.Row
|
||||||
|
conn.execute("PRAGMA journal_mode=WAL")
|
||||||
|
try:
|
||||||
|
yield conn
|
||||||
|
conn.commit()
|
||||||
|
except Exception:
|
||||||
|
conn.rollback()
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _init_db() -> None:
|
||||||
|
with _db() as conn:
|
||||||
|
conn.executescript(_SCHEMA)
|
||||||
|
_seed_sources(conn)
|
||||||
|
|
||||||
|
|
||||||
|
def _pipeline_ingest_dir() -> Path | None:
|
||||||
|
"""Return the configured pipeline log ingest directory, or None if unset."""
|
||||||
|
f = _config_file()
|
||||||
|
if not f.exists():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
raw = yaml.safe_load(f.read_text(encoding="utf-8")) or {}
|
||||||
|
except yaml.YAMLError:
|
||||||
|
return None
|
||||||
|
val = raw.get("corpus", {}).get("pipeline_ingest_dir", "") or ""
|
||||||
|
return Path(val) if val else None
|
||||||
|
|
||||||
|
|
||||||
|
def _load_corpus_config() -> list[dict]:
|
||||||
|
f = _config_file()
|
||||||
|
if not f.exists():
|
||||||
|
return []
|
||||||
|
try:
|
||||||
|
raw = yaml.safe_load(f.read_text(encoding="utf-8")) or {}
|
||||||
|
except yaml.YAMLError as exc:
|
||||||
|
logger.warning("Failed to parse corpus config: %s", exc)
|
||||||
|
return []
|
||||||
|
return raw.get("corpus", {}).get("sources", []) or []
|
||||||
|
|
||||||
|
|
||||||
|
def _seed_sources(conn: sqlite3.Connection) -> None:
|
||||||
|
for src in _load_corpus_config():
|
||||||
|
conn.execute(
|
||||||
|
"INSERT OR IGNORE INTO corpus_sources (token, source_host, owner, consent_date, consent_method) "
|
||||||
|
"VALUES (?, ?, ?, ?, ?)",
|
||||||
|
(src["token"], src["source_host"], src["owner"],
|
||||||
|
src["consent_date"], src["consent_method"]),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_token(token: str, conn: sqlite3.Connection) -> str:
|
||||||
|
"""Return source_host for token, or raise 403."""
|
||||||
|
row = conn.execute(
|
||||||
|
"SELECT source_host FROM corpus_sources WHERE token = ? AND active = 1",
|
||||||
|
(token,),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(status_code=403, detail="Unknown or revoked consent token")
|
||||||
|
return row["source_host"]
|
||||||
|
|
||||||
|
|
||||||
|
def _extract_bearer(request: Request) -> str:
|
||||||
|
auth = request.headers.get("Authorization", "")
|
||||||
|
if not auth.startswith("Bearer "):
|
||||||
|
raise HTTPException(status_code=401, detail="Bearer token required")
|
||||||
|
return auth.removeprefix("Bearer ").strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _now_iso() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat()
|
||||||
|
|
||||||
|
|
||||||
|
# ── Startup ────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
_init_db()
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST /api/corpus/log-batch ─────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.post("/log-batch")
|
||||||
|
def receive_batch(request: Request, payload: dict) -> dict:
|
||||||
|
"""Accept a push batch from a Turnstone node."""
|
||||||
|
token = _extract_bearer(request)
|
||||||
|
|
||||||
|
batch_type = payload.get("batch_type", "raw_entries")
|
||||||
|
entries_raw = payload.get("entries", [])
|
||||||
|
batch_id = payload.get("batch_id") or str(uuid.uuid4())
|
||||||
|
|
||||||
|
with _db() as conn:
|
||||||
|
source_host = _validate_token(token, conn)
|
||||||
|
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO corpus_batches (id, source_host, batch_type, received_at, entry_count, "
|
||||||
|
"watermark_from, watermark_to, raw_json) VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
|
||||||
|
(batch_id, source_host, batch_type, _now_iso(), len(entries_raw),
|
||||||
|
str(payload.get("watermark_from", "")),
|
||||||
|
str(payload.get("watermark_to", "")),
|
||||||
|
json.dumps(payload)),
|
||||||
|
)
|
||||||
|
|
||||||
|
stored = 0
|
||||||
|
for entry in entries_raw:
|
||||||
|
text = entry.get("text", "").strip()
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
conn.execute(
|
||||||
|
"INSERT OR IGNORE INTO corpus_entries "
|
||||||
|
"(id, batch_id, source_host, origin_entry_id, timestamp_iso, severity, "
|
||||||
|
"source_id, text, matched_patterns) "
|
||||||
|
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
|
||||||
|
(str(uuid.uuid4()), batch_id, source_host,
|
||||||
|
entry.get("entry_id") or entry.get("id"),
|
||||||
|
entry.get("timestamp_iso"),
|
||||||
|
entry.get("severity"),
|
||||||
|
entry.get("source_id"),
|
||||||
|
text,
|
||||||
|
json.dumps(entry.get("matched_patterns", []))),
|
||||||
|
)
|
||||||
|
stored += 1
|
||||||
|
|
||||||
|
logger.info("Received batch %s from %s: %d/%d entries stored",
|
||||||
|
batch_id, source_host, stored, len(entries_raw))
|
||||||
|
return {"received": True, "batch_id": batch_id, "entries_stored": stored}
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET /api/corpus/entries ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/entries")
|
||||||
|
def list_entries(
|
||||||
|
state: str = "unlabeled",
|
||||||
|
source_host: str | None = None,
|
||||||
|
limit: int = 25,
|
||||||
|
) -> dict:
|
||||||
|
"""Return entries for labeling. Default: unlabeled entries, oldest first."""
|
||||||
|
with _db() as conn:
|
||||||
|
query = "SELECT * FROM corpus_entries WHERE label_state = ?"
|
||||||
|
params: list = [state]
|
||||||
|
if source_host:
|
||||||
|
query += " AND source_host = ?"
|
||||||
|
params.append(source_host)
|
||||||
|
query += " ORDER BY rowid LIMIT ?"
|
||||||
|
params.append(min(limit, 100))
|
||||||
|
rows = conn.execute(query, params).fetchall()
|
||||||
|
return {"entries": [dict(r) for r in rows], "count": len(rows)}
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST /api/corpus/entries/{id}/label ───────────────────────────────────────
|
||||||
|
|
||||||
|
@router.post("/entries/{entry_id}/label")
|
||||||
|
def label_entry(entry_id: str, body: dict) -> dict:
|
||||||
|
"""Submit a label for a corpus entry."""
|
||||||
|
failure_type = body.get("failure_type")
|
||||||
|
plain_explanation = body.get("plain_explanation", "").strip()
|
||||||
|
known_pattern = body.get("known_pattern")
|
||||||
|
pii_flagged = int(bool(body.get("pii_flagged", False)))
|
||||||
|
|
||||||
|
if not failure_type:
|
||||||
|
raise HTTPException(status_code=422, detail="failure_type is required")
|
||||||
|
valid_types = {"hardware", "software", "network", "security", "application", "none", "other"}
|
||||||
|
if failure_type not in valid_types:
|
||||||
|
raise HTTPException(status_code=422, detail=f"failure_type must be one of {sorted(valid_types)}")
|
||||||
|
|
||||||
|
with _db() as conn:
|
||||||
|
row = conn.execute("SELECT id FROM corpus_entries WHERE id = ?", (entry_id,)).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(status_code=404, detail="Entry not found")
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE corpus_entries SET label_state='labeled', failure_type=?, plain_explanation=?, "
|
||||||
|
"known_pattern=?, labeled_at=?, pii_flagged=? WHERE id=?",
|
||||||
|
(failure_type, plain_explanation, known_pattern, _now_iso(), pii_flagged, entry_id),
|
||||||
|
)
|
||||||
|
return {"labeled": True, "entry_id": entry_id}
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST /api/corpus/entries/{id}/skip ────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.post("/entries/{entry_id}/skip")
|
||||||
|
def skip_entry(entry_id: str) -> dict:
|
||||||
|
with _db() as conn:
|
||||||
|
row = conn.execute("SELECT id FROM corpus_entries WHERE id = ?", (entry_id,)).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(status_code=404, detail="Entry not found")
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE corpus_entries SET label_state='skipped' WHERE id=?", (entry_id,)
|
||||||
|
)
|
||||||
|
return {"skipped": True, "entry_id": entry_id}
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET /api/corpus/stats ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/stats")
|
||||||
|
def get_stats() -> dict:
|
||||||
|
with _db() as conn:
|
||||||
|
total = conn.execute("SELECT COUNT(*) FROM corpus_entries").fetchone()[0]
|
||||||
|
by_state = {
|
||||||
|
r["label_state"]: r["cnt"]
|
||||||
|
for r in conn.execute(
|
||||||
|
"SELECT label_state, COUNT(*) AS cnt FROM corpus_entries GROUP BY label_state"
|
||||||
|
).fetchall()
|
||||||
|
}
|
||||||
|
by_source = {
|
||||||
|
r["source_host"]: r["cnt"]
|
||||||
|
for r in conn.execute(
|
||||||
|
"SELECT source_host, COUNT(*) AS cnt FROM corpus_entries GROUP BY source_host"
|
||||||
|
).fetchall()
|
||||||
|
}
|
||||||
|
by_severity = {
|
||||||
|
r["severity"]: r["cnt"]
|
||||||
|
for r in conn.execute(
|
||||||
|
"SELECT severity, COUNT(*) AS cnt FROM corpus_entries "
|
||||||
|
"WHERE severity IS NOT NULL GROUP BY severity"
|
||||||
|
).fetchall()
|
||||||
|
}
|
||||||
|
batch_count = conn.execute("SELECT COUNT(*) FROM corpus_batches").fetchone()[0]
|
||||||
|
return {
|
||||||
|
"total_entries": total,
|
||||||
|
"batch_count": batch_count,
|
||||||
|
"by_label_state": by_state,
|
||||||
|
"by_source": by_source,
|
||||||
|
"by_severity": by_severity,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET /api/corpus/export ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/export")
|
||||||
|
def export_labeled() -> StreamingResponse:
|
||||||
|
"""Stream labeled, non-PII entries as JSONL for SFT harness."""
|
||||||
|
with _db() as conn:
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT source_host, source_id, severity, text, failure_type, plain_explanation, known_pattern "
|
||||||
|
"FROM corpus_entries "
|
||||||
|
"WHERE label_state = 'labeled' AND pii_flagged = 0 AND plain_explanation != ''"
|
||||||
|
"ORDER BY rowid"
|
||||||
|
).fetchall()
|
||||||
|
|
||||||
|
def _generate():
|
||||||
|
for row in rows:
|
||||||
|
record = {
|
||||||
|
"input": row["text"],
|
||||||
|
"output": row["plain_explanation"],
|
||||||
|
"metadata": {
|
||||||
|
"failure_type": row["failure_type"],
|
||||||
|
"source": row["source_host"],
|
||||||
|
"source_id": row["source_id"],
|
||||||
|
"severity": row["severity"],
|
||||||
|
"known_pattern": row["known_pattern"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
yield json.dumps(record) + "\n"
|
||||||
|
|
||||||
|
return StreamingResponse(
|
||||||
|
_generate(),
|
||||||
|
media_type="application/x-ndjson",
|
||||||
|
headers={"Content-Disposition": "attachment; filename=log_corpus_labeled.jsonl"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST /api/corpus/pipeline-ingest ─────────────────────────────────────────
|
||||||
|
|
||||||
|
def _ingest_one_file(conn: sqlite3.Connection, path: Path) -> int:
|
||||||
|
"""Parse a pipeline JSONL file and insert entries. Returns count stored."""
|
||||||
|
batch_id = str(uuid.uuid4())
|
||||||
|
lines = path.read_text(encoding="utf-8").splitlines()
|
||||||
|
entries_raw: list[dict] = []
|
||||||
|
for line in lines:
|
||||||
|
line = line.strip()
|
||||||
|
if not line:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
entries_raw.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
logger.debug("Skipping malformed line in %s", path.name)
|
||||||
|
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO corpus_batches (id, source_host, batch_type, received_at, entry_count, raw_json) "
|
||||||
|
"VALUES (?, ?, ?, ?, ?, ?)",
|
||||||
|
(batch_id, _PIPELINE_SOURCE_HOST, "pipeline_log", _now_iso(),
|
||||||
|
len(entries_raw), json.dumps({"file": path.name})),
|
||||||
|
)
|
||||||
|
|
||||||
|
stored = 0
|
||||||
|
for entry in entries_raw:
|
||||||
|
text = (entry.get("msg") or "").strip()
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
conn.execute(
|
||||||
|
"INSERT OR IGNORE INTO corpus_entries "
|
||||||
|
"(id, batch_id, source_host, timestamp_iso, severity, source_id, text, matched_patterns) "
|
||||||
|
"VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
|
||||||
|
(str(uuid.uuid4()), batch_id, _PIPELINE_SOURCE_HOST,
|
||||||
|
entry.get("ts"),
|
||||||
|
entry.get("level"),
|
||||||
|
entry.get("logger"),
|
||||||
|
text,
|
||||||
|
json.dumps([entry["extra"]] if entry.get("extra") else [])),
|
||||||
|
)
|
||||||
|
stored += 1
|
||||||
|
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO ingested_pipeline_files (filename, ingested_at, entry_count) VALUES (?, ?, ?)",
|
||||||
|
(path.name, _now_iso(), stored),
|
||||||
|
)
|
||||||
|
return stored
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/pipeline-ingest")
|
||||||
|
def pipeline_ingest() -> dict:
|
||||||
|
"""Walk the configured pipeline log directory and ingest new JSONL files.
|
||||||
|
|
||||||
|
Skips files already recorded in ingested_pipeline_files. Safe to call
|
||||||
|
repeatedly — idempotent by filename.
|
||||||
|
"""
|
||||||
|
ingest_dir = _pipeline_ingest_dir()
|
||||||
|
if ingest_dir is None:
|
||||||
|
raise HTTPException(404, "pipeline_ingest_dir not configured in label_tool.yaml")
|
||||||
|
|
||||||
|
ingested = 0
|
||||||
|
skipped = 0
|
||||||
|
total_stored = 0
|
||||||
|
files_detail: list[dict] = []
|
||||||
|
|
||||||
|
with _db() as conn:
|
||||||
|
already_done: set[str] = {
|
||||||
|
row[0]
|
||||||
|
for row in conn.execute("SELECT filename FROM ingested_pipeline_files").fetchall()
|
||||||
|
}
|
||||||
|
|
||||||
|
for path in sorted(ingest_dir.glob("*.jsonl")):
|
||||||
|
if path.name in already_done:
|
||||||
|
skipped += 1
|
||||||
|
continue
|
||||||
|
stored = _ingest_one_file(conn, path)
|
||||||
|
ingested += 1
|
||||||
|
total_stored += stored
|
||||||
|
files_detail.append({"file": path.name, "entries_stored": stored})
|
||||||
|
|
||||||
|
logger.info("Pipeline ingest: %d files ingested, %d skipped, %d entries stored",
|
||||||
|
ingested, skipped, total_stored)
|
||||||
|
return {
|
||||||
|
"ingested_files": ingested,
|
||||||
|
"skipped_files": skipped,
|
||||||
|
"entries_stored": total_stored,
|
||||||
|
"files": files_detail,
|
||||||
|
}
|
||||||
313
app/data/recipe_scan.py
Normal file
313
app/data/recipe_scan.py
Normal file
|
|
@ -0,0 +1,313 @@
|
||||||
|
"""Avocet — Recipe scan labeling API (avocet#65).
|
||||||
|
|
||||||
|
Receives recipe scan items from the Kiwi pipeline (scanner/phone image +
|
||||||
|
docuvision OCR extraction + ground-truth structured recipe), presents them
|
||||||
|
for human review, and exports approved/edited pairs in the messages chat
|
||||||
|
format for the vision fine-tune harness.
|
||||||
|
|
||||||
|
DB: data/recipe_scan.db (separate from corpus.db — different lifecycle)
|
||||||
|
No auth required — local admin tool, not a push endpoint.
|
||||||
|
|
||||||
|
All endpoints registered on `router`. api.py includes this with
|
||||||
|
prefix="/api/recipe-scan".
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import sqlite3
|
||||||
|
import uuid
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Generator, Literal
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
from pydantic import BaseModel, field_validator
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_ROOT = Path(__file__).parent.parent.parent
|
||||||
|
_DB_PATH: Path = _ROOT / "data" / "recipe_scan.db"
|
||||||
|
|
||||||
|
_VALID_MODALITIES = {"scanner", "phone", "handwritten"}
|
||||||
|
_VALID_STATUSES = {"pending", "approved", "edited", "rejected"}
|
||||||
|
|
||||||
|
_SCHEMA = """
|
||||||
|
CREATE TABLE IF NOT EXISTS recipe_scan_items (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
image_path TEXT NOT NULL,
|
||||||
|
modality TEXT NOT NULL DEFAULT 'scanner',
|
||||||
|
source TEXT NOT NULL DEFAULT 'purple_carrot',
|
||||||
|
extracted TEXT NOT NULL,
|
||||||
|
ground_truth TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL DEFAULT 'pending',
|
||||||
|
corrected TEXT,
|
||||||
|
labeled_at TEXT,
|
||||||
|
rejected_reason TEXT
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_rsi_status ON recipe_scan_items(status);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_rsi_modality ON recipe_scan_items(modality);
|
||||||
|
"""
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
# ── Testability seam ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def set_db_path(path: Path) -> None:
|
||||||
|
global _DB_PATH
|
||||||
|
_DB_PATH = path
|
||||||
|
|
||||||
|
|
||||||
|
# ── Internal helpers ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def _db() -> Generator[sqlite3.Connection, None, None]:
|
||||||
|
conn = sqlite3.connect(str(_DB_PATH))
|
||||||
|
conn.row_factory = sqlite3.Row
|
||||||
|
conn.execute("PRAGMA journal_mode=WAL")
|
||||||
|
try:
|
||||||
|
yield conn
|
||||||
|
conn.commit()
|
||||||
|
except Exception:
|
||||||
|
conn.rollback()
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _init_db() -> None:
|
||||||
|
with _db() as conn:
|
||||||
|
conn.executescript(_SCHEMA)
|
||||||
|
|
||||||
|
|
||||||
|
def _now_iso() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat()
|
||||||
|
|
||||||
|
|
||||||
|
def _build_training_pair(row: sqlite3.Row) -> dict:
|
||||||
|
"""Build a messages-format training pair from a labeled row.
|
||||||
|
|
||||||
|
user message: correction prompt + the docuvision-extracted JSON draft.
|
||||||
|
Trains the model to review and correct an existing extraction, which is
|
||||||
|
more data-efficient than producing from scratch when OCR is usually close.
|
||||||
|
|
||||||
|
assistant message: the approved ground truth (or human-corrected JSON).
|
||||||
|
"""
|
||||||
|
target_str = row["corrected"] if row["corrected"] else row["ground_truth"]
|
||||||
|
extracted = json.loads(row["extracted"])
|
||||||
|
target = json.loads(target_str)
|
||||||
|
user_content = (
|
||||||
|
"Review and correct this recipe extraction. "
|
||||||
|
"Return valid JSON with fields: title, description, ingredients, steps, "
|
||||||
|
"prep_time, cook_time, servings.\n\n"
|
||||||
|
f"Extraction to review:\n{json.dumps(extracted, ensure_ascii=False, indent=2)}"
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"id": row["id"],
|
||||||
|
"modality": row["modality"],
|
||||||
|
"source": row["source"],
|
||||||
|
"image_path": row["image_path"],
|
||||||
|
"messages": [
|
||||||
|
{"role": "user", "content": user_content},
|
||||||
|
{"role": "assistant", "content": json.dumps(target, ensure_ascii=False)},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
_init_db()
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST /import ───────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
class ImportItem(BaseModel):
|
||||||
|
id: str = ""
|
||||||
|
image_path: str
|
||||||
|
modality: Literal["scanner", "phone", "handwritten"] = "scanner"
|
||||||
|
source: str = "purple_carrot"
|
||||||
|
extracted: dict
|
||||||
|
ground_truth: dict
|
||||||
|
|
||||||
|
@field_validator("id", mode="before")
|
||||||
|
@classmethod
|
||||||
|
def default_id(cls, v: str) -> str:
|
||||||
|
return v or str(uuid.uuid4())
|
||||||
|
|
||||||
|
|
||||||
|
class ImportRequest(BaseModel):
|
||||||
|
items: list[ImportItem]
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/import")
|
||||||
|
def import_items(body: ImportRequest) -> dict:
|
||||||
|
"""Bulk-import scan items from the Kiwi pipeline. Idempotent by item id."""
|
||||||
|
stored = 0
|
||||||
|
with _db() as conn:
|
||||||
|
for item in body.items:
|
||||||
|
result = conn.execute(
|
||||||
|
"INSERT OR IGNORE INTO recipe_scan_items "
|
||||||
|
"(id, image_path, modality, source, extracted, ground_truth) "
|
||||||
|
"VALUES (?, ?, ?, ?, ?, ?)",
|
||||||
|
(item.id, item.image_path, item.modality, item.source,
|
||||||
|
json.dumps(item.extracted), json.dumps(item.ground_truth)),
|
||||||
|
)
|
||||||
|
stored += result.rowcount
|
||||||
|
return {"imported": stored, "total_submitted": len(body.items)}
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET /next ─────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/next")
|
||||||
|
def get_next() -> dict:
|
||||||
|
"""Return the next pending item for review, oldest-first."""
|
||||||
|
with _db() as conn:
|
||||||
|
row = conn.execute(
|
||||||
|
"SELECT * FROM recipe_scan_items WHERE status = 'pending' ORDER BY rowid LIMIT 1"
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(404, "No pending items in queue")
|
||||||
|
return {
|
||||||
|
**dict(row),
|
||||||
|
"extracted": json.loads(row["extracted"]),
|
||||||
|
"ground_truth": json.loads(row["ground_truth"]),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST /items/{id}/approve ──────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.post("/items/{item_id}/approve")
|
||||||
|
def approve_item(item_id: str) -> dict:
|
||||||
|
"""Mark item as approved — extracted JSON is close enough to ground truth."""
|
||||||
|
with _db() as conn:
|
||||||
|
row = conn.execute("SELECT id FROM recipe_scan_items WHERE id = ?", (item_id,)).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(404, "Item not found")
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE recipe_scan_items SET status='approved', labeled_at=? WHERE id=?",
|
||||||
|
(_now_iso(), item_id),
|
||||||
|
)
|
||||||
|
return {"status": "approved", "id": item_id}
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST /items/{id}/edit ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
class EditBody(BaseModel):
|
||||||
|
corrected: dict
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/items/{item_id}/edit")
|
||||||
|
def edit_item(item_id: str, body: EditBody) -> dict:
|
||||||
|
"""Approve with a human-corrected JSON. corrected overrides extracted in export."""
|
||||||
|
with _db() as conn:
|
||||||
|
row = conn.execute("SELECT id FROM recipe_scan_items WHERE id = ?", (item_id,)).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(404, "Item not found")
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE recipe_scan_items SET status='edited', corrected=?, labeled_at=? WHERE id=?",
|
||||||
|
(json.dumps(body.corrected), _now_iso(), item_id),
|
||||||
|
)
|
||||||
|
return {"status": "edited", "id": item_id}
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST /items/{id}/reject ───────────────────────────────────────────────────
|
||||||
|
|
||||||
|
class RejectBody(BaseModel):
|
||||||
|
reason: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/items/{item_id}/reject")
|
||||||
|
def reject_item(item_id: str, body: RejectBody = RejectBody()) -> dict:
|
||||||
|
"""Reject item — extraction too broken to use for training."""
|
||||||
|
with _db() as conn:
|
||||||
|
row = conn.execute("SELECT id FROM recipe_scan_items WHERE id = ?", (item_id,)).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(404, "Item not found")
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE recipe_scan_items SET status='rejected', rejected_reason=?, labeled_at=? WHERE id=?",
|
||||||
|
(body.reason or None, _now_iso(), item_id),
|
||||||
|
)
|
||||||
|
return {"status": "rejected", "id": item_id}
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET /stats ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/stats")
|
||||||
|
def get_stats() -> dict:
|
||||||
|
with _db() as conn:
|
||||||
|
total = conn.execute("SELECT COUNT(*) FROM recipe_scan_items").fetchone()[0]
|
||||||
|
by_status = {
|
||||||
|
r["status"]: r["cnt"]
|
||||||
|
for r in conn.execute(
|
||||||
|
"SELECT status, COUNT(*) AS cnt FROM recipe_scan_items GROUP BY status"
|
||||||
|
).fetchall()
|
||||||
|
}
|
||||||
|
by_modality = {
|
||||||
|
r["modality"]: r["cnt"]
|
||||||
|
for r in conn.execute(
|
||||||
|
"SELECT modality, COUNT(*) AS cnt FROM recipe_scan_items GROUP BY modality"
|
||||||
|
).fetchall()
|
||||||
|
}
|
||||||
|
export_ready = conn.execute(
|
||||||
|
"SELECT COUNT(*) FROM recipe_scan_items WHERE status IN ('approved', 'edited')"
|
||||||
|
).fetchone()[0]
|
||||||
|
return {
|
||||||
|
"total": total,
|
||||||
|
"by_status": by_status,
|
||||||
|
"by_modality": by_modality,
|
||||||
|
"export_ready": export_ready,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET /export ───────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/export")
|
||||||
|
def export_pairs() -> StreamingResponse:
|
||||||
|
"""Stream approved/edited items as JSONL training pairs (messages format)."""
|
||||||
|
with _db() as conn:
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT * FROM recipe_scan_items WHERE status IN ('approved', 'edited') ORDER BY rowid"
|
||||||
|
).fetchall()
|
||||||
|
|
||||||
|
def _generate():
|
||||||
|
for row in rows:
|
||||||
|
yield json.dumps(_build_training_pair(row), ensure_ascii=False) + "\n"
|
||||||
|
|
||||||
|
return StreamingResponse(
|
||||||
|
_generate(),
|
||||||
|
media_type="application/x-ndjson",
|
||||||
|
headers={"Content-Disposition": "attachment; filename=recipe_scan_pairs.jsonl"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET /image ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
_IMAGE_ROOT = Path("/Library/Assets/kiwi")
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/image")
|
||||||
|
def serve_image(path: str) -> StreamingResponse:
|
||||||
|
"""Serve a scan image from /Library/Assets/kiwi/.
|
||||||
|
|
||||||
|
path must resolve within /Library/Assets/kiwi/ — rejects traversal attempts.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
resolved = Path(path).resolve()
|
||||||
|
_IMAGE_ROOT.resolve() # ensure root itself is valid
|
||||||
|
resolved.relative_to(_IMAGE_ROOT.resolve())
|
||||||
|
except (ValueError, OSError):
|
||||||
|
raise HTTPException(403, "Path outside allowed image directory")
|
||||||
|
|
||||||
|
if not resolved.exists():
|
||||||
|
raise HTTPException(404, "Image not found")
|
||||||
|
|
||||||
|
suffix = resolved.suffix.lower()
|
||||||
|
media_types = {".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".png": "image/png", ".webp": "image/webp"}
|
||||||
|
media_type = media_types.get(suffix, "application/octet-stream")
|
||||||
|
|
||||||
|
return StreamingResponse(
|
||||||
|
open(resolved, "rb"),
|
||||||
|
media_type=media_type,
|
||||||
|
headers={"Cache-Control": "public, max-age=86400"},
|
||||||
|
)
|
||||||
|
|
@ -12,27 +12,33 @@ Route prefixes when mounted at /api in api.py:
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
from fastapi import APIRouter
|
from fastapi import APIRouter
|
||||||
|
|
||||||
from app.cforch import router as _cforch_router
|
from app.cforch import router as _cforch_router
|
||||||
from app.style import router as _style_router
|
from app.style import router as _style_router
|
||||||
from app.voice import router as _voice_router
|
from app.voice import router as _voice_router
|
||||||
from app.plans_bench import router as _plans_router
|
from app.plans_bench import router as _plans_router
|
||||||
|
from app.eval.embed_bench import router as _embed_router
|
||||||
|
|
||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
router.include_router(_cforch_router, prefix="/cforch")
|
router.include_router(_cforch_router, prefix="/cforch")
|
||||||
router.include_router(_style_router, prefix="/style")
|
router.include_router(_style_router, prefix="/style")
|
||||||
router.include_router(_voice_router, prefix="/voice")
|
router.include_router(_voice_router, prefix="/voice")
|
||||||
router.include_router(_plans_router, prefix="/plans-bench")
|
router.include_router(_plans_router, prefix="/plans-bench")
|
||||||
|
router.include_router(_embed_router, prefix="/embed-bench")
|
||||||
|
|
||||||
|
|
||||||
def set_config_dir(path) -> None:
|
def set_config_dir(path: Path | None) -> None:
|
||||||
"""Propagate config dir override to all sub-modules -- used by tests."""
|
"""Propagate config dir override to all sub-modules -- used by tests."""
|
||||||
import app.cforch as _cforch_mod
|
import app.cforch as _cforch_mod
|
||||||
import app.style as _style_mod
|
import app.style as _style_mod
|
||||||
import app.voice as _voice_mod
|
import app.voice as _voice_mod
|
||||||
import app.plans_bench as _plans_mod
|
import app.plans_bench as _plans_mod
|
||||||
|
import app.eval.embed_bench as _embed_mod
|
||||||
_cforch_mod.set_config_dir(path)
|
_cforch_mod.set_config_dir(path)
|
||||||
_style_mod.set_config_dir(path)
|
_style_mod.set_config_dir(path)
|
||||||
_voice_mod.set_config_dir(path)
|
_voice_mod.set_config_dir(path)
|
||||||
_plans_mod.set_config_dir(path)
|
_plans_mod.set_config_dir(path)
|
||||||
|
_embed_mod.set_config_dir(path)
|
||||||
|
|
|
||||||
293
app/eval/embed_bench.py
Normal file
293
app/eval/embed_bench.py
Normal file
|
|
@ -0,0 +1,293 @@
|
||||||
|
"""Avocet — embedding model comparison harness.
|
||||||
|
|
||||||
|
Exposes FastAPI routes under /api/embed-bench (mounted via app/eval/cforch.py).
|
||||||
|
All computation is local: no LLM inference, Ollama only. MIT tier throughout.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import csv
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import math
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
import yaml
|
||||||
|
from fastapi import APIRouter, HTTPException
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
from pydantic import BaseModel, field_validator
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_ROOT = Path(__file__).parent.parent.parent
|
||||||
|
_CONFIG_DIR: Path | None = None # override via set_config_dir() in tests
|
||||||
|
_RUN_ACTIVE: bool = False
|
||||||
|
_RATINGS_FILE = _ROOT / "data" / "embed_bench_ratings.jsonl"
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
# ── Testability seam ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def set_config_dir(path: Path | None) -> None:
|
||||||
|
global _CONFIG_DIR
|
||||||
|
_CONFIG_DIR = path
|
||||||
|
|
||||||
|
|
||||||
|
# ── Internal helpers ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _config_file() -> Path:
|
||||||
|
if _CONFIG_DIR is not None:
|
||||||
|
return _CONFIG_DIR / "label_tool.yaml"
|
||||||
|
return _ROOT / "config" / "label_tool.yaml"
|
||||||
|
|
||||||
|
|
||||||
|
def _load_config() -> dict[str, Any]:
|
||||||
|
f = _config_file()
|
||||||
|
if not f.exists():
|
||||||
|
return {}
|
||||||
|
try:
|
||||||
|
return yaml.safe_load(f.read_text(encoding="utf-8")) or {}
|
||||||
|
except yaml.YAMLError as exc:
|
||||||
|
logger.warning("Failed to parse embed_bench config %s: %s", f, exc)
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def _ollama_url() -> str:
|
||||||
|
cfg = _load_config()
|
||||||
|
embed_cfg = cfg.get("embed_bench", {}) or {}
|
||||||
|
cforch_cfg = cfg.get("cforch", {}) or {}
|
||||||
|
return (
|
||||||
|
embed_cfg.get("ollama_url")
|
||||||
|
or cforch_cfg.get("ollama_url", "http://localhost:11434")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _ratings_path() -> Path:
|
||||||
|
if _CONFIG_DIR is not None:
|
||||||
|
return _CONFIG_DIR / "embed_bench_ratings.jsonl"
|
||||||
|
return _RATINGS_FILE
|
||||||
|
|
||||||
|
|
||||||
|
def _cosine(a: list[float], b: list[float]) -> float:
|
||||||
|
if len(a) != len(b):
|
||||||
|
raise ValueError(
|
||||||
|
f"Embedding dimension mismatch: {len(a)} vs {len(b)}"
|
||||||
|
)
|
||||||
|
dot = sum(x * y for x, y in zip(a, b))
|
||||||
|
mag_a = math.sqrt(sum(x * x for x in a))
|
||||||
|
mag_b = math.sqrt(sum(x * x for x in b))
|
||||||
|
if mag_a == 0.0 or mag_b == 0.0:
|
||||||
|
return 0.0
|
||||||
|
return dot / (mag_a * mag_b)
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET /models ───────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/models")
|
||||||
|
def get_models() -> dict:
|
||||||
|
"""Return Ollama embedding models available on the configured instance."""
|
||||||
|
ollama = _ollama_url()
|
||||||
|
models: list[dict] = []
|
||||||
|
try:
|
||||||
|
resp = httpx.get(f"{ollama}/api/tags", timeout=5.0)
|
||||||
|
resp.raise_for_status()
|
||||||
|
for entry in resp.json().get("models", []):
|
||||||
|
models.append({
|
||||||
|
"name": entry.get("name", ""),
|
||||||
|
"size": entry.get("size", 0),
|
||||||
|
})
|
||||||
|
except httpx.HTTPStatusError as exc:
|
||||||
|
logger.warning("Ollama /api/tags returned HTTP %s: %s", exc.response.status_code, exc)
|
||||||
|
except httpx.RequestError as exc:
|
||||||
|
logger.warning("Failed to reach Ollama for model list: %s", exc)
|
||||||
|
return {"models": models, "ollama_url": ollama}
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST /run ─────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
class RunRequest(BaseModel):
|
||||||
|
corpus: list[str]
|
||||||
|
queries: list[str]
|
||||||
|
models: list[str]
|
||||||
|
top_k: int = 5
|
||||||
|
ollama_url: str = ""
|
||||||
|
|
||||||
|
@field_validator("corpus")
|
||||||
|
@classmethod
|
||||||
|
def corpus_nonempty(cls, v: list[str]) -> list[str]:
|
||||||
|
if not v:
|
||||||
|
raise ValueError("corpus must not be empty")
|
||||||
|
return v
|
||||||
|
|
||||||
|
@field_validator("queries")
|
||||||
|
@classmethod
|
||||||
|
def queries_nonempty(cls, v: list[str]) -> list[str]:
|
||||||
|
if not v:
|
||||||
|
raise ValueError("queries must not be empty")
|
||||||
|
return v
|
||||||
|
|
||||||
|
@field_validator("models")
|
||||||
|
@classmethod
|
||||||
|
def models_nonempty(cls, v: list[str]) -> list[str]:
|
||||||
|
if not v:
|
||||||
|
raise ValueError("models must contain at least one model name")
|
||||||
|
return v
|
||||||
|
|
||||||
|
|
||||||
|
def _embed_texts(ollama: str, model: str, texts: list[str]) -> list[list[float]]:
|
||||||
|
"""Batch-embed texts via Ollama /v1/embeddings. Returns one vector per text."""
|
||||||
|
resp = httpx.post(
|
||||||
|
f"{ollama}/v1/embeddings",
|
||||||
|
json={"model": model, "input": texts},
|
||||||
|
timeout=120.0,
|
||||||
|
)
|
||||||
|
resp.raise_for_status()
|
||||||
|
data = resp.json().get("data", [])
|
||||||
|
return [item["embedding"] for item in data]
|
||||||
|
|
||||||
|
|
||||||
|
def _sse(event: dict) -> str:
|
||||||
|
return f"data: {json.dumps(event)}\n\n"
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/run")
|
||||||
|
def run_embed_bench(req: RunRequest) -> StreamingResponse:
|
||||||
|
"""Embed corpus + queries with each model; stream SSE results."""
|
||||||
|
global _RUN_ACTIVE
|
||||||
|
|
||||||
|
if _RUN_ACTIVE:
|
||||||
|
raise HTTPException(409, "An embedding benchmark run is already active")
|
||||||
|
|
||||||
|
ollama = req.ollama_url or _ollama_url()
|
||||||
|
|
||||||
|
def _generate():
|
||||||
|
global _RUN_ACTIVE
|
||||||
|
_RUN_ACTIVE = True
|
||||||
|
try:
|
||||||
|
for model_idx, model in enumerate(req.models, start=1):
|
||||||
|
yield _sse({
|
||||||
|
"type": "progress",
|
||||||
|
"msg": f"Indexing corpus with {model} ({model_idx}/{len(req.models)})...",
|
||||||
|
})
|
||||||
|
try:
|
||||||
|
corpus_vecs = _embed_texts(ollama, model, req.corpus)
|
||||||
|
except Exception as exc:
|
||||||
|
yield _sse({"type": "error", "msg": f"Ollama error for {model}: {exc}"})
|
||||||
|
continue
|
||||||
|
|
||||||
|
yield _sse({
|
||||||
|
"type": "progress",
|
||||||
|
"msg": f"Running queries with {model}...",
|
||||||
|
})
|
||||||
|
|
||||||
|
for q_idx, query in enumerate(req.queries):
|
||||||
|
try:
|
||||||
|
q_vecs = _embed_texts(ollama, model, [query])
|
||||||
|
except Exception as exc:
|
||||||
|
yield _sse({"type": "error", "msg": f"Query embed error ({model}): {exc}"})
|
||||||
|
continue
|
||||||
|
q_vec = q_vecs[0]
|
||||||
|
scored = sorted(
|
||||||
|
[
|
||||||
|
{"chunk_idx": i, "text": chunk, "score": round(_cosine(q_vec, cv), 4)}
|
||||||
|
for i, (chunk, cv) in enumerate(zip(req.corpus, corpus_vecs))
|
||||||
|
],
|
||||||
|
key=lambda h: h["score"],
|
||||||
|
reverse=True,
|
||||||
|
)[: req.top_k]
|
||||||
|
yield _sse({
|
||||||
|
"type": "result",
|
||||||
|
"query_idx": q_idx,
|
||||||
|
"query": query,
|
||||||
|
"model": model,
|
||||||
|
"hits": scored,
|
||||||
|
})
|
||||||
|
|
||||||
|
yield _sse({"type": "done"})
|
||||||
|
finally:
|
||||||
|
_RUN_ACTIVE = False
|
||||||
|
|
||||||
|
return StreamingResponse(_generate(), media_type="text/event-stream")
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST /rate ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
_VALID_RATINGS = {"relevant", "not_relevant"}
|
||||||
|
|
||||||
|
|
||||||
|
class RatingRequest(BaseModel):
|
||||||
|
query: str
|
||||||
|
model: str
|
||||||
|
chunk_text: str
|
||||||
|
chunk_idx: int
|
||||||
|
rating: str
|
||||||
|
|
||||||
|
@field_validator("rating")
|
||||||
|
@classmethod
|
||||||
|
def rating_valid(cls, v: str) -> str:
|
||||||
|
if v not in _VALID_RATINGS:
|
||||||
|
raise ValueError(f"rating must be one of {_VALID_RATINGS}")
|
||||||
|
return v
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/rate")
|
||||||
|
def rate_result(req: RatingRequest) -> dict:
|
||||||
|
"""Append one rating to the JSONL ratings file."""
|
||||||
|
entry = {
|
||||||
|
"timestamp": datetime.now(timezone.utc).isoformat(),
|
||||||
|
"query": req.query,
|
||||||
|
"model": req.model,
|
||||||
|
"chunk_idx": req.chunk_idx,
|
||||||
|
"chunk_text": req.chunk_text,
|
||||||
|
"rating": req.rating,
|
||||||
|
}
|
||||||
|
path = _ratings_path()
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with path.open("a", encoding="utf-8") as fh:
|
||||||
|
fh.write(json.dumps(entry) + "\n")
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET /export ───────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
_CSV_FIELDS = ["timestamp", "query", "model", "chunk_idx", "chunk_text", "rating"]
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/export")
|
||||||
|
def export_ratings(format: str = "csv") -> Any:
|
||||||
|
"""Download ratings as CSV or JSON."""
|
||||||
|
path = _ratings_path()
|
||||||
|
rows: list[dict] = []
|
||||||
|
if path.exists():
|
||||||
|
for raw in path.read_text(encoding="utf-8").splitlines():
|
||||||
|
raw = raw.strip()
|
||||||
|
if raw:
|
||||||
|
try:
|
||||||
|
rows.append(json.loads(raw))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
date_str = datetime.now(timezone.utc).strftime("%Y-%m-%d")
|
||||||
|
|
||||||
|
if format == "json":
|
||||||
|
content = json.dumps(rows, ensure_ascii=False, indent=2)
|
||||||
|
return StreamingResponse(
|
||||||
|
iter([content]),
|
||||||
|
media_type="application/json",
|
||||||
|
headers={"Content-Disposition": f'attachment; filename="embed_comparison_{date_str}.json"'},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Default: CSV
|
||||||
|
buf = io.StringIO()
|
||||||
|
writer = csv.DictWriter(buf, fieldnames=_CSV_FIELDS, extrasaction="ignore")
|
||||||
|
writer.writeheader()
|
||||||
|
writer.writerows(rows)
|
||||||
|
return StreamingResponse(
|
||||||
|
iter([buf.getvalue()]),
|
||||||
|
media_type="text/csv",
|
||||||
|
headers={"Content-Disposition": f'attachment; filename="embed_comparison_{date_str}.csv"'},
|
||||||
|
)
|
||||||
|
|
@ -38,13 +38,17 @@ except ImportError: # pragma: no cover
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
_ROOT = Path(__file__).parent.parent
|
_ROOT = Path(__file__).parent.parent
|
||||||
_MODELS_DIR: Path = _ROOT / "models"
|
_MODELS_DIR: Path = Path(
|
||||||
|
os.environ.get("AVOCET_MODELS_DIR", str(_ROOT / "models"))
|
||||||
|
)
|
||||||
_QUEUE_DIR: Path = _ROOT / "data"
|
_QUEUE_DIR: Path = _ROOT / "data"
|
||||||
|
|
||||||
# Service-specific model destinations.
|
# Service-specific model destinations.
|
||||||
# cf-text models land on the NFS-mounted shared asset store so every cluster
|
# cf-text models land on the NFS-mounted shared asset store so every cluster
|
||||||
# node can reach them without a separate download. Avocet classifiers stay local
|
# node can reach them without a separate download. Avocet classifiers default
|
||||||
# because they are fine-tuned in-place and are only consumed by avocet itself.
|
# to a local path but can be redirected via AVOCET_MODELS_DIR — set this to
|
||||||
|
# /Library/Assets/LLM/avocet/models on NFS-connected nodes to keep all model
|
||||||
|
# weights out of the repo directory.
|
||||||
# Override via CF_TEXT_MODELS_DIR env var (useful for dev / non-NFS setups).
|
# Override via CF_TEXT_MODELS_DIR env var (useful for dev / non-NFS setups).
|
||||||
_CF_TEXT_MODELS_DIR: Path = Path(
|
_CF_TEXT_MODELS_DIR: Path = Path(
|
||||||
os.environ.get("CF_TEXT_MODELS_DIR", "/Library/Assets/LLM/cf-text/models")
|
os.environ.get("CF_TEXT_MODELS_DIR", "/Library/Assets/LLM/cf-text/models")
|
||||||
|
|
@ -120,11 +124,12 @@ _TAG_TO_INFO: dict[str, _TagInfo] = {
|
||||||
"image-classification": {"adapter": None, "role": "vision", "service": "cf-vision"},
|
"image-classification": {"adapter": None, "role": "vision", "service": "cf-vision"},
|
||||||
"zero-shot-image-classification": {"adapter": None, "role": "vision", "service": "cf-vision"},
|
"zero-shot-image-classification": {"adapter": None, "role": "vision", "service": "cf-vision"},
|
||||||
"image-feature-extraction": {"adapter": None, "role": "embedding", "service": "cf-vision"},
|
"image-feature-extraction": {"adapter": None, "role": "embedding", "service": "cf-vision"},
|
||||||
# Generative VLMs (image+text → text) — run under vllm, not cf-vision.
|
# Generative VLMs (image+text → text) — GGUF quants run via llama.cpp (cf-text).
|
||||||
# cf-vision is a classifier/embedder service; generative VLMs like Qwen-VL,
|
# cf-vision is a classifier/embedder service; generative VLMs like Qwen2-VL
|
||||||
# LLaVA, and InternVL are textgen models that happen to accept image inputs.
|
# and LLaVA accept image inputs but are textgen at the backend level.
|
||||||
"image-text-to-text": {"adapter": None, "role": "vlm", "service": "vllm"},
|
# Full-precision HF-format VLMs would use vllm, but our fleet uses GGUF quants.
|
||||||
"visual-question-answering": {"adapter": None, "role": "vlm", "service": "vllm"},
|
"image-text-to-text": {"adapter": None, "role": "vlm", "service": "cf-text"},
|
||||||
|
"visual-question-answering": {"adapter": None, "role": "vlm", "service": "cf-text"},
|
||||||
# Image generation — cf-image (text → image; distinct from cf-vision image understanding)
|
# Image generation — cf-image (text → image; distinct from cf-vision image understanding)
|
||||||
"text-to-image": {"adapter": None, "role": "image-gen", "service": "cf-image"},
|
"text-to-image": {"adapter": None, "role": "image-gen", "service": "cf-image"},
|
||||||
# Embedding — cf-core shared embedding layer
|
# Embedding — cf-core shared embedding layer
|
||||||
|
|
@ -139,6 +144,11 @@ def set_models_dir(path: Path) -> None:
|
||||||
_MODELS_DIR = path
|
_MODELS_DIR = path
|
||||||
|
|
||||||
|
|
||||||
|
def set_cf_text_models_dir(path: Path) -> None:
|
||||||
|
global _CF_TEXT_MODELS_DIR
|
||||||
|
_CF_TEXT_MODELS_DIR = path
|
||||||
|
|
||||||
|
|
||||||
def set_queue_dir(path: Path) -> None:
|
def set_queue_dir(path: Path) -> None:
|
||||||
global _QUEUE_DIR
|
global _QUEUE_DIR
|
||||||
_QUEUE_DIR = path
|
_QUEUE_DIR = path
|
||||||
|
|
|
||||||
180
app/nodes.py
180
app/nodes.py
|
|
@ -120,7 +120,7 @@ def list_nodes() -> list:
|
||||||
try:
|
try:
|
||||||
r = httpx.get(f"{coordinator_url}/api/nodes", timeout=5.0)
|
r = httpx.get(f"{coordinator_url}/api/nodes", timeout=5.0)
|
||||||
r.raise_for_status()
|
r.raise_for_status()
|
||||||
coord_nodes: list[dict] = r.json()
|
coord_nodes: list[dict] = r.json().get("nodes", [])
|
||||||
except httpx.HTTPError as exc:
|
except httpx.HTTPError as exc:
|
||||||
logger.warning("Coordinator unreachable: %s", exc)
|
logger.warning("Coordinator unreachable: %s", exc)
|
||||||
return []
|
return []
|
||||||
|
|
@ -128,7 +128,7 @@ def list_nodes() -> list:
|
||||||
try:
|
try:
|
||||||
sr = httpx.get(f"{coordinator_url}/api/services", timeout=5.0)
|
sr = httpx.get(f"{coordinator_url}/api/services", timeout=5.0)
|
||||||
sr.raise_for_status()
|
sr.raise_for_status()
|
||||||
services_data: list[dict] = sr.json()
|
services_data: list[dict] = sr.json().get("services", [])
|
||||||
except httpx.HTTPError:
|
except httpx.HTTPError:
|
||||||
logger.warning("Services API unreachable for %s, skipping", coordinator_url)
|
logger.warning("Services API unreachable for %s, skipping", coordinator_url)
|
||||||
services_data = []
|
services_data = []
|
||||||
|
|
@ -294,6 +294,99 @@ def update_gpu_services(node_id: str, gpu_id: int, body: UpdateServicesRequest)
|
||||||
|
|
||||||
return {"ok": True, "reloaded": reloaded, "warnings": []}
|
return {"ok": True, "reloaded": reloaded, "warnings": []}
|
||||||
|
|
||||||
|
# ── Profile save / generate ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
class SaveProfileRequest(BaseModel):
|
||||||
|
profile: dict
|
||||||
|
|
||||||
|
|
||||||
|
@router.put("/nodes/{node_id}/profile", status_code=200)
|
||||||
|
def save_profile(node_id: str, body: SaveProfileRequest) -> dict:
|
||||||
|
"""Write a full profile dict to disk as YAML, then trigger coordinator reload."""
|
||||||
|
p = _profile_path(node_id)
|
||||||
|
if p is None:
|
||||||
|
raise HTTPException(500, "profiles_dir not configured in label_tool.yaml")
|
||||||
|
p.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
tmp = Path(str(p) + ".tmp")
|
||||||
|
tmp.write_text(
|
||||||
|
yaml.dump(body.profile, default_flow_style=False, allow_unicode=True, sort_keys=False),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
os.replace(tmp, p)
|
||||||
|
|
||||||
|
cfg = _load_config()
|
||||||
|
coordinator_url = cfg.get("coordinator_url", "") or ""
|
||||||
|
reloaded = False
|
||||||
|
if coordinator_url:
|
||||||
|
try:
|
||||||
|
import httpx
|
||||||
|
rr = httpx.post(f"{coordinator_url}/api/nodes/{node_id}/reload-profile", timeout=5.0)
|
||||||
|
reloaded = rr.status_code < 300
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("Coordinator reload failed for %s: %s", node_id, exc)
|
||||||
|
return {"ok": True, "reloaded": reloaded}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/nodes/{node_id}/profile/generate")
|
||||||
|
def generate_profile(node_id: str) -> dict:
|
||||||
|
"""Return a profile skeleton seeded from coordinator GPU data.
|
||||||
|
|
||||||
|
If a profile already exists, preserves its services section and only
|
||||||
|
refreshes the nodes hardware section. Never writes to disk — the caller
|
||||||
|
must call PUT /profile to persist.
|
||||||
|
"""
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
cfg = _load_config()
|
||||||
|
coordinator_url = cfg.get("coordinator_url", "") or ""
|
||||||
|
if not coordinator_url:
|
||||||
|
raise HTTPException(503, "coordinator_url not configured")
|
||||||
|
|
||||||
|
try:
|
||||||
|
r = httpx.get(f"{coordinator_url}/api/nodes", timeout=5.0)
|
||||||
|
r.raise_for_status()
|
||||||
|
coord_nodes: list[dict] = r.json().get("nodes", [])
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
raise HTTPException(502, f"Coordinator unreachable: {exc}")
|
||||||
|
|
||||||
|
node = next((n for n in coord_nodes if n.get("node_id") == node_id), None)
|
||||||
|
if node is None:
|
||||||
|
raise HTTPException(404, f"Node {node_id!r} not found in coordinator")
|
||||||
|
|
||||||
|
gpus = [
|
||||||
|
{
|
||||||
|
"id": g.get("gpu_id", i),
|
||||||
|
"vram_mb": g.get("vram_total_mb", 0),
|
||||||
|
"compute_cap": g.get("compute_cap", 0.0),
|
||||||
|
"card": g.get("card", g.get("name", "")),
|
||||||
|
"role": "inference",
|
||||||
|
"services": [],
|
||||||
|
}
|
||||||
|
for i, g in enumerate(node.get("gpus", []))
|
||||||
|
]
|
||||||
|
vram_total = max((g["vram_mb"] for g in gpus), default=0)
|
||||||
|
|
||||||
|
existing = _load_profile(node_id) or {}
|
||||||
|
return {
|
||||||
|
"schema_version": existing.get("schema_version", 1),
|
||||||
|
"name": existing.get("name", f"node-{node_id}"),
|
||||||
|
"vram_total_mb": vram_total,
|
||||||
|
"eviction_timeout_s": existing.get("eviction_timeout_s", 10.0),
|
||||||
|
"services": existing.get("services", {}),
|
||||||
|
"nodes": {
|
||||||
|
node_id: {
|
||||||
|
"local_model_root": (
|
||||||
|
(existing.get("nodes", {}) or {})
|
||||||
|
.get(node_id, {})
|
||||||
|
.get("local_model_root", "")
|
||||||
|
),
|
||||||
|
"gpus": gpus,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"model_size_hints": existing.get("model_size_hints", {}),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
# ── Ollama model management ────────────────────────────────────────────────────
|
# ── Ollama model management ────────────────────────────────────────────────────
|
||||||
|
|
||||||
class PullRequest(BaseModel):
|
class PullRequest(BaseModel):
|
||||||
|
|
@ -357,3 +450,86 @@ def delete_ollama_model(node_id: str, name: str) -> dict:
|
||||||
raise
|
raise
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
raise HTTPException(502, f"Ollama unreachable: {exc}")
|
raise HTTPException(502, f"Ollama unreachable: {exc}")
|
||||||
|
|
||||||
|
|
||||||
|
# ── Model deploy (add catalog entry) ──────────────────────────────────────────
|
||||||
|
|
||||||
|
class DeployModelRequest(BaseModel):
|
||||||
|
model_id: str
|
||||||
|
service_type: str
|
||||||
|
vram_mb: int
|
||||||
|
description: str = ""
|
||||||
|
hf_repo: str = ""
|
||||||
|
path: str = "" # explicit path; if empty, constructed from model_base_path + hf_repo slug
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/nodes/{node_id}/models/deploy", status_code=200)
|
||||||
|
def deploy_model(node_id: str, body: DeployModelRequest) -> dict:
|
||||||
|
"""Register a model in the node's service catalog.
|
||||||
|
|
||||||
|
Adds (or updates) the catalog entry for body.model_id under the given
|
||||||
|
service_type in the node's profile YAML, then triggers a coordinator reload.
|
||||||
|
Does not download the model — that is the user's responsibility.
|
||||||
|
Returns the resolved path so the caller can see where the model should land.
|
||||||
|
"""
|
||||||
|
p = _profile_path(node_id)
|
||||||
|
if p is None or not p.exists():
|
||||||
|
raise HTTPException(404, f"No profile found for node {node_id!r}")
|
||||||
|
|
||||||
|
try:
|
||||||
|
profile = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
|
||||||
|
except yaml.YAMLError as exc:
|
||||||
|
raise HTTPException(500, f"Malformed profile YAML: {exc}")
|
||||||
|
|
||||||
|
services_def = profile.get("services", {}) or {}
|
||||||
|
svc = services_def.get(body.service_type)
|
||||||
|
if svc is None:
|
||||||
|
raise HTTPException(
|
||||||
|
422,
|
||||||
|
f"Service '{body.service_type}' not defined in node '{node_id}' profile; "
|
||||||
|
"add it first via the profile editor",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Resolve path: explicit > model_base_path + hf slug > model_id slug
|
||||||
|
model_path = body.path.strip()
|
||||||
|
if not model_path:
|
||||||
|
base = (svc.get("model_base_path", "") or "").rstrip("/")
|
||||||
|
if not base:
|
||||||
|
raise HTTPException(
|
||||||
|
422,
|
||||||
|
f"Service '{body.service_type}' has no model_base_path; supply an explicit path",
|
||||||
|
)
|
||||||
|
slug_src = body.hf_repo.strip() if body.hf_repo.strip() else body.model_id
|
||||||
|
hf_slug = slug_src.replace("/", "--")
|
||||||
|
model_path = f"{base}/{hf_slug}"
|
||||||
|
|
||||||
|
# Immutable catalog update — spread, never mutate
|
||||||
|
entry: dict = {"path": model_path, "vram_mb": body.vram_mb}
|
||||||
|
if body.description:
|
||||||
|
entry["description"] = body.description
|
||||||
|
new_catalog = {**(svc.get("catalog") or {}), body.model_id: entry}
|
||||||
|
new_svc = {**svc, "catalog": new_catalog}
|
||||||
|
new_services = {**services_def, body.service_type: new_svc}
|
||||||
|
new_profile = {**profile, "services": new_services}
|
||||||
|
|
||||||
|
# Atomic write
|
||||||
|
tmp = Path(str(p) + ".tmp")
|
||||||
|
tmp.write_text(
|
||||||
|
yaml.dump(new_profile, default_flow_style=False, allow_unicode=True, sort_keys=False),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
os.replace(tmp, p)
|
||||||
|
|
||||||
|
# Trigger coordinator reload
|
||||||
|
cfg = _load_config()
|
||||||
|
coordinator_url = cfg.get("coordinator_url", "") or ""
|
||||||
|
reloaded = False
|
||||||
|
if coordinator_url:
|
||||||
|
try:
|
||||||
|
import httpx
|
||||||
|
rr = httpx.post(f"{coordinator_url}/api/nodes/{node_id}/reload-profile", timeout=5.0)
|
||||||
|
reloaded = rr.status_code < 300
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("Coordinator reload failed for %s: %s", node_id, exc)
|
||||||
|
|
||||||
|
return {"ok": True, "reloaded": reloaded, "path": model_path}
|
||||||
|
|
|
||||||
|
|
@ -38,11 +38,15 @@ router = APIRouter()
|
||||||
# Kept here so the UI can list them without importing the script.
|
# Kept here so the UI can list them without importing the script.
|
||||||
|
|
||||||
MODEL_REGISTRY: dict[str, str] = {
|
MODEL_REGISTRY: dict[str, str] = {
|
||||||
"llama3.2-3b": "Llama 3.2 3B Instruct (local via cf-text)",
|
"deepseek-r1-1.5b": "DeepSeek R1 1.5B distill (cf-orch catalog key)",
|
||||||
"llama3.2-1b": "Llama 3.2 1B Instruct (local via cf-text)",
|
"deepseek-r1-7b-4bit": "DeepSeek R1 7B distill, 4-bit (cf-orch catalog key)",
|
||||||
"mistral-7b": "Mistral 7B Instruct (local via cf-text)",
|
"deepseek-r1-0528-qwen3-8b-gguf": "DeepSeek R1 0528 Qwen3 8B GGUF (4 nodes)",
|
||||||
"phi3-mini": "Phi-3 Mini 3.8B (local via cf-text)",
|
"deepseek-coder-6.7b-4bit": "DeepSeek Coder 6.7B instruct, 4-bit (cf-orch catalog key)",
|
||||||
"qwen2.5-3b": "Qwen 2.5 3B Instruct (local via cf-text)",
|
"granite-4.1-8b": "IBM Granite 4.1 8B, 4-bit (cf-orch catalog key)",
|
||||||
|
"qwen2.5-3b": "Qwen 2.5 3B Q4 GGUF (cf-orch catalog key)",
|
||||||
|
"qwen2.5-7b": "Qwen 2.5 7B Q4 GGUF (cf-orch catalog key)",
|
||||||
|
"capybarahermes-2.5-mistral-7b-gguf": "CapybaraHermes 2.5 Mistral 7B GGUF (4 nodes)",
|
||||||
|
"darwin-9b-opus-gguf": "Darwin 9B Opus GGUF -- long-form writing (3 nodes)",
|
||||||
}
|
}
|
||||||
|
|
||||||
RUBRIC_LABELS: dict[str, str] = {
|
RUBRIC_LABELS: dict[str, str] = {
|
||||||
|
|
|
||||||
|
|
@ -110,3 +110,35 @@ imitate:
|
||||||
sample_endpoint: /api/listings
|
sample_endpoint: /api/listings
|
||||||
text_fields: [title, description, seller_info]
|
text_fields: [title, description, seller_info]
|
||||||
prompt_template: "Evaluate the trustworthiness of this listing and flag any red flags:\n\n{text}"
|
prompt_template: "Evaluate the trustworthiness of this listing and flag any red flags:\n\n{text}"
|
||||||
|
|
||||||
|
- id: pagepiper
|
||||||
|
name: Pagepiper
|
||||||
|
icon: "📄"
|
||||||
|
description: "PDF/rulebook RAG tool: page-level text chunks"
|
||||||
|
base_url: http://localhost:8511
|
||||||
|
health_path: /api/health
|
||||||
|
sample_endpoint: /api/library
|
||||||
|
chunk_endpoint: /api/library/sample-chunks?limit=50 # requires pagepiper#6
|
||||||
|
text_fields: [title]
|
||||||
|
prompt_template: "Summarize the key rules described in this passage:\n\n{text}"
|
||||||
|
|
||||||
|
# ── Log corpus (Turnstone training data) ──────────────────────────────────────
|
||||||
|
corpus:
|
||||||
|
# Directory containing pipeline JSONL log files to ingest (pull-side).
|
||||||
|
# Files named <script>_<ts>.jsonl; one structured record per line.
|
||||||
|
# POST /api/corpus/pipeline-ingest walks this dir and imports new files.
|
||||||
|
# NFS-mounted on both Heimdall and Sif at /Library/Assets/
|
||||||
|
pipeline_ingest_dir: /Library/Assets/logs/pipeline/
|
||||||
|
|
||||||
|
# Turnstone push sources (consent-gated, token-authenticated).
|
||||||
|
# sources:
|
||||||
|
# - token: "your-bearer-token"
|
||||||
|
# source_host: "node.local"
|
||||||
|
# owner: YourName
|
||||||
|
# consent_date: "2026-05-17"
|
||||||
|
# consent_method: signal_chat
|
||||||
|
|
||||||
|
# ── Embedding model comparison harness ────────────────────────────────────────
|
||||||
|
embed_bench:
|
||||||
|
# ollama_url: http://localhost:11434 # optional; falls back to cforch.ollama_url
|
||||||
|
# top_k: 5 # default hits per model per query
|
||||||
|
|
|
||||||
|
|
@ -3,3 +3,6 @@ testpaths = tests
|
||||||
python_files = test_*.py
|
python_files = test_*.py
|
||||||
python_classes = Test*
|
python_classes = Test*
|
||||||
python_functions = test_*
|
python_functions = test_*
|
||||||
|
markers =
|
||||||
|
gpu: requires an idle GPU; excluded from default runs
|
||||||
|
slow: long-running test; excluded from default CI runs
|
||||||
|
|
|
||||||
|
|
@ -23,16 +23,16 @@ Usage
|
||||||
python scripts/benchmark_plans.py --list-models
|
python scripts/benchmark_plans.py --list-models
|
||||||
|
|
||||||
# Run all held-out prompts against a single model, print report
|
# Run all held-out prompts against a single model, print report
|
||||||
python scripts/benchmark_plans.py --model llama3.2-3b
|
python scripts/benchmark_plans.py --model granite-4.1-8b
|
||||||
|
|
||||||
# Compare two models side-by-side
|
# Compare two models side-by-side
|
||||||
python scripts/benchmark_plans.py --compare llama3.2-3b mistral-7b
|
python scripts/benchmark_plans.py --compare granite-4.1-8b deepseek-r1-7b-4bit
|
||||||
|
|
||||||
# Run with a custom API base (cf-text default: http://localhost:8080/v1)
|
# Run with a custom API base (cf-text default: http://localhost:8080/v1)
|
||||||
python scripts/benchmark_plans.py --model llama3.2-3b --api-base http://localhost:8080/v1
|
python scripts/benchmark_plans.py --model granite-4.1-8b --api-base http://localhost:8080/v1
|
||||||
|
|
||||||
# Export detailed results JSON
|
# Export detailed results JSON
|
||||||
python scripts/benchmark_plans.py --model llama3.2-3b --output data/bench_results.json
|
python scripts/benchmark_plans.py --model granite-4.1-8b --output data/bench_results.json
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
@ -290,6 +290,11 @@ MODEL_REGISTRY: dict[str, dict[str, str]] = {
|
||||||
"model": "deepseek-r1-7b-4bit",
|
"model": "deepseek-r1-7b-4bit",
|
||||||
"description": "DeepSeek R1 7B distill, 4-bit (cf-orch catalog key)",
|
"description": "DeepSeek R1 7B distill, 4-bit (cf-orch catalog key)",
|
||||||
},
|
},
|
||||||
|
"deepseek-r1-0528-qwen3-8b-gguf": {
|
||||||
|
"api_base": CF_TEXT_BASE,
|
||||||
|
"model": "deepseek-r1-0528-qwen3-8b-gguf",
|
||||||
|
"description": "DeepSeek R1 0528 Qwen3 8B GGUF -- current reasoning model (4 nodes)",
|
||||||
|
},
|
||||||
"deepseek-coder-6.7b-4bit": {
|
"deepseek-coder-6.7b-4bit": {
|
||||||
"api_base": CF_TEXT_BASE,
|
"api_base": CF_TEXT_BASE,
|
||||||
"model": "deepseek-coder-6.7b-4bit",
|
"model": "deepseek-coder-6.7b-4bit",
|
||||||
|
|
@ -298,17 +303,27 @@ MODEL_REGISTRY: dict[str, dict[str, str]] = {
|
||||||
"granite-4.1-8b": {
|
"granite-4.1-8b": {
|
||||||
"api_base": CF_TEXT_BASE,
|
"api_base": CF_TEXT_BASE,
|
||||||
"model": "granite-4.1-8b",
|
"model": "granite-4.1-8b",
|
||||||
"description": "IBM Granite 4.1 8B, 4-bit (cf-orch catalog key)",
|
"description": "IBM Granite 4.1 8B, 4-bit -- safety-trained (cf-orch catalog key)",
|
||||||
|
},
|
||||||
|
"capybarahermes-2.5-mistral-7b-gguf": {
|
||||||
|
"api_base": CF_TEXT_BASE,
|
||||||
|
"model": "capybarahermes-2.5-mistral-7b-gguf",
|
||||||
|
"description": "CapybaraHermes 2.5 Mistral 7B GGUF -- conversational/creative (4 nodes)",
|
||||||
|
},
|
||||||
|
"darwin-9b-opus-gguf": {
|
||||||
|
"api_base": CF_TEXT_BASE,
|
||||||
|
"model": "darwin-9b-opus-gguf",
|
||||||
|
"description": "Darwin 9B Opus GGUF -- high-quality long-form writing (3 nodes)",
|
||||||
},
|
},
|
||||||
"qwen2.5-3b": {
|
"qwen2.5-3b": {
|
||||||
"api_base": CF_TEXT_BASE,
|
"api_base": CF_TEXT_BASE,
|
||||||
"model": "qwen2.5-3b",
|
"model": "qwen2.5-3b",
|
||||||
"description": "Qwen 2.5 3B Q4 GGUF (cf-orch catalog key, navi only)",
|
"description": "Qwen 2.5 3B Q4 GGUF (cf-orch catalog key)",
|
||||||
},
|
},
|
||||||
"qwen2.5-7b": {
|
"qwen2.5-7b": {
|
||||||
"api_base": CF_TEXT_BASE,
|
"api_base": CF_TEXT_BASE,
|
||||||
"model": "qwen2.5-7b",
|
"model": "qwen2.5-7b",
|
||||||
"description": "Qwen 2.5 7B Q4 GGUF (cf-orch catalog key, navi only)",
|
"description": "Qwen 2.5 7B Q4 GGUF (cf-orch catalog key)",
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -176,9 +176,14 @@ def test_models_merges_installed_generators(client, config_dir, tmp_path):
|
||||||
# ── GET /run ───────────────────────────────────────────────────────────────────
|
# ── GET /run ───────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def test_run_returns_409_when_already_running(client):
|
def test_run_returns_409_when_already_running(client):
|
||||||
"""If _BENCH_RUNNING is True, GET /run returns 409."""
|
"""If a benchmark subprocess is actively running, GET /run returns 409."""
|
||||||
|
from unittest.mock import MagicMock
|
||||||
from app import cforch as cforch_module
|
from app import cforch as cforch_module
|
||||||
|
|
||||||
|
mock_proc = MagicMock()
|
||||||
|
mock_proc.poll.return_value = None # process still alive
|
||||||
cforch_module._BENCH_RUNNING = True
|
cforch_module._BENCH_RUNNING = True
|
||||||
|
cforch_module._bench_proc = mock_proc
|
||||||
|
|
||||||
r = client.get("/api/cforch/run")
|
r = client.get("/api/cforch/run")
|
||||||
assert r.status_code == 409
|
assert r.status_code == 409
|
||||||
|
|
@ -212,16 +217,15 @@ def test_run_streams_progress_events(client, config_dir, tmp_path):
|
||||||
"python_bin": "/usr/bin/python3",
|
"python_bin": "/usr/bin/python3",
|
||||||
})
|
})
|
||||||
|
|
||||||
|
mock_stdout = MagicMock()
|
||||||
|
mock_stdout.readline.side_effect = ["Running task 1\n", "Running task 2\n", ""]
|
||||||
mock_proc = MagicMock()
|
mock_proc = MagicMock()
|
||||||
mock_proc.stdout = iter(["Running task 1\n", "Running task 2\n"])
|
mock_proc.stdout = mock_stdout
|
||||||
mock_proc.returncode = 1 # non-zero so we don't need summary.json
|
mock_proc.returncode = 1 # non-zero so we don't need summary.json
|
||||||
|
mock_proc.wait = MagicMock()
|
||||||
|
|
||||||
def mock_wait():
|
with patch("app.cforch._subprocess.Popen", return_value=mock_proc), \
|
||||||
pass
|
patch("app.cforch._select.select", return_value=([mock_stdout], [], [])):
|
||||||
|
|
||||||
mock_proc.wait = mock_wait
|
|
||||||
|
|
||||||
with patch("app.cforch._subprocess.Popen", return_value=mock_proc):
|
|
||||||
r = client.get("/api/cforch/run")
|
r = client.get("/api/cforch/run")
|
||||||
|
|
||||||
assert r.status_code == 200
|
assert r.status_code == 200
|
||||||
|
|
@ -254,12 +258,15 @@ def test_run_emits_result_on_success(client, config_dir, tmp_path):
|
||||||
"python_bin": "/usr/bin/python3",
|
"python_bin": "/usr/bin/python3",
|
||||||
})
|
})
|
||||||
|
|
||||||
|
mock_stdout = MagicMock()
|
||||||
|
mock_stdout.readline.side_effect = [""] # no output lines, immediate EOF
|
||||||
mock_proc = MagicMock()
|
mock_proc = MagicMock()
|
||||||
mock_proc.stdout = iter([])
|
mock_proc.stdout = mock_stdout
|
||||||
mock_proc.returncode = 0
|
mock_proc.returncode = 0
|
||||||
mock_proc.wait = MagicMock()
|
mock_proc.wait = MagicMock()
|
||||||
|
|
||||||
with patch("app.cforch._subprocess.Popen", return_value=mock_proc):
|
with patch("app.cforch._subprocess.Popen", return_value=mock_proc), \
|
||||||
|
patch("app.cforch._select.select", return_value=([mock_stdout], [], [])):
|
||||||
r = client.get("/api/cforch/run")
|
r = client.get("/api/cforch/run")
|
||||||
|
|
||||||
assert r.status_code == 200
|
assert r.status_code == 200
|
||||||
|
|
|
||||||
234
tests/test_embed_bench.py
Normal file
234
tests/test_embed_bench.py
Normal file
|
|
@ -0,0 +1,234 @@
|
||||||
|
"""Tests for app/eval/embed_bench.py."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
|
||||||
|
# ── Fixtures ──────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def reset_embed_bench_globals(tmp_path):
|
||||||
|
"""Redirect config dir to tmp_path and reset running flag."""
|
||||||
|
from app.eval import embed_bench as mod
|
||||||
|
|
||||||
|
prev_config_dir = mod._CONFIG_DIR
|
||||||
|
prev_running = mod._RUN_ACTIVE
|
||||||
|
|
||||||
|
mod.set_config_dir(tmp_path)
|
||||||
|
mod._RUN_ACTIVE = False
|
||||||
|
|
||||||
|
yield tmp_path
|
||||||
|
|
||||||
|
mod.set_config_dir(prev_config_dir)
|
||||||
|
mod._RUN_ACTIVE = prev_running
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client():
|
||||||
|
from app.api import app
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
# ── cosine helper ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_cosine_identical():
|
||||||
|
from app.eval.embed_bench import _cosine
|
||||||
|
assert _cosine([1.0, 0.0], [1.0, 0.0]) == pytest.approx(1.0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_cosine_orthogonal():
|
||||||
|
from app.eval.embed_bench import _cosine
|
||||||
|
assert _cosine([1.0, 0.0], [0.0, 1.0]) == pytest.approx(0.0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_cosine_opposite():
|
||||||
|
from app.eval.embed_bench import _cosine
|
||||||
|
assert _cosine([1.0, 0.0], [-1.0, 0.0]) == pytest.approx(-1.0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_cosine_zero_vector_returns_zero():
|
||||||
|
from app.eval.embed_bench import _cosine
|
||||||
|
assert _cosine([0.0, 0.0], [1.0, 0.0]) == pytest.approx(0.0)
|
||||||
|
|
||||||
|
|
||||||
|
# ── models endpoint ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_models_returns_list_with_mock(client, tmp_path):
|
||||||
|
"""GET /api/embed-bench/models returns list from Ollama tags endpoint."""
|
||||||
|
import yaml
|
||||||
|
cfg = {"cforch": {"ollama_url": "http://localhost:11434"}}
|
||||||
|
(tmp_path / "label_tool.yaml").write_text(yaml.dump(cfg))
|
||||||
|
|
||||||
|
mock_resp = MagicMock()
|
||||||
|
mock_resp.status_code = 200
|
||||||
|
mock_resp.json.return_value = {
|
||||||
|
"models": [
|
||||||
|
{"name": "nomic-embed-text", "size": 274302480},
|
||||||
|
{"name": "mxbai-embed-large", "size": 669000000},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
mock_resp.raise_for_status = MagicMock()
|
||||||
|
|
||||||
|
with patch("app.eval.embed_bench.httpx.get", return_value=mock_resp):
|
||||||
|
r = client.get("/api/embed-bench/models")
|
||||||
|
|
||||||
|
assert r.status_code == 200
|
||||||
|
data = r.json()
|
||||||
|
assert isinstance(data["models"], list)
|
||||||
|
assert any(m["name"] == "nomic-embed-text" for m in data["models"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_models_returns_empty_on_ollama_error(client, tmp_path):
|
||||||
|
"""GET /api/embed-bench/models returns empty list if Ollama unreachable."""
|
||||||
|
import httpx
|
||||||
|
with patch("app.eval.embed_bench.httpx.get", side_effect=httpx.ConnectError("refused")):
|
||||||
|
r = client.get("/api/embed-bench/models")
|
||||||
|
assert r.status_code == 200
|
||||||
|
assert r.json()["models"] == []
|
||||||
|
|
||||||
|
|
||||||
|
# ── run endpoint ───────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_run_empty_corpus_returns_422(client):
|
||||||
|
r = client.post("/api/embed-bench/run", json={
|
||||||
|
"corpus": [], "queries": ["test"], "models": ["nomic-embed-text"], "top_k": 3
|
||||||
|
})
|
||||||
|
assert r.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_empty_queries_returns_422(client):
|
||||||
|
r = client.post("/api/embed-bench/run", json={
|
||||||
|
"corpus": ["chunk 1"], "queries": [], "models": ["nomic-embed-text"], "top_k": 3
|
||||||
|
})
|
||||||
|
assert r.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_empty_models_returns_422(client):
|
||||||
|
r = client.post("/api/embed-bench/run", json={
|
||||||
|
"corpus": ["chunk 1"], "queries": ["test"], "models": [], "top_k": 3
|
||||||
|
})
|
||||||
|
assert r.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def _fake_embed_response(texts: list[str]) -> MagicMock:
|
||||||
|
"""Build a mock httpx.post response returning unit vectors for each text."""
|
||||||
|
resp = MagicMock()
|
||||||
|
resp.raise_for_status = MagicMock()
|
||||||
|
resp.json.return_value = {
|
||||||
|
"data": [{"embedding": [1.0, 0.0, 0.0] if i % 2 == 0 else [0.0, 1.0, 0.0]}
|
||||||
|
for i, _ in enumerate(texts)]
|
||||||
|
}
|
||||||
|
return resp
|
||||||
|
|
||||||
|
|
||||||
|
def _collect_sse(raw: bytes) -> list[dict]:
|
||||||
|
"""Parse SSE stream bytes into a list of decoded event dicts."""
|
||||||
|
events = []
|
||||||
|
for line in raw.decode().splitlines():
|
||||||
|
if line.startswith("data: "):
|
||||||
|
events.append(json.loads(line[6:]))
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_single_model_returns_result_and_done(client, tmp_path):
|
||||||
|
import yaml
|
||||||
|
(tmp_path / "label_tool.yaml").write_text(yaml.dump({"cforch": {"ollama_url": "http://localhost:11434"}}))
|
||||||
|
|
||||||
|
with patch("app.eval.embed_bench.httpx.post", return_value=_fake_embed_response(["chunk 1", "chunk 2"])):
|
||||||
|
r = client.post("/api/embed-bench/run", json={
|
||||||
|
"corpus": ["chunk 1", "chunk 2"],
|
||||||
|
"queries": ["what is chunk one?"],
|
||||||
|
"models": ["nomic-embed-text"],
|
||||||
|
"top_k": 2,
|
||||||
|
})
|
||||||
|
|
||||||
|
assert r.status_code == 200
|
||||||
|
events = _collect_sse(r.content)
|
||||||
|
types = [e["type"] for e in events]
|
||||||
|
assert "result" in types
|
||||||
|
assert types[-1] == "done"
|
||||||
|
result_events = [e for e in events if e["type"] == "result"]
|
||||||
|
assert result_events[0]["model"] == "nomic-embed-text"
|
||||||
|
assert result_events[0]["query_idx"] == 0
|
||||||
|
assert len(result_events[0]["hits"]) <= 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_two_models_returns_two_result_events_per_query(client, tmp_path):
|
||||||
|
import yaml
|
||||||
|
(tmp_path / "label_tool.yaml").write_text(yaml.dump({"cforch": {"ollama_url": "http://localhost:11434"}}))
|
||||||
|
|
||||||
|
with patch("app.eval.embed_bench.httpx.post", return_value=_fake_embed_response(["chunk A", "chunk B"])):
|
||||||
|
r = client.post("/api/embed-bench/run", json={
|
||||||
|
"corpus": ["chunk A", "chunk B"],
|
||||||
|
"queries": ["find it"],
|
||||||
|
"models": ["nomic-embed-text", "mxbai-embed-large"],
|
||||||
|
"top_k": 2,
|
||||||
|
})
|
||||||
|
|
||||||
|
events = _collect_sse(r.content)
|
||||||
|
result_events = [e for e in events if e["type"] == "result"]
|
||||||
|
models_seen = {e["model"] for e in result_events}
|
||||||
|
assert "nomic-embed-text" in models_seen
|
||||||
|
assert "mxbai-embed-large" in models_seen
|
||||||
|
|
||||||
|
|
||||||
|
# ── rate + export ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_rate_appends_jsonl_line(client, tmp_path):
|
||||||
|
r = client.post("/api/embed-bench/rate", json={
|
||||||
|
"query": "test query",
|
||||||
|
"model": "nomic-embed-text",
|
||||||
|
"chunk_text": "some text",
|
||||||
|
"chunk_idx": 2,
|
||||||
|
"rating": "relevant",
|
||||||
|
})
|
||||||
|
assert r.status_code == 200
|
||||||
|
assert r.json() == {"ok": True}
|
||||||
|
ratings_file = tmp_path / "embed_bench_ratings.jsonl"
|
||||||
|
assert ratings_file.exists()
|
||||||
|
line = json.loads(ratings_file.read_text().strip())
|
||||||
|
assert line["query"] == "test query"
|
||||||
|
assert line["rating"] == "relevant"
|
||||||
|
assert line["chunk_idx"] == 2
|
||||||
|
assert "timestamp" in line
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_csv_two_rows(client, tmp_path):
|
||||||
|
for i in range(2):
|
||||||
|
client.post("/api/embed-bench/rate", json={
|
||||||
|
"query": f"q{i}", "model": "nomic-embed-text",
|
||||||
|
"chunk_text": f"chunk {i}", "chunk_idx": i, "rating": "relevant",
|
||||||
|
})
|
||||||
|
r = client.get("/api/embed-bench/export?format=csv")
|
||||||
|
assert r.status_code == 200
|
||||||
|
assert "text/csv" in r.headers["content-type"]
|
||||||
|
lines = r.text.strip().splitlines()
|
||||||
|
assert len(lines) == 3 # header + 2 rows
|
||||||
|
assert "query" in lines[0]
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_json_two_entries(client, tmp_path):
|
||||||
|
for i in range(2):
|
||||||
|
client.post("/api/embed-bench/rate", json={
|
||||||
|
"query": f"q{i}", "model": "nomic-embed-text",
|
||||||
|
"chunk_text": f"chunk {i}", "chunk_idx": i, "rating": "not_relevant",
|
||||||
|
})
|
||||||
|
r = client.get("/api/embed-bench/export?format=json")
|
||||||
|
assert r.status_code == 200
|
||||||
|
data = r.json()
|
||||||
|
assert isinstance(data, list)
|
||||||
|
assert len(data) == 2
|
||||||
|
assert data[0]["rating"] == "not_relevant"
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_empty_returns_csv_header_only(client):
|
||||||
|
r = client.get("/api/embed-bench/export?format=csv")
|
||||||
|
assert r.status_code == 200
|
||||||
|
lines = r.text.strip().splitlines()
|
||||||
|
assert len(lines) == 1 # header only
|
||||||
|
assert "query" in lines[0]
|
||||||
|
|
@ -321,6 +321,7 @@ def test_load_and_prepare_data_single_path_still_works(tmp_path):
|
||||||
|
|
||||||
# ---- Integration test ----
|
# ---- Integration test ----
|
||||||
|
|
||||||
|
@pytest.mark.gpu
|
||||||
def test_integration_finetune_on_example_data(tmp_path):
|
def test_integration_finetune_on_example_data(tmp_path):
|
||||||
"""Fine-tune deberta-small on example data for 1 epoch.
|
"""Fine-tune deberta-small on example data for 1 epoch.
|
||||||
|
|
||||||
|
|
|
||||||
454
tests/test_log_corpus.py
Normal file
454
tests/test_log_corpus.py
Normal file
|
|
@ -0,0 +1,454 @@
|
||||||
|
"""Tests for app/data/log_corpus.py — corpus receiver and labeling endpoints."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import uuid
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from app.data import log_corpus as lc
|
||||||
|
|
||||||
|
|
||||||
|
VALID_TOKEN = str(uuid.uuid4())
|
||||||
|
VALID_HOST = "testnode.local"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def isolated_db(tmp_path, monkeypatch):
|
||||||
|
"""Each test gets its own fresh corpus DB and config dir."""
|
||||||
|
monkeypatch.setattr(lc, "_DATA_DIR", tmp_path)
|
||||||
|
monkeypatch.setattr(lc, "_DB_PATH", tmp_path / "corpus.db")
|
||||||
|
# Config dir pointing to a temp yaml with one test source
|
||||||
|
config_dir = tmp_path / "config"
|
||||||
|
config_dir.mkdir()
|
||||||
|
(config_dir / "label_tool.yaml").write_text(
|
||||||
|
f"corpus:\n sources:\n"
|
||||||
|
f" - token: \"{VALID_TOKEN}\"\n"
|
||||||
|
f" source_host: \"{VALID_HOST}\"\n"
|
||||||
|
f" owner: TestOwner\n"
|
||||||
|
f" consent_date: \"2026-05-11\"\n"
|
||||||
|
f" consent_method: signal_chat\n"
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
|
||||||
|
lc._init_db()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def client():
|
||||||
|
from fastapi import FastAPI
|
||||||
|
app = FastAPI()
|
||||||
|
app.include_router(lc.router, prefix="/api/corpus")
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
def _batch(batch_type="raw_entries", entries=None, source_host=VALID_HOST):
|
||||||
|
return {
|
||||||
|
"batch_version": 1,
|
||||||
|
"batch_id": str(uuid.uuid4()),
|
||||||
|
"pushed_at": "2026-05-11T10:00:00Z",
|
||||||
|
"source_host": source_host,
|
||||||
|
"batch_type": batch_type,
|
||||||
|
"watermark_from": 0,
|
||||||
|
"watermark_to": 5,
|
||||||
|
"entries": entries or [
|
||||||
|
{
|
||||||
|
"entry_id": str(uuid.uuid4()),
|
||||||
|
"source_id": "sonarr",
|
||||||
|
"timestamp_iso": "2026-05-11T09:58:00Z",
|
||||||
|
"severity": "ERROR",
|
||||||
|
"text": "Connection refused to indexer",
|
||||||
|
"matched_patterns": [],
|
||||||
|
}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── Receive endpoint ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_receive_missing_auth(client):
|
||||||
|
resp = client.post("/api/corpus/log-batch", json=_batch())
|
||||||
|
assert resp.status_code == 401
|
||||||
|
|
||||||
|
|
||||||
|
def test_receive_invalid_token(client):
|
||||||
|
resp = client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=_batch(),
|
||||||
|
headers={"Authorization": "Bearer bad-token"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 403
|
||||||
|
|
||||||
|
|
||||||
|
def test_receive_valid_batch(client):
|
||||||
|
resp = client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=_batch(),
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.json()
|
||||||
|
assert data["received"] is True
|
||||||
|
assert data["entries_stored"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_receive_stores_source_host_from_token_not_payload(client):
|
||||||
|
"""source_host is always taken from the DB lookup, not the payload."""
|
||||||
|
payload = _batch(source_host="attacker-injected-host")
|
||||||
|
resp = client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=payload,
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
entries_resp = client.get("/api/corpus/entries")
|
||||||
|
entry = entries_resp.json()["entries"][0]
|
||||||
|
assert entry["source_host"] == VALID_HOST
|
||||||
|
|
||||||
|
|
||||||
|
def test_receive_skips_empty_text_entries(client):
|
||||||
|
payload = _batch(entries=[
|
||||||
|
{"entry_id": "e1", "source_id": "svc", "severity": "ERROR", "text": ""},
|
||||||
|
{"entry_id": "e2", "source_id": "svc", "severity": "ERROR", "text": " "},
|
||||||
|
{"entry_id": "e3", "source_id": "svc", "severity": "ERROR", "text": "real error"},
|
||||||
|
])
|
||||||
|
resp = client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=payload,
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
assert resp.json()["entries_stored"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_receive_incident_bundle(client):
|
||||||
|
payload = _batch(batch_type="incident_bundles", entries=[
|
||||||
|
{"id": "inc-1", "label": "plex crash", "issue_type": "plex",
|
||||||
|
"started_at": "2026-05-11T09:00:00", "ended_at": "2026-05-11T09:30:00",
|
||||||
|
"notes": "audio dropped", "created_at": "2026-05-11T09:35:00",
|
||||||
|
"severity": "high", "text": "plex crash"},
|
||||||
|
])
|
||||||
|
resp = client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=payload,
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.json()["entries_stored"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ── Labeling endpoints ─────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_label_entry(client):
|
||||||
|
client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=_batch(),
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
|
||||||
|
|
||||||
|
resp = client.post(f"/api/corpus/entries/{entry_id}/label", json={
|
||||||
|
"failure_type": "software",
|
||||||
|
"plain_explanation": "Sonarr lost connection to its indexer — restart the service.",
|
||||||
|
"known_pattern": "y",
|
||||||
|
})
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.json()["labeled"] is True
|
||||||
|
|
||||||
|
entries = client.get("/api/corpus/entries", params={"state": "labeled"}).json()["entries"]
|
||||||
|
assert len(entries) == 1
|
||||||
|
assert entries[0]["failure_type"] == "software"
|
||||||
|
|
||||||
|
|
||||||
|
def test_label_entry_invalid_failure_type(client):
|
||||||
|
client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=_batch(),
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
|
||||||
|
resp = client.post(f"/api/corpus/entries/{entry_id}/label", json={"failure_type": "aliens"})
|
||||||
|
assert resp.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_label_entry_missing_failure_type(client):
|
||||||
|
client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=_batch(),
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
|
||||||
|
resp = client.post(f"/api/corpus/entries/{entry_id}/label", json={})
|
||||||
|
assert resp.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_label_entry_not_found(client):
|
||||||
|
resp = client.post("/api/corpus/entries/nonexistent/label", json={"failure_type": "software"})
|
||||||
|
assert resp.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_skip_entry(client):
|
||||||
|
client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=_batch(),
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
|
||||||
|
resp = client.post(f"/api/corpus/entries/{entry_id}/skip")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
|
||||||
|
unlabeled = client.get("/api/corpus/entries").json()["entries"]
|
||||||
|
assert len(unlabeled) == 0
|
||||||
|
|
||||||
|
|
||||||
|
# ── Stats ──────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_stats_empty(client):
|
||||||
|
stats = client.get("/api/corpus/stats").json()
|
||||||
|
assert stats["total_entries"] == 0
|
||||||
|
assert stats["batch_count"] == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_stats_after_receive(client):
|
||||||
|
client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=_batch(),
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
stats = client.get("/api/corpus/stats").json()
|
||||||
|
assert stats["total_entries"] == 1
|
||||||
|
assert stats["batch_count"] == 1
|
||||||
|
assert stats["by_label_state"].get("unlabeled", 0) == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ── Export ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_export_excludes_unlabeled(client):
|
||||||
|
client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=_batch(),
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
resp = client.get("/api/corpus/export")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.text.strip() == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_includes_labeled(client):
|
||||||
|
client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=_batch(),
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
|
||||||
|
client.post(f"/api/corpus/entries/{entry_id}/label", json={
|
||||||
|
"failure_type": "software",
|
||||||
|
"plain_explanation": "Sonarr lost connection to indexer.",
|
||||||
|
})
|
||||||
|
|
||||||
|
resp = client.get("/api/corpus/export")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
lines = [l for l in resp.text.strip().splitlines() if l]
|
||||||
|
assert len(lines) == 1
|
||||||
|
record = json.loads(lines[0])
|
||||||
|
assert record["output"] == "Sonarr lost connection to indexer."
|
||||||
|
assert record["metadata"]["failure_type"] == "software"
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_excludes_pii_flagged(client):
|
||||||
|
client.post(
|
||||||
|
"/api/corpus/log-batch",
|
||||||
|
json=_batch(),
|
||||||
|
headers={"Authorization": f"Bearer {VALID_TOKEN}"},
|
||||||
|
)
|
||||||
|
entry_id = client.get("/api/corpus/entries").json()["entries"][0]["id"]
|
||||||
|
client.post(f"/api/corpus/entries/{entry_id}/label", json={
|
||||||
|
"failure_type": "software",
|
||||||
|
"plain_explanation": "Contains username — should not export.",
|
||||||
|
"pii_flagged": True,
|
||||||
|
})
|
||||||
|
|
||||||
|
resp = client.get("/api/corpus/export")
|
||||||
|
assert resp.text.strip() == ""
|
||||||
|
|
||||||
|
|
||||||
|
# ── Pipeline ingest endpoint ───────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _make_pipeline_file(directory: Path, name: str, lines: list[dict]) -> Path:
|
||||||
|
"""Write a JSONL pipeline log file to directory."""
|
||||||
|
p = directory / name
|
||||||
|
p.write_text("\n".join(json.dumps(l) for l in lines), encoding="utf-8")
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
_PIPELINE_LINE = {
|
||||||
|
"ts": "2026-05-17T10:00:00Z",
|
||||||
|
"level": "INFO",
|
||||||
|
"logger": "scripts.pipeline.purple_carrot_scraper",
|
||||||
|
"msg": "Fetched recipe page",
|
||||||
|
"extra": {"url": "https://example.com/recipe/1", "status": 200},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_pipeline_ingest_returns_404_when_dir_not_configured(client, tmp_path):
|
||||||
|
"""No pipeline_ingest_dir in config — endpoint returns 404."""
|
||||||
|
resp = client.post("/api/corpus/pipeline-ingest")
|
||||||
|
assert resp.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_pipeline_ingest_empty_dir(client, tmp_path, monkeypatch):
|
||||||
|
"""Configured dir exists but is empty — returns zeros, no error."""
|
||||||
|
ingest_dir = tmp_path / "pipeline_logs"
|
||||||
|
ingest_dir.mkdir()
|
||||||
|
config_dir = tmp_path / "config"
|
||||||
|
config_dir.mkdir(exist_ok=True)
|
||||||
|
(config_dir / "label_tool.yaml").write_text(
|
||||||
|
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
|
||||||
|
|
||||||
|
resp = client.post("/api/corpus/pipeline-ingest")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.json()
|
||||||
|
assert data["ingested_files"] == 0
|
||||||
|
assert data["skipped_files"] == 0
|
||||||
|
assert data["entries_stored"] == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_pipeline_ingest_ingests_valid_file(client, tmp_path, monkeypatch):
|
||||||
|
"""Valid JSONL file is ingested; entries appear in corpus."""
|
||||||
|
ingest_dir = tmp_path / "pipeline_logs"
|
||||||
|
ingest_dir.mkdir()
|
||||||
|
_make_pipeline_file(ingest_dir, "scraper_20260517.jsonl", [
|
||||||
|
_PIPELINE_LINE,
|
||||||
|
{**_PIPELINE_LINE, "msg": "Saved 3 recipes", "level": "INFO"},
|
||||||
|
])
|
||||||
|
|
||||||
|
config_dir = tmp_path / "config"
|
||||||
|
config_dir.mkdir(exist_ok=True)
|
||||||
|
(config_dir / "label_tool.yaml").write_text(
|
||||||
|
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
|
||||||
|
|
||||||
|
resp = client.post("/api/corpus/pipeline-ingest")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.json()
|
||||||
|
assert data["ingested_files"] == 1
|
||||||
|
assert data["entries_stored"] == 2
|
||||||
|
|
||||||
|
entries = client.get("/api/corpus/entries", params={"limit": 10}).json()["entries"]
|
||||||
|
assert len(entries) == 2
|
||||||
|
assert all(e["source_host"] == "pipeline_scrape" for e in entries)
|
||||||
|
|
||||||
|
|
||||||
|
def test_pipeline_ingest_source_id_from_logger(client, tmp_path, monkeypatch):
|
||||||
|
"""source_id is populated from the 'logger' field of each log line."""
|
||||||
|
ingest_dir = tmp_path / "pipeline_logs"
|
||||||
|
ingest_dir.mkdir()
|
||||||
|
_make_pipeline_file(ingest_dir, "run_20260517.jsonl", [_PIPELINE_LINE])
|
||||||
|
|
||||||
|
config_dir = tmp_path / "config"
|
||||||
|
config_dir.mkdir(exist_ok=True)
|
||||||
|
(config_dir / "label_tool.yaml").write_text(
|
||||||
|
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
|
||||||
|
|
||||||
|
client.post("/api/corpus/pipeline-ingest")
|
||||||
|
entries = client.get("/api/corpus/entries", params={"limit": 10}).json()["entries"]
|
||||||
|
assert entries[0]["source_id"] == "scripts.pipeline.purple_carrot_scraper"
|
||||||
|
|
||||||
|
|
||||||
|
def test_pipeline_ingest_idempotent(client, tmp_path, monkeypatch):
|
||||||
|
"""Calling the endpoint twice does not re-ingest already-processed files."""
|
||||||
|
ingest_dir = tmp_path / "pipeline_logs"
|
||||||
|
ingest_dir.mkdir()
|
||||||
|
_make_pipeline_file(ingest_dir, "scraper_20260517.jsonl", [_PIPELINE_LINE])
|
||||||
|
|
||||||
|
config_dir = tmp_path / "config"
|
||||||
|
config_dir.mkdir(exist_ok=True)
|
||||||
|
(config_dir / "label_tool.yaml").write_text(
|
||||||
|
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
|
||||||
|
|
||||||
|
client.post("/api/corpus/pipeline-ingest")
|
||||||
|
resp2 = client.post("/api/corpus/pipeline-ingest")
|
||||||
|
|
||||||
|
data = resp2.json()
|
||||||
|
assert data["ingested_files"] == 0
|
||||||
|
assert data["skipped_files"] == 1
|
||||||
|
assert data["entries_stored"] == 0
|
||||||
|
|
||||||
|
entries = client.get("/api/corpus/entries", params={"limit": 10}).json()["entries"]
|
||||||
|
assert len(entries) == 1 # still just the one from the first ingest
|
||||||
|
|
||||||
|
|
||||||
|
def test_pipeline_ingest_skips_non_jsonl(client, tmp_path, monkeypatch):
|
||||||
|
"""Non-.jsonl files in the dir are silently ignored."""
|
||||||
|
ingest_dir = tmp_path / "pipeline_logs"
|
||||||
|
ingest_dir.mkdir()
|
||||||
|
(ingest_dir / "notes.txt").write_text("this is not a log file")
|
||||||
|
(ingest_dir / "run.csv").write_text("a,b,c\n1,2,3")
|
||||||
|
_make_pipeline_file(ingest_dir, "valid_20260517.jsonl", [_PIPELINE_LINE])
|
||||||
|
|
||||||
|
config_dir = tmp_path / "config"
|
||||||
|
config_dir.mkdir(exist_ok=True)
|
||||||
|
(config_dir / "label_tool.yaml").write_text(
|
||||||
|
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
|
||||||
|
|
||||||
|
resp = client.post("/api/corpus/pipeline-ingest")
|
||||||
|
assert resp.json()["ingested_files"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_pipeline_ingest_skips_malformed_lines(client, tmp_path, monkeypatch):
|
||||||
|
"""Lines that are not valid JSON are skipped; valid lines in the same file still land."""
|
||||||
|
ingest_dir = tmp_path / "pipeline_logs"
|
||||||
|
ingest_dir.mkdir()
|
||||||
|
p = ingest_dir / "mixed_20260517.jsonl"
|
||||||
|
p.write_text(
|
||||||
|
json.dumps(_PIPELINE_LINE) + "\n"
|
||||||
|
"this is not json\n"
|
||||||
|
+ json.dumps({**_PIPELINE_LINE, "msg": "another valid line"}),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
config_dir = tmp_path / "config"
|
||||||
|
config_dir.mkdir(exist_ok=True)
|
||||||
|
(config_dir / "label_tool.yaml").write_text(
|
||||||
|
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
|
||||||
|
|
||||||
|
resp = client.post("/api/corpus/pipeline-ingest")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.json()["entries_stored"] == 2 # 2 valid lines, 1 skipped
|
||||||
|
|
||||||
|
|
||||||
|
def test_pipeline_ingest_new_file_after_first_run(client, tmp_path, monkeypatch):
|
||||||
|
"""A new file added after the first ingest is picked up on the next call."""
|
||||||
|
ingest_dir = tmp_path / "pipeline_logs"
|
||||||
|
ingest_dir.mkdir()
|
||||||
|
_make_pipeline_file(ingest_dir, "run_a.jsonl", [_PIPELINE_LINE])
|
||||||
|
|
||||||
|
config_dir = tmp_path / "config"
|
||||||
|
config_dir.mkdir(exist_ok=True)
|
||||||
|
(config_dir / "label_tool.yaml").write_text(
|
||||||
|
f"corpus:\n pipeline_ingest_dir: \"{ingest_dir}\"\n sources: []\n"
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(lc, "_CONFIG_DIR", config_dir)
|
||||||
|
|
||||||
|
client.post("/api/corpus/pipeline-ingest") # ingest run_a.jsonl
|
||||||
|
|
||||||
|
_make_pipeline_file(ingest_dir, "run_b.jsonl", [
|
||||||
|
{**_PIPELINE_LINE, "msg": "Second run line"},
|
||||||
|
])
|
||||||
|
|
||||||
|
resp2 = client.post("/api/corpus/pipeline-ingest")
|
||||||
|
data = resp2.json()
|
||||||
|
assert data["ingested_files"] == 1
|
||||||
|
assert data["skipped_files"] == 1
|
||||||
|
assert data["entries_stored"] == 1
|
||||||
|
|
@ -17,6 +17,7 @@ def reset_models_globals(tmp_path):
|
||||||
from app import models as models_module
|
from app import models as models_module
|
||||||
|
|
||||||
prev_models = models_module._MODELS_DIR
|
prev_models = models_module._MODELS_DIR
|
||||||
|
prev_cf_text = models_module._CF_TEXT_MODELS_DIR
|
||||||
prev_queue = models_module._QUEUE_DIR
|
prev_queue = models_module._QUEUE_DIR
|
||||||
prev_progress = dict(models_module._download_progress)
|
prev_progress = dict(models_module._download_progress)
|
||||||
|
|
||||||
|
|
@ -26,12 +27,14 @@ def reset_models_globals(tmp_path):
|
||||||
queue_dir.mkdir()
|
queue_dir.mkdir()
|
||||||
|
|
||||||
models_module.set_models_dir(models_dir)
|
models_module.set_models_dir(models_dir)
|
||||||
|
models_module.set_cf_text_models_dir(tmp_path / "cf-text-models")
|
||||||
models_module.set_queue_dir(queue_dir)
|
models_module.set_queue_dir(queue_dir)
|
||||||
models_module._download_progress = {}
|
models_module._download_progress = {}
|
||||||
|
|
||||||
yield
|
yield
|
||||||
|
|
||||||
models_module.set_models_dir(prev_models)
|
models_module.set_models_dir(prev_models)
|
||||||
|
models_module.set_cf_text_models_dir(prev_cf_text)
|
||||||
models_module.set_queue_dir(prev_queue)
|
models_module.set_queue_dir(prev_queue)
|
||||||
models_module._download_progress = prev_progress
|
models_module._download_progress = prev_progress
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -55,11 +55,11 @@ def _fake_nodes_response(nodes_json: list, services_json: list | None = None):
|
||||||
"""Build side_effect list for two httpx.get calls: nodes then services."""
|
"""Build side_effect list for two httpx.get calls: nodes then services."""
|
||||||
mock_nodes = MagicMock()
|
mock_nodes = MagicMock()
|
||||||
mock_nodes.raise_for_status = MagicMock()
|
mock_nodes.raise_for_status = MagicMock()
|
||||||
mock_nodes.json.return_value = nodes_json
|
mock_nodes.json.return_value = {"nodes": nodes_json}
|
||||||
|
|
||||||
mock_services = MagicMock()
|
mock_services = MagicMock()
|
||||||
mock_services.raise_for_status = MagicMock()
|
mock_services.raise_for_status = MagicMock()
|
||||||
mock_services.json.return_value = services_json or []
|
mock_services.json.return_value = {"services": services_json or []}
|
||||||
|
|
||||||
return [mock_nodes, mock_services]
|
return [mock_nodes, mock_services]
|
||||||
|
|
||||||
|
|
@ -469,3 +469,107 @@ def test_delete_ollama_model_404_when_not_found(client, tmp_path):
|
||||||
r = client.delete("/api/nodes-mgmt/nodes/heimdall/models/ollama/missing-model")
|
r = client.delete("/api/nodes-mgmt/nodes/heimdall/models/ollama/missing-model")
|
||||||
|
|
||||||
assert r.status_code == 404
|
assert r.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
# ── Deploy model endpoint ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
_DEPLOY_PROFILE = {
|
||||||
|
"services": {
|
||||||
|
"cf-text": {
|
||||||
|
"max_mb": 20000,
|
||||||
|
"min_compute_cap": 7.0,
|
||||||
|
"model_base_path": "/devl/Assets/LLM/cf-text/models",
|
||||||
|
"catalog": {},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"nodes": {
|
||||||
|
"heimdall": {
|
||||||
|
"gpus": [],
|
||||||
|
"agent_url": "http://10.1.10.71:7701",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_deploy_model_adds_catalog_entry(client, tmp_path):
|
||||||
|
"""Deploy endpoint should add the model to the service catalog."""
|
||||||
|
profiles_dir = tmp_path / "profiles"
|
||||||
|
_write_config(tmp_path, {
|
||||||
|
"coordinator_url": "http://fake-coord:7700",
|
||||||
|
"profiles_dir": str(profiles_dir),
|
||||||
|
})
|
||||||
|
_write_profile(profiles_dir, "heimdall", _DEPLOY_PROFILE)
|
||||||
|
|
||||||
|
mock_reload = MagicMock()
|
||||||
|
mock_reload.status_code = 200
|
||||||
|
|
||||||
|
with patch("httpx.post", return_value=mock_reload):
|
||||||
|
r = client.post(
|
||||||
|
"/api/nodes-mgmt/nodes/heimdall/models/deploy",
|
||||||
|
json={
|
||||||
|
"model_id": "fdtn-ai--Foundation-Sec-8B-Q4",
|
||||||
|
"service_type": "cf-text",
|
||||||
|
"vram_mb": 5180,
|
||||||
|
"hf_repo": "fdtn-ai/Foundation-Sec-8B-Q4_K_M-GGUF",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert r.status_code == 200
|
||||||
|
data = r.json()
|
||||||
|
assert data["ok"] is True
|
||||||
|
assert data["reloaded"] is True
|
||||||
|
assert "fdtn-ai--Foundation-Sec-8B-Q4_K_M-GGUF" in data["path"]
|
||||||
|
|
||||||
|
saved = yaml.safe_load((profiles_dir / "heimdall.yaml").read_text())
|
||||||
|
catalog = saved["services"]["cf-text"]["catalog"]
|
||||||
|
assert "fdtn-ai--Foundation-Sec-8B-Q4" in catalog
|
||||||
|
entry = catalog["fdtn-ai--Foundation-Sec-8B-Q4"]
|
||||||
|
assert entry["vram_mb"] == 5180
|
||||||
|
assert entry["path"].endswith("fdtn-ai--Foundation-Sec-8B-Q4_K_M-GGUF")
|
||||||
|
|
||||||
|
|
||||||
|
def test_deploy_model_explicit_path_overrides_base(client, tmp_path):
|
||||||
|
"""An explicit path in the request body takes precedence over model_base_path."""
|
||||||
|
profiles_dir = tmp_path / "profiles"
|
||||||
|
_write_config(tmp_path, {
|
||||||
|
"coordinator_url": "http://fake-coord:7700",
|
||||||
|
"profiles_dir": str(profiles_dir),
|
||||||
|
})
|
||||||
|
_write_profile(profiles_dir, "heimdall", _DEPLOY_PROFILE)
|
||||||
|
|
||||||
|
with patch("httpx.post", return_value=MagicMock(status_code=200)):
|
||||||
|
r = client.post(
|
||||||
|
"/api/nodes-mgmt/nodes/heimdall/models/deploy",
|
||||||
|
json={
|
||||||
|
"model_id": "my-model",
|
||||||
|
"service_type": "cf-text",
|
||||||
|
"vram_mb": 8000,
|
||||||
|
"path": "/custom/path/to/model",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert r.status_code == 200
|
||||||
|
assert r.json()["path"] == "/custom/path/to/model"
|
||||||
|
|
||||||
|
|
||||||
|
def test_deploy_model_unknown_service_returns_422(client, tmp_path):
|
||||||
|
"""Service type not in profile → 422."""
|
||||||
|
profiles_dir = tmp_path / "profiles"
|
||||||
|
_write_config(tmp_path, {"profiles_dir": str(profiles_dir)})
|
||||||
|
_write_profile(profiles_dir, "heimdall", _DEPLOY_PROFILE)
|
||||||
|
|
||||||
|
r = client.post(
|
||||||
|
"/api/nodes-mgmt/nodes/heimdall/models/deploy",
|
||||||
|
json={"model_id": "x", "service_type": "vllm", "vram_mb": 8000},
|
||||||
|
)
|
||||||
|
assert r.status_code == 422
|
||||||
|
assert "vllm" in r.json()["detail"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_deploy_model_missing_profile_returns_404(client, tmp_path):
|
||||||
|
_write_config(tmp_path, {"profiles_dir": str(tmp_path / "profiles")})
|
||||||
|
r = client.post(
|
||||||
|
"/api/nodes-mgmt/nodes/nonexistent/models/deploy",
|
||||||
|
json={"model_id": "x", "service_type": "cf-text", "vram_mb": 100},
|
||||||
|
)
|
||||||
|
assert r.status_code == 404
|
||||||
|
|
|
||||||
227
tests/test_recipe_scan.py
Normal file
227
tests/test_recipe_scan.py
Normal file
|
|
@ -0,0 +1,227 @@
|
||||||
|
"""Tests for app/data/recipe_scan.py — recipe scan labeling endpoints."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import uuid
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from app.data import recipe_scan as rs
|
||||||
|
|
||||||
|
|
||||||
|
EXTRACTED = {"title": "Shepherd's Pie", "ingredients": ["lamb", "potato"], "steps": ["brown meat", "mash potato"]}
|
||||||
|
GROUND_TRUTH = {"title": "Shepherd's Pie", "ingredients": ["ground lamb", "mashed potato", "peas"], "steps": ["brown meat", "add veg", "mash potato", "bake"]}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def isolated_db(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setattr(rs, "_DB_PATH", tmp_path / "recipe_scan.db")
|
||||||
|
rs._init_db()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def client():
|
||||||
|
from fastapi import FastAPI
|
||||||
|
app = FastAPI()
|
||||||
|
app.include_router(rs.router, prefix="/api/recipe-scan")
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
def _item(**kwargs) -> dict:
|
||||||
|
return {
|
||||||
|
"id": str(uuid.uuid4()),
|
||||||
|
"image_path": "/Library/Assets/kiwi/scans/pc_test.jpg",
|
||||||
|
"modality": kwargs.get("modality", "scanner"),
|
||||||
|
"source": kwargs.get("source", "purple_carrot"),
|
||||||
|
"extracted": kwargs.get("extracted", EXTRACTED),
|
||||||
|
"ground_truth": kwargs.get("ground_truth", GROUND_TRUTH),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _import(client, items: list[dict]) -> None:
|
||||||
|
resp = client.post("/api/recipe-scan/import", json={"items": items})
|
||||||
|
assert resp.status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
# ── Import ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_import_stores_items(client):
|
||||||
|
_import(client, [_item()])
|
||||||
|
stats = client.get("/api/recipe-scan/stats").json()
|
||||||
|
assert stats["total"] == 1
|
||||||
|
assert stats["by_status"]["pending"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_import_rejects_unknown_modality(client):
|
||||||
|
bad = _item()
|
||||||
|
bad["modality"] = "telepathy"
|
||||||
|
resp = client.post("/api/recipe-scan/import", json={"items": [bad]})
|
||||||
|
assert resp.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_import_is_idempotent(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
_import(client, [item]) # same id — should not duplicate
|
||||||
|
stats = client.get("/api/recipe-scan/stats").json()
|
||||||
|
assert stats["total"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_import_multiple_items(client):
|
||||||
|
_import(client, [_item(), _item(), _item()])
|
||||||
|
assert client.get("/api/recipe-scan/stats").json()["total"] == 3
|
||||||
|
|
||||||
|
|
||||||
|
# ── Next ───────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_next_returns_404_when_queue_empty(client):
|
||||||
|
resp = client.get("/api/recipe-scan/next")
|
||||||
|
assert resp.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_next_returns_pending_item(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
resp = client.get("/api/recipe-scan/next")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.json()
|
||||||
|
assert data["id"] == item["id"]
|
||||||
|
assert data["status"] == "pending"
|
||||||
|
assert "extracted" in data
|
||||||
|
assert "ground_truth" in data
|
||||||
|
|
||||||
|
|
||||||
|
def test_next_skips_non_pending(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
client.post(f"/api/recipe-scan/items/{item['id']}/reject")
|
||||||
|
resp = client.get("/api/recipe-scan/next")
|
||||||
|
assert resp.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
# ── Approve ────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_approve_marks_item_approved(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
resp = client.post(f"/api/recipe-scan/items/{item['id']}/approve")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.json()["status"] == "approved"
|
||||||
|
stats = client.get("/api/recipe-scan/stats").json()
|
||||||
|
assert stats["by_status"]["approved"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_approve_returns_404_for_unknown_id(client):
|
||||||
|
resp = client.post("/api/recipe-scan/items/no-such-id/approve")
|
||||||
|
assert resp.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
# ── Edit ───────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_edit_stores_corrected_json(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
corrected = {**GROUND_TRUTH, "servings": 4}
|
||||||
|
resp = client.post(
|
||||||
|
f"/api/recipe-scan/items/{item['id']}/edit",
|
||||||
|
json={"corrected": corrected},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.json()["status"] == "edited"
|
||||||
|
stats = client.get("/api/recipe-scan/stats").json()
|
||||||
|
assert stats["by_status"]["edited"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_edit_requires_corrected_field(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
resp = client.post(f"/api/recipe-scan/items/{item['id']}/edit", json={})
|
||||||
|
assert resp.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
# ── Reject ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_reject_marks_item_rejected(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
resp = client.post(
|
||||||
|
f"/api/recipe-scan/items/{item['id']}/reject",
|
||||||
|
json={"reason": "OCR completely unreadable"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.json()["status"] == "rejected"
|
||||||
|
|
||||||
|
|
||||||
|
def test_reject_without_reason_is_valid(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
resp = client.post(f"/api/recipe-scan/items/{item['id']}/reject")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
# ── Export ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_export_empty_when_nothing_approved(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
resp = client.get("/api/recipe-scan/export")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.text.strip() == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_includes_approved_item(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
client.post(f"/api/recipe-scan/items/{item['id']}/approve")
|
||||||
|
resp = client.get("/api/recipe-scan/export")
|
||||||
|
lines = [l for l in resp.text.strip().splitlines() if l]
|
||||||
|
assert len(lines) == 1
|
||||||
|
pair = json.loads(lines[0])
|
||||||
|
assert pair["id"] == item["id"]
|
||||||
|
assert pair["modality"] == "scanner"
|
||||||
|
assert "messages" in pair
|
||||||
|
assert len(pair["messages"]) == 2
|
||||||
|
assert pair["messages"][0]["role"] == "user"
|
||||||
|
assert pair["messages"][1]["role"] == "assistant"
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_includes_edited_item_with_correction(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
corrected = {**GROUND_TRUTH, "servings": 4}
|
||||||
|
client.post(
|
||||||
|
f"/api/recipe-scan/items/{item['id']}/edit",
|
||||||
|
json={"corrected": corrected},
|
||||||
|
)
|
||||||
|
resp = client.get("/api/recipe-scan/export")
|
||||||
|
lines = [l for l in resp.text.strip().splitlines() if l]
|
||||||
|
pair = json.loads(lines[0])
|
||||||
|
assistant_content = json.loads(pair["messages"][1]["content"])
|
||||||
|
assert assistant_content["servings"] == 4
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_excludes_rejected_items(client):
|
||||||
|
item = _item()
|
||||||
|
_import(client, [item])
|
||||||
|
client.post(f"/api/recipe-scan/items/{item['id']}/reject")
|
||||||
|
resp = client.get("/api/recipe-scan/export")
|
||||||
|
assert resp.text.strip() == ""
|
||||||
|
|
||||||
|
|
||||||
|
# ── Stats ──────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_stats_counts_all_statuses(client):
|
||||||
|
items = [_item(), _item(), _item(), _item()]
|
||||||
|
_import(client, items)
|
||||||
|
client.post(f"/api/recipe-scan/items/{items[0]['id']}/approve")
|
||||||
|
client.post(f"/api/recipe-scan/items/{items[1]['id']}/edit", json={"corrected": GROUND_TRUTH})
|
||||||
|
client.post(f"/api/recipe-scan/items/{items[2]['id']}/reject")
|
||||||
|
stats = client.get("/api/recipe-scan/stats").json()
|
||||||
|
assert stats["total"] == 4
|
||||||
|
assert stats["by_status"]["pending"] == 1
|
||||||
|
assert stats["by_status"]["approved"] == 1
|
||||||
|
assert stats["by_status"]["edited"] == 1
|
||||||
|
assert stats["by_status"]["rejected"] == 1
|
||||||
|
assert stats["export_ready"] == 2 # approved + edited
|
||||||
42
web/package-lock.json
generated
42
web/package-lock.json
generated
|
|
@ -2676,9 +2676,9 @@
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/brace-expansion": {
|
"node_modules/brace-expansion": {
|
||||||
"version": "2.0.2",
|
"version": "2.1.0",
|
||||||
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz",
|
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.0.tgz",
|
||||||
"integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==",
|
"integrity": "sha512-TN1kCZAgdgweJhWWpgKYrQaMNHcDULHkWwQIspdtjV4Y5aurRdZpjAqn6yX3FPqTA9ngHCc4hJxMAMgGfve85w==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
|
@ -2890,9 +2890,9 @@
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
"node_modules/defu": {
|
"node_modules/defu": {
|
||||||
"version": "6.1.4",
|
"version": "6.1.7",
|
||||||
"resolved": "https://registry.npmjs.org/defu/-/defu-6.1.4.tgz",
|
"resolved": "https://registry.npmjs.org/defu/-/defu-6.1.7.tgz",
|
||||||
"integrity": "sha512-mEQCMmwJu317oSz8CwdIOdwf3xMif1ttiM8LTufzc3g6kR+9Pe236twL8j3IYT1F7GfRgGcW6MWxzZjLIkuHIg==",
|
"integrity": "sha512-7z22QmUWiQ/2d0KkdYmANbRUVABpZ9SNYyH5vx6PZ+nE5bcC0l7uFvEfHlyld/HcGBFTL536ClDt3DEcSlEJAQ==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
|
|
@ -3725,9 +3725,9 @@
|
||||||
"license": "ISC"
|
"license": "ISC"
|
||||||
},
|
},
|
||||||
"node_modules/picomatch": {
|
"node_modules/picomatch": {
|
||||||
"version": "4.0.3",
|
"version": "4.0.4",
|
||||||
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
|
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.4.tgz",
|
||||||
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
|
"integrity": "sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A==",
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"engines": {
|
"engines": {
|
||||||
"node": ">=12"
|
"node": ">=12"
|
||||||
|
|
@ -3769,9 +3769,9 @@
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/postcss": {
|
"node_modules/postcss": {
|
||||||
"version": "8.5.8",
|
"version": "8.5.14",
|
||||||
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.8.tgz",
|
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.14.tgz",
|
||||||
"integrity": "sha512-OW/rX8O/jXnm82Ey1k44pObPtdblfiuWnrd8X7GJ7emImCOstunGbXUpp7HdBrFQX6rJzn3sPT397Wp5aCwCHg==",
|
"integrity": "sha512-SoSL4+OSEtR99LHFZQiJLkT59C5B1amGO1NzTwj7TT1qCUgUO6hxOvzkOYxD+vMrXBM3XJIKzokoERdqQq/Zmg==",
|
||||||
"funding": [
|
"funding": [
|
||||||
{
|
{
|
||||||
"type": "opencollective",
|
"type": "opencollective",
|
||||||
|
|
@ -4325,9 +4325,9 @@
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/undici": {
|
"node_modules/undici": {
|
||||||
"version": "7.22.0",
|
"version": "7.25.0",
|
||||||
"resolved": "https://registry.npmjs.org/undici/-/undici-7.22.0.tgz",
|
"resolved": "https://registry.npmjs.org/undici/-/undici-7.25.0.tgz",
|
||||||
"integrity": "sha512-RqslV2Us5BrllB+JeiZnK4peryVTndy9Dnqq62S3yYRRTj0tFQCwEniUy2167skdGOy3vqRzEvl1Dm4sV2ReDg==",
|
"integrity": "sha512-xXnp4kTyor2Zq+J1FfPI6Eq3ew5h6Vl0F/8d9XU5zZQf1tX9s2Su1/3PiMmUANFULpmksxkClamIZcaUqryHsQ==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"engines": {
|
"engines": {
|
||||||
|
|
@ -4422,9 +4422,9 @@
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/vite": {
|
"node_modules/vite": {
|
||||||
"version": "7.3.1",
|
"version": "7.3.2",
|
||||||
"resolved": "https://registry.npmjs.org/vite/-/vite-7.3.1.tgz",
|
"resolved": "https://registry.npmjs.org/vite/-/vite-7.3.2.tgz",
|
||||||
"integrity": "sha512-w+N7Hifpc3gRjZ63vYBXA56dvvRlNWRczTdmCBBa+CotUzAPf5b7YMdMR/8CQoeYE5LX3W4wj6RYTgonm1b9DA==",
|
"integrity": "sha512-Bby3NOsna2jsjfLVOHKes8sGwgl4TT0E6vvpYgnAYDIF/tie7MRaFthmKuHx1NSXjiTueXH3do80FMQgvEktRg==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
|
@ -4921,9 +4921,9 @@
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
"node_modules/yaml": {
|
"node_modules/yaml": {
|
||||||
"version": "2.8.2",
|
"version": "2.8.4",
|
||||||
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.8.2.tgz",
|
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.8.4.tgz",
|
||||||
"integrity": "sha512-mplynKqc1C2hTVYxd0PU2xQAc22TI1vShAYGksCCfxbn/dFwnHTNi1bvYsBTkhdUNtGIf5xNOg938rrSSYvS9A==",
|
"integrity": "sha512-ml/JPOj9fOQK8RNnWojA67GbZ0ApXAUlN2UQclwv2eVgTgn7O9gg9o7paZWKMp4g0H3nTLtS9LVzhkpOFIKzog==",
|
||||||
"license": "ISC",
|
"license": "ISC",
|
||||||
"bin": {
|
"bin": {
|
||||||
"yaml": "bin.mjs"
|
"yaml": "bin.mjs"
|
||||||
|
|
|
||||||
|
|
@ -220,11 +220,13 @@ const dataItems: NavItem[] = [
|
||||||
{ path: '/data/fetch', icon: '📬', label: 'Fetch' },
|
{ path: '/data/fetch', icon: '📬', label: 'Fetch' },
|
||||||
{ path: '/data/corrections', icon: '✏️', label: 'Corrections' },
|
{ path: '/data/corrections', icon: '✏️', label: 'Corrections' },
|
||||||
{ path: '/data/imitate', icon: '🪞', label: 'Imitate' },
|
{ path: '/data/imitate', icon: '🪞', label: 'Imitate' },
|
||||||
|
{ path: '/data/recipe-scan', icon: '📷', label: 'Recipe Scan' },
|
||||||
]
|
]
|
||||||
|
|
||||||
const evalItems: NavItem[] = [
|
const evalItems: NavItem[] = [
|
||||||
{ path: '/eval/benchmark', icon: '📊', label: 'Benchmark' },
|
{ path: '/eval/benchmark', icon: '📊', label: 'Benchmark' },
|
||||||
{ path: '/eval/compare', icon: '🔍', label: 'Compare' },
|
{ path: '/eval/compare', icon: '🔍', label: 'Compare' },
|
||||||
|
{ path: '/eval/embed-compare', icon: '🧮', label: 'Embed Compare' },
|
||||||
]
|
]
|
||||||
|
|
||||||
const trainItems: NavItem[] = [
|
const trainItems: NavItem[] = [
|
||||||
|
|
|
||||||
170
web/src/components/nodes/CatalogEntryFormModal.vue
Normal file
170
web/src/components/nodes/CatalogEntryFormModal.vue
Normal file
|
|
@ -0,0 +1,170 @@
|
||||||
|
<script setup lang="ts">
|
||||||
|
import { ref, watch } from 'vue'
|
||||||
|
import type { CatalogEntryFull } from '../../types/nodes'
|
||||||
|
|
||||||
|
const props = defineProps<{
|
||||||
|
svcName: string
|
||||||
|
modelName?: string
|
||||||
|
entry?: CatalogEntryFull
|
||||||
|
}>()
|
||||||
|
const emit = defineEmits<{
|
||||||
|
save: [svcName: string, modelName: string, entry: CatalogEntryFull]
|
||||||
|
cancel: []
|
||||||
|
}>()
|
||||||
|
|
||||||
|
const name = ref(props.modelName ?? '')
|
||||||
|
const path = ref(props.entry?.path ?? '')
|
||||||
|
const vramMb = ref(props.entry?.vram_mb ?? 0)
|
||||||
|
const description = ref(props.entry?.description ?? '')
|
||||||
|
const multiGpu = ref(props.entry?.multi_gpu ?? false)
|
||||||
|
const envPairs = ref<{ k: string; v: string }[]>(
|
||||||
|
Object.entries(props.entry?.env ?? {}).map(([k, v]) => ({ k, v }))
|
||||||
|
)
|
||||||
|
const formError = ref('')
|
||||||
|
|
||||||
|
watch(() => props.entry, (e) => {
|
||||||
|
name.value = props.modelName ?? ''
|
||||||
|
path.value = e?.path ?? ''
|
||||||
|
vramMb.value = e?.vram_mb ?? 0
|
||||||
|
description.value = e?.description ?? ''
|
||||||
|
multiGpu.value = e?.multi_gpu ?? false
|
||||||
|
envPairs.value = Object.entries(e?.env ?? {}).map(([k, v]) => ({ k, v }))
|
||||||
|
})
|
||||||
|
|
||||||
|
function addEnvPair() {
|
||||||
|
envPairs.value = [...envPairs.value, { k: '', v: '' }]
|
||||||
|
}
|
||||||
|
function removeEnvPair(i: number) {
|
||||||
|
envPairs.value = envPairs.value.filter((_, idx) => idx !== i)
|
||||||
|
}
|
||||||
|
|
||||||
|
function submit() {
|
||||||
|
formError.value = ''
|
||||||
|
if (!name.value.trim()) { formError.value = 'Model name is required.'; return }
|
||||||
|
if (!path.value.trim()) { formError.value = 'Path is required.'; return }
|
||||||
|
if (!vramMb.value || vramMb.value < 0) { formError.value = 'vram_mb must be a positive number.'; return }
|
||||||
|
|
||||||
|
const envObj: Record<string, string> = {}
|
||||||
|
for (const { k, v } of envPairs.value) {
|
||||||
|
if (k.trim()) envObj[k.trim()] = v
|
||||||
|
}
|
||||||
|
|
||||||
|
const entry: CatalogEntryFull = { path: path.value.trim(), vram_mb: vramMb.value }
|
||||||
|
if (description.value.trim()) entry.description = description.value.trim()
|
||||||
|
if (multiGpu.value) entry.multi_gpu = true
|
||||||
|
if (Object.keys(envObj).length) entry.env = envObj
|
||||||
|
|
||||||
|
emit('save', props.svcName, name.value.trim(), entry)
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<div class="modal-backdrop" role="dialog" aria-modal="true" :aria-label="`${modelName ? 'Edit' : 'Add'} catalog entry`">
|
||||||
|
<div class="modal-box">
|
||||||
|
<h3 class="modal-title">{{ modelName ? 'Edit' : 'Add' }} Catalog Entry — {{ svcName }}</h3>
|
||||||
|
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="ce-name">Model name</label>
|
||||||
|
<input id="ce-name" v-model="name" class="field-input" :readonly="!!modelName" placeholder="deepseek-r1-7b" />
|
||||||
|
</div>
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="ce-path">Path</label>
|
||||||
|
<input id="ce-path" v-model="path" class="field-input" placeholder="/devl/Assets/LLM/cf-text/models/..." />
|
||||||
|
</div>
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="ce-vram">VRAM (MB)</label>
|
||||||
|
<input id="ce-vram" v-model.number="vramMb" type="number" min="0" class="field-input field-input--sm" />
|
||||||
|
</div>
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="ce-desc">Description</label>
|
||||||
|
<input id="ce-desc" v-model="description" class="field-input" placeholder="Short description" />
|
||||||
|
</div>
|
||||||
|
<div class="field-row field-row--check">
|
||||||
|
<input id="ce-mgpu" v-model="multiGpu" type="checkbox" />
|
||||||
|
<label for="ce-mgpu">Multi-GPU span</label>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="env-section">
|
||||||
|
<div class="env-header">
|
||||||
|
<span class="field-label">Env vars</span>
|
||||||
|
<button type="button" class="btn-link" @click="addEnvPair">+ Add</button>
|
||||||
|
</div>
|
||||||
|
<div v-for="(pair, i) in envPairs" :key="i" class="env-row">
|
||||||
|
<input v-model="pair.k" class="field-input field-input--sm" placeholder="CF_TEXT_4BIT" />
|
||||||
|
<span>=</span>
|
||||||
|
<input v-model="pair.v" class="field-input field-input--sm" placeholder="1" />
|
||||||
|
<button type="button" class="btn-icon" @click="removeEnvPair(i)" aria-label="Remove">✕</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-if="formError" class="form-error" role="alert">{{ formError }}</div>
|
||||||
|
|
||||||
|
<div class="modal-actions">
|
||||||
|
<button class="btn-secondary" @click="emit('cancel')">Cancel</button>
|
||||||
|
<button class="btn-primary" @click="submit">Save</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<style scoped>
|
||||||
|
.modal-backdrop {
|
||||||
|
position: fixed; inset: 0;
|
||||||
|
background: rgba(0,0,0,0.5);
|
||||||
|
display: flex; align-items: center; justify-content: center;
|
||||||
|
z-index: 200;
|
||||||
|
}
|
||||||
|
.modal-box {
|
||||||
|
background: var(--color-surface-raised);
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: 8px;
|
||||||
|
padding: 1.5rem;
|
||||||
|
width: 100%; max-width: 500px;
|
||||||
|
max-height: 90vh; overflow-y: auto;
|
||||||
|
display: flex; flex-direction: column; gap: 0.75rem;
|
||||||
|
color: var(--color-text);
|
||||||
|
}
|
||||||
|
.modal-title { margin: 0 0 0.25rem; font-size: 1rem; font-weight: 600; color: var(--color-text); }
|
||||||
|
.field-row { display: flex; align-items: center; gap: 0.5rem; }
|
||||||
|
.field-row--check { gap: 0.4rem; color: var(--color-text); }
|
||||||
|
.field-label { min-width: 8rem; font-size: 0.85rem; color: var(--color-text-muted); }
|
||||||
|
.field-input {
|
||||||
|
flex: 1;
|
||||||
|
background: var(--color-surface-alt);
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.3rem 0.5rem;
|
||||||
|
color: var(--color-text);
|
||||||
|
font-size: 0.85rem;
|
||||||
|
}
|
||||||
|
.field-input--sm { flex: 0 0 8rem; }
|
||||||
|
.env-section { display: flex; flex-direction: column; gap: 0.35rem; }
|
||||||
|
.env-header { display: flex; align-items: center; justify-content: space-between; }
|
||||||
|
.env-row { display: flex; align-items: center; gap: 0.4rem; }
|
||||||
|
.btn-link { background: none; border: none; color: var(--app-primary); cursor: pointer; font-size: 0.8rem; padding: 0; }
|
||||||
|
.btn-link:hover { color: var(--app-primary-hover); }
|
||||||
|
.btn-icon { background: none; border: none; color: var(--color-text-muted); cursor: pointer; padding: 0 0.2rem; font-size: 0.85rem; }
|
||||||
|
.btn-icon:hover { color: var(--color-error); }
|
||||||
|
.form-error { color: var(--color-error); font-size: 0.8rem; }
|
||||||
|
.modal-actions { display: flex; justify-content: flex-end; gap: 0.5rem; margin-top: 0.25rem; }
|
||||||
|
.btn-primary {
|
||||||
|
background: var(--app-primary);
|
||||||
|
color: var(--color-text-inverse);
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.4rem 1rem;
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 0.875rem;
|
||||||
|
}
|
||||||
|
.btn-primary:hover { background: var(--app-primary-hover); }
|
||||||
|
.btn-secondary {
|
||||||
|
background: transparent;
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
color: var(--color-text);
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.4rem 0.75rem;
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 0.875rem;
|
||||||
|
}
|
||||||
|
.btn-secondary:hover { background: var(--color-surface-alt); }
|
||||||
|
</style>
|
||||||
|
|
@ -106,24 +106,24 @@ async function toggleService(svcName: string) {
|
||||||
.gpu-row {
|
.gpu-row {
|
||||||
padding: 0.5rem 0.75rem;
|
padding: 0.5rem 0.75rem;
|
||||||
border-radius: 4px;
|
border-radius: 4px;
|
||||||
background: var(--bg-secondary, #111);
|
background: var(--color-surface-alt);
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
gap: 0.4rem;
|
gap: 0.4rem;
|
||||||
}
|
}
|
||||||
.gpu-info { display: flex; gap: 0.75rem; align-items: center; flex-wrap: wrap; font-size: 0.875rem; }
|
.gpu-info { display: flex; gap: 0.75rem; align-items: center; flex-wrap: wrap; font-size: 0.875rem; }
|
||||||
.gpu-label { font-weight: 500; }
|
.gpu-label { font-weight: 500; color: var(--color-text); }
|
||||||
.gpu-meta { color: var(--text-secondary, #888); font-size: 0.8rem; }
|
.gpu-meta { color: var(--color-text-muted); font-size: 0.8rem; }
|
||||||
.vram-wrap { display: flex; align-items: center; gap: 0.5rem; }
|
.vram-wrap { display: flex; align-items: center; gap: 0.5rem; }
|
||||||
.vram-bar {
|
.vram-bar {
|
||||||
flex: 1;
|
flex: 1;
|
||||||
height: 8px;
|
height: 8px;
|
||||||
background: var(--bg-bar, #2a2a2a);
|
background: var(--color-border);
|
||||||
border-radius: 4px;
|
border-radius: 4px;
|
||||||
overflow: hidden;
|
overflow: hidden;
|
||||||
}
|
}
|
||||||
.vram-fill { height: 100%; background: var(--color-primary, #4080ff); transition: width 0.3s; }
|
.vram-fill { height: 100%; background: var(--app-primary); transition: width 0.3s; }
|
||||||
.vram-text { font-size: 0.75rem; color: var(--text-secondary, #888); white-space: nowrap; }
|
.vram-text { font-size: 0.75rem; color: var(--color-text-muted); white-space: nowrap; }
|
||||||
.services-row { display: flex; flex-wrap: wrap; gap: 0.4rem; }
|
.services-row { display: flex; flex-wrap: wrap; gap: 0.4rem; }
|
||||||
.save-msg { color: var(--color-warning, #ed8936); font-size: 0.8rem; }
|
.save-msg { color: var(--color-warning); font-size: 0.8rem; }
|
||||||
</style>
|
</style>
|
||||||
|
|
|
||||||
|
|
@ -99,19 +99,21 @@ onUnmounted(() => { fetchAbort?.abort() })
|
||||||
.hf-panel {
|
.hf-panel {
|
||||||
margin-top: 0.75rem;
|
margin-top: 0.75rem;
|
||||||
padding: 0.75rem;
|
padding: 0.75rem;
|
||||||
border: 1px solid var(--border, #333);
|
border: 1px solid var(--color-border);
|
||||||
border-radius: 6px;
|
border-radius: 6px;
|
||||||
|
color: var(--color-text);
|
||||||
}
|
}
|
||||||
.panel-title { margin: 0 0 0.5rem; font-size: 0.9rem; }
|
.panel-title { margin: 0 0 0.5rem; font-size: 0.9rem; color: var(--color-text); }
|
||||||
.hf-hint { font-size: 0.8rem; color: var(--text-secondary, #888); margin: 0 0 0.75rem; }
|
.hf-hint { font-size: 0.8rem; color: var(--color-text-muted); margin: 0 0 0.75rem; }
|
||||||
.hf-link { color: var(--color-primary, #4080ff); }
|
.hf-link { color: var(--app-primary); }
|
||||||
|
.hf-link:hover { color: var(--app-primary-hover); }
|
||||||
.svc-section { margin-bottom: 0.75rem; }
|
.svc-section { margin-bottom: 0.75rem; }
|
||||||
.svc-name {
|
.svc-name {
|
||||||
margin: 0 0 0.25rem;
|
margin: 0 0 0.25rem;
|
||||||
font-size: 0.75rem;
|
font-size: 0.75rem;
|
||||||
text-transform: uppercase;
|
text-transform: uppercase;
|
||||||
letter-spacing: 0.05em;
|
letter-spacing: 0.05em;
|
||||||
color: var(--text-secondary, #888);
|
color: var(--color-text-muted);
|
||||||
}
|
}
|
||||||
.catalog-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.2rem; }
|
.catalog-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.2rem; }
|
||||||
.catalog-item {
|
.catalog-item {
|
||||||
|
|
@ -119,14 +121,14 @@ onUnmounted(() => { fetchAbort?.abort() })
|
||||||
align-items: center;
|
align-items: center;
|
||||||
gap: 0.5rem;
|
gap: 0.5rem;
|
||||||
padding: 0.25rem 0.5rem;
|
padding: 0.25rem 0.5rem;
|
||||||
background: var(--bg-secondary, #111);
|
background: var(--color-surface-alt);
|
||||||
border-radius: 4px;
|
border-radius: 4px;
|
||||||
font-size: 0.8rem;
|
font-size: 0.8rem;
|
||||||
}
|
}
|
||||||
.catalog-model { font-family: monospace; flex: 1; }
|
.catalog-model { font-family: var(--font-mono, monospace); flex: 1; }
|
||||||
.catalog-vram { color: var(--text-secondary, #888); white-space: nowrap; }
|
.catalog-vram { color: var(--color-text-muted); white-space: nowrap; }
|
||||||
.catalog-desc { color: var(--text-secondary, #888); font-size: 0.75rem; flex: 2; }
|
.catalog-desc { color: var(--color-text-muted); font-size: 0.75rem; flex: 2; }
|
||||||
.catalog-empty, .panel-empty { color: var(--text-secondary, #888); font-size: 0.875rem; }
|
.catalog-empty, .panel-empty { color: var(--color-text-muted); font-size: 0.875rem; }
|
||||||
.sr-announce { min-height: 1.2em; }
|
.sr-announce { min-height: 1.2em; }
|
||||||
.panel-error { color: var(--color-error, #fc8181); font-size: 0.8rem; }
|
.panel-error { color: var(--color-error); font-size: 0.8rem; }
|
||||||
</style>
|
</style>
|
||||||
|
|
|
||||||
|
|
@ -2,14 +2,43 @@
|
||||||
import { ref } from 'vue'
|
import { ref } from 'vue'
|
||||||
import GpuRow from './GpuRow.vue'
|
import GpuRow from './GpuRow.vue'
|
||||||
import OllamaModelPanel from './OllamaModelPanel.vue'
|
import OllamaModelPanel from './OllamaModelPanel.vue'
|
||||||
import HfNodeModelPanel from './HfNodeModelPanel.vue'
|
import ProfileEditorPanel from './ProfileEditorPanel.vue'
|
||||||
import type { NodeSummary } from '../../types/nodes'
|
import type { NodeSummary, FullProfile } from '../../types/nodes'
|
||||||
|
|
||||||
const props = defineProps<{ node: NodeSummary }>()
|
const props = defineProps<{ node: NodeSummary }>()
|
||||||
const emit = defineEmits<{ updated: [] }>()
|
const emit = defineEmits<{ updated: [] }>()
|
||||||
|
|
||||||
const showOllama = ref(false)
|
const showOllama = ref(false)
|
||||||
const showHf = ref(false)
|
const showEditor = ref(false)
|
||||||
|
const loadedProfile = ref<FullProfile | null>(null)
|
||||||
|
const profileLoading = ref(false)
|
||||||
|
const profileError = ref('')
|
||||||
|
|
||||||
|
async function openEditor() {
|
||||||
|
if (showEditor.value) { showEditor.value = false; return }
|
||||||
|
profileLoading.value = true
|
||||||
|
profileError.value = ''
|
||||||
|
try {
|
||||||
|
const r = await fetch(`/api/nodes-mgmt/nodes/${props.node.node_id}/profile`)
|
||||||
|
if (r.status === 404) {
|
||||||
|
loadedProfile.value = null
|
||||||
|
} else if (!r.ok) {
|
||||||
|
throw new Error(`HTTP ${r.status}`)
|
||||||
|
} else {
|
||||||
|
loadedProfile.value = await r.json() as FullProfile
|
||||||
|
}
|
||||||
|
showEditor.value = true
|
||||||
|
} catch (e) {
|
||||||
|
profileError.value = e instanceof Error ? e.message : 'Failed to load profile'
|
||||||
|
} finally {
|
||||||
|
profileLoading.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function onProfileSaved() {
|
||||||
|
showEditor.value = false
|
||||||
|
emit('updated')
|
||||||
|
}
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
<template>
|
<template>
|
||||||
|
|
@ -25,12 +54,20 @@ const showHf = ref(false)
|
||||||
<h2 class="node-name">{{ node.node_id }}</h2>
|
<h2 class="node-name">{{ node.node_id }}</h2>
|
||||||
<span class="node-agent">{{ node.agent_url }}</span>
|
<span class="node-agent">{{ node.agent_url }}</span>
|
||||||
</div>
|
</div>
|
||||||
<div v-if="node.profile_loaded" class="node-actions">
|
<div class="node-actions">
|
||||||
<button class="btn-secondary btn-sm" @click="showOllama = !showOllama">
|
<button
|
||||||
|
v-if="node.profile_loaded"
|
||||||
|
class="btn-secondary btn-sm"
|
||||||
|
@click="showOllama = !showOllama"
|
||||||
|
>
|
||||||
{{ showOllama ? 'Hide Ollama' : 'Ollama' }}
|
{{ showOllama ? 'Hide Ollama' : 'Ollama' }}
|
||||||
</button>
|
</button>
|
||||||
<button class="btn-secondary btn-sm" @click="showHf = !showHf">
|
<button
|
||||||
{{ showHf ? 'Hide Catalog' : 'Catalog' }}
|
class="btn-secondary btn-sm"
|
||||||
|
:disabled="profileLoading"
|
||||||
|
@click="openEditor"
|
||||||
|
>
|
||||||
|
{{ profileLoading ? 'Loading…' : node.profile_loaded ? (showEditor ? 'Close Editor' : 'Edit Profile') : 'Create Profile' }}
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
</header>
|
</header>
|
||||||
|
|
@ -52,16 +89,24 @@ const showHf = ref(false)
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<OllamaModelPanel v-if="showOllama" :node-id="node.node_id" />
|
<OllamaModelPanel v-if="showOllama" :node-id="node.node_id" />
|
||||||
<HfNodeModelPanel v-if="showHf" :node-id="node.node_id" />
|
<div v-if="profileError" class="profile-load-error" role="alert">{{ profileError }}</div>
|
||||||
|
<ProfileEditorPanel
|
||||||
|
v-if="showEditor"
|
||||||
|
:node-id="node.node_id"
|
||||||
|
:initial-profile="loadedProfile"
|
||||||
|
@saved="onProfileSaved"
|
||||||
|
@close="showEditor = false"
|
||||||
|
/>
|
||||||
</section>
|
</section>
|
||||||
</template>
|
</template>
|
||||||
|
|
||||||
<style scoped>
|
<style scoped>
|
||||||
.node-card {
|
.node-card {
|
||||||
border: 1px solid var(--border, #333);
|
border: 1px solid var(--color-border);
|
||||||
border-radius: 8px;
|
border-radius: 8px;
|
||||||
padding: 1rem;
|
padding: 1rem;
|
||||||
background: var(--bg-card, #1a1a1a);
|
background: var(--color-surface-raised);
|
||||||
|
color: var(--color-text);
|
||||||
}
|
}
|
||||||
.node-card.offline { opacity: 0.65; }
|
.node-card.offline { opacity: 0.65; }
|
||||||
.node-card-header {
|
.node-card-header {
|
||||||
|
|
@ -72,19 +117,32 @@ const showHf = ref(false)
|
||||||
margin-bottom: 0.75rem;
|
margin-bottom: 0.75rem;
|
||||||
}
|
}
|
||||||
.node-identity { display: flex; align-items: center; gap: 0.5rem; flex-wrap: wrap; }
|
.node-identity { display: flex; align-items: center; gap: 0.5rem; flex-wrap: wrap; }
|
||||||
.node-name { margin: 0; font-size: 1rem; font-weight: 600; }
|
.node-name { margin: 0; font-size: 1rem; font-weight: 600; color: var(--color-text); }
|
||||||
.node-agent { color: var(--text-secondary, #888); font-size: 0.8rem; font-family: monospace; }
|
.node-agent { color: var(--color-text-muted); font-size: 0.8rem; font-family: var(--font-mono, monospace); }
|
||||||
.status-dot { width: 10px; height: 10px; border-radius: 50%; flex-shrink: 0; }
|
.status-dot { width: 10px; height: 10px; border-radius: 50%; flex-shrink: 0; }
|
||||||
.status-dot.online { background: var(--color-success, #48bb78); }
|
.status-dot.online { background: var(--color-success); }
|
||||||
.status-dot.offline { background: var(--color-warning, #ed8936); }
|
.status-dot.offline { background: var(--color-warning); }
|
||||||
.node-actions { display: flex; gap: 0.5rem; flex-shrink: 0; }
|
.node-actions { display: flex; gap: 0.5rem; flex-shrink: 0; }
|
||||||
|
.btn-secondary {
|
||||||
|
background: transparent;
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
color: var(--color-text);
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.3rem 0.65rem;
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 0.8rem;
|
||||||
|
}
|
||||||
|
.btn-secondary:hover { background: var(--color-surface-alt); }
|
||||||
|
.btn-secondary:disabled { opacity: 0.5; cursor: not-allowed; }
|
||||||
|
.btn-sm { font-size: 0.8rem; padding: 0.25rem 0.6rem; }
|
||||||
.no-profile {
|
.no-profile {
|
||||||
padding: 0.6rem 0.75rem;
|
padding: 0.6rem 0.75rem;
|
||||||
background: var(--bg-notice, #1e1e1e);
|
background: var(--color-surface-alt);
|
||||||
border-radius: 4px;
|
border-radius: 4px;
|
||||||
color: var(--text-secondary, #888);
|
color: var(--color-text-muted);
|
||||||
font-size: 0.875rem;
|
font-size: 0.875rem;
|
||||||
margin-bottom: 0.5rem;
|
margin-bottom: 0.5rem;
|
||||||
}
|
}
|
||||||
.gpu-list { display: flex; flex-direction: column; gap: 0.5rem; }
|
.gpu-list { display: flex; flex-direction: column; gap: 0.5rem; }
|
||||||
|
.profile-load-error { color: var(--color-error); font-size: 0.8rem; margin-top: 0.5rem; }
|
||||||
</style>
|
</style>
|
||||||
|
|
|
||||||
|
|
@ -198,44 +198,45 @@ onUnmounted(() => {
|
||||||
.ollama-panel {
|
.ollama-panel {
|
||||||
margin-top: 0.75rem;
|
margin-top: 0.75rem;
|
||||||
padding: 0.75rem;
|
padding: 0.75rem;
|
||||||
border: 1px solid var(--border, #333);
|
border: 1px solid var(--color-border);
|
||||||
border-radius: 6px;
|
border-radius: 6px;
|
||||||
|
color: var(--color-text);
|
||||||
}
|
}
|
||||||
.panel-title { margin: 0 0 0.75rem; font-size: 0.9rem; }
|
.panel-title { margin: 0 0 0.75rem; font-size: 0.9rem; color: var(--color-text); }
|
||||||
.pull-form { display: flex; gap: 0.5rem; margin-bottom: 0.5rem; }
|
.pull-form { display: flex; gap: 0.5rem; margin-bottom: 0.5rem; }
|
||||||
.pull-input {
|
.pull-input {
|
||||||
flex: 1;
|
flex: 1;
|
||||||
padding: 0.3rem 0.5rem;
|
padding: 0.3rem 0.5rem;
|
||||||
background: var(--bg-input, #111);
|
background: var(--color-surface-alt);
|
||||||
border: 1px solid var(--border, #333);
|
border: 1px solid var(--color-border);
|
||||||
border-radius: 4px;
|
border-radius: 4px;
|
||||||
color: inherit;
|
color: var(--color-text);
|
||||||
font-size: 0.875rem;
|
font-size: 0.875rem;
|
||||||
}
|
}
|
||||||
.pull-progress { margin-bottom: 0.5rem; }
|
.pull-progress { margin-bottom: 0.5rem; }
|
||||||
.progress-bar {
|
.progress-bar {
|
||||||
height: 8px;
|
height: 8px;
|
||||||
background: var(--bg-bar, #2a2a2a);
|
background: var(--color-border);
|
||||||
border-radius: 4px;
|
border-radius: 4px;
|
||||||
overflow: hidden;
|
overflow: hidden;
|
||||||
margin-bottom: 0.25rem;
|
margin-bottom: 0.25rem;
|
||||||
}
|
}
|
||||||
.progress-fill { height: 100%; background: var(--color-primary, #4080ff); transition: width 0.2s; }
|
.progress-fill { height: 100%; background: var(--app-primary); transition: width 0.2s; }
|
||||||
.progress-label { font-size: 0.75rem; color: var(--text-secondary, #888); }
|
.progress-label { font-size: 0.75rem; color: var(--color-text-muted); }
|
||||||
.pull-error, .panel-error { color: var(--color-error, #fc8181); font-size: 0.8rem; margin-bottom: 0.5rem; }
|
.pull-error, .panel-error { color: var(--color-error); font-size: 0.8rem; margin-bottom: 0.5rem; }
|
||||||
.sr-announce { min-height: 1.2em; }
|
.sr-announce { min-height: 1.2em; }
|
||||||
.panel-loading { color: var(--text-secondary, #888); font-size: 0.875rem; }
|
.panel-loading { color: var(--color-text-muted); font-size: 0.875rem; }
|
||||||
.model-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.3rem; }
|
.model-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.3rem; }
|
||||||
.model-item {
|
.model-item {
|
||||||
display: flex;
|
display: flex;
|
||||||
align-items: center;
|
align-items: center;
|
||||||
gap: 0.5rem;
|
gap: 0.5rem;
|
||||||
padding: 0.3rem 0.5rem;
|
padding: 0.3rem 0.5rem;
|
||||||
background: var(--bg-secondary, #111);
|
background: var(--color-surface-alt);
|
||||||
border-radius: 4px;
|
border-radius: 4px;
|
||||||
font-size: 0.875rem;
|
font-size: 0.875rem;
|
||||||
}
|
}
|
||||||
.model-name { flex: 1; font-family: monospace; }
|
.model-name { flex: 1; font-family: var(--font-mono, monospace); }
|
||||||
.model-size { color: var(--text-secondary, #888); font-size: 0.8rem; }
|
.model-size { color: var(--color-text-muted); font-size: 0.8rem; }
|
||||||
.model-empty { color: var(--text-secondary, #888); font-size: 0.875rem; padding: 0.25rem 0; }
|
.model-empty { color: var(--color-text-muted); font-size: 0.875rem; padding: 0.25rem 0; }
|
||||||
</style>
|
</style>
|
||||||
|
|
|
||||||
597
web/src/components/nodes/ProfileEditorPanel.vue
Normal file
597
web/src/components/nodes/ProfileEditorPanel.vue
Normal file
|
|
@ -0,0 +1,597 @@
|
||||||
|
<script setup lang="ts">
|
||||||
|
import { ref, onMounted } from 'vue'
|
||||||
|
import type { FullProfile, ServiceDefinition, CatalogEntryFull } from '../../types/nodes'
|
||||||
|
import ServiceFormModal from './ServiceFormModal.vue'
|
||||||
|
import CatalogEntryFormModal from './CatalogEntryFormModal.vue'
|
||||||
|
|
||||||
|
const props = defineProps<{
|
||||||
|
nodeId: string
|
||||||
|
initialProfile: FullProfile | null
|
||||||
|
}>()
|
||||||
|
const emit = defineEmits<{ saved: []; close: [] }>()
|
||||||
|
|
||||||
|
// Deep-clone initial profile so edits don't mutate the parent's data
|
||||||
|
const profile = ref<FullProfile>(
|
||||||
|
props.initialProfile
|
||||||
|
? JSON.parse(JSON.stringify(props.initialProfile))
|
||||||
|
: { services: {}, nodes: {} }
|
||||||
|
)
|
||||||
|
|
||||||
|
const saving = ref(false)
|
||||||
|
const generating = ref(false)
|
||||||
|
const opError = ref('')
|
||||||
|
const expandedSvcs = ref<Set<string>>(new Set())
|
||||||
|
|
||||||
|
// Service modal
|
||||||
|
const showSvcModal = ref(false)
|
||||||
|
const editingSvcName = ref<string | undefined>()
|
||||||
|
const editingSvcDef = ref<ServiceDefinition | undefined>()
|
||||||
|
|
||||||
|
// Catalog modal
|
||||||
|
const showCatalogModal = ref(false)
|
||||||
|
const catalogTargetSvc = ref('')
|
||||||
|
const editingModelName = ref<string | undefined>()
|
||||||
|
const editingEntry = ref<CatalogEntryFull | undefined>()
|
||||||
|
|
||||||
|
// ── Generate nodes section from coordinator ────────────────────────────────────
|
||||||
|
|
||||||
|
async function generate() {
|
||||||
|
generating.value = true
|
||||||
|
opError.value = ''
|
||||||
|
try {
|
||||||
|
const r = await fetch(`/api/nodes-mgmt/nodes/${props.nodeId}/profile/generate`, { method: 'POST' })
|
||||||
|
if (!r.ok) { const d = await r.json().catch(() => ({})); throw new Error((d as {detail?: string}).detail ?? `HTTP ${r.status}`) }
|
||||||
|
const generated = await r.json() as FullProfile
|
||||||
|
// Merge: keep current services edits, replace nodes section
|
||||||
|
profile.value = { ...generated, services: profile.value.services }
|
||||||
|
} catch (e) {
|
||||||
|
opError.value = e instanceof Error ? e.message : 'Generate failed'
|
||||||
|
} finally {
|
||||||
|
generating.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Save full profile ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
async function save() {
|
||||||
|
saving.value = true
|
||||||
|
opError.value = ''
|
||||||
|
try {
|
||||||
|
const r = await fetch(`/api/nodes-mgmt/nodes/${props.nodeId}/profile`, {
|
||||||
|
method: 'PUT',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ profile: profile.value }),
|
||||||
|
})
|
||||||
|
if (!r.ok) { const d = await r.json().catch(() => ({})); throw new Error((d as {detail?: string}).detail ?? `HTTP ${r.status}`) }
|
||||||
|
emit('saved')
|
||||||
|
} catch (e) {
|
||||||
|
opError.value = e instanceof Error ? e.message : 'Save failed'
|
||||||
|
} finally {
|
||||||
|
saving.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Service CRUD ───────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
function openAddService() {
|
||||||
|
editingSvcName.value = undefined
|
||||||
|
editingSvcDef.value = undefined
|
||||||
|
showSvcModal.value = true
|
||||||
|
}
|
||||||
|
|
||||||
|
function openEditService(name: string) {
|
||||||
|
editingSvcName.value = name
|
||||||
|
editingSvcDef.value = JSON.parse(JSON.stringify(profile.value.services[name]))
|
||||||
|
showSvcModal.value = true
|
||||||
|
}
|
||||||
|
|
||||||
|
function onServiceSaved(name: string, def: ServiceDefinition) {
|
||||||
|
profile.value = { ...profile.value, services: { ...profile.value.services, [name]: def } }
|
||||||
|
expandedSvcs.value = new Set([...expandedSvcs.value, name])
|
||||||
|
showSvcModal.value = false
|
||||||
|
}
|
||||||
|
|
||||||
|
function deleteService(name: string) {
|
||||||
|
if (!confirm(`Remove service "${name}" from this profile?`)) return
|
||||||
|
const svcs = { ...profile.value.services }
|
||||||
|
delete svcs[name]
|
||||||
|
profile.value = { ...profile.value, services: svcs }
|
||||||
|
expandedSvcs.value = new Set([...expandedSvcs.value].filter(s => s !== name))
|
||||||
|
}
|
||||||
|
|
||||||
|
function toggleSvc(name: string) {
|
||||||
|
const s = new Set(expandedSvcs.value)
|
||||||
|
s.has(name) ? s.delete(name) : s.add(name)
|
||||||
|
expandedSvcs.value = s
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Catalog CRUD ───────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
function openAddCatalogEntry(svcName: string) {
|
||||||
|
catalogTargetSvc.value = svcName
|
||||||
|
editingModelName.value = undefined
|
||||||
|
editingEntry.value = undefined
|
||||||
|
showCatalogModal.value = true
|
||||||
|
}
|
||||||
|
|
||||||
|
function openEditCatalogEntry(svcName: string, modelName: string) {
|
||||||
|
catalogTargetSvc.value = svcName
|
||||||
|
editingModelName.value = modelName
|
||||||
|
editingEntry.value = JSON.parse(JSON.stringify(profile.value.services[svcName].catalog![modelName]))
|
||||||
|
showCatalogModal.value = true
|
||||||
|
}
|
||||||
|
|
||||||
|
function onCatalogEntrySaved(svcName: string, modelName: string, entry: CatalogEntryFull) {
|
||||||
|
const svcs = { ...profile.value.services }
|
||||||
|
const svc = { ...svcs[svcName], catalog: { ...(svcs[svcName].catalog ?? {}), [modelName]: entry } }
|
||||||
|
svcs[svcName] = svc
|
||||||
|
profile.value = { ...profile.value, services: svcs }
|
||||||
|
showCatalogModal.value = false
|
||||||
|
}
|
||||||
|
|
||||||
|
function deleteCatalogEntry(svcName: string, modelName: string) {
|
||||||
|
if (!confirm(`Remove model "${modelName}" from ${svcName} catalog?`)) return
|
||||||
|
const svcs = { ...profile.value.services }
|
||||||
|
const catalog = { ...(svcs[svcName].catalog ?? {}) }
|
||||||
|
delete catalog[modelName]
|
||||||
|
svcs[svcName] = { ...svcs[svcName], catalog }
|
||||||
|
profile.value = { ...profile.value, services: svcs }
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Helpers ────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
function gpuList() {
|
||||||
|
return (profile.value.nodes[props.nodeId]?.gpus ?? [])
|
||||||
|
}
|
||||||
|
|
||||||
|
function serviceCount() {
|
||||||
|
return Object.keys(profile.value.services).length
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Ollama model suggestions ───────────────────────────────────────────────────
|
||||||
|
|
||||||
|
interface OllamaModel { name: string; size: number }
|
||||||
|
const ollamaModels = ref<OllamaModel[]>([])
|
||||||
|
const ollamaLoading = ref(false)
|
||||||
|
|
||||||
|
onMounted(async () => {
|
||||||
|
ollamaLoading.value = true
|
||||||
|
try {
|
||||||
|
const r = await fetch(`/api/nodes-mgmt/nodes/${props.nodeId}/models/ollama`)
|
||||||
|
if (r.ok) {
|
||||||
|
const d = await r.json() as { models?: OllamaModel[] }
|
||||||
|
ollamaModels.value = d.models ?? []
|
||||||
|
}
|
||||||
|
} catch { /* Ollama offline — silently skip */ }
|
||||||
|
finally { ollamaLoading.value = false }
|
||||||
|
})
|
||||||
|
|
||||||
|
function ollamaNotInCatalog(svcName: string): OllamaModel[] {
|
||||||
|
const catalog = profile.value.services[svcName]?.catalog ?? {}
|
||||||
|
return ollamaModels.value.filter(m => !(m.name in catalog))
|
||||||
|
}
|
||||||
|
|
||||||
|
function openAddFromOllama(svcName: string, modelName: string) {
|
||||||
|
catalogTargetSvc.value = svcName
|
||||||
|
editingModelName.value = modelName
|
||||||
|
editingEntry.value = {
|
||||||
|
path: profile.value.services[svcName]?.model_base_path
|
||||||
|
? `${profile.value.services[svcName].model_base_path}/${modelName}`
|
||||||
|
: '',
|
||||||
|
vram_mb: 0,
|
||||||
|
}
|
||||||
|
showCatalogModal.value = true
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatMb(bytes: number): string {
|
||||||
|
return bytes >= 1_000_000_000
|
||||||
|
? `${(bytes / 1_073_741_824).toFixed(1)} GB`
|
||||||
|
: `${Math.round(bytes / 1_048_576)} MB`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Pull model onto node ───────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
const pullName = ref('')
|
||||||
|
const pulling = ref(false)
|
||||||
|
const pullStatus = ref('')
|
||||||
|
const pullPct = ref(0)
|
||||||
|
const pullError = ref('')
|
||||||
|
let pullAbort: AbortController | null = null
|
||||||
|
|
||||||
|
async function doPull() {
|
||||||
|
const name = pullName.value.trim()
|
||||||
|
if (!name || pulling.value) return
|
||||||
|
pulling.value = true
|
||||||
|
pullStatus.value = 'Starting…'
|
||||||
|
pullError.value = ''
|
||||||
|
pullPct.value = 0
|
||||||
|
pullAbort?.abort()
|
||||||
|
pullAbort = new AbortController()
|
||||||
|
|
||||||
|
try {
|
||||||
|
const resp = await fetch(`/api/nodes-mgmt/nodes/${props.nodeId}/models/ollama/pull`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ name }),
|
||||||
|
signal: pullAbort.signal,
|
||||||
|
})
|
||||||
|
if (!resp.ok || !resp.body) {
|
||||||
|
pullError.value = `HTTP ${resp.status}`
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const reader = resp.body.getReader()
|
||||||
|
const dec = new TextDecoder()
|
||||||
|
let buf = ''
|
||||||
|
while (true) {
|
||||||
|
const { done, value } = await reader.read()
|
||||||
|
if (done) break
|
||||||
|
buf += dec.decode(value, { stream: true })
|
||||||
|
const lines = buf.split('\n')
|
||||||
|
buf = lines.pop() ?? ''
|
||||||
|
for (const line of lines) {
|
||||||
|
if (!line.startsWith('data:')) continue
|
||||||
|
try {
|
||||||
|
const d = JSON.parse(line.slice(5)) as {
|
||||||
|
status?: string; completed?: number; total?: number; error?: string; done?: boolean
|
||||||
|
}
|
||||||
|
if (d.error) { pullError.value = d.error; return }
|
||||||
|
pullStatus.value = d.status ?? ''
|
||||||
|
if (d.total && d.total > 0) pullPct.value = Math.round((d.completed ?? 0) / d.total * 100)
|
||||||
|
if (d.done) {
|
||||||
|
pullName.value = ''
|
||||||
|
pullPct.value = 100
|
||||||
|
// Refresh Ollama model list so new model appears in suggest chips
|
||||||
|
const r = await fetch(`/api/nodes-mgmt/nodes/${props.nodeId}/models/ollama`)
|
||||||
|
if (r.ok) { const d2 = await r.json() as { models?: OllamaModel[] }; ollamaModels.value = d2.models ?? [] }
|
||||||
|
}
|
||||||
|
} catch { /* skip malformed SSE line */ }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
if (e instanceof Error && e.name !== 'AbortError') pullError.value = e.message
|
||||||
|
} finally {
|
||||||
|
pulling.value = false
|
||||||
|
if (pullPct.value === 100) setTimeout(() => { pullStatus.value = ''; pullPct.value = 0 }, 2000)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<section class="pep" aria-label="Profile editor">
|
||||||
|
<!-- Header -->
|
||||||
|
<div class="pep-header">
|
||||||
|
<div class="pep-title-row">
|
||||||
|
<h3 class="pep-title">Profile — {{ nodeId }}</h3>
|
||||||
|
<span class="pep-svc-count">{{ serviceCount() }} service{{ serviceCount() === 1 ? '' : 's' }}</span>
|
||||||
|
</div>
|
||||||
|
<div class="pep-actions">
|
||||||
|
<button class="btn-secondary btn-sm" :disabled="generating" @click="generate">
|
||||||
|
{{ generating ? 'Refreshing…' : 'Refresh Hardware' }}
|
||||||
|
</button>
|
||||||
|
<button class="btn-primary btn-sm" :disabled="saving" @click="save">
|
||||||
|
{{ saving ? 'Saving…' : 'Save Profile' }}
|
||||||
|
</button>
|
||||||
|
<button class="btn-icon-lg" aria-label="Close editor" @click="emit('close')">✕</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-if="opError" class="pep-error" role="alert">{{ opError }}</div>
|
||||||
|
|
||||||
|
<!-- Meta fields -->
|
||||||
|
<div class="pep-meta">
|
||||||
|
<label class="meta-label" for="pep-vram">vram_total_mb</label>
|
||||||
|
<input id="pep-vram" v-model.number="profile.vram_total_mb" type="number" min="0" class="meta-input" />
|
||||||
|
<label class="meta-label" for="pep-evict">eviction_timeout_s</label>
|
||||||
|
<input id="pep-evict" v-model.number="profile.eviction_timeout_s" type="number" min="0" step="0.5" class="meta-input" />
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Hardware summary -->
|
||||||
|
<div v-if="gpuList().length" class="hw-section">
|
||||||
|
<span class="hw-label">Hardware</span>
|
||||||
|
<span v-for="g in gpuList()" :key="g.id" class="hw-gpu">
|
||||||
|
GPU {{ g.id }}: {{ g.card || 'unknown' }} · {{ g.vram_mb }} MB · sm{{ g.compute_cap ?? '?' }}
|
||||||
|
</span>
|
||||||
|
<span v-if="!gpuList().length" class="hw-none">No hardware data — click Refresh Hardware.</span>
|
||||||
|
</div>
|
||||||
|
<div v-else class="hw-section">
|
||||||
|
<span class="hw-none">No hardware data — click Refresh Hardware to seed from coordinator.</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Services -->
|
||||||
|
<div class="svcs-header">
|
||||||
|
<span class="svcs-title">Services</span>
|
||||||
|
<button class="btn-secondary btn-sm" @click="openAddService">+ Add Service</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-if="serviceCount() === 0" class="svcs-empty">
|
||||||
|
No services defined. Add a service to configure what can run on this node.
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<ul class="svcs-list" role="list">
|
||||||
|
<li
|
||||||
|
v-for="(def, svcName) in profile.services"
|
||||||
|
:key="String(svcName)"
|
||||||
|
class="svc-item"
|
||||||
|
>
|
||||||
|
<!-- Service row header -->
|
||||||
|
<div class="svc-row">
|
||||||
|
<button
|
||||||
|
class="svc-toggle"
|
||||||
|
:aria-expanded="expandedSvcs.has(String(svcName))"
|
||||||
|
@click="toggleSvc(String(svcName))"
|
||||||
|
>
|
||||||
|
<span class="svc-arrow">{{ expandedSvcs.has(String(svcName)) ? '▾' : '▸' }}</span>
|
||||||
|
<span class="svc-name">{{ svcName }}</span>
|
||||||
|
</button>
|
||||||
|
<span class="svc-badges">
|
||||||
|
<span class="badge">{{ def.max_mb }} MB</span>
|
||||||
|
<span class="badge">p{{ def.priority }}</span>
|
||||||
|
<span v-if="def.shared" class="badge badge--blue">shared</span>
|
||||||
|
<span v-if="def.managed" class="badge badge--dim">managed</span>
|
||||||
|
<span v-if="def.catalog" class="badge badge--dim">{{ Object.keys(def.catalog).length }} models</span>
|
||||||
|
</span>
|
||||||
|
<div class="svc-btns">
|
||||||
|
<button class="btn-secondary btn-xs" @click="openEditService(String(svcName))">Edit</button>
|
||||||
|
<button class="btn-danger btn-xs" @click="deleteService(String(svcName))">Delete</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Expanded catalog -->
|
||||||
|
<div v-if="expandedSvcs.has(String(svcName))" class="svc-detail">
|
||||||
|
<div class="svc-detail-meta">
|
||||||
|
<span v-if="def.min_compute_cap">min sm{{ def.min_compute_cap }}</span>
|
||||||
|
<span v-if="def.max_concurrent">max_concurrent: {{ def.max_concurrent }}</span>
|
||||||
|
<span v-if="def.idle_stop_after_s">idle_stop: {{ def.idle_stop_after_s }}s</span>
|
||||||
|
<span v-if="def.always_on" class="badge badge--blue">always_on</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Ollama model suggestions + pull -->
|
||||||
|
<div class="ollama-suggest">
|
||||||
|
<div class="suggest-row">
|
||||||
|
<span class="suggest-label">On node (Ollama):</span>
|
||||||
|
<span v-if="ollamaLoading" class="suggest-loading">loading…</span>
|
||||||
|
<template v-else-if="ollamaNotInCatalog(String(svcName)).length">
|
||||||
|
<button
|
||||||
|
v-for="m in ollamaNotInCatalog(String(svcName))"
|
||||||
|
:key="m.name"
|
||||||
|
class="suggest-chip"
|
||||||
|
:title="`Add ${m.name} (${formatMb(m.size)}) to this service catalog`"
|
||||||
|
@click="openAddFromOllama(String(svcName), m.name)"
|
||||||
|
>
|
||||||
|
+ {{ m.name }} <span class="chip-size">{{ formatMb(m.size) }}</span>
|
||||||
|
</button>
|
||||||
|
</template>
|
||||||
|
<span v-else-if="!ollamaLoading" class="suggest-none">All Ollama models already in catalog.</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Pull model onto this node -->
|
||||||
|
<div class="pull-row">
|
||||||
|
<input
|
||||||
|
v-model="pullName"
|
||||||
|
class="pull-input"
|
||||||
|
placeholder="Pull model on node (e.g. llama3:8b)"
|
||||||
|
:disabled="pulling"
|
||||||
|
@keyup.enter="doPull"
|
||||||
|
/>
|
||||||
|
<button class="btn-pull" :disabled="pulling || !pullName.trim()" @click="doPull">
|
||||||
|
{{ pulling ? 'Pulling…' : 'Pull' }}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
<div v-if="pulling || pullPct > 0" class="pull-progress">
|
||||||
|
<div class="pull-bar"><div class="pull-fill" :style="{ width: pullPct + '%' }" /></div>
|
||||||
|
<span class="pull-status">{{ pullStatus }}</span>
|
||||||
|
</div>
|
||||||
|
<div v-if="pullError" class="pull-err" role="alert">{{ pullError }}</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="catalog-header">
|
||||||
|
<span class="catalog-title">Catalog</span>
|
||||||
|
<button class="btn-link" @click="openAddCatalogEntry(String(svcName))">+ Add Model</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-if="!def.catalog || !Object.keys(def.catalog).length" class="catalog-empty">
|
||||||
|
No catalog entries. Only services like cf-text need a catalog.
|
||||||
|
</div>
|
||||||
|
<ul v-else class="catalog-list" role="list">
|
||||||
|
<li
|
||||||
|
v-for="(entry, modelName) in def.catalog"
|
||||||
|
:key="String(modelName)"
|
||||||
|
class="catalog-item"
|
||||||
|
>
|
||||||
|
<span class="catalog-model">{{ modelName }}</span>
|
||||||
|
<span class="catalog-vram">{{ entry.vram_mb }} MB</span>
|
||||||
|
<span v-if="entry.multi_gpu" class="badge badge--dim">multi-gpu</span>
|
||||||
|
<span v-if="entry.description" class="catalog-desc">{{ entry.description }}</span>
|
||||||
|
<div class="catalog-btns">
|
||||||
|
<button class="btn-secondary btn-xs" @click="openEditCatalogEntry(String(svcName), String(modelName))">Edit</button>
|
||||||
|
<button class="btn-danger btn-xs" @click="deleteCatalogEntry(String(svcName), String(modelName))">✕</button>
|
||||||
|
</div>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<!-- Service form modal -->
|
||||||
|
<ServiceFormModal
|
||||||
|
v-if="showSvcModal"
|
||||||
|
:service-name="editingSvcName"
|
||||||
|
:definition="editingSvcDef"
|
||||||
|
@save="onServiceSaved"
|
||||||
|
@cancel="showSvcModal = false"
|
||||||
|
/>
|
||||||
|
|
||||||
|
<!-- Catalog entry form modal -->
|
||||||
|
<CatalogEntryFormModal
|
||||||
|
v-if="showCatalogModal"
|
||||||
|
:svc-name="catalogTargetSvc"
|
||||||
|
:model-name="editingModelName"
|
||||||
|
:entry="editingEntry"
|
||||||
|
@save="onCatalogEntrySaved"
|
||||||
|
@cancel="showCatalogModal = false"
|
||||||
|
/>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<style scoped>
|
||||||
|
.pep {
|
||||||
|
margin-top: 0.75rem;
|
||||||
|
padding: 1rem;
|
||||||
|
border: 1px solid var(--color-primary);
|
||||||
|
border-radius: 6px;
|
||||||
|
background: var(--color-surface-raised);
|
||||||
|
color: var(--color-text);
|
||||||
|
}
|
||||||
|
.pep-header {
|
||||||
|
display: flex; align-items: center; justify-content: space-between; gap: 0.5rem;
|
||||||
|
margin-bottom: 0.75rem; flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
.pep-title-row { display: flex; align-items: baseline; gap: 0.5rem; }
|
||||||
|
.pep-title { margin: 0; font-size: 0.95rem; font-weight: 600; color: var(--color-text); }
|
||||||
|
.pep-svc-count { font-size: 0.75rem; color: var(--color-text-muted); }
|
||||||
|
.pep-actions { display: flex; align-items: center; gap: 0.4rem; flex-wrap: wrap; }
|
||||||
|
.pep-error { color: var(--color-error); font-size: 0.8rem; margin-bottom: 0.5rem; }
|
||||||
|
.pep-meta {
|
||||||
|
display: flex; align-items: center; gap: 0.5rem; flex-wrap: wrap;
|
||||||
|
padding: 0.5rem; background: var(--color-surface-alt); border-radius: 4px; margin-bottom: 0.75rem;
|
||||||
|
}
|
||||||
|
.meta-label { font-size: 0.8rem; color: var(--color-text-muted); }
|
||||||
|
.meta-input {
|
||||||
|
width: 7rem; background: var(--color-surface); border: 1px solid var(--color-border);
|
||||||
|
border-radius: 4px; padding: 0.2rem 0.4rem; color: var(--color-text); font-size: 0.8rem;
|
||||||
|
}
|
||||||
|
.hw-section {
|
||||||
|
display: flex; flex-wrap: wrap; align-items: center; gap: 0.5rem;
|
||||||
|
font-size: 0.8rem; color: var(--color-text-muted);
|
||||||
|
padding: 0.4rem 0.5rem; border-radius: 4px; background: var(--color-surface-alt);
|
||||||
|
margin-bottom: 0.75rem;
|
||||||
|
}
|
||||||
|
.hw-label { font-weight: 600; color: var(--color-text); }
|
||||||
|
.hw-gpu { font-family: monospace; color: var(--color-text); }
|
||||||
|
.hw-none { font-style: italic; }
|
||||||
|
.svcs-header {
|
||||||
|
display: flex; align-items: center; justify-content: space-between;
|
||||||
|
margin-bottom: 0.5rem;
|
||||||
|
}
|
||||||
|
.svcs-title { font-size: 0.85rem; font-weight: 600; color: var(--color-text); }
|
||||||
|
.svcs-empty { color: var(--color-text-muted); font-size: 0.85rem; padding: 0.5rem 0; }
|
||||||
|
.svcs-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.4rem; }
|
||||||
|
.svc-item { border: 1px solid var(--color-border); border-radius: 4px; overflow: hidden; }
|
||||||
|
.svc-row {
|
||||||
|
display: flex; align-items: center; gap: 0.5rem; padding: 0.4rem 0.5rem;
|
||||||
|
background: var(--color-surface-alt); flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
.svc-toggle {
|
||||||
|
display: flex; align-items: center; gap: 0.35rem;
|
||||||
|
background: none; border: none; cursor: pointer; color: var(--color-text); padding: 0; flex: 1; min-width: 0;
|
||||||
|
}
|
||||||
|
.svc-arrow { font-size: 0.7rem; color: var(--color-text-muted); }
|
||||||
|
.svc-name { font-size: 0.875rem; font-weight: 500; font-family: monospace; }
|
||||||
|
.svc-badges { display: flex; gap: 0.3rem; flex-wrap: wrap; }
|
||||||
|
.svc-btns { display: flex; gap: 0.3rem; margin-left: auto; }
|
||||||
|
.svc-detail { padding: 0.5rem 0.75rem; display: flex; flex-direction: column; gap: 0.5rem; background: var(--color-surface-raised); }
|
||||||
|
.svc-detail-meta {
|
||||||
|
display: flex; gap: 0.5rem; flex-wrap: wrap;
|
||||||
|
font-size: 0.78rem; color: var(--color-text-muted);
|
||||||
|
}
|
||||||
|
.ollama-suggest {
|
||||||
|
display: flex; flex-direction: column; gap: 0.35rem;
|
||||||
|
padding: 0.4rem 0.5rem;
|
||||||
|
background: var(--color-primary-light);
|
||||||
|
border: 1px solid var(--color-border-light);
|
||||||
|
border-radius: 4px;
|
||||||
|
font-size: 0.78rem;
|
||||||
|
}
|
||||||
|
.suggest-row { display: flex; flex-wrap: wrap; align-items: center; gap: 0.35rem; }
|
||||||
|
.suggest-label { color: var(--color-text-muted); font-weight: 500; white-space: nowrap; }
|
||||||
|
.suggest-loading { color: var(--color-text-muted); font-style: italic; }
|
||||||
|
.suggest-none { color: var(--color-text-muted); font-style: italic; }
|
||||||
|
.suggest-chip {
|
||||||
|
display: inline-flex; align-items: center; gap: 0.25rem;
|
||||||
|
padding: 0.15rem 0.45rem;
|
||||||
|
background: var(--color-surface-raised);
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: 3px;
|
||||||
|
color: var(--color-text);
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 0.78rem;
|
||||||
|
transition: border-color 0.15s, background 0.15s;
|
||||||
|
}
|
||||||
|
.suggest-chip:hover { border-color: var(--app-primary); background: var(--color-surface-alt); }
|
||||||
|
.chip-size { color: var(--color-text-muted); font-size: 0.72rem; }
|
||||||
|
.pull-row { display: flex; gap: 0.4rem; align-items: center; }
|
||||||
|
.pull-input {
|
||||||
|
flex: 1;
|
||||||
|
padding: 0.25rem 0.5rem;
|
||||||
|
background: var(--color-surface-raised);
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: 4px;
|
||||||
|
color: var(--color-text);
|
||||||
|
font-size: 0.78rem;
|
||||||
|
font-family: var(--font-mono, monospace);
|
||||||
|
}
|
||||||
|
.pull-input:disabled { opacity: 0.5; }
|
||||||
|
.btn-pull {
|
||||||
|
padding: 0.25rem 0.6rem;
|
||||||
|
background: var(--app-primary);
|
||||||
|
color: var(--color-text-inverse);
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 0.78rem;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
.btn-pull:hover:not(:disabled) { background: var(--app-primary-hover); }
|
||||||
|
.btn-pull:disabled { opacity: 0.5; cursor: not-allowed; }
|
||||||
|
.pull-progress { display: flex; align-items: center; gap: 0.4rem; }
|
||||||
|
.pull-bar {
|
||||||
|
flex: 1; height: 6px;
|
||||||
|
background: var(--color-border);
|
||||||
|
border-radius: 3px; overflow: hidden;
|
||||||
|
}
|
||||||
|
.pull-fill { height: 100%; background: var(--app-primary); transition: width 0.2s; }
|
||||||
|
.pull-status { color: var(--color-text-muted); font-size: 0.72rem; white-space: nowrap; max-width: 14rem; overflow: hidden; text-overflow: ellipsis; }
|
||||||
|
.pull-err { color: var(--color-error); font-size: 0.75rem; }
|
||||||
|
.catalog-header { display: flex; align-items: center; justify-content: space-between; }
|
||||||
|
.catalog-title { font-size: 0.8rem; font-weight: 600; color: var(--color-text-muted); text-transform: uppercase; letter-spacing: 0.05em; }
|
||||||
|
.catalog-empty { font-size: 0.8rem; color: var(--color-text-muted); font-style: italic; }
|
||||||
|
.catalog-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.25rem; }
|
||||||
|
.catalog-item {
|
||||||
|
display: flex; align-items: center; gap: 0.4rem; flex-wrap: wrap;
|
||||||
|
padding: 0.25rem 0.5rem; background: var(--color-surface-alt); border-radius: 3px; font-size: 0.8rem;
|
||||||
|
color: var(--color-text);
|
||||||
|
}
|
||||||
|
.catalog-model { font-family: monospace; flex: 1; min-width: 12rem; }
|
||||||
|
.catalog-vram { color: var(--color-text-muted); white-space: nowrap; }
|
||||||
|
.catalog-desc { color: var(--color-text-muted); flex: 2; font-size: 0.75rem; }
|
||||||
|
.catalog-btns { display: flex; gap: 0.25rem; margin-left: auto; }
|
||||||
|
.badge {
|
||||||
|
padding: 0.1rem 0.4rem; border-radius: 3px; font-size: 0.72rem;
|
||||||
|
background: var(--color-surface); border: 1px solid var(--color-border); color: var(--color-text);
|
||||||
|
}
|
||||||
|
.badge--blue { border-color: var(--color-primary); color: var(--color-primary); background: var(--color-primary-light); }
|
||||||
|
.badge--dim { opacity: 0.75; }
|
||||||
|
.btn-link { background: none; border: none; color: var(--color-accent); cursor: pointer; font-size: 0.8rem; padding: 0; }
|
||||||
|
.btn-link:hover { color: var(--color-accent-hover); }
|
||||||
|
.btn-primary {
|
||||||
|
background: var(--color-primary); color: var(--color-text-inverse); border: none;
|
||||||
|
border-radius: 4px; cursor: pointer; font-size: 0.8rem;
|
||||||
|
}
|
||||||
|
.btn-primary:hover { background: var(--color-primary-hover); }
|
||||||
|
.btn-primary:disabled { opacity: 0.6; cursor: not-allowed; }
|
||||||
|
.btn-secondary {
|
||||||
|
background: transparent; border: 1px solid var(--color-border); color: var(--color-text);
|
||||||
|
border-radius: 4px; cursor: pointer; font-size: 0.8rem;
|
||||||
|
}
|
||||||
|
.btn-secondary:hover { background: var(--color-surface-alt); }
|
||||||
|
.btn-secondary:disabled { opacity: 0.6; cursor: not-allowed; }
|
||||||
|
.btn-danger {
|
||||||
|
background: transparent; border: 1px solid var(--color-error); color: var(--color-error);
|
||||||
|
border-radius: 4px; cursor: pointer; font-size: 0.8rem;
|
||||||
|
}
|
||||||
|
.btn-danger:hover { background: var(--color-surface-alt); }
|
||||||
|
.btn-sm { padding: 0.3rem 0.6rem; }
|
||||||
|
.btn-xs { padding: 0.15rem 0.4rem; }
|
||||||
|
.btn-icon-lg { background: none; border: none; color: var(--color-text-muted); cursor: pointer; font-size: 1rem; padding: 0.2rem 0.3rem; }
|
||||||
|
.btn-icon-lg:hover { color: var(--color-text); }
|
||||||
|
</style>
|
||||||
|
|
@ -64,18 +64,19 @@ function handleToggle() {
|
||||||
gap: 0.3rem;
|
gap: 0.3rem;
|
||||||
padding: 0.2rem 0.5rem;
|
padding: 0.2rem 0.5rem;
|
||||||
border-radius: 4px;
|
border-radius: 4px;
|
||||||
border: 1px solid var(--border, #333);
|
border: 1px solid var(--color-border);
|
||||||
background: var(--bg-badge, #1e1e1e);
|
background: var(--color-surface);
|
||||||
|
color: var(--color-text);
|
||||||
font-size: 0.75rem;
|
font-size: 0.75rem;
|
||||||
cursor: pointer;
|
cursor: pointer;
|
||||||
transition: opacity 0.1s, border-color 0.1s;
|
transition: opacity 0.1s, border-color 0.1s;
|
||||||
}
|
}
|
||||||
.service-badge:hover:not(.is-disabled) { opacity: 0.8; }
|
.service-badge:hover:not(.is-disabled) { opacity: 0.8; }
|
||||||
.service-badge.is-disabled { cursor: not-allowed; opacity: 0.5; }
|
.service-badge.is-disabled { cursor: not-allowed; opacity: 0.5; }
|
||||||
.service-badge.state-running { border-color: var(--color-success, #48bb78); }
|
.service-badge.state-running { border-color: var(--color-success); }
|
||||||
.service-badge.state-stopped { border-color: var(--color-warning, #ed8936); }
|
.service-badge.state-stopped { border-color: var(--color-warning); }
|
||||||
.service-badge.state-assigned-only { border-color: var(--color-info, #4299e1); }
|
.service-badge.state-assigned-only { border-color: var(--color-info); }
|
||||||
.service-badge.state-incompatible { border-color: var(--color-error, #fc8181); }
|
.service-badge.state-incompatible { border-color: var(--color-error); }
|
||||||
.service-badge.state-vram-tight { border-color: var(--color-warning, #ed8936); }
|
.service-badge.state-vram-tight { border-color: var(--color-warning); }
|
||||||
.badge-state { color: var(--text-secondary, #888); }
|
.badge-state { color: var(--color-text-muted); }
|
||||||
</style>
|
</style>
|
||||||
|
|
|
||||||
231
web/src/components/nodes/ServiceFormModal.vue
Normal file
231
web/src/components/nodes/ServiceFormModal.vue
Normal file
|
|
@ -0,0 +1,231 @@
|
||||||
|
<script setup lang="ts">
|
||||||
|
import { ref, watch, computed } from 'vue'
|
||||||
|
import type { ServiceDefinition } from '../../types/nodes'
|
||||||
|
|
||||||
|
const props = defineProps<{
|
||||||
|
serviceName?: string
|
||||||
|
definition?: ServiceDefinition
|
||||||
|
}>()
|
||||||
|
const emit = defineEmits<{
|
||||||
|
save: [name: string, def: ServiceDefinition]
|
||||||
|
cancel: []
|
||||||
|
}>()
|
||||||
|
|
||||||
|
const name = ref(props.serviceName ?? '')
|
||||||
|
const maxMb = ref(props.definition?.max_mb ?? 0)
|
||||||
|
const priority = ref(props.definition?.priority ?? 1)
|
||||||
|
const minCap = ref(props.definition?.min_compute_cap ?? 0)
|
||||||
|
const prefCap = ref<number | ''>(props.definition?.preferred_compute_cap ?? '')
|
||||||
|
const shared = ref(props.definition?.shared ?? false)
|
||||||
|
const maxConcurrent = ref<number | ''>(props.definition?.max_concurrent ?? '')
|
||||||
|
const idleStop = ref<number | ''>(props.definition?.idle_stop_after_s ?? '')
|
||||||
|
const alwaysOn = ref(props.definition?.always_on ?? false)
|
||||||
|
const modelBasePath = ref(props.definition?.model_base_path ?? '')
|
||||||
|
const hasManaged = ref(!!props.definition?.managed)
|
||||||
|
const managedJson = ref(
|
||||||
|
props.definition?.managed ? JSON.stringify(props.definition.managed, null, 2) : ''
|
||||||
|
)
|
||||||
|
const formError = ref('')
|
||||||
|
|
||||||
|
watch(() => props.definition, (d) => {
|
||||||
|
name.value = props.serviceName ?? ''
|
||||||
|
maxMb.value = d?.max_mb ?? 0
|
||||||
|
priority.value = d?.priority ?? 1
|
||||||
|
minCap.value = d?.min_compute_cap ?? 0
|
||||||
|
prefCap.value = d?.preferred_compute_cap ?? ''
|
||||||
|
shared.value = d?.shared ?? false
|
||||||
|
maxConcurrent.value = d?.max_concurrent ?? ''
|
||||||
|
idleStop.value = d?.idle_stop_after_s ?? ''
|
||||||
|
alwaysOn.value = d?.always_on ?? false
|
||||||
|
modelBasePath.value = d?.model_base_path ?? ''
|
||||||
|
hasManaged.value = !!d?.managed
|
||||||
|
managedJson.value = d?.managed ? JSON.stringify(d.managed, null, 2) : ''
|
||||||
|
})
|
||||||
|
|
||||||
|
const managedJsonError = computed(() => {
|
||||||
|
if (!hasManaged.value || !managedJson.value.trim()) return ''
|
||||||
|
try { JSON.parse(managedJson.value); return '' }
|
||||||
|
catch { return 'Invalid JSON' }
|
||||||
|
})
|
||||||
|
|
||||||
|
function submit() {
|
||||||
|
formError.value = ''
|
||||||
|
if (!name.value.trim()) { formError.value = 'Service name is required.'; return }
|
||||||
|
if (!maxMb.value || maxMb.value <= 0) { formError.value = 'max_mb must be > 0.'; return }
|
||||||
|
if (managedJsonError.value) { formError.value = 'Fix the managed JSON before saving.'; return }
|
||||||
|
|
||||||
|
const def: ServiceDefinition = { max_mb: maxMb.value, priority: priority.value }
|
||||||
|
if (minCap.value) def.min_compute_cap = minCap.value
|
||||||
|
if (prefCap.value !== '') def.preferred_compute_cap = Number(prefCap.value)
|
||||||
|
if (shared.value) def.shared = true
|
||||||
|
if (maxConcurrent.value !== '') def.max_concurrent = Number(maxConcurrent.value)
|
||||||
|
if (idleStop.value !== '') def.idle_stop_after_s = Number(idleStop.value)
|
||||||
|
if (alwaysOn.value) def.always_on = true
|
||||||
|
if (modelBasePath.value.trim()) def.model_base_path = modelBasePath.value.trim()
|
||||||
|
if (hasManaged.value && managedJson.value.trim()) {
|
||||||
|
def.managed = JSON.parse(managedJson.value)
|
||||||
|
}
|
||||||
|
// Preserve existing catalog when editing
|
||||||
|
if (props.definition?.catalog) def.catalog = props.definition.catalog
|
||||||
|
|
||||||
|
emit('save', name.value.trim(), def)
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<div class="modal-backdrop" role="dialog" aria-modal="true" :aria-label="`${serviceName ? 'Edit' : 'Add'} service`">
|
||||||
|
<div class="modal-box">
|
||||||
|
<h3 class="modal-title">{{ serviceName ? 'Edit' : 'Add' }} Service</h3>
|
||||||
|
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="sf-name">Service name</label>
|
||||||
|
<input id="sf-name" v-model="name" class="field-input" :readonly="!!serviceName" placeholder="cf-text" />
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="sf-maxmb">max_mb</label>
|
||||||
|
<input id="sf-maxmb" v-model.number="maxMb" type="number" min="0" class="field-input field-input--sm" />
|
||||||
|
<span class="field-hint">VRAM ceiling</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="sf-prio">priority</label>
|
||||||
|
<input id="sf-prio" v-model.number="priority" type="number" min="1" max="10" class="field-input field-input--sm" />
|
||||||
|
<span class="field-hint">1 = highest</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="sf-mincap">min_compute_cap</label>
|
||||||
|
<input id="sf-mincap" v-model.number="minCap" type="number" step="0.1" min="0" class="field-input field-input--sm" placeholder="0.0" />
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="sf-prefcap">preferred_cap</label>
|
||||||
|
<input id="sf-prefcap" v-model="prefCap" type="number" step="0.1" min="0" class="field-input field-input--sm" placeholder="optional" />
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="field-row field-row--check">
|
||||||
|
<input id="sf-shared" v-model="shared" type="checkbox" />
|
||||||
|
<label for="sf-shared">shared (multiple concurrent users)</label>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="sf-maxcon">max_concurrent</label>
|
||||||
|
<input id="sf-maxcon" v-model="maxConcurrent" type="number" min="1" class="field-input field-input--sm" placeholder="optional" />
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="sf-idle">idle_stop_after_s</label>
|
||||||
|
<input id="sf-idle" v-model="idleStop" type="number" min="0" class="field-input field-input--sm" placeholder="optional" />
|
||||||
|
<span class="field-hint">seconds</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="field-row field-row--check">
|
||||||
|
<input id="sf-always" v-model="alwaysOn" type="checkbox" />
|
||||||
|
<label for="sf-always">always_on (never evict)</label>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="field-row">
|
||||||
|
<label class="field-label" for="sf-base">model_base_path</label>
|
||||||
|
<input id="sf-base" v-model="modelBasePath" class="field-input" placeholder="/devl/Assets/LLM/cf-text/models (optional)" />
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="managed-section">
|
||||||
|
<div class="field-row field-row--check">
|
||||||
|
<input id="sf-has-managed" v-model="hasManaged" type="checkbox" />
|
||||||
|
<label for="sf-has-managed">Has managed process config</label>
|
||||||
|
</div>
|
||||||
|
<div v-if="hasManaged" class="managed-body">
|
||||||
|
<label class="field-label" for="sf-managed">managed (JSON)</label>
|
||||||
|
<textarea
|
||||||
|
id="sf-managed"
|
||||||
|
v-model="managedJson"
|
||||||
|
class="field-textarea"
|
||||||
|
rows="6"
|
||||||
|
spellcheck="false"
|
||||||
|
placeholder='{"type": "process", "exec_path": "...", "args_template": "...", "port": 8008, "host_port": 8008}'
|
||||||
|
/>
|
||||||
|
<span v-if="managedJsonError" class="json-error" role="alert">{{ managedJsonError }}</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-if="formError" class="form-error" role="alert">{{ formError }}</div>
|
||||||
|
|
||||||
|
<div class="modal-actions">
|
||||||
|
<button class="btn-secondary" @click="emit('cancel')">Cancel</button>
|
||||||
|
<button class="btn-primary" @click="submit">Save</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<style scoped>
|
||||||
|
.modal-backdrop {
|
||||||
|
position: fixed; inset: 0;
|
||||||
|
background: rgba(0,0,0,0.5);
|
||||||
|
display: flex; align-items: center; justify-content: center;
|
||||||
|
z-index: 200;
|
||||||
|
}
|
||||||
|
.modal-box {
|
||||||
|
background: var(--color-surface-raised);
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: 8px;
|
||||||
|
padding: 1.5rem;
|
||||||
|
width: 100%; max-width: 540px;
|
||||||
|
max-height: 90vh; overflow-y: auto;
|
||||||
|
display: flex; flex-direction: column; gap: 0.65rem;
|
||||||
|
color: var(--color-text);
|
||||||
|
}
|
||||||
|
.modal-title { margin: 0 0 0.25rem; font-size: 1rem; font-weight: 600; color: var(--color-text); }
|
||||||
|
.field-row { display: flex; align-items: center; gap: 0.5rem; }
|
||||||
|
.field-row--check { gap: 0.4rem; font-size: 0.875rem; color: var(--color-text); }
|
||||||
|
.field-label { min-width: 9rem; font-size: 0.85rem; color: var(--color-text-muted); flex-shrink: 0; }
|
||||||
|
.field-hint { font-size: 0.75rem; color: var(--color-text-muted); }
|
||||||
|
.field-input {
|
||||||
|
flex: 1;
|
||||||
|
background: var(--color-surface-alt);
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.3rem 0.5rem;
|
||||||
|
color: var(--color-text);
|
||||||
|
font-size: 0.85rem;
|
||||||
|
}
|
||||||
|
.field-input--sm { flex: 0 0 8rem; }
|
||||||
|
.managed-section { display: flex; flex-direction: column; gap: 0.4rem; border-top: 1px solid var(--color-border); padding-top: 0.5rem; }
|
||||||
|
.managed-body { display: flex; flex-direction: column; gap: 0.3rem; }
|
||||||
|
.field-textarea {
|
||||||
|
width: 100%;
|
||||||
|
background: var(--color-surface-alt);
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.4rem 0.5rem;
|
||||||
|
color: var(--color-text);
|
||||||
|
font-size: 0.8rem;
|
||||||
|
font-family: var(--font-mono, monospace);
|
||||||
|
resize: vertical;
|
||||||
|
box-sizing: border-box;
|
||||||
|
}
|
||||||
|
.json-error { color: var(--color-error); font-size: 0.78rem; }
|
||||||
|
.form-error { color: var(--color-error); font-size: 0.8rem; }
|
||||||
|
.modal-actions { display: flex; justify-content: flex-end; gap: 0.5rem; margin-top: 0.25rem; }
|
||||||
|
.btn-primary {
|
||||||
|
background: var(--app-primary);
|
||||||
|
color: var(--color-text-inverse);
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.4rem 1rem;
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 0.875rem;
|
||||||
|
}
|
||||||
|
.btn-primary:hover { background: var(--app-primary-hover); }
|
||||||
|
.btn-secondary {
|
||||||
|
background: transparent;
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
color: var(--color-text);
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.4rem 0.75rem;
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 0.875rem;
|
||||||
|
}
|
||||||
|
.btn-secondary:hover { background: var(--color-surface-alt); }
|
||||||
|
</style>
|
||||||
|
|
@ -26,10 +26,12 @@ export const routes = [
|
||||||
{ path: '/data/fetch', component: FetchView, meta: { title: 'Fetch' } },
|
{ path: '/data/fetch', component: FetchView, meta: { title: 'Fetch' } },
|
||||||
{ path: '/data/corrections', component: CorrectionsView, meta: { title: 'Corrections' } },
|
{ path: '/data/corrections', component: CorrectionsView, meta: { title: 'Corrections' } },
|
||||||
{ path: '/data/imitate', component: ImitateView, meta: { title: 'Imitate' } },
|
{ path: '/data/imitate', component: ImitateView, meta: { title: 'Imitate' } },
|
||||||
|
{ path: '/data/recipe-scan', component: () => import('../views/RecipeScanView.vue'), meta: { title: 'Recipe Scan' } },
|
||||||
|
|
||||||
// ── Eval domain ──────────────────────────────────────────
|
// ── Eval domain ──────────────────────────────────────────
|
||||||
{ path: '/eval/benchmark', component: BenchmarkView, meta: { title: 'Benchmark' } },
|
{ path: '/eval/benchmark', component: BenchmarkView, meta: { title: 'Benchmark' } },
|
||||||
{ path: '/eval/compare', component: CompareView, meta: { title: 'Compare' } },
|
{ path: '/eval/compare', component: CompareView, meta: { title: 'Compare' } },
|
||||||
|
{ path: '/eval/embed-compare', component: () => import('../views/EmbedCompareView.vue'), meta: { title: 'Embed Compare' } },
|
||||||
|
|
||||||
// ── Train domain ─────────────────────────────────────────
|
// ── Train domain ─────────────────────────────────────────
|
||||||
{ path: '/train/jobs', component: TrainJobsView, meta: { title: 'Training Jobs' } },
|
{ path: '/train/jobs', component: TrainJobsView, meta: { title: 'Training Jobs' } },
|
||||||
|
|
|
||||||
|
|
@ -25,3 +25,65 @@ export interface NodeSummary {
|
||||||
profile_loaded: boolean
|
profile_loaded: boolean
|
||||||
services_catalog: Record<string, ServiceInfo>
|
services_catalog: Record<string, ServiceInfo>
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── Full profile types (for profile editor) ────────────────────────────────────
|
||||||
|
|
||||||
|
export interface ServiceManaged {
|
||||||
|
type: string
|
||||||
|
exec_path?: string
|
||||||
|
args_template?: string
|
||||||
|
port?: number
|
||||||
|
host_port?: number
|
||||||
|
base_port?: number
|
||||||
|
health_path?: string
|
||||||
|
cwd?: string
|
||||||
|
adopt?: boolean
|
||||||
|
[key: string]: unknown
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface CatalogEntryFull {
|
||||||
|
path: string
|
||||||
|
vram_mb: number
|
||||||
|
description?: string
|
||||||
|
multi_gpu?: boolean
|
||||||
|
env?: Record<string, string>
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface ServiceDefinition {
|
||||||
|
max_mb: number
|
||||||
|
priority: number
|
||||||
|
min_compute_cap?: number
|
||||||
|
preferred_compute_cap?: number
|
||||||
|
shared?: boolean
|
||||||
|
max_concurrent?: number
|
||||||
|
idle_stop_after_s?: number
|
||||||
|
always_on?: boolean
|
||||||
|
model_base_path?: string
|
||||||
|
managed?: ServiceManaged
|
||||||
|
catalog?: Record<string, CatalogEntryFull>
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface NodeHardwareGpu {
|
||||||
|
id: number
|
||||||
|
vram_mb: number
|
||||||
|
compute_cap?: number
|
||||||
|
card?: string
|
||||||
|
role?: string
|
||||||
|
services?: string[]
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface NodeHardwareEntry {
|
||||||
|
local_model_root?: string
|
||||||
|
agent_url?: string
|
||||||
|
gpus: NodeHardwareGpu[]
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface FullProfile {
|
||||||
|
schema_version?: number
|
||||||
|
name?: string
|
||||||
|
vram_total_mb?: number
|
||||||
|
eviction_timeout_s?: number
|
||||||
|
services: Record<string, ServiceDefinition>
|
||||||
|
nodes: Record<string, NodeHardwareEntry>
|
||||||
|
model_size_hints?: Record<string, string>
|
||||||
|
}
|
||||||
|
|
|
||||||
987
web/src/views/AssignmentsTab.vue
Normal file
987
web/src/views/AssignmentsTab.vue
Normal file
|
|
@ -0,0 +1,987 @@
|
||||||
|
<template>
|
||||||
|
<div class="assignments-tab">
|
||||||
|
|
||||||
|
<!-- ── Toast ───────────────────────────────────────────── -->
|
||||||
|
<div v-if="toast" class="toast" :class="toast.type" role="status" aria-live="polite">
|
||||||
|
{{ toast.message }}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ── Assignments section ─────────────────────────────── -->
|
||||||
|
<div class="section-header">
|
||||||
|
<h2 class="section-title">Task Assignments</h2>
|
||||||
|
<button class="btn-primary btn-sm" @click="openNewAssignment">+ New Assignment</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="filter-row">
|
||||||
|
<label for="product-filter" class="filter-label">Product</label>
|
||||||
|
<select id="product-filter" v-model="productFilter" class="filter-select">
|
||||||
|
<option value="">All products</option>
|
||||||
|
<option v-for="p in allProducts" :key="p" :value="p">{{ p }}</option>
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-if="assignmentsLoading" class="empty-state">Loading assignments…</div>
|
||||||
|
<div v-else-if="assignmentsError" class="error-notice" role="alert">{{ assignmentsError }}</div>
|
||||||
|
<div v-else-if="filteredGroups.length === 0" class="empty-state">No assignments yet. Add one above.</div>
|
||||||
|
<div v-else class="product-groups">
|
||||||
|
<div v-for="group in filteredGroups" :key="group.product" class="product-group">
|
||||||
|
<h3 class="product-name">{{ group.product.toUpperCase() }}</h3>
|
||||||
|
<div class="assignment-list">
|
||||||
|
<div v-for="a in group.assignments" :key="`${a.product}/${a.task}`" class="assignment-row">
|
||||||
|
<div class="assignment-main">
|
||||||
|
<span class="task-id">{{ a.task }}</span>
|
||||||
|
<span
|
||||||
|
class="model-name"
|
||||||
|
:title="a.model_id"
|
||||||
|
>{{ displayModelId(a) }}</span>
|
||||||
|
<span v-if="a.vram_mb" class="chip chip-vram">{{ formatVram(a.vram_mb) }}</span>
|
||||||
|
<span v-if="a.service_type" class="chip" :class="serviceChipClass(a.service_type)">{{ a.service_type }}</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Node deployment status -->
|
||||||
|
<div v-if="deploymentMap[`${a.product}/${a.task}`]" class="node-statuses">
|
||||||
|
<span
|
||||||
|
v-for="ns in deploymentMap[`${a.product}/${a.task}`]"
|
||||||
|
:key="ns.node_id"
|
||||||
|
class="node-badge-wrap"
|
||||||
|
>
|
||||||
|
<span
|
||||||
|
class="node-badge"
|
||||||
|
:class="ns.status"
|
||||||
|
:title="`${ns.node_id}: ${ns.status}`"
|
||||||
|
>
|
||||||
|
<span class="node-icon">{{ nodeIcon(ns.status) }}</span>
|
||||||
|
{{ ns.node_id }}
|
||||||
|
</span>
|
||||||
|
<button
|
||||||
|
v-if="ns.status === 'absent'"
|
||||||
|
class="btn-deploy"
|
||||||
|
:disabled="deploying.has(`${a.product}/${a.task}/${ns.node_id}`)"
|
||||||
|
:title="`Register ${a.model_id} in ${ns.node_id} catalog`"
|
||||||
|
@click="deployModel(a, ns.node_id)"
|
||||||
|
>{{ deploying.has(`${a.product}/${a.task}/${ns.node_id}`) ? '…' : 'Register' }}</button>
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="assignment-actions">
|
||||||
|
<button
|
||||||
|
v-if="editingKey !== `${a.product}/${a.task}`"
|
||||||
|
class="btn-ghost btn-sm"
|
||||||
|
@click="startEdit(a)"
|
||||||
|
>Edit</button>
|
||||||
|
<button
|
||||||
|
class="btn-ghost btn-sm btn-danger"
|
||||||
|
@click="deleteAssignment(a.product, a.task)"
|
||||||
|
>Delete</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Inline edit form -->
|
||||||
|
<div v-if="editingKey === `${a.product}/${a.task}`" class="inline-edit">
|
||||||
|
<select v-model="editDraft.model_id" class="edit-select" aria-label="Model">
|
||||||
|
<option value="" disabled>Select model…</option>
|
||||||
|
<option v-for="m in registryModels" :key="m.model_id" :value="m.model_id">
|
||||||
|
{{ m.alias || truncate(m.model_id, 40) }}
|
||||||
|
</option>
|
||||||
|
</select>
|
||||||
|
<input
|
||||||
|
v-model="editDraft.description"
|
||||||
|
type="text"
|
||||||
|
class="edit-input"
|
||||||
|
placeholder="Description (optional)"
|
||||||
|
/>
|
||||||
|
<div class="inline-edit-btns">
|
||||||
|
<button class="btn-primary btn-sm" :disabled="!editDraft.model_id" @click="saveEdit(a)">Save</button>
|
||||||
|
<button class="btn-ghost btn-sm" @click="editingKey = null">Cancel</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ── Model Registry section ───────────────────────────── -->
|
||||||
|
<div class="section-header section-header-mt">
|
||||||
|
<h2 class="section-title">Model Registry</h2>
|
||||||
|
<button class="btn-primary btn-sm" @click="showRegisterModal = true">Register Model</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-if="registryLoading" class="empty-state">Loading model registry…</div>
|
||||||
|
<div v-else-if="registryError" class="error-notice" role="alert">{{ registryError }}</div>
|
||||||
|
<div v-else-if="registryModels.length === 0" class="empty-state">No models registered yet.</div>
|
||||||
|
<div v-else class="registry-table-wrap">
|
||||||
|
<table class="registry-table">
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>Alias</th>
|
||||||
|
<th>Model ID</th>
|
||||||
|
<th>VRAM</th>
|
||||||
|
<th>Service</th>
|
||||||
|
<th class="col-hf">HF Repo</th>
|
||||||
|
<th></th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
<tr v-for="m in registryModels" :key="m.model_id">
|
||||||
|
<td>{{ m.alias || '—' }}</td>
|
||||||
|
<td>
|
||||||
|
<span class="truncated" :title="m.model_id">{{ truncate(m.model_id, 36) }}</span>
|
||||||
|
</td>
|
||||||
|
<td>{{ formatVram(m.vram_mb) }}</td>
|
||||||
|
<td><span class="chip" :class="serviceChipClass(m.service_type)">{{ m.service_type }}</span></td>
|
||||||
|
<td class="col-hf">
|
||||||
|
<a
|
||||||
|
v-if="m.hf_repo"
|
||||||
|
:href="`https://huggingface.co/${m.hf_repo}`"
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
class="hf-link"
|
||||||
|
>{{ truncate(m.hf_repo, 30) }}</a>
|
||||||
|
<span v-else class="text-muted">—</span>
|
||||||
|
</td>
|
||||||
|
<td>
|
||||||
|
<button class="btn-ghost btn-sm btn-danger" @click="deleteModel(m.model_id)">Delete</button>
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ── New Assignment modal ─────────────────────────────── -->
|
||||||
|
<div v-if="showNewAssignmentModal" class="modal-backdrop" @click.self="showNewAssignmentModal = false">
|
||||||
|
<div class="modal" role="dialog" aria-modal="true" aria-labelledby="modal-new-assignment-title">
|
||||||
|
<h3 id="modal-new-assignment-title" class="modal-title">New Assignment</h3>
|
||||||
|
<label class="form-label">Product</label>
|
||||||
|
<input
|
||||||
|
v-model="newAssignment.product"
|
||||||
|
list="product-list"
|
||||||
|
class="form-input"
|
||||||
|
placeholder="e.g. peregrine"
|
||||||
|
autocomplete="off"
|
||||||
|
/>
|
||||||
|
<datalist id="product-list">
|
||||||
|
<option v-for="p in allProducts" :key="p" :value="p" />
|
||||||
|
</datalist>
|
||||||
|
|
||||||
|
<label class="form-label">Task ID</label>
|
||||||
|
<input
|
||||||
|
v-model="newAssignment.task"
|
||||||
|
type="text"
|
||||||
|
class="form-input"
|
||||||
|
placeholder="e.g. cover_letter"
|
||||||
|
/>
|
||||||
|
|
||||||
|
<label class="form-label">Model</label>
|
||||||
|
<select v-model="newAssignment.model_id" class="form-select">
|
||||||
|
<option value="" disabled>Select from registry…</option>
|
||||||
|
<option v-for="m in registryModels" :key="m.model_id" :value="m.model_id">
|
||||||
|
{{ m.alias || truncate(m.model_id, 50) }}
|
||||||
|
</option>
|
||||||
|
</select>
|
||||||
|
|
||||||
|
<label class="form-label">Description <span class="optional">(optional)</span></label>
|
||||||
|
<input
|
||||||
|
v-model="newAssignment.description"
|
||||||
|
type="text"
|
||||||
|
class="form-input"
|
||||||
|
placeholder="Human-readable note for operators"
|
||||||
|
/>
|
||||||
|
|
||||||
|
<div class="modal-actions">
|
||||||
|
<button
|
||||||
|
class="btn-primary"
|
||||||
|
:disabled="!newAssignment.product || !newAssignment.task || !newAssignment.model_id || saving"
|
||||||
|
@click="saveNewAssignment"
|
||||||
|
>{{ saving ? 'Saving…' : 'Save' }}</button>
|
||||||
|
<button class="btn-ghost" @click="showNewAssignmentModal = false">Cancel</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ── Register Model modal ─────────────────────────────── -->
|
||||||
|
<div v-if="showRegisterModal" class="modal-backdrop" @click.self="showRegisterModal = false">
|
||||||
|
<div class="modal" role="dialog" aria-modal="true" aria-labelledby="modal-register-title">
|
||||||
|
<h3 id="modal-register-title" class="modal-title">Register Model</h3>
|
||||||
|
|
||||||
|
<label class="form-label">Model ID <span class="hint">(HuggingFace slug, e.g. ibm-granite/granite-4.1-8b)</span></label>
|
||||||
|
<input v-model="newModel.model_id" type="text" class="form-input" placeholder="org/model-name" />
|
||||||
|
|
||||||
|
<label class="form-label">Alias <span class="optional">(optional, short name for assignments)</span></label>
|
||||||
|
<input v-model="newModel.alias" type="text" class="form-input" placeholder="e.g. granite-8b" />
|
||||||
|
|
||||||
|
<label class="form-label">Service type</label>
|
||||||
|
<select v-model="newModel.service_type" class="form-select">
|
||||||
|
<option value="" disabled>Select service…</option>
|
||||||
|
<option value="cf-text">cf-text — Language Models</option>
|
||||||
|
<option value="cf-stt">cf-stt — Speech Recognition</option>
|
||||||
|
<option value="cf-tts">cf-tts — Text to Speech</option>
|
||||||
|
<option value="cf-vision">cf-vision — Vision / VLM</option>
|
||||||
|
<option value="cf-image">cf-image — Image Generation</option>
|
||||||
|
<option value="cf-voice">cf-voice — Audio Classification</option>
|
||||||
|
<option value="vllm">vllm — vLLM inference</option>
|
||||||
|
<option value="ollama">ollama — Ollama inference</option>
|
||||||
|
</select>
|
||||||
|
|
||||||
|
<label class="form-label">VRAM required (MB)</label>
|
||||||
|
<input v-model.number="newModel.vram_mb" type="number" min="0" class="form-input" placeholder="e.g. 16384" />
|
||||||
|
|
||||||
|
<label class="form-label">HF Repo <span class="optional">(optional)</span></label>
|
||||||
|
<input v-model="newModel.hf_repo" type="text" class="form-input" placeholder="org/repo-name" />
|
||||||
|
|
||||||
|
<label class="form-label">Description <span class="optional">(optional)</span></label>
|
||||||
|
<input v-model="newModel.description" type="text" class="form-input" placeholder="Human-readable note" />
|
||||||
|
|
||||||
|
<div class="modal-actions">
|
||||||
|
<button
|
||||||
|
class="btn-primary"
|
||||||
|
:disabled="!newModel.model_id || !newModel.service_type || !newModel.vram_mb || saving"
|
||||||
|
@click="saveNewModel"
|
||||||
|
>{{ saving ? 'Saving…' : 'Register' }}</button>
|
||||||
|
<button class="btn-ghost" @click="showRegisterModal = false">Cancel</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<script setup lang="ts">
|
||||||
|
import { ref, computed, onMounted } from 'vue'
|
||||||
|
|
||||||
|
// ── Types ──────────────────────────────────────────────
|
||||||
|
|
||||||
|
interface AssignmentNode {
|
||||||
|
node_id: string
|
||||||
|
status: 'present' | 'absent' | 'vram_tight'
|
||||||
|
}
|
||||||
|
|
||||||
|
interface DeployingKey {
|
||||||
|
nodeId: string
|
||||||
|
assignmentKey: string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface Assignment {
|
||||||
|
product: string
|
||||||
|
task: string
|
||||||
|
model_id: string
|
||||||
|
description: string
|
||||||
|
alias?: string
|
||||||
|
service_type?: string
|
||||||
|
vram_mb?: number
|
||||||
|
nodes?: AssignmentNode[]
|
||||||
|
}
|
||||||
|
|
||||||
|
interface RegistryModel {
|
||||||
|
model_id: string
|
||||||
|
alias: string
|
||||||
|
service_type: string
|
||||||
|
vram_mb: number
|
||||||
|
hf_repo: string
|
||||||
|
description: string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ProductGroup {
|
||||||
|
product: string
|
||||||
|
assignments: Assignment[]
|
||||||
|
}
|
||||||
|
|
||||||
|
interface Toast {
|
||||||
|
message: string
|
||||||
|
type: 'success' | 'error'
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── State ──────────────────────────────────────────────
|
||||||
|
|
||||||
|
const assignments = ref<Assignment[]>([])
|
||||||
|
const assignmentsLoading = ref(false)
|
||||||
|
const assignmentsError = ref<string | null>(null)
|
||||||
|
|
||||||
|
const registryModels = ref<RegistryModel[]>([])
|
||||||
|
const registryLoading = ref(false)
|
||||||
|
const registryError = ref<string | null>(null)
|
||||||
|
|
||||||
|
const productFilter = ref('')
|
||||||
|
const editingKey = ref<string | null>(null)
|
||||||
|
const editDraft = ref({ model_id: '', description: '' })
|
||||||
|
|
||||||
|
const showNewAssignmentModal = ref(false)
|
||||||
|
const newAssignment = ref({ product: '', task: '', model_id: '', description: '' })
|
||||||
|
|
||||||
|
const showRegisterModal = ref(false)
|
||||||
|
const newModel = ref({ model_id: '', alias: '', service_type: '', vram_mb: 0, hf_repo: '', description: '' })
|
||||||
|
|
||||||
|
const saving = ref(false)
|
||||||
|
const toast = ref<Toast | null>(null)
|
||||||
|
let toastTimer: ReturnType<typeof setTimeout> | null = null
|
||||||
|
|
||||||
|
const deploying = ref<Set<string>>(new Set())
|
||||||
|
|
||||||
|
// ── Derived ────────────────────────────────────────────
|
||||||
|
|
||||||
|
const allProducts = computed(() => {
|
||||||
|
const seen = new Set<string>()
|
||||||
|
for (const a of assignments.value) seen.add(a.product)
|
||||||
|
return [...seen].sort()
|
||||||
|
})
|
||||||
|
|
||||||
|
const deploymentMap = computed(() => {
|
||||||
|
const map: Record<string, AssignmentNode[]> = {}
|
||||||
|
for (const a of assignments.value) {
|
||||||
|
if (a.nodes) map[`${a.product}/${a.task}`] = a.nodes
|
||||||
|
}
|
||||||
|
return map
|
||||||
|
})
|
||||||
|
|
||||||
|
const filteredGroups = computed((): ProductGroup[] => {
|
||||||
|
const filtered = productFilter.value
|
||||||
|
? assignments.value.filter(a => a.product === productFilter.value)
|
||||||
|
: assignments.value
|
||||||
|
|
||||||
|
const byProduct: Record<string, Assignment[]> = {}
|
||||||
|
for (const a of filtered) {
|
||||||
|
if (!byProduct[a.product]) byProduct[a.product] = []
|
||||||
|
byProduct[a.product].push(a)
|
||||||
|
}
|
||||||
|
return Object.keys(byProduct)
|
||||||
|
.sort()
|
||||||
|
.map(product => ({ product, assignments: byProduct[product] }))
|
||||||
|
})
|
||||||
|
|
||||||
|
// ── Helpers ────────────────────────────────────────────
|
||||||
|
|
||||||
|
function truncate(s: string, max: number): string {
|
||||||
|
return s.length > max ? s.slice(0, max - 1) + '…' : s
|
||||||
|
}
|
||||||
|
|
||||||
|
function displayModelId(a: Assignment): string {
|
||||||
|
if (a.alias) return a.alias
|
||||||
|
const id = a.model_id
|
||||||
|
// Show only the model name part (after /) and truncate long slugs
|
||||||
|
const short = id.includes('/') ? id.split('/').slice(1).join('/') : id
|
||||||
|
return truncate(short, 36)
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatVram(mb: number | undefined): string {
|
||||||
|
if (!mb) return ''
|
||||||
|
if (mb >= 1024) return `${(mb / 1024).toFixed(1)} GB`
|
||||||
|
return `${mb} MB`
|
||||||
|
}
|
||||||
|
|
||||||
|
function serviceChipClass(service: string): string {
|
||||||
|
return `chip-service-${service.replace(/[^a-z0-9]/g, '-')}`
|
||||||
|
}
|
||||||
|
|
||||||
|
function nodeIcon(status: string): string {
|
||||||
|
if (status === 'present') return '✓'
|
||||||
|
if (status === 'vram_tight') return '~'
|
||||||
|
return '✗'
|
||||||
|
}
|
||||||
|
|
||||||
|
function showToast(message: string, type: 'success' | 'error' = 'success') {
|
||||||
|
if (toastTimer) clearTimeout(toastTimer)
|
||||||
|
toast.value = { message, type }
|
||||||
|
toastTimer = setTimeout(() => { toast.value = null }, 3500)
|
||||||
|
}
|
||||||
|
|
||||||
|
function openNewAssignment() {
|
||||||
|
newAssignment.value = { product: '', task: '', model_id: '', description: '' }
|
||||||
|
showNewAssignmentModal.value = true
|
||||||
|
}
|
||||||
|
|
||||||
|
function startEdit(a: Assignment) {
|
||||||
|
editingKey.value = `${a.product}/${a.task}`
|
||||||
|
editDraft.value = { model_id: a.model_id, description: a.description }
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── API ────────────────────────────────────────────────
|
||||||
|
|
||||||
|
async function loadAssignments() {
|
||||||
|
assignmentsLoading.value = true
|
||||||
|
assignmentsError.value = null
|
||||||
|
try {
|
||||||
|
// Fetch both list and deployment status in parallel
|
||||||
|
const [listRes, statusRes] = await Promise.all([
|
||||||
|
fetch('/api/cforch/assignments'),
|
||||||
|
fetch('/api/cforch/assignments/deployment-status'),
|
||||||
|
])
|
||||||
|
if (!listRes.ok) throw new Error(`HTTP ${listRes.status}`)
|
||||||
|
const list: Assignment[] = (await listRes.json()).assignments ?? []
|
||||||
|
|
||||||
|
// Merge deployment status into assignments if available
|
||||||
|
if (statusRes.ok) {
|
||||||
|
const statusList: Assignment[] = (await statusRes.json()).deployment_status ?? []
|
||||||
|
const statusMap: Record<string, AssignmentNode[]> = {}
|
||||||
|
for (const s of statusList) {
|
||||||
|
statusMap[`${s.product}/${s.task}`] = s.nodes ?? []
|
||||||
|
}
|
||||||
|
for (const a of list) {
|
||||||
|
a.nodes = statusMap[`${a.product}/${a.task}`] ?? []
|
||||||
|
// Enrich with service_type/vram_mb from status payload
|
||||||
|
const s = statusList.find(x => x.product === a.product && x.task === a.task)
|
||||||
|
if (s) {
|
||||||
|
a.service_type = s.service_type
|
||||||
|
a.vram_mb = s.vram_mb
|
||||||
|
a.alias = s.alias
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assignments.value = list
|
||||||
|
} catch (e) {
|
||||||
|
assignmentsError.value = `Could not load assignments: ${e}`
|
||||||
|
} finally {
|
||||||
|
assignmentsLoading.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadRegistry() {
|
||||||
|
registryLoading.value = true
|
||||||
|
registryError.value = null
|
||||||
|
try {
|
||||||
|
const res = await fetch('/api/cforch/model-registry')
|
||||||
|
if (!res.ok) throw new Error(`HTTP ${res.status}`)
|
||||||
|
registryModels.value = (await res.json()).models ?? []
|
||||||
|
} catch (e) {
|
||||||
|
registryError.value = `Could not load model registry: ${e}`
|
||||||
|
} finally {
|
||||||
|
registryLoading.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function saveNewAssignment() {
|
||||||
|
saving.value = true
|
||||||
|
try {
|
||||||
|
const res = await fetch('/api/cforch/assignments', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify(newAssignment.value),
|
||||||
|
})
|
||||||
|
if (!res.ok) throw new Error(await res.text())
|
||||||
|
showNewAssignmentModal.value = false
|
||||||
|
showToast('Assignment saved')
|
||||||
|
await loadAssignments()
|
||||||
|
} catch (e) {
|
||||||
|
showToast(`Save failed: ${e}`, 'error')
|
||||||
|
} finally {
|
||||||
|
saving.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function saveEdit(a: Assignment) {
|
||||||
|
saving.value = true
|
||||||
|
try {
|
||||||
|
const res = await fetch('/api/cforch/assignments', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({
|
||||||
|
product: a.product,
|
||||||
|
task: a.task,
|
||||||
|
model_id: editDraft.value.model_id,
|
||||||
|
description: editDraft.value.description,
|
||||||
|
}),
|
||||||
|
})
|
||||||
|
if (!res.ok) throw new Error(await res.text())
|
||||||
|
editingKey.value = null
|
||||||
|
showToast('Assignment updated')
|
||||||
|
await loadAssignments()
|
||||||
|
} catch (e) {
|
||||||
|
showToast(`Update failed: ${e}`, 'error')
|
||||||
|
} finally {
|
||||||
|
saving.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function deleteAssignment(product: string, task: string) {
|
||||||
|
if (!confirm(`Delete assignment ${product}.${task}?`)) return
|
||||||
|
try {
|
||||||
|
const res = await fetch(
|
||||||
|
`/api/cforch/assignments/${encodeURIComponent(product)}/${encodeURIComponent(task)}`,
|
||||||
|
{ method: 'DELETE' },
|
||||||
|
)
|
||||||
|
if (!res.ok) throw new Error(await res.text())
|
||||||
|
showToast('Assignment deleted')
|
||||||
|
await loadAssignments()
|
||||||
|
} catch (e) {
|
||||||
|
showToast(`Delete failed: ${e}`, 'error')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function saveNewModel() {
|
||||||
|
saving.value = true
|
||||||
|
try {
|
||||||
|
const res = await fetch('/api/cforch/model-registry', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify(newModel.value),
|
||||||
|
})
|
||||||
|
if (!res.ok) throw new Error(await res.text())
|
||||||
|
showRegisterModal.value = false
|
||||||
|
showToast('Model registered')
|
||||||
|
await loadRegistry()
|
||||||
|
} catch (e) {
|
||||||
|
showToast(`Register failed: ${e}`, 'error')
|
||||||
|
} finally {
|
||||||
|
saving.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function deleteModel(model_id: string) {
|
||||||
|
if (!confirm(`Remove ${model_id} from the registry?`)) return
|
||||||
|
try {
|
||||||
|
const res = await fetch(
|
||||||
|
`/api/cforch/model-registry/${encodeURIComponent(model_id)}`,
|
||||||
|
{ method: 'DELETE' },
|
||||||
|
)
|
||||||
|
if (!res.ok) throw new Error(await res.text())
|
||||||
|
showToast('Model removed')
|
||||||
|
await loadRegistry()
|
||||||
|
} catch (e) {
|
||||||
|
showToast(`Delete failed: ${e}`, 'error')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function deployModel(a: Assignment, nodeId: string) {
|
||||||
|
const key = `${a.product}/${a.task}/${nodeId}`
|
||||||
|
if (deploying.value.has(key)) return
|
||||||
|
|
||||||
|
// Look up hf_repo from registry for cleaner path construction
|
||||||
|
const regEntry = registryModels.value.find(m => m.model_id === a.model_id)
|
||||||
|
const hf_repo = regEntry?.hf_repo ?? ''
|
||||||
|
const service_type = a.service_type ?? regEntry?.service_type ?? ''
|
||||||
|
const vram_mb = a.vram_mb ?? regEntry?.vram_mb ?? 0
|
||||||
|
const description = regEntry?.alias ? `${regEntry.alias} (via assignments)` : ''
|
||||||
|
|
||||||
|
if (!service_type) {
|
||||||
|
showToast(`No service type for model ${a.model_id}`, 'error')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
deploying.value = new Set([...deploying.value, key])
|
||||||
|
try {
|
||||||
|
const res = await fetch(`/api/nodes-mgmt/nodes/${encodeURIComponent(nodeId)}/models/deploy`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ model_id: a.model_id, service_type, vram_mb, hf_repo, description }),
|
||||||
|
})
|
||||||
|
if (!res.ok) throw new Error(await res.text())
|
||||||
|
const data = await res.json()
|
||||||
|
showToast(`Registered ${a.model_id} on ${nodeId} at ${data.path}`)
|
||||||
|
|
||||||
|
// Optimistic update: flip node to 'present' immediately so the Register button
|
||||||
|
// disappears before the coordinator reload confirms. loadAssignments() reconciles
|
||||||
|
// with real server state on the next round-trip.
|
||||||
|
assignments.value = assignments.value.map(asgn => {
|
||||||
|
if (asgn.product !== a.product || asgn.task !== a.task) return asgn
|
||||||
|
return {
|
||||||
|
...asgn,
|
||||||
|
nodes: (asgn.nodes ?? []).map(ns =>
|
||||||
|
ns.node_id === nodeId ? { ...ns, status: 'present' as const } : ns
|
||||||
|
),
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
await loadAssignments()
|
||||||
|
} catch (e) {
|
||||||
|
showToast(`Deploy failed: ${e}`, 'error')
|
||||||
|
} finally {
|
||||||
|
deploying.value = new Set([...deploying.value].filter(k => k !== key))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
onMounted(() => {
|
||||||
|
loadAssignments()
|
||||||
|
loadRegistry()
|
||||||
|
})
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<style scoped>
|
||||||
|
.assignments-tab {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 1.25rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Toast ── */
|
||||||
|
.toast {
|
||||||
|
position: fixed;
|
||||||
|
bottom: 1.5rem;
|
||||||
|
right: 1.5rem;
|
||||||
|
padding: 0.65rem 1.1rem;
|
||||||
|
border-radius: 0.5rem;
|
||||||
|
font-size: 0.88rem;
|
||||||
|
font-weight: 500;
|
||||||
|
z-index: 200;
|
||||||
|
box-shadow: 0 2px 8px rgba(0,0,0,0.15);
|
||||||
|
}
|
||||||
|
.toast.success {
|
||||||
|
background: var(--color-success, #2a8050);
|
||||||
|
color: #fff;
|
||||||
|
}
|
||||||
|
.toast.error {
|
||||||
|
background: var(--color-danger, #b03030);
|
||||||
|
color: #fff;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Section headers ── */
|
||||||
|
.section-header {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 1rem;
|
||||||
|
}
|
||||||
|
.section-header-mt {
|
||||||
|
margin-top: 1.5rem;
|
||||||
|
}
|
||||||
|
.section-title {
|
||||||
|
font-size: 1rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: var(--app-primary, #2A6080);
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Filter row ── */
|
||||||
|
.filter-row {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.6rem;
|
||||||
|
}
|
||||||
|
.filter-label {
|
||||||
|
font-size: 0.85rem;
|
||||||
|
color: var(--color-text-muted, #6b7a99);
|
||||||
|
}
|
||||||
|
.filter-select {
|
||||||
|
padding: 0.3rem 0.6rem;
|
||||||
|
font-size: 0.85rem;
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
border-radius: 0.4rem;
|
||||||
|
background: var(--color-surface, #fff);
|
||||||
|
color: var(--color-text, #1a2030);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Product groups ── */
|
||||||
|
.product-groups {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 1rem;
|
||||||
|
}
|
||||||
|
.product-group {}
|
||||||
|
.product-name {
|
||||||
|
font-size: 0.75rem;
|
||||||
|
font-weight: 700;
|
||||||
|
letter-spacing: 0.08em;
|
||||||
|
color: var(--color-text-muted, #6b7a99);
|
||||||
|
text-transform: uppercase;
|
||||||
|
margin: 0 0 0.4rem;
|
||||||
|
}
|
||||||
|
.assignment-list {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.4rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Assignment rows ── */
|
||||||
|
.assignment-row {
|
||||||
|
background: var(--color-surface-raised, #f0f4fa);
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
border-radius: 0.5rem;
|
||||||
|
padding: 0.65rem 0.85rem;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.4rem;
|
||||||
|
}
|
||||||
|
.assignment-main {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
.task-id {
|
||||||
|
font-family: var(--font-mono, monospace);
|
||||||
|
font-size: 0.88rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: var(--color-text, #1a2030);
|
||||||
|
min-width: 0;
|
||||||
|
}
|
||||||
|
.model-name {
|
||||||
|
font-size: 0.85rem;
|
||||||
|
color: var(--color-text-muted, #6b7a99);
|
||||||
|
white-space: nowrap;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
max-width: 280px;
|
||||||
|
cursor: default;
|
||||||
|
}
|
||||||
|
.assignment-actions {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.4rem;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Node status badges ── */
|
||||||
|
.node-statuses {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.35rem;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
.node-badge-wrap {
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.2rem;
|
||||||
|
}
|
||||||
|
.node-badge {
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.2rem;
|
||||||
|
font-size: 0.78rem;
|
||||||
|
padding: 0.15rem 0.5rem;
|
||||||
|
border-radius: 0.35rem;
|
||||||
|
font-weight: 500;
|
||||||
|
}
|
||||||
|
.node-badge.present {
|
||||||
|
background: color-mix(in srgb, var(--color-success, #2a8050) 15%, transparent);
|
||||||
|
color: var(--color-success, #2a8050);
|
||||||
|
border: 1px solid color-mix(in srgb, var(--color-success, #2a8050) 30%, transparent);
|
||||||
|
}
|
||||||
|
.node-badge.absent {
|
||||||
|
background: color-mix(in srgb, var(--color-danger, #b03030) 12%, transparent);
|
||||||
|
color: var(--color-danger, #b03030);
|
||||||
|
border: 1px solid color-mix(in srgb, var(--color-danger, #b03030) 25%, transparent);
|
||||||
|
}
|
||||||
|
.node-badge.vram_tight {
|
||||||
|
background: color-mix(in srgb, #c08030 15%, transparent);
|
||||||
|
color: #8a5500;
|
||||||
|
border: 1px solid color-mix(in srgb, #c08030 30%, transparent);
|
||||||
|
}
|
||||||
|
.node-icon {
|
||||||
|
font-size: 0.85em;
|
||||||
|
}
|
||||||
|
.btn-deploy {
|
||||||
|
padding: 0.1rem 0.4rem;
|
||||||
|
font-size: 0.72rem;
|
||||||
|
font-weight: 600;
|
||||||
|
background: color-mix(in srgb, var(--app-primary, #2A6080) 12%, transparent);
|
||||||
|
color: var(--app-primary, #2A6080);
|
||||||
|
border: 1px solid color-mix(in srgb, var(--app-primary, #2A6080) 30%, transparent);
|
||||||
|
border-radius: 0.3rem;
|
||||||
|
cursor: pointer;
|
||||||
|
white-space: nowrap;
|
||||||
|
transition: background 0.15s;
|
||||||
|
}
|
||||||
|
.btn-deploy:hover:not(:disabled) {
|
||||||
|
background: color-mix(in srgb, var(--app-primary, #2A6080) 22%, transparent);
|
||||||
|
}
|
||||||
|
.btn-deploy:disabled { opacity: 0.5; cursor: default; }
|
||||||
|
|
||||||
|
/* ── Inline edit ── */
|
||||||
|
.inline-edit {
|
||||||
|
display: flex;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
gap: 0.4rem;
|
||||||
|
padding-top: 0.35rem;
|
||||||
|
border-top: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
}
|
||||||
|
.edit-select,
|
||||||
|
.edit-input {
|
||||||
|
flex: 1;
|
||||||
|
min-width: 160px;
|
||||||
|
padding: 0.35rem 0.55rem;
|
||||||
|
font-size: 0.85rem;
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
border-radius: 0.4rem;
|
||||||
|
background: var(--color-surface, #fff);
|
||||||
|
color: var(--color-text, #1a2030);
|
||||||
|
}
|
||||||
|
.inline-edit-btns {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.35rem;
|
||||||
|
align-items: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Registry table ── */
|
||||||
|
.registry-table-wrap {
|
||||||
|
overflow-x: auto;
|
||||||
|
border-radius: 0.5rem;
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
}
|
||||||
|
.registry-table {
|
||||||
|
width: 100%;
|
||||||
|
border-collapse: collapse;
|
||||||
|
font-size: 0.85rem;
|
||||||
|
}
|
||||||
|
.registry-table th {
|
||||||
|
text-align: left;
|
||||||
|
padding: 0.5rem 0.75rem;
|
||||||
|
font-size: 0.78rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: var(--color-text-muted, #6b7a99);
|
||||||
|
background: var(--color-surface-raised, #f0f4fa);
|
||||||
|
border-bottom: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
.registry-table td {
|
||||||
|
padding: 0.5rem 0.75rem;
|
||||||
|
border-bottom: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
vertical-align: middle;
|
||||||
|
}
|
||||||
|
.registry-table tbody tr:last-child td {
|
||||||
|
border-bottom: none;
|
||||||
|
}
|
||||||
|
.truncated {
|
||||||
|
display: inline-block;
|
||||||
|
max-width: 220px;
|
||||||
|
white-space: nowrap;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
vertical-align: bottom;
|
||||||
|
cursor: default;
|
||||||
|
}
|
||||||
|
.hf-link {
|
||||||
|
color: var(--app-primary, #2A6080);
|
||||||
|
text-decoration: none;
|
||||||
|
font-size: 0.82rem;
|
||||||
|
}
|
||||||
|
.hf-link:hover { text-decoration: underline; }
|
||||||
|
.text-muted { color: var(--color-text-muted, #6b7a99); }
|
||||||
|
|
||||||
|
/* ── Chips ── */
|
||||||
|
.chip {
|
||||||
|
display: inline-block;
|
||||||
|
padding: 0.15rem 0.5rem;
|
||||||
|
border-radius: 0.35rem;
|
||||||
|
font-size: 0.75rem;
|
||||||
|
font-weight: 600;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
.chip-vram {
|
||||||
|
background: color-mix(in srgb, var(--app-primary, #2A6080) 12%, transparent);
|
||||||
|
color: var(--app-primary, #2A6080);
|
||||||
|
border: 1px solid color-mix(in srgb, var(--app-primary, #2A6080) 25%, transparent);
|
||||||
|
}
|
||||||
|
/* service chips — match ModelsView convention */
|
||||||
|
.chip-service-cf-text { background: #e8f0fe; color: #1a5276; border: 1px solid #a9c4e8; }
|
||||||
|
.chip-service-cf-stt { background: #eaf6ea; color: #1e6b3a; border: 1px solid #a2d9b1; }
|
||||||
|
.chip-service-cf-tts { background: #fdf3e3; color: #7d4e00; border: 1px solid #e8c98a; }
|
||||||
|
.chip-service-cf-vision { background: #f3e8fd; color: #5b2d8e; border: 1px solid #c8a0e8; }
|
||||||
|
.chip-service-cf-image { background: #fce8f0; color: #8e1a4f; border: 1px solid #e8a0c0; }
|
||||||
|
.chip-service-cf-voice { background: #e8f8fc; color: #0a5c6e; border: 1px solid #88d0e0; }
|
||||||
|
.chip-service-vllm { background: #f5ece0; color: #7a3800; border: 1px solid #d4a87a; }
|
||||||
|
.chip-service-ollama { background: #eeeeee; color: #444; border: 1px solid #ccc; }
|
||||||
|
|
||||||
|
/* ── Buttons ── */
|
||||||
|
.btn-primary {
|
||||||
|
padding: 0.45rem 1rem;
|
||||||
|
background: var(--app-primary, #2A6080);
|
||||||
|
color: #fff;
|
||||||
|
border: none;
|
||||||
|
border-radius: 0.4rem;
|
||||||
|
font-size: 0.85rem;
|
||||||
|
font-weight: 600;
|
||||||
|
cursor: pointer;
|
||||||
|
transition: opacity 0.15s;
|
||||||
|
}
|
||||||
|
.btn-primary:disabled { opacity: 0.5; cursor: default; }
|
||||||
|
.btn-primary:not(:disabled):hover { opacity: 0.88; }
|
||||||
|
|
||||||
|
.btn-ghost {
|
||||||
|
padding: 0.35rem 0.75rem;
|
||||||
|
background: transparent;
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
border-radius: 0.4rem;
|
||||||
|
font-size: 0.82rem;
|
||||||
|
color: var(--color-text-muted, #6b7a99);
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background 0.15s;
|
||||||
|
}
|
||||||
|
.btn-ghost:hover { background: var(--color-surface-raised, #e4ebf5); }
|
||||||
|
.btn-ghost.btn-danger { color: var(--color-danger, #b03030); border-color: color-mix(in srgb, var(--color-danger, #b03030) 30%, transparent); }
|
||||||
|
.btn-ghost.btn-danger:hover { background: color-mix(in srgb, var(--color-danger, #b03030) 10%, transparent); }
|
||||||
|
|
||||||
|
.btn-sm { padding: 0.3rem 0.65rem; font-size: 0.8rem; }
|
||||||
|
|
||||||
|
/* ── Empty / error states ── */
|
||||||
|
.empty-state {
|
||||||
|
padding: 1.5rem;
|
||||||
|
text-align: center;
|
||||||
|
color: var(--color-text-muted, #6b7a99);
|
||||||
|
font-size: 0.9rem;
|
||||||
|
background: var(--color-surface-raised, #f0f4fa);
|
||||||
|
border: 1px dashed var(--color-border, #d0d7e8);
|
||||||
|
border-radius: 0.5rem;
|
||||||
|
}
|
||||||
|
.error-notice {
|
||||||
|
padding: 0.75rem 1rem;
|
||||||
|
background: color-mix(in srgb, var(--color-danger, #b03030) 10%, transparent);
|
||||||
|
color: var(--color-danger, #b03030);
|
||||||
|
border: 1px solid color-mix(in srgb, var(--color-danger, #b03030) 25%, transparent);
|
||||||
|
border-radius: 0.4rem;
|
||||||
|
font-size: 0.87rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Modal ── */
|
||||||
|
.modal-backdrop {
|
||||||
|
position: fixed;
|
||||||
|
inset: 0;
|
||||||
|
background: rgba(0,0,0,0.35);
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
z-index: 100;
|
||||||
|
padding: 1rem;
|
||||||
|
}
|
||||||
|
.modal {
|
||||||
|
background: var(--color-surface, #fff);
|
||||||
|
border-radius: 0.65rem;
|
||||||
|
padding: 1.5rem;
|
||||||
|
width: 100%;
|
||||||
|
max-width: 480px;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.65rem;
|
||||||
|
box-shadow: 0 8px 32px rgba(0,0,0,0.18);
|
||||||
|
max-height: 90vh;
|
||||||
|
overflow-y: auto;
|
||||||
|
}
|
||||||
|
.modal-title {
|
||||||
|
font-size: 1rem;
|
||||||
|
font-weight: 700;
|
||||||
|
color: var(--app-primary, #2A6080);
|
||||||
|
margin: 0 0 0.25rem;
|
||||||
|
}
|
||||||
|
.form-label {
|
||||||
|
font-size: 0.82rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: var(--color-text-muted, #6b7a99);
|
||||||
|
}
|
||||||
|
.form-input,
|
||||||
|
.form-select {
|
||||||
|
padding: 0.4rem 0.65rem;
|
||||||
|
font-size: 0.88rem;
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
border-radius: 0.4rem;
|
||||||
|
background: var(--color-surface, #fff);
|
||||||
|
color: var(--color-text, #1a2030);
|
||||||
|
width: 100%;
|
||||||
|
box-sizing: border-box;
|
||||||
|
}
|
||||||
|
.form-input:focus, .form-select:focus {
|
||||||
|
outline: 2px solid var(--app-primary, #2A6080);
|
||||||
|
outline-offset: 1px;
|
||||||
|
}
|
||||||
|
.modal-actions {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.5rem;
|
||||||
|
justify-content: flex-end;
|
||||||
|
margin-top: 0.25rem;
|
||||||
|
}
|
||||||
|
.optional, .hint {
|
||||||
|
font-weight: 400;
|
||||||
|
color: var(--color-text-muted, #6b7a99);
|
||||||
|
font-size: 0.78rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Responsive ── */
|
||||||
|
@media (max-width: 600px) {
|
||||||
|
.assignment-main { flex-direction: column; align-items: flex-start; }
|
||||||
|
.col-hf { display: none; }
|
||||||
|
.model-name { max-width: 100%; }
|
||||||
|
.modal { padding: 1rem; }
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
|
@ -325,7 +325,7 @@ function toggleCategory(models: AvailableModel[], checked: boolean) {
|
||||||
|
|
||||||
async function loadModelCategories() {
|
async function loadModelCategories() {
|
||||||
modelsLoading.value = true
|
modelsLoading.value = true
|
||||||
const { data } = await useApiFetch<ModelCategoriesResponse>('/api/benchmark/models')
|
const { data } = await useApiFetch<ModelCategoriesResponse>('/api/cforch/models')
|
||||||
modelsLoading.value = false
|
modelsLoading.value = false
|
||||||
if (data?.categories) {
|
if (data?.categories) {
|
||||||
modelCategories.value = data.categories
|
modelCategories.value = data.categories
|
||||||
|
|
@ -342,7 +342,7 @@ const modelCount = computed(() => modelNames.value.length)
|
||||||
const labelNames = computed(() => {
|
const labelNames = computed(() => {
|
||||||
const canonical = Object.keys(LABEL_META)
|
const canonical = Object.keys(LABEL_META)
|
||||||
const inResults = new Set(
|
const inResults = new Set(
|
||||||
modelNames.value.flatMap(n => Object.keys(results.value!.models[n].per_label))
|
modelNames.value.flatMap(n => Object.keys(results.value?.models[n]?.per_label ?? {}))
|
||||||
)
|
)
|
||||||
return [...canonical.filter(l => inResults.has(l)), ...[...inResults].filter(l => !canonical.includes(l))]
|
return [...canonical.filter(l => inResults.has(l)), ...[...inResults].filter(l => !canonical.includes(l))]
|
||||||
})
|
})
|
||||||
|
|
@ -401,16 +401,16 @@ function formatDate(iso: string | null): string {
|
||||||
// ── Data loading ─────────────────────────────────────────────────────────────
|
// ── Data loading ─────────────────────────────────────────────────────────────
|
||||||
async function loadResults() {
|
async function loadResults() {
|
||||||
loading.value = true
|
loading.value = true
|
||||||
const { data } = await useApiFetch<BenchResults>('/api/benchmark/results')
|
const { data } = await useApiFetch<BenchResults>('/api/cforch/results')
|
||||||
loading.value = false
|
loading.value = false
|
||||||
if (data && Object.keys(data.models).length > 0) {
|
if (data?.models && Object.keys(data.models).length > 0) {
|
||||||
results.value = data
|
results.value = data
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async function loadFineTunedModels() {
|
async function loadFineTunedModels() {
|
||||||
const { data } = await useApiFetch<FineTunedModel[]>('/api/finetune/status')
|
const { data } = await useApiFetch<{ results: FineTunedModel[] }>('/api/train/results')
|
||||||
if (Array.isArray(data)) fineTunedModels.value = data
|
if (Array.isArray(data?.results)) fineTunedModels.value = data.results
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Benchmark run ────────────────────────────────────────────────────────────
|
// ── Benchmark run ────────────────────────────────────────────────────────────
|
||||||
|
|
@ -428,7 +428,7 @@ function startBenchmark() {
|
||||||
params.set('model_names', [...selectedModels.value].join(','))
|
params.set('model_names', [...selectedModels.value].join(','))
|
||||||
}
|
}
|
||||||
const qs = params.toString()
|
const qs = params.toString()
|
||||||
const url = `/api/benchmark/run${qs ? `?${qs}` : ''}`
|
const url = `/api/cforch/run${qs ? `?${qs}` : ''}`
|
||||||
useApiSSE(
|
useApiSSE(
|
||||||
url,
|
url,
|
||||||
async (event) => {
|
async (event) => {
|
||||||
|
|
@ -457,7 +457,7 @@ function startBenchmark() {
|
||||||
}
|
}
|
||||||
|
|
||||||
async function cancelBenchmark() {
|
async function cancelBenchmark() {
|
||||||
await fetch('/api/benchmark/cancel', { method: 'POST' }).catch(() => {})
|
await fetch('/api/cforch/cancel', { method: 'POST' }).catch(() => {})
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Fine-tune ─────────────────────────────────────────────────────────────────
|
// ── Fine-tune ─────────────────────────────────────────────────────────────────
|
||||||
|
|
|
||||||
|
|
@ -71,32 +71,35 @@
|
||||||
rows="6"
|
rows="6"
|
||||||
/>
|
/>
|
||||||
|
|
||||||
<!-- Ollama model picker -->
|
<!-- LLM model picker (ollama + vllm + cf-text) -->
|
||||||
<details class="model-picker" open>
|
<details class="model-picker" open>
|
||||||
<summary class="picker-summary">
|
<summary class="picker-summary">
|
||||||
<span class="picker-title">🤖 Ollama Models</span>
|
<span class="picker-title">🤖 LLM Models</span>
|
||||||
<span class="picker-badge">{{ cmpSelectedModels.size }} / {{ ollamaLlmModels.length }}</span>
|
<span class="picker-badge">{{ cmpSelectedModels.size }} / {{ llmSelectableModels.length }}</span>
|
||||||
</summary>
|
</summary>
|
||||||
<div class="picker-body">
|
<div class="picker-body">
|
||||||
<label class="picker-cat-header">
|
<label class="picker-cat-header">
|
||||||
<input
|
<input
|
||||||
type="checkbox"
|
type="checkbox"
|
||||||
:checked="cmpSelectedModels.size === ollamaLlmModels.length"
|
:checked="cmpSelectedModels.size === llmSelectableModels.length"
|
||||||
:indeterminate="cmpSelectedModels.size > 0 && cmpSelectedModels.size < ollamaLlmModels.length"
|
:indeterminate="cmpSelectedModels.size > 0 && cmpSelectedModels.size < llmSelectableModels.length"
|
||||||
@change="toggleAllCmpModels(($event.target as HTMLInputElement).checked)"
|
@change="toggleAllCmpModels(($event.target as HTMLInputElement).checked)"
|
||||||
/>
|
/>
|
||||||
<span class="picker-cat-name">All ollama models</span>
|
<span class="picker-cat-name">All LLM models</span>
|
||||||
</label>
|
</label>
|
||||||
<div class="picker-model-list">
|
<div v-for="(models, service) in llmModelsByService" :key="service" class="picker-category">
|
||||||
<label v-for="m in ollamaLlmModels" :key="m.id" class="picker-model-row">
|
<span class="picker-cat-section">{{ service }}</span>
|
||||||
<input
|
<div class="picker-model-list">
|
||||||
type="checkbox"
|
<label v-for="m in models" :key="m.id" class="picker-model-row">
|
||||||
:checked="cmpSelectedModels.has(m.id)"
|
<input
|
||||||
@change="toggleCmpModel(m.id, ($event.target as HTMLInputElement).checked)"
|
type="checkbox"
|
||||||
/>
|
:checked="cmpSelectedModels.has(m.id)"
|
||||||
<span class="picker-model-name">{{ m.name }}</span>
|
@change="toggleCmpModel(m.id, ($event.target as HTMLInputElement).checked)"
|
||||||
<span class="picker-adapter-type">{{ m.tags.slice(0, 3).join(', ') }}</span>
|
/>
|
||||||
</label>
|
<span class="picker-model-name">{{ m.name }}</span>
|
||||||
|
<span class="picker-adapter-type">{{ m.tags.slice(0, 2).join(', ') }}</span>
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</details>
|
</details>
|
||||||
|
|
@ -232,10 +235,22 @@ const cmpResults = ref<CmpResult[]>([])
|
||||||
const cmpEventSource = ref<EventSource | null>(null)
|
const cmpEventSource = ref<EventSource | null>(null)
|
||||||
|
|
||||||
// ── Computed ────────────────────────────────────────────────────────────────
|
// ── Computed ────────────────────────────────────────────────────────────────
|
||||||
const ollamaLlmModels = computed(() =>
|
const LLM_SERVICES = new Set(['ollama', 'vllm', 'cf-text'])
|
||||||
llmModels.value.filter(m => m.service === 'ollama')
|
|
||||||
|
const llmSelectableModels = computed(() =>
|
||||||
|
llmModels.value.filter(m => LLM_SERVICES.has(m.service))
|
||||||
)
|
)
|
||||||
|
|
||||||
|
/** Group selectable models by service for the picker UI */
|
||||||
|
const llmModelsByService = computed((): Record<string, CfOrchModel[]> => {
|
||||||
|
const groups: Record<string, CfOrchModel[]> = {}
|
||||||
|
for (const m of llmSelectableModels.value) {
|
||||||
|
if (!groups[m.service]) groups[m.service] = []
|
||||||
|
groups[m.service].push(m)
|
||||||
|
}
|
||||||
|
return groups
|
||||||
|
})
|
||||||
|
|
||||||
const llmTasksByType = computed((): Record<string, CfOrchTask[]> => {
|
const llmTasksByType = computed((): Record<string, CfOrchTask[]> => {
|
||||||
const groups: Record<string, CfOrchTask[]> = {}
|
const groups: Record<string, CfOrchTask[]> = {}
|
||||||
for (const t of llmTasks.value) {
|
for (const t of llmTasks.value) {
|
||||||
|
|
@ -270,7 +285,7 @@ function toggleCmpModel(id: string, checked: boolean) {
|
||||||
|
|
||||||
function toggleAllCmpModels(checked: boolean) {
|
function toggleAllCmpModels(checked: boolean) {
|
||||||
cmpSelectedModels.value = checked
|
cmpSelectedModels.value = checked
|
||||||
? new Set(ollamaLlmModels.value.map(m => m.id))
|
? new Set(llmSelectableModels.value.map(m => m.id))
|
||||||
: new Set()
|
: new Set()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -288,9 +303,8 @@ async function loadLlmModels() {
|
||||||
const { data } = await useApiFetch<{ models: CfOrchModel[] }>('/api/cforch/models')
|
const { data } = await useApiFetch<{ models: CfOrchModel[] }>('/api/cforch/models')
|
||||||
if (data?.models) {
|
if (data?.models) {
|
||||||
llmModels.value = data.models
|
llmModels.value = data.models
|
||||||
// Pre-select all ollama models
|
|
||||||
cmpSelectedModels.value = new Set(
|
cmpSelectedModels.value = new Set(
|
||||||
data.models.filter(m => m.service === 'ollama').map(m => m.id)
|
data.models.filter(m => LLM_SERVICES.has(m.service)).map(m => m.id)
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -28,9 +28,6 @@
|
||||||
<span class="metric-label"> labeled since last eval</span>
|
<span class="metric-label"> labeled since last eval</span>
|
||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
<div v-if="data.signals.data_to_eval" class="card-cta">
|
|
||||||
<RouterLink to="/eval/benchmark" class="cta-btn">Run Eval</RouterLink>
|
|
||||||
</div>
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- ② Eval card -->
|
<!-- ② Eval card -->
|
||||||
|
|
@ -40,18 +37,28 @@
|
||||||
<h2 class="card-title">Eval</h2>
|
<h2 class="card-title">Eval</h2>
|
||||||
</div>
|
</div>
|
||||||
<div class="card-body">
|
<div class="card-body">
|
||||||
<p class="card-metric">
|
<div class="bench-run-table">
|
||||||
<span class="metric-label">Last run: </span>
|
<div
|
||||||
<strong class="metric-value">{{ formattedEvalTime }}</strong>
|
v-for="(run, type) in data.recent_bench_runs"
|
||||||
</p>
|
:key="type"
|
||||||
<p v-if="data.last_eval_best_score != null" class="card-metric">
|
class="bench-run-row"
|
||||||
<span class="metric-label">Best score: </span>
|
>
|
||||||
<strong class="metric-value">{{ formatScore(data.last_eval_best_score) }}</strong>
|
<span class="bench-type-label">{{ BENCH_LABELS[type as BenchType] ?? type }}</span>
|
||||||
</p>
|
<span class="bench-run-time" :class="{ 'metric-muted': !run.timestamp }">
|
||||||
|
{{ run.timestamp ? formatBenchTs(run.timestamp) : '—' }}
|
||||||
|
</span>
|
||||||
|
<span v-if="run.score != null" class="bench-run-score">
|
||||||
|
{{ formatScore(run.score) }}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div v-if="data.signals.eval_to_train" class="card-cta">
|
<div v-if="data.signals.eval_to_train" class="card-cta">
|
||||||
<RouterLink to="/train/jobs" class="cta-btn">Queue Finetune</RouterLink>
|
<RouterLink to="/train/jobs" class="cta-btn">Queue Finetune</RouterLink>
|
||||||
</div>
|
</div>
|
||||||
|
<div v-if="data.signals.data_to_eval" class="card-cta">
|
||||||
|
<RouterLink to="/eval/benchmark" class="cta-btn">Run Eval</RouterLink>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- ③ Train card -->
|
<!-- ③ Train card -->
|
||||||
|
|
@ -104,33 +111,49 @@ interface DashboardSignals {
|
||||||
train_to_fleet: boolean
|
train_to_fleet: boolean
|
||||||
}
|
}
|
||||||
|
|
||||||
|
interface BenchRun {
|
||||||
|
timestamp: string | null
|
||||||
|
metric: string | null
|
||||||
|
score: number | null
|
||||||
|
}
|
||||||
|
|
||||||
|
type BenchType = 'classifier' | 'llm' | 'style' | 'plans'
|
||||||
|
|
||||||
interface DashboardData {
|
interface DashboardData {
|
||||||
labeled_since_last_eval: number
|
labeled_since_last_eval: number
|
||||||
last_eval_timestamp: string | null
|
last_eval_timestamp: string | null
|
||||||
last_eval_best_score: number | null
|
last_eval_best_score: number | null
|
||||||
active_jobs: ActiveJob[]
|
active_jobs: ActiveJob[]
|
||||||
corrections_export_ready: number
|
corrections_export_ready: number
|
||||||
|
recent_bench_runs: Record<BenchType, BenchRun>
|
||||||
signals: DashboardSignals
|
signals: DashboardSignals
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const BENCH_LABELS: Record<BenchType, string> = {
|
||||||
|
classifier: 'Classifier',
|
||||||
|
llm: 'LLM Eval',
|
||||||
|
style: 'Style',
|
||||||
|
plans: 'Planning',
|
||||||
|
}
|
||||||
|
|
||||||
const data = ref<DashboardData | null>(null)
|
const data = ref<DashboardData | null>(null)
|
||||||
const loading = ref(false)
|
const loading = ref(false)
|
||||||
const error = ref<string | null>(null)
|
const error = ref<string | null>(null)
|
||||||
|
|
||||||
const formattedEvalTime = computed(() => {
|
function formatBenchTs(ts: string): string {
|
||||||
if (!data.value?.last_eval_timestamp) return 'Never'
|
const date = new Date(ts)
|
||||||
const date = new Date(data.value.last_eval_timestamp)
|
if (!isNaN(date.getTime())) {
|
||||||
if (isNaN(date.getTime())) return 'Unknown'
|
const diff = Date.now() - date.getTime()
|
||||||
const now = Date.now()
|
const mins = Math.floor(diff / 60000)
|
||||||
const diff = now - date.getTime()
|
if (mins < 1) return 'just now'
|
||||||
const mins = Math.floor(diff / 60000)
|
if (mins < 60) return `${mins}m ago`
|
||||||
if (mins < 1) return 'just now'
|
const hrs = Math.floor(mins / 60)
|
||||||
if (mins < 60) return `${mins}m ago`
|
if (hrs < 24) return `${hrs}h ago`
|
||||||
const hrs = Math.floor(mins / 60)
|
return `${Math.floor(hrs / 24)}d ago`
|
||||||
if (hrs < 24) return `${hrs}h ago`
|
}
|
||||||
const days = Math.floor(hrs / 24)
|
// Non-ISO: show as-is (plans bench uses "YYYY-MM-DD HH:MM")
|
||||||
return `${days}d ago`
|
return ts.length > 16 ? ts.slice(0, 16) : ts
|
||||||
})
|
}
|
||||||
|
|
||||||
function formatScore(score: number): string {
|
function formatScore(score: number): string {
|
||||||
return `${(score * 100).toFixed(1)}%`
|
return `${(score * 100).toFixed(1)}%`
|
||||||
|
|
@ -285,6 +308,42 @@ onMounted(() => load())
|
||||||
|
|
||||||
.cta-btn:hover { background: color-mix(in srgb, var(--app-primary, #2A6080) 85%, black); }
|
.cta-btn:hover { background: color-mix(in srgb, var(--app-primary, #2A6080) 85%, black); }
|
||||||
|
|
||||||
|
/* ── Bench run table ── */
|
||||||
|
.bench-run-table {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.3rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.bench-run-row {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 6rem 1fr auto;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
font-size: 0.82rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.bench-type-label {
|
||||||
|
font-weight: 600;
|
||||||
|
color: var(--color-text, #1a2338);
|
||||||
|
font-size: 0.78rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.bench-run-time {
|
||||||
|
color: var(--color-text-secondary, #6b7a99);
|
||||||
|
font-size: 0.78rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.bench-run-score {
|
||||||
|
font-family: var(--font-mono, monospace);
|
||||||
|
font-size: 0.75rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: var(--app-primary, #2A6080);
|
||||||
|
background: color-mix(in srgb, var(--app-primary, #2A6080) 10%, transparent);
|
||||||
|
padding: 0.1rem 0.35rem;
|
||||||
|
border-radius: 0.25rem;
|
||||||
|
}
|
||||||
|
|
||||||
/* ── Job pills ── */
|
/* ── Job pills ── */
|
||||||
.job-row {
|
.job-row {
|
||||||
display: flex;
|
display: flex;
|
||||||
|
|
|
||||||
705
web/src/views/EmbedCompareTab.vue
Normal file
705
web/src/views/EmbedCompareTab.vue
Normal file
|
|
@ -0,0 +1,705 @@
|
||||||
|
<template>
|
||||||
|
<div class="embed-compare-page">
|
||||||
|
<!-- Step indicator (non-interactive) -->
|
||||||
|
<ol class="step-indicator" aria-label="Setup progress">
|
||||||
|
<li :class="{ complete: corpus.length > 0 }">Corpus</li>
|
||||||
|
<li :class="{ complete: queries.length > 0 }">Queries</li>
|
||||||
|
<li :class="{ complete: selectedModels.length > 0 }">Models</li>
|
||||||
|
<li :class="{ complete: hasResults }">Run & Rate</li>
|
||||||
|
</ol>
|
||||||
|
|
||||||
|
<!-- Persistent aria-live region — always in DOM, never v-if -->
|
||||||
|
<div
|
||||||
|
ref="liveRegion"
|
||||||
|
class="sr-live"
|
||||||
|
aria-live="polite"
|
||||||
|
aria-atomic="true"
|
||||||
|
v-text="liveMessage"
|
||||||
|
/>
|
||||||
|
|
||||||
|
<!-- ① Corpus section -->
|
||||||
|
<section class="card" aria-labelledby="corpus-heading">
|
||||||
|
<h2 id="corpus-heading">① Corpus</h2>
|
||||||
|
<div class="corpus-controls">
|
||||||
|
<div class="field">
|
||||||
|
<label for="corpus-paste">Paste chunks (one per line)</label>
|
||||||
|
<textarea
|
||||||
|
id="corpus-paste"
|
||||||
|
v-model="rawCorpus"
|
||||||
|
rows="6"
|
||||||
|
placeholder="Paste one chunk per line, or use Import below..."
|
||||||
|
@change="onCorpusPaste"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<div class="import-row">
|
||||||
|
<label for="imitate-product-select">Import from product</label>
|
||||||
|
<select id="imitate-product-select" v-model="selectedProduct">
|
||||||
|
<option value="">-- select product --</option>
|
||||||
|
<option
|
||||||
|
v-for="p in imitateProducts"
|
||||||
|
:key="p.id"
|
||||||
|
:value="p.id"
|
||||||
|
>{{ p.name }}</option>
|
||||||
|
</select>
|
||||||
|
<button
|
||||||
|
class="btn-secondary"
|
||||||
|
:disabled="!selectedProduct || importing"
|
||||||
|
@click="importCorpus"
|
||||||
|
>
|
||||||
|
{{ importing ? 'Importing…' : 'Import' }}
|
||||||
|
</button>
|
||||||
|
<span v-if="importError" class="error-text" role="alert">{{ importError }}</span>
|
||||||
|
</div>
|
||||||
|
<p v-if="corpus.length > 0" class="corpus-count">
|
||||||
|
{{ corpus.length }} chunk{{ corpus.length === 1 ? '' : 's' }} loaded.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<!-- ② Queries section -->
|
||||||
|
<section class="card" aria-labelledby="queries-heading">
|
||||||
|
<h2 id="queries-heading">② Queries</h2>
|
||||||
|
<div class="field">
|
||||||
|
<label for="query-input">Enter queries (one per line)</label>
|
||||||
|
<textarea
|
||||||
|
id="query-input"
|
||||||
|
v-model="rawQueries"
|
||||||
|
rows="4"
|
||||||
|
placeholder="One query per line..."
|
||||||
|
@change="onQueriesChange"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<p v-if="queries.length > 0" class="query-count">
|
||||||
|
{{ queries.length }} quer{{ queries.length === 1 ? 'y' : 'ies' }}.
|
||||||
|
</p>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<!-- ③ Model selection -->
|
||||||
|
<section class="card" aria-labelledby="models-heading">
|
||||||
|
<h2 id="models-heading">③ Models</h2>
|
||||||
|
<p v-if="loadingModels" class="muted">Loading models from Ollama…</p>
|
||||||
|
<p v-else-if="modelsError" class="error-text" role="alert">{{ modelsError }}</p>
|
||||||
|
<ul v-else class="model-list" role="list">
|
||||||
|
<li v-for="m in availableModels" :key="m.name">
|
||||||
|
<label class="model-checkbox">
|
||||||
|
<input
|
||||||
|
type="checkbox"
|
||||||
|
:value="m.name"
|
||||||
|
v-model="selectedModels"
|
||||||
|
/>
|
||||||
|
{{ m.name }}
|
||||||
|
<span class="model-size muted" aria-label="model size">
|
||||||
|
{{ formatBytes(m.size) }}
|
||||||
|
</span>
|
||||||
|
</label>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
<p v-if="availableModels.length === 0 && !loadingModels && !modelsError" class="muted">
|
||||||
|
No Ollama models found. Pull an embedding model first.
|
||||||
|
</p>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<!-- ④ Run controls -->
|
||||||
|
<section class="card run-controls" aria-labelledby="run-heading">
|
||||||
|
<h2 id="run-heading">④ Run</h2>
|
||||||
|
<div class="run-row">
|
||||||
|
<div class="field-inline">
|
||||||
|
<label for="top-k-input">Results per query</label>
|
||||||
|
<input
|
||||||
|
id="top-k-input"
|
||||||
|
type="number"
|
||||||
|
v-model.number="topK"
|
||||||
|
min="1"
|
||||||
|
max="20"
|
||||||
|
style="width: 5rem"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<button
|
||||||
|
class="btn-primary"
|
||||||
|
:disabled="!canRun || running"
|
||||||
|
@click="startRun"
|
||||||
|
>
|
||||||
|
{{ running ? 'Running…' : 'Run' }}
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
v-if="running"
|
||||||
|
class="btn-danger"
|
||||||
|
aria-label="Cancel embedding run"
|
||||||
|
@click="cancelRun"
|
||||||
|
>
|
||||||
|
Cancel
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
<p v-if="!canRun && !running" class="muted">
|
||||||
|
Fill corpus, at least one query, and select at least one model to run.
|
||||||
|
</p>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<!-- Results -->
|
||||||
|
<section
|
||||||
|
v-if="hasResults"
|
||||||
|
class="card results-section"
|
||||||
|
aria-labelledby="results-heading"
|
||||||
|
>
|
||||||
|
<h2 id="results-heading">Results</h2>
|
||||||
|
|
||||||
|
<!-- Query pagination -->
|
||||||
|
<div class="query-nav" role="navigation" aria-label="Query navigation">
|
||||||
|
<button
|
||||||
|
class="btn-secondary"
|
||||||
|
aria-label="Previous query"
|
||||||
|
:disabled="currentQueryIdx === 0"
|
||||||
|
@click="currentQueryIdx--"
|
||||||
|
>‹</button>
|
||||||
|
<span class="query-counter">
|
||||||
|
Query {{ currentQueryIdx + 1 }} of {{ uniqueQueries.length }}:
|
||||||
|
<em>{{ uniqueQueries[currentQueryIdx] }}</em>
|
||||||
|
</span>
|
||||||
|
<button
|
||||||
|
class="btn-secondary"
|
||||||
|
aria-label="Next query"
|
||||||
|
:disabled="currentQueryIdx >= uniqueQueries.length - 1"
|
||||||
|
@click="currentQueryIdx++"
|
||||||
|
>›</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Results table: one column per model -->
|
||||||
|
<div class="table-wrap">
|
||||||
|
<table class="results-table">
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th scope="col" class="rank-col">#</th>
|
||||||
|
<th
|
||||||
|
v-for="model in selectedModels"
|
||||||
|
:key="model"
|
||||||
|
scope="col"
|
||||||
|
>{{ model }}</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
<tr v-for="rank in topK" :key="rank">
|
||||||
|
<td class="rank-col muted">{{ rank }}</td>
|
||||||
|
<td
|
||||||
|
v-for="model in selectedModels"
|
||||||
|
:key="model"
|
||||||
|
class="hit-cell"
|
||||||
|
>
|
||||||
|
<template v-if="getHit(currentQueryIdx, model, rank - 1) as hit">
|
||||||
|
<div class="hit-text">{{ hit.text }}</div>
|
||||||
|
<!-- Visual score bar: decorative only -->
|
||||||
|
<div class="score-row">
|
||||||
|
<div class="score-bar-wrap" aria-hidden="true">
|
||||||
|
<div class="score-bar" :style="{ width: `${hit.score * 100}%` }" />
|
||||||
|
</div>
|
||||||
|
<span class="score-label">{{ hit.score.toFixed(3) }}</span>
|
||||||
|
</div>
|
||||||
|
<!-- Rating buttons -->
|
||||||
|
<div class="rating-row">
|
||||||
|
<button
|
||||||
|
class="rate-btn"
|
||||||
|
:class="{ active: getRating(currentQueryIdx, model, hit.chunk_idx) === 'relevant' }"
|
||||||
|
:aria-pressed="getRating(currentQueryIdx, model, hit.chunk_idx) === 'relevant'"
|
||||||
|
aria-label="Mark as relevant"
|
||||||
|
@click="rate(currentQueryIdx, model, hit, 'relevant')"
|
||||||
|
>
|
||||||
|
👍 Relevant
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
class="rate-btn rate-btn-neg"
|
||||||
|
:class="{ active: getRating(currentQueryIdx, model, hit.chunk_idx) === 'not_relevant' }"
|
||||||
|
:aria-pressed="getRating(currentQueryIdx, model, hit.chunk_idx) === 'not_relevant'"
|
||||||
|
aria-label="Mark as not relevant"
|
||||||
|
@click="rate(currentQueryIdx, model, hit, 'not_relevant')"
|
||||||
|
>
|
||||||
|
👎 Not relevant
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
<span v-else class="muted">—</span>
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<!-- Export -->
|
||||||
|
<section
|
||||||
|
v-if="hasResults"
|
||||||
|
class="card export-section"
|
||||||
|
aria-labelledby="export-heading"
|
||||||
|
>
|
||||||
|
<h2 id="export-heading">Export Ratings</h2>
|
||||||
|
<div class="export-row">
|
||||||
|
<fieldset class="export-format-group">
|
||||||
|
<legend>Format</legend>
|
||||||
|
<label><input type="radio" v-model="exportFormat" value="csv" /> CSV</label>
|
||||||
|
<label><input type="radio" v-model="exportFormat" value="json" /> JSON</label>
|
||||||
|
</fieldset>
|
||||||
|
<button class="btn-secondary" @click="exportRatings">Export</button>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<script setup lang="ts">
|
||||||
|
import { ref, computed, onMounted } from 'vue'
|
||||||
|
|
||||||
|
// ── Types ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
interface OllamaModel { name: string; size: number }
|
||||||
|
interface ImitateProduct { id: string; name: string }
|
||||||
|
interface HitResult { chunk_idx: number; text: string; score: number }
|
||||||
|
interface ResultEvent {
|
||||||
|
type: 'result'
|
||||||
|
query_idx: number
|
||||||
|
query: string
|
||||||
|
model: string
|
||||||
|
hits: HitResult[]
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── State ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
const rawCorpus = ref('')
|
||||||
|
const corpus = ref<string[]>([])
|
||||||
|
const rawQueries = ref('')
|
||||||
|
const queries = ref<string[]>([])
|
||||||
|
const selectedModels = ref<string[]>([])
|
||||||
|
const topK = ref(5)
|
||||||
|
const availableModels = ref<OllamaModel[]>([])
|
||||||
|
const loadingModels = ref(false)
|
||||||
|
const modelsError = ref('')
|
||||||
|
const imitateProducts = ref<ImitateProduct[]>([])
|
||||||
|
const selectedProduct = ref('')
|
||||||
|
const importing = ref(false)
|
||||||
|
const importError = ref('')
|
||||||
|
const running = ref(false)
|
||||||
|
const liveMessage = ref('')
|
||||||
|
const resultEvents = ref<ResultEvent[]>([])
|
||||||
|
const runController = ref<AbortController | null>(null)
|
||||||
|
|
||||||
|
const currentQueryIdx = ref(0)
|
||||||
|
const exportFormat = ref<'csv' | 'json'>('csv')
|
||||||
|
|
||||||
|
type RatingMap = Record<string, Record<string, Record<number, 'relevant' | 'not_relevant'>>>
|
||||||
|
const ratings = ref<RatingMap>({})
|
||||||
|
|
||||||
|
const uniqueQueries = computed(() => {
|
||||||
|
const seen = new Set<string>()
|
||||||
|
const out: string[] = []
|
||||||
|
for (const e of resultEvents.value) {
|
||||||
|
if (!seen.has(e.query)) { seen.add(e.query); out.push(e.query) }
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
})
|
||||||
|
|
||||||
|
const hasResults = computed(() => resultEvents.value.length > 0)
|
||||||
|
const canRun = computed(
|
||||||
|
() => corpus.value.length > 0 && queries.value.length > 0 && selectedModels.value.length > 0
|
||||||
|
)
|
||||||
|
|
||||||
|
// ── Corpus helpers ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
function onCorpusPaste() {
|
||||||
|
const chunks = rawCorpus.value.split('\n').map(l => l.trim()).filter(Boolean)
|
||||||
|
corpus.value = chunks
|
||||||
|
if (chunks.length > 0) {
|
||||||
|
liveMessage.value = `${chunks.length} chunk${chunks.length === 1 ? '' : 's'} loaded.`
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function onQueriesChange() {
|
||||||
|
queries.value = rawQueries.value.split('\n').map(l => l.trim()).filter(Boolean)
|
||||||
|
}
|
||||||
|
|
||||||
|
async function importCorpus() {
|
||||||
|
if (!selectedProduct.value) return
|
||||||
|
importing.value = true
|
||||||
|
importError.value = ''
|
||||||
|
try {
|
||||||
|
const r = await fetch(`/api/imitate/products/${selectedProduct.value}/sample-chunks`)
|
||||||
|
if (!r.ok) {
|
||||||
|
const text = await r.text()
|
||||||
|
throw new Error(text || `HTTP ${r.status}`)
|
||||||
|
}
|
||||||
|
const data = await r.json() as { chunks?: string[] }
|
||||||
|
const chunks = data.chunks ?? []
|
||||||
|
corpus.value = chunks
|
||||||
|
rawCorpus.value = chunks.join('\n')
|
||||||
|
liveMessage.value = `${chunks.length} chunk${chunks.length === 1 ? '' : 's'} loaded from import.`
|
||||||
|
} catch (err) {
|
||||||
|
importError.value = String(err)
|
||||||
|
} finally {
|
||||||
|
importing.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Model loading ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
async function loadModels() {
|
||||||
|
loadingModels.value = true
|
||||||
|
modelsError.value = ''
|
||||||
|
try {
|
||||||
|
const r = await fetch('/api/embed-bench/models')
|
||||||
|
if (!r.ok) throw new Error(`HTTP ${r.status}`)
|
||||||
|
const data = await r.json() as { models: OllamaModel[] }
|
||||||
|
availableModels.value = data.models
|
||||||
|
} catch (err) {
|
||||||
|
modelsError.value = `Failed to load models: ${err}`
|
||||||
|
} finally {
|
||||||
|
loadingModels.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Run ───────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
async function startRun() {
|
||||||
|
if (!canRun.value) return
|
||||||
|
running.value = true
|
||||||
|
resultEvents.value = []
|
||||||
|
liveMessage.value = 'Starting embedding run…'
|
||||||
|
runController.value = new AbortController()
|
||||||
|
|
||||||
|
try {
|
||||||
|
const resp = await fetch('/api/embed-bench/run', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({
|
||||||
|
corpus: corpus.value,
|
||||||
|
queries: queries.value,
|
||||||
|
models: selectedModels.value,
|
||||||
|
top_k: topK.value,
|
||||||
|
}),
|
||||||
|
signal: runController.value.signal,
|
||||||
|
})
|
||||||
|
|
||||||
|
const reader = resp.body!.getReader()
|
||||||
|
const decoder = new TextDecoder()
|
||||||
|
let buf = ''
|
||||||
|
|
||||||
|
while (true) {
|
||||||
|
const { done, value } = await reader.read()
|
||||||
|
if (done) break
|
||||||
|
buf += decoder.decode(value, { stream: true })
|
||||||
|
const lines = buf.split('\n')
|
||||||
|
buf = lines.pop() ?? ''
|
||||||
|
for (const line of lines) {
|
||||||
|
if (!line.startsWith('data: ')) continue
|
||||||
|
const event = JSON.parse(line.slice(6))
|
||||||
|
if (event.type === 'progress') {
|
||||||
|
liveMessage.value = event.msg
|
||||||
|
} else if (event.type === 'result') {
|
||||||
|
resultEvents.value.push(event as ResultEvent)
|
||||||
|
} else if (event.type === 'done') {
|
||||||
|
liveMessage.value = 'Run complete.'
|
||||||
|
} else if (event.type === 'error') {
|
||||||
|
liveMessage.value = `Error: ${event.msg}`
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
if ((err as Error).name !== 'AbortError') {
|
||||||
|
liveMessage.value = `Run failed: ${err}`
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
running.value = false
|
||||||
|
runController.value = null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function cancelRun() {
|
||||||
|
runController.value?.abort()
|
||||||
|
liveMessage.value = 'Run cancelled.'
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Utilities ─────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
function formatBytes(bytes: number): string {
|
||||||
|
if (bytes < 1_000_000) return `${(bytes / 1000).toFixed(0)} KB`
|
||||||
|
if (bytes < 1_000_000_000) return `${(bytes / 1_000_000).toFixed(0)} MB`
|
||||||
|
return `${(bytes / 1_000_000_000).toFixed(1)} GB`
|
||||||
|
}
|
||||||
|
|
||||||
|
function getHit(queryIdx: number, model: string, rank: number): HitResult | null {
|
||||||
|
const query = uniqueQueries.value[queryIdx]
|
||||||
|
if (!query) return null
|
||||||
|
const ev = resultEvents.value.find(e => e.query === query && e.model === model)
|
||||||
|
return ev?.hits[rank] ?? null
|
||||||
|
}
|
||||||
|
|
||||||
|
function getRating(queryIdx: number, model: string, chunkIdx: number): string | undefined {
|
||||||
|
const query = uniqueQueries.value[queryIdx]
|
||||||
|
return ratings.value[query]?.[model]?.[chunkIdx]
|
||||||
|
}
|
||||||
|
|
||||||
|
async function rate(
|
||||||
|
queryIdx: number,
|
||||||
|
model: string,
|
||||||
|
hit: HitResult,
|
||||||
|
rating: 'relevant' | 'not_relevant',
|
||||||
|
) {
|
||||||
|
const query = uniqueQueries.value[queryIdx]
|
||||||
|
// Optimistic update
|
||||||
|
if (!ratings.value[query]) ratings.value[query] = {}
|
||||||
|
if (!ratings.value[query][model]) ratings.value[query][model] = {}
|
||||||
|
ratings.value[query][model][hit.chunk_idx] = rating
|
||||||
|
|
||||||
|
try {
|
||||||
|
await fetch('/api/embed-bench/rate', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({
|
||||||
|
query,
|
||||||
|
model,
|
||||||
|
chunk_text: hit.text,
|
||||||
|
chunk_idx: hit.chunk_idx,
|
||||||
|
rating,
|
||||||
|
}),
|
||||||
|
})
|
||||||
|
liveMessage.value = `Rated chunk ${hit.chunk_idx + 1} as ${rating}.`
|
||||||
|
} catch (err) {
|
||||||
|
liveMessage.value = `Rating failed: ${err}`
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function exportRatings() {
|
||||||
|
const r = await fetch(`/api/embed-bench/export?format=${exportFormat.value}`)
|
||||||
|
if (!r.ok) {
|
||||||
|
liveMessage.value = `Export failed: HTTP ${r.status}`
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const blob = await r.blob()
|
||||||
|
const disposition = r.headers.get('Content-Disposition') ?? ''
|
||||||
|
const filenameMatch = disposition.match(/filename="([^"]+)"/)
|
||||||
|
const filename = filenameMatch ? filenameMatch[1] : `embed_comparison.${exportFormat.value}`
|
||||||
|
const url = URL.createObjectURL(blob)
|
||||||
|
const a = document.createElement('a')
|
||||||
|
a.href = url
|
||||||
|
a.download = filename
|
||||||
|
a.click()
|
||||||
|
URL.revokeObjectURL(url)
|
||||||
|
liveMessage.value = `Exported ${filename}.`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Lifecycle ─────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
onMounted(() => {
|
||||||
|
loadModels()
|
||||||
|
})
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<style scoped>
|
||||||
|
.embed-compare-page {
|
||||||
|
padding: var(--space-4, 1.5rem);
|
||||||
|
max-width: 1100px;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Step indicator */
|
||||||
|
.step-indicator {
|
||||||
|
display: flex;
|
||||||
|
gap: 0;
|
||||||
|
list-style: none;
|
||||||
|
margin: 0 0 var(--space-4, 1.5rem);
|
||||||
|
padding: 0;
|
||||||
|
border-bottom: 2px solid var(--color-border, #d0d7e8);
|
||||||
|
}
|
||||||
|
.step-indicator li {
|
||||||
|
padding: 0.4rem 1rem;
|
||||||
|
font-size: 0.8rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: var(--color-text-muted, #4a5c7a);
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.05em;
|
||||||
|
border-bottom: 2px solid transparent;
|
||||||
|
margin-bottom: -2px;
|
||||||
|
}
|
||||||
|
.step-indicator li.complete {
|
||||||
|
color: var(--app-primary, #2A6080);
|
||||||
|
border-bottom-color: var(--app-primary, #2A6080);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Accessibility: screen-reader live region — visually hidden but always present */
|
||||||
|
.sr-live {
|
||||||
|
position: absolute;
|
||||||
|
width: 1px; height: 1px;
|
||||||
|
overflow: hidden;
|
||||||
|
clip: rect(0 0 0 0);
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Cards */
|
||||||
|
.card {
|
||||||
|
background: var(--color-surface-raised, #e4ebf5);
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
border-radius: var(--radius-md, 0.5rem);
|
||||||
|
padding: var(--space-4, 1.5rem);
|
||||||
|
margin-bottom: var(--space-4, 1.5rem);
|
||||||
|
}
|
||||||
|
.card h2 {
|
||||||
|
font-size: 1rem;
|
||||||
|
font-weight: 700;
|
||||||
|
margin: 0 0 var(--space-3, 1rem);
|
||||||
|
color: var(--color-text, #1a2338);
|
||||||
|
}
|
||||||
|
|
||||||
|
.field { display: flex; flex-direction: column; gap: 0.3rem; margin-bottom: 0.75rem; }
|
||||||
|
.field label { font-size: 0.85rem; font-weight: 600; }
|
||||||
|
textarea, input[type="number"] {
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
border-radius: var(--radius-sm, 0.25rem);
|
||||||
|
padding: 0.5rem;
|
||||||
|
font-size: 0.875rem;
|
||||||
|
background: var(--color-surface, #f0f4fb);
|
||||||
|
color: var(--color-text, #1a2338);
|
||||||
|
resize: vertical;
|
||||||
|
}
|
||||||
|
|
||||||
|
.corpus-controls { display: flex; flex-direction: column; gap: 0.5rem; }
|
||||||
|
.import-row {
|
||||||
|
display: flex; flex-wrap: wrap; gap: 0.5rem; align-items: center;
|
||||||
|
}
|
||||||
|
.import-row label { font-size: 0.85rem; font-weight: 600; }
|
||||||
|
.corpus-count, .query-count { font-size: 0.875rem; color: var(--app-primary, #2A6080); margin: 0; }
|
||||||
|
|
||||||
|
.model-list { list-style: none; padding: 0; margin: 0; display: flex; flex-wrap: wrap; gap: 0.5rem; }
|
||||||
|
.model-checkbox {
|
||||||
|
display: flex; align-items: center; gap: 0.4rem;
|
||||||
|
font-size: 0.875rem; cursor: pointer;
|
||||||
|
padding: 0.3rem 0.6rem;
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
border-radius: var(--radius-sm, 0.25rem);
|
||||||
|
background: var(--color-surface, #f0f4fb);
|
||||||
|
}
|
||||||
|
.model-size { font-size: 0.75rem; }
|
||||||
|
|
||||||
|
.run-row { display: flex; flex-wrap: wrap; gap: 0.75rem; align-items: flex-end; }
|
||||||
|
.field-inline { display: flex; align-items: center; gap: 0.4rem; }
|
||||||
|
.field-inline label { font-size: 0.85rem; font-weight: 600; white-space: nowrap; }
|
||||||
|
|
||||||
|
.btn-primary, .btn-secondary, .btn-danger {
|
||||||
|
padding: 0.4rem 1rem;
|
||||||
|
border-radius: var(--radius-sm, 0.25rem);
|
||||||
|
border: 1px solid transparent;
|
||||||
|
font-size: 0.875rem;
|
||||||
|
font-weight: 600;
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background 0.15s;
|
||||||
|
}
|
||||||
|
.btn-primary { background: var(--app-primary, #2A6080); color: #fff; }
|
||||||
|
.btn-primary:hover:not(:disabled) { filter: brightness(1.1); }
|
||||||
|
.btn-primary:disabled { opacity: 0.5; cursor: not-allowed; }
|
||||||
|
.btn-secondary { background: var(--color-surface, #f0f4fb); color: var(--color-text, #1a2338); border-color: var(--color-border, #d0d7e8); }
|
||||||
|
.btn-secondary:hover:not(:disabled) { background: var(--color-border, #d0d7e8); }
|
||||||
|
.btn-secondary:disabled { opacity: 0.5; cursor: not-allowed; }
|
||||||
|
.btn-danger { background: var(--color-error, #c0392b); color: #fff; }
|
||||||
|
|
||||||
|
.muted { color: var(--color-text-muted, #4a5c7a); font-size: 0.875rem; }
|
||||||
|
.error-text { color: var(--color-error, #c0392b); font-size: 0.875rem; }
|
||||||
|
|
||||||
|
@media (max-width: 768px) {
|
||||||
|
.import-row { flex-direction: column; align-items: flex-start; }
|
||||||
|
.run-row { flex-direction: column; }
|
||||||
|
.model-list { flex-direction: column; }
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Results table */
|
||||||
|
.table-wrap { overflow-x: auto; }
|
||||||
|
.results-table {
|
||||||
|
width: 100%;
|
||||||
|
border-collapse: collapse;
|
||||||
|
font-size: 0.875rem;
|
||||||
|
}
|
||||||
|
.results-table thead th {
|
||||||
|
position: sticky;
|
||||||
|
top: 0;
|
||||||
|
background: var(--color-surface-raised, #e4ebf5);
|
||||||
|
border-bottom: 2px solid var(--color-border, #d0d7e8);
|
||||||
|
padding: 0.5rem 0.75rem;
|
||||||
|
text-align: left;
|
||||||
|
font-weight: 700;
|
||||||
|
white-space: nowrap;
|
||||||
|
z-index: 1;
|
||||||
|
}
|
||||||
|
.results-table td {
|
||||||
|
padding: 0.5rem 0.75rem;
|
||||||
|
vertical-align: top;
|
||||||
|
border-bottom: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
}
|
||||||
|
.rank-col { width: 2rem; text-align: center; }
|
||||||
|
|
||||||
|
.hit-text { margin-bottom: 0.25rem; line-height: 1.4; }
|
||||||
|
|
||||||
|
.score-row { display: flex; align-items: center; gap: 0.4rem; margin-bottom: 0.25rem; }
|
||||||
|
.score-bar-wrap {
|
||||||
|
flex: 1;
|
||||||
|
height: 6px;
|
||||||
|
background: var(--color-border, #d0d7e8);
|
||||||
|
border-radius: 3px;
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
.score-bar {
|
||||||
|
height: 100%;
|
||||||
|
background: var(--app-primary, #2A6080);
|
||||||
|
border-radius: 3px;
|
||||||
|
transition: width 0.3s ease;
|
||||||
|
}
|
||||||
|
.score-label { font-size: 0.75rem; color: var(--color-text-muted, #4a5c7a); min-width: 3rem; text-align: right; }
|
||||||
|
|
||||||
|
.rating-row { display: flex; gap: 0.4rem; flex-wrap: wrap; }
|
||||||
|
.rate-btn {
|
||||||
|
padding: 0.2rem 0.5rem;
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
border-radius: var(--radius-sm, 0.25rem);
|
||||||
|
background: var(--color-surface, #f0f4fb);
|
||||||
|
color: var(--color-text, #1a2338);
|
||||||
|
font-size: 0.75rem;
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background 0.15s, border-color 0.15s;
|
||||||
|
}
|
||||||
|
.rate-btn.active {
|
||||||
|
background: color-mix(in srgb, var(--app-primary, #2A6080) 20%, transparent);
|
||||||
|
border-color: var(--app-primary, #2A6080);
|
||||||
|
font-weight: 700;
|
||||||
|
}
|
||||||
|
.rate-btn-neg.active {
|
||||||
|
background: color-mix(in srgb, var(--color-error, #c0392b) 15%, transparent);
|
||||||
|
border-color: var(--color-error, #c0392b);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Query nav */
|
||||||
|
.query-nav {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
margin-bottom: 0.75rem;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
.query-counter { font-size: 0.875rem; flex: 1; }
|
||||||
|
|
||||||
|
/* Export */
|
||||||
|
.export-row { display: flex; gap: 1rem; align-items: center; flex-wrap: wrap; }
|
||||||
|
.export-format-group {
|
||||||
|
border: none;
|
||||||
|
padding: 0;
|
||||||
|
display: flex;
|
||||||
|
gap: 0.75rem;
|
||||||
|
}
|
||||||
|
.export-format-group legend {
|
||||||
|
font-size: 0.85rem;
|
||||||
|
font-weight: 600;
|
||||||
|
margin-bottom: 0.25rem;
|
||||||
|
float: left;
|
||||||
|
margin-right: 0.5rem;
|
||||||
|
}
|
||||||
|
.export-format-group label { font-size: 0.875rem; display: flex; align-items: center; gap: 0.3rem; }
|
||||||
|
|
||||||
|
@media (max-width: 768px) {
|
||||||
|
.results-table thead th,
|
||||||
|
.results-table td { padding: 0.35rem 0.4rem; font-size: 0.8rem; }
|
||||||
|
.query-nav { flex-direction: column; align-items: flex-start; }
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (prefers-reduced-motion: reduce) {
|
||||||
|
.score-bar { transition: none; }
|
||||||
|
}
|
||||||
|
</style>
|
||||||
7
web/src/views/EmbedCompareView.vue
Normal file
7
web/src/views/EmbedCompareView.vue
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
<template>
|
||||||
|
<EmbedCompareTab />
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<script setup lang="ts">
|
||||||
|
import EmbedCompareTab from './EmbedCompareTab.vue'
|
||||||
|
</script>
|
||||||
|
|
@ -302,7 +302,7 @@ const llmModelBadge = computed(() => {
|
||||||
const llmTaskTypeCols = computed(() => {
|
const llmTaskTypeCols = computed(() => {
|
||||||
const types = new Set<string>()
|
const types = new Set<string>()
|
||||||
for (const r of llmResults.value) {
|
for (const r of llmResults.value) {
|
||||||
for (const k of Object.keys(r.quality_by_task_type)) types.add(k)
|
for (const k of Object.keys(r.quality_by_task_type ?? {})) types.add(k)
|
||||||
}
|
}
|
||||||
return [...types].sort()
|
return [...types].sort()
|
||||||
})
|
})
|
||||||
|
|
@ -338,7 +338,7 @@ const llmBestByCol = computed((): Record<string, string> => {
|
||||||
for (const col of llmTaskTypeCols.value) {
|
for (const col of llmTaskTypeCols.value) {
|
||||||
bestId = ''; bestVal = -Infinity
|
bestId = ''; bestVal = -Infinity
|
||||||
for (const r of llmResults.value) {
|
for (const r of llmResults.value) {
|
||||||
const v = r.quality_by_task_type[col]
|
const v = r.quality_by_task_type?.[col]
|
||||||
if (v != null && v > bestVal) { bestVal = v; bestId = r.model_id }
|
if (v != null && v > bestVal) { bestVal = v; bestId = r.model_id }
|
||||||
}
|
}
|
||||||
best[col] = bestId
|
best[col] = bestId
|
||||||
|
|
|
||||||
|
|
@ -2,6 +2,24 @@
|
||||||
<div class="models-view">
|
<div class="models-view">
|
||||||
<h1 class="page-title">🤗 Models</h1>
|
<h1 class="page-title">🤗 Models</h1>
|
||||||
|
|
||||||
|
<!-- ── Fleet tab bar ─────────────────────────────── -->
|
||||||
|
<div class="mode-toggle" role="group" aria-label="Fleet view">
|
||||||
|
<button
|
||||||
|
class="mode-btn"
|
||||||
|
:class="{ active: fleetTab === 'models' }"
|
||||||
|
@click="fleetTab = 'models'"
|
||||||
|
>Models</button>
|
||||||
|
<button
|
||||||
|
class="mode-btn"
|
||||||
|
:class="{ active: fleetTab === 'assignments' }"
|
||||||
|
@click="fleetTab = 'assignments'"
|
||||||
|
>Assignments</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<AssignmentsTab v-if="fleetTab === 'assignments'" />
|
||||||
|
|
||||||
|
<template v-if="fleetTab === 'models'">
|
||||||
|
|
||||||
<!-- ── 1. HF Lookup ───────────────────────────────── -->
|
<!-- ── 1. HF Lookup ───────────────────────────────── -->
|
||||||
<section class="section">
|
<section class="section">
|
||||||
<h2 class="section-title">HuggingFace Lookup</h2>
|
<h2 class="section-title">HuggingFace Lookup</h2>
|
||||||
|
|
@ -297,11 +315,17 @@
|
||||||
</div>
|
</div>
|
||||||
</template>
|
</template>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
|
</template><!-- end fleetTab === 'models' -->
|
||||||
</div>
|
</div>
|
||||||
</template>
|
</template>
|
||||||
|
|
||||||
<script setup lang="ts">
|
<script setup lang="ts">
|
||||||
import { ref, computed, onMounted, onUnmounted } from 'vue'
|
import { ref, computed, onMounted, onUnmounted } from 'vue'
|
||||||
|
import AssignmentsTab from './AssignmentsTab.vue'
|
||||||
|
|
||||||
|
type FleetTab = 'models' | 'assignments'
|
||||||
|
const fleetTab = ref<FleetTab>('models')
|
||||||
|
|
||||||
// ── Type definitions ──────────────────────────────────
|
// ── Type definitions ──────────────────────────────────
|
||||||
|
|
||||||
|
|
@ -738,6 +762,39 @@ onUnmounted(() => {
|
||||||
color: var(--color-primary, #2d5a27);
|
color: var(--color-primary, #2d5a27);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* ── Fleet tab bar (mode-toggle pattern from BenchmarkView) ── */
|
||||||
|
.mode-toggle {
|
||||||
|
display: inline-flex;
|
||||||
|
border: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
border-radius: 0.5rem;
|
||||||
|
overflow: hidden;
|
||||||
|
align-self: flex-start;
|
||||||
|
}
|
||||||
|
.mode-btn {
|
||||||
|
padding: 0.4rem 1.1rem;
|
||||||
|
font-size: 0.85rem;
|
||||||
|
font-family: var(--font-body, sans-serif);
|
||||||
|
font-weight: 500;
|
||||||
|
border: none;
|
||||||
|
background: var(--color-surface, #fff);
|
||||||
|
color: var(--color-text-secondary, #6b7a99);
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background 0.15s, color 0.15s;
|
||||||
|
}
|
||||||
|
.mode-btn:not(:last-child) {
|
||||||
|
border-right: 1px solid var(--color-border, #d0d7e8);
|
||||||
|
}
|
||||||
|
.mode-btn.active {
|
||||||
|
background: var(--app-primary, #2A6080);
|
||||||
|
color: #fff;
|
||||||
|
}
|
||||||
|
.mode-btn:not(.active):hover {
|
||||||
|
background: var(--color-surface-raised, #e4ebf5);
|
||||||
|
}
|
||||||
|
@media (max-width: 600px) {
|
||||||
|
.mode-btn { padding: 0.4rem 0.65rem; font-size: 0.78rem; }
|
||||||
|
}
|
||||||
|
|
||||||
/* ── Sections ── */
|
/* ── Sections ── */
|
||||||
.section {
|
.section {
|
||||||
display: flex;
|
display: flex;
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,12 @@
|
||||||
<script setup lang="ts">
|
<script setup lang="ts">
|
||||||
import { ref, onMounted } from 'vue'
|
import { ref, onMounted } from 'vue'
|
||||||
import NodeCard from '../components/nodes/NodeCard.vue'
|
import NodeCard from '../components/nodes/NodeCard.vue'
|
||||||
|
import AssignmentsTab from './AssignmentsTab.vue'
|
||||||
import type { NodeSummary } from '../types/nodes'
|
import type { NodeSummary } from '../types/nodes'
|
||||||
|
|
||||||
|
type Tab = 'nodes' | 'assignments'
|
||||||
|
|
||||||
|
const activeTab = ref<Tab>('nodes')
|
||||||
const nodes = ref<NodeSummary[]>([])
|
const nodes = ref<NodeSummary[]>([])
|
||||||
const loading = ref(true)
|
const loading = ref(true)
|
||||||
const error = ref('')
|
const error = ref('')
|
||||||
|
|
@ -25,45 +29,137 @@ onMounted(fetchNodes)
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
<template>
|
<template>
|
||||||
<main class="nodes-page">
|
<main class="fleet-page">
|
||||||
<header class="nodes-header">
|
<header class="fleet-header">
|
||||||
<h1>Nodes</h1>
|
<h1 class="fleet-title">Fleet</h1>
|
||||||
<button class="btn-secondary" @click="fetchNodes" :disabled="loading">Refresh</button>
|
|
||||||
</header>
|
</header>
|
||||||
|
|
||||||
<div aria-live="polite" aria-atomic="true" class="sr-announce">
|
<!-- Tab bar -->
|
||||||
<span v-if="loading">Loading nodes...</span>
|
<nav class="tab-bar" role="tablist" aria-label="Fleet sections">
|
||||||
</div>
|
<button
|
||||||
<div v-if="error" class="nodes-status nodes-error" role="alert">{{ error }}</div>
|
id="tab-nodes"
|
||||||
<div v-else-if="!loading && nodes.length === 0" class="nodes-status">
|
role="tab"
|
||||||
No nodes found. Check <code>coordinator_url</code> in config.
|
:aria-selected="activeTab === 'nodes'"
|
||||||
</div>
|
:class="['tab', { active: activeTab === 'nodes' }]"
|
||||||
<div v-else-if="!loading" class="nodes-grid">
|
@click="activeTab = 'nodes'"
|
||||||
<NodeCard
|
>Nodes</button>
|
||||||
v-for="node in nodes"
|
<button
|
||||||
:key="node.node_id"
|
id="tab-assignments"
|
||||||
:node="node"
|
role="tab"
|
||||||
@updated="fetchNodes"
|
:aria-selected="activeTab === 'assignments'"
|
||||||
/>
|
:class="['tab', { active: activeTab === 'assignments' }]"
|
||||||
</div>
|
@click="activeTab = 'assignments'"
|
||||||
|
>Assignments</button>
|
||||||
|
</nav>
|
||||||
|
|
||||||
|
<!-- Nodes tab -->
|
||||||
|
<section
|
||||||
|
v-if="activeTab === 'nodes'"
|
||||||
|
role="tabpanel"
|
||||||
|
aria-labelledby="tab-nodes"
|
||||||
|
class="tab-panel"
|
||||||
|
>
|
||||||
|
<div class="nodes-toolbar">
|
||||||
|
<button class="btn-secondary btn-sm" @click="fetchNodes" :disabled="loading">Refresh</button>
|
||||||
|
</div>
|
||||||
|
<div aria-live="polite" aria-atomic="true" class="sr-announce">
|
||||||
|
<span v-if="loading">Loading nodes...</span>
|
||||||
|
</div>
|
||||||
|
<div v-if="error" class="nodes-status nodes-error" role="alert">{{ error }}</div>
|
||||||
|
<div v-else-if="!loading && nodes.length === 0" class="nodes-status">
|
||||||
|
No nodes found. Check <code>coordinator_url</code> in config.
|
||||||
|
</div>
|
||||||
|
<div v-else-if="!loading" class="nodes-grid">
|
||||||
|
<NodeCard
|
||||||
|
v-for="node in nodes"
|
||||||
|
:key="node.node_id"
|
||||||
|
:node="node"
|
||||||
|
@updated="fetchNodes"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<!-- Assignments tab -->
|
||||||
|
<section
|
||||||
|
v-else-if="activeTab === 'assignments'"
|
||||||
|
role="tabpanel"
|
||||||
|
aria-labelledby="tab-assignments"
|
||||||
|
class="tab-panel"
|
||||||
|
>
|
||||||
|
<AssignmentsTab />
|
||||||
|
</section>
|
||||||
</main>
|
</main>
|
||||||
</template>
|
</template>
|
||||||
|
|
||||||
<style scoped>
|
<style scoped>
|
||||||
.nodes-page { padding: 1.5rem; }
|
.fleet-page { padding: 1.5rem; }
|
||||||
.nodes-header {
|
|
||||||
display: flex;
|
.fleet-header {
|
||||||
align-items: center;
|
margin-bottom: 1rem;
|
||||||
justify-content: space-between;
|
|
||||||
margin-bottom: 1.5rem;
|
|
||||||
}
|
}
|
||||||
.nodes-header h1 { margin: 0; font-size: 1.5rem; }
|
.fleet-title {
|
||||||
|
margin: 0;
|
||||||
|
font-size: 1.5rem;
|
||||||
|
color: var(--color-text);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Tab bar ── */
|
||||||
|
.tab-bar {
|
||||||
|
display: flex;
|
||||||
|
gap: 0;
|
||||||
|
border-bottom: 2px solid var(--color-border);
|
||||||
|
margin-bottom: 1.25rem;
|
||||||
|
}
|
||||||
|
.tab {
|
||||||
|
padding: 0.55rem 1.1rem;
|
||||||
|
font-size: 0.88rem;
|
||||||
|
font-weight: 600;
|
||||||
|
background: none;
|
||||||
|
border: none;
|
||||||
|
border-bottom: 2px solid transparent;
|
||||||
|
margin-bottom: -2px;
|
||||||
|
cursor: pointer;
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
transition: color 0.15s, border-color 0.15s;
|
||||||
|
}
|
||||||
|
.tab:hover { color: var(--color-text); }
|
||||||
|
.tab.active {
|
||||||
|
color: var(--app-primary);
|
||||||
|
border-bottom-color: var(--app-primary);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Tab panel ── */
|
||||||
|
.tab-panel { min-height: 200px; }
|
||||||
|
|
||||||
|
/* ── Nodes toolbar ── */
|
||||||
|
.nodes-toolbar {
|
||||||
|
display: flex;
|
||||||
|
justify-content: flex-end;
|
||||||
|
margin-bottom: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Nodes grid / status ── */
|
||||||
.nodes-grid { display: flex; flex-direction: column; gap: 1.5rem; }
|
.nodes-grid { display: flex; flex-direction: column; gap: 1.5rem; }
|
||||||
.nodes-status {
|
.nodes-status {
|
||||||
color: var(--text-secondary, #888);
|
color: var(--color-text-muted);
|
||||||
padding: 2rem;
|
padding: 2rem;
|
||||||
text-align: center;
|
text-align: center;
|
||||||
}
|
}
|
||||||
.nodes-error { color: var(--color-error, #fc8181); }
|
.nodes-error { color: var(--color-error); }
|
||||||
.sr-announce { min-height: 1.2em; }
|
.sr-announce { min-height: 1.2em; }
|
||||||
|
|
||||||
|
/* ── Shared button ── */
|
||||||
|
.btn-secondary {
|
||||||
|
padding: 0.4rem 0.9rem;
|
||||||
|
background: var(--color-surface-alt);
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: 0.4rem;
|
||||||
|
font-size: 0.85rem;
|
||||||
|
color: var(--color-text);
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background 0.15s;
|
||||||
|
}
|
||||||
|
.btn-secondary:hover:not(:disabled) { background: var(--color-surface-raised); }
|
||||||
|
.btn-secondary:disabled { opacity: 0.5; cursor: default; }
|
||||||
|
.btn-sm { padding: 0.3rem 0.65rem; font-size: 0.8rem; }
|
||||||
</style>
|
</style>
|
||||||
|
|
|
||||||
536
web/src/views/RecipeScanView.vue
Normal file
536
web/src/views/RecipeScanView.vue
Normal file
|
|
@ -0,0 +1,536 @@
|
||||||
|
<template>
|
||||||
|
<div class="rsv">
|
||||||
|
<!-- Header -->
|
||||||
|
<header class="rsv-header">
|
||||||
|
<h1 class="rsv-title">Recipe Scan Review</h1>
|
||||||
|
<div class="rsv-stats" v-if="stats">
|
||||||
|
<span class="stat-chip">{{ stats.by_status?.pending ?? 0 }} pending</span>
|
||||||
|
<span class="stat-chip stat-chip--ok">{{ stats.by_status?.approved ?? 0 }} approved</span>
|
||||||
|
<span class="stat-chip stat-chip--edited">{{ stats.by_status?.edited ?? 0 }} edited</span>
|
||||||
|
<span class="stat-chip stat-chip--bad">{{ stats.by_status?.rejected ?? 0 }} rejected</span>
|
||||||
|
<a
|
||||||
|
v-if="(stats.export_ready ?? 0) > 0"
|
||||||
|
:href="`${apiBase}/api/recipe-scan/export`"
|
||||||
|
download
|
||||||
|
class="btn-export"
|
||||||
|
>
|
||||||
|
⬇ Export {{ stats.export_ready }} pairs
|
||||||
|
</a>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<!-- Loading -->
|
||||||
|
<div v-if="loading" class="rsv-state" aria-label="Loading">
|
||||||
|
<div class="skeleton-block" />
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Error -->
|
||||||
|
<div v-else-if="apiError" class="rsv-state rsv-error" role="alert">
|
||||||
|
<p>{{ apiError }}</p>
|
||||||
|
<button class="btn-action" @click="fetchNext">Retry</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Queue empty -->
|
||||||
|
<div v-else-if="!item" class="rsv-state rsv-empty">
|
||||||
|
<p>Queue is empty — all items reviewed.</p>
|
||||||
|
<p class="rsv-hint">Import items from the Kiwi pipeline to continue.</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Review panel -->
|
||||||
|
<div v-else class="rsv-workspace">
|
||||||
|
<!-- Left: image -->
|
||||||
|
<section class="rsv-image-panel" aria-label="Scan image">
|
||||||
|
<div class="rsv-panel-label">
|
||||||
|
<span class="modality-badge">{{ item.modality }}</span>
|
||||||
|
<span class="source-badge">{{ item.source }}</span>
|
||||||
|
</div>
|
||||||
|
<div class="rsv-image-wrap">
|
||||||
|
<img
|
||||||
|
v-if="imageUrl"
|
||||||
|
:src="imageUrl"
|
||||||
|
:alt="`Recipe scan — ${item.source}`"
|
||||||
|
class="rsv-image"
|
||||||
|
/>
|
||||||
|
<div v-else class="rsv-image-placeholder">
|
||||||
|
<span>Image not available</span>
|
||||||
|
<code class="rsv-path">{{ item.image_path }}</code>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<!-- Right: JSON comparison -->
|
||||||
|
<section class="rsv-json-panel" aria-label="Extraction review">
|
||||||
|
|
||||||
|
<!-- Ground truth (read-only reference) -->
|
||||||
|
<div class="rsv-json-block">
|
||||||
|
<h2 class="rsv-json-label">Ground truth <span class="label-tag">reference</span></h2>
|
||||||
|
<pre class="rsv-json rsv-json--ground-truth" tabindex="0" aria-label="Ground truth JSON">{{ prettyJson(item.ground_truth) }}</pre>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Extracted / editable -->
|
||||||
|
<div class="rsv-json-block">
|
||||||
|
<h2 class="rsv-json-label">
|
||||||
|
Extracted
|
||||||
|
<span class="label-tag label-tag--edit">edit before approving</span>
|
||||||
|
</h2>
|
||||||
|
<textarea
|
||||||
|
v-model="draftJson"
|
||||||
|
class="rsv-json rsv-json--edit"
|
||||||
|
spellcheck="false"
|
||||||
|
aria-label="Extracted JSON — edit to correct"
|
||||||
|
:class="{ 'rsv-json--invalid': jsonError }"
|
||||||
|
/>
|
||||||
|
<p v-if="jsonError" class="rsv-json-error" role="alert">{{ jsonError }}</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Actions -->
|
||||||
|
<div class="rsv-actions" role="group" aria-label="Review actions">
|
||||||
|
<button
|
||||||
|
class="btn-approve"
|
||||||
|
:disabled="acting"
|
||||||
|
@click="handleApprove"
|
||||||
|
title="Extracted JSON is accurate — approve as-is (A)"
|
||||||
|
>
|
||||||
|
✓ Approve
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
class="btn-edit"
|
||||||
|
:disabled="acting || !!jsonError"
|
||||||
|
@click="handleEdit"
|
||||||
|
title="Approve the edited JSON in the text area (E)"
|
||||||
|
>
|
||||||
|
✎ Approve edited
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
class="btn-reject"
|
||||||
|
:disabled="acting"
|
||||||
|
@click="handleReject"
|
||||||
|
title="Extraction too broken to use — reject (R)"
|
||||||
|
>
|
||||||
|
✕ Reject
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
</section>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Feedback toast -->
|
||||||
|
<Transition name="toast">
|
||||||
|
<div v-if="toast" class="rsv-toast" role="status" aria-live="polite">
|
||||||
|
{{ toast }}
|
||||||
|
</div>
|
||||||
|
</Transition>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<script setup lang="ts">
|
||||||
|
import { ref, computed, watch, onMounted, onUnmounted } from 'vue'
|
||||||
|
|
||||||
|
const apiBase = window.location.origin
|
||||||
|
|
||||||
|
interface RecipeScanItem {
|
||||||
|
id: string
|
||||||
|
image_path: string
|
||||||
|
modality: string
|
||||||
|
source: string
|
||||||
|
extracted: Record<string, unknown>
|
||||||
|
ground_truth: Record<string, unknown>
|
||||||
|
status: string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface Stats {
|
||||||
|
total: number
|
||||||
|
by_status: Record<string, number>
|
||||||
|
by_modality: Record<string, number>
|
||||||
|
export_ready: number
|
||||||
|
}
|
||||||
|
|
||||||
|
const item = ref<RecipeScanItem | null>(null)
|
||||||
|
const stats = ref<Stats | null>(null)
|
||||||
|
const loading = ref(true)
|
||||||
|
const acting = ref(false)
|
||||||
|
const apiError = ref('')
|
||||||
|
const draftJson = ref('')
|
||||||
|
const toast = ref('')
|
||||||
|
let toastTimer: ReturnType<typeof setTimeout> | null = null
|
||||||
|
|
||||||
|
const jsonError = computed(() => {
|
||||||
|
if (!draftJson.value.trim()) return ''
|
||||||
|
try {
|
||||||
|
JSON.parse(draftJson.value)
|
||||||
|
return ''
|
||||||
|
} catch (e) {
|
||||||
|
return 'Invalid JSON — fix before approving'
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
const imageUrl = computed(() => {
|
||||||
|
if (!item.value) return ''
|
||||||
|
const encoded = encodeURIComponent(item.value.image_path)
|
||||||
|
return `${apiBase}/api/recipe-scan/image?path=${encoded}`
|
||||||
|
})
|
||||||
|
|
||||||
|
function prettyJson(obj: unknown): string {
|
||||||
|
return JSON.stringify(obj, null, 2)
|
||||||
|
}
|
||||||
|
|
||||||
|
function showToast(msg: string) {
|
||||||
|
toast.value = msg
|
||||||
|
if (toastTimer) clearTimeout(toastTimer)
|
||||||
|
toastTimer = setTimeout(() => { toast.value = '' }, 2500)
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchNext() {
|
||||||
|
loading.value = true
|
||||||
|
apiError.value = ''
|
||||||
|
try {
|
||||||
|
const r = await fetch(`${apiBase}/api/recipe-scan/next`)
|
||||||
|
if (r.status === 404) {
|
||||||
|
item.value = null
|
||||||
|
} else if (!r.ok) {
|
||||||
|
throw new Error(`API error ${r.status}`)
|
||||||
|
} else {
|
||||||
|
item.value = await r.json()
|
||||||
|
draftJson.value = prettyJson(item.value!.extracted)
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
apiError.value = e instanceof Error ? e.message : 'Could not reach API'
|
||||||
|
} finally {
|
||||||
|
loading.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchStats() {
|
||||||
|
try {
|
||||||
|
const r = await fetch(`${apiBase}/api/recipe-scan/stats`)
|
||||||
|
if (r.ok) stats.value = await r.json()
|
||||||
|
} catch { /* non-critical */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
async function act(endpoint: string, body?: unknown) {
|
||||||
|
if (!item.value || acting.value) return
|
||||||
|
acting.value = true
|
||||||
|
try {
|
||||||
|
const r = await fetch(`${apiBase}/api/recipe-scan/items/${item.value.id}/${endpoint}`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: body ? { 'Content-Type': 'application/json' } : {},
|
||||||
|
body: body ? JSON.stringify(body) : undefined,
|
||||||
|
})
|
||||||
|
if (!r.ok) throw new Error(`API error ${r.status}`)
|
||||||
|
} catch (e) {
|
||||||
|
showToast(e instanceof Error ? e.message : 'Action failed')
|
||||||
|
acting.value = false
|
||||||
|
return
|
||||||
|
}
|
||||||
|
acting.value = false
|
||||||
|
await Promise.all([fetchNext(), fetchStats()])
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleApprove() {
|
||||||
|
showToast('Approved')
|
||||||
|
await act('approve')
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleEdit() {
|
||||||
|
if (jsonError.value) return
|
||||||
|
let corrected: unknown
|
||||||
|
try {
|
||||||
|
corrected = JSON.parse(draftJson.value)
|
||||||
|
} catch {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
showToast('Saved edit')
|
||||||
|
await act('edit', { corrected })
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleReject() {
|
||||||
|
showToast('Rejected')
|
||||||
|
await act('reject')
|
||||||
|
}
|
||||||
|
|
||||||
|
// Keyboard shortcuts: A = approve, E = edit+approve, R = reject
|
||||||
|
function handleKey(e: KeyboardEvent) {
|
||||||
|
const tag = (e.target as HTMLElement)?.tagName?.toLowerCase()
|
||||||
|
if (tag === 'textarea' || tag === 'input') return
|
||||||
|
if (e.key === 'a' || e.key === 'A') handleApprove()
|
||||||
|
if (e.key === 'e' || e.key === 'E') handleEdit()
|
||||||
|
if (e.key === 'r' || e.key === 'R') handleReject()
|
||||||
|
}
|
||||||
|
|
||||||
|
watch(item, (newItem) => {
|
||||||
|
if (newItem) draftJson.value = prettyJson(newItem.extracted)
|
||||||
|
})
|
||||||
|
|
||||||
|
onMounted(() => {
|
||||||
|
fetchNext()
|
||||||
|
fetchStats()
|
||||||
|
window.addEventListener('keydown', handleKey)
|
||||||
|
})
|
||||||
|
|
||||||
|
onUnmounted(() => {
|
||||||
|
window.removeEventListener('keydown', handleKey)
|
||||||
|
if (toastTimer) clearTimeout(toastTimer)
|
||||||
|
})
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<style scoped>
|
||||||
|
.rsv {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
height: 100%;
|
||||||
|
padding: var(--space-md, 1rem);
|
||||||
|
gap: var(--space-md, 1rem);
|
||||||
|
box-sizing: border-box;
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Header */
|
||||||
|
.rsv-header {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: var(--space-md, 1rem);
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
.rsv-title {
|
||||||
|
font-size: 1.1rem;
|
||||||
|
font-weight: 600;
|
||||||
|
margin: 0;
|
||||||
|
color: var(--color-text, #fff);
|
||||||
|
}
|
||||||
|
.rsv-stats {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
.stat-chip {
|
||||||
|
font-size: 0.75rem;
|
||||||
|
padding: 2px 8px;
|
||||||
|
border-radius: 12px;
|
||||||
|
background: var(--color-surface-alt, #2a2a2a);
|
||||||
|
color: var(--color-text-muted, #aaa);
|
||||||
|
}
|
||||||
|
.stat-chip--ok { background: #1a3a1a; color: #6fcf97; }
|
||||||
|
.stat-chip--edited { background: #2a2a00; color: #f2c94c; }
|
||||||
|
.stat-chip--bad { background: #3a1a1a; color: #eb5757; }
|
||||||
|
.btn-export {
|
||||||
|
font-size: 0.8rem;
|
||||||
|
padding: 4px 12px;
|
||||||
|
border-radius: 6px;
|
||||||
|
background: var(--color-accent, #4a9eff);
|
||||||
|
color: #fff;
|
||||||
|
text-decoration: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* State panels */
|
||||||
|
.rsv-state {
|
||||||
|
flex: 1;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
color: var(--color-text-muted, #aaa);
|
||||||
|
}
|
||||||
|
.rsv-error { color: var(--color-danger, #eb5757); }
|
||||||
|
.rsv-empty { font-size: 1rem; }
|
||||||
|
.rsv-hint { font-size: 0.85rem; opacity: 0.7; margin: 0; }
|
||||||
|
.skeleton-block {
|
||||||
|
width: 100%; height: 300px;
|
||||||
|
border-radius: 8px;
|
||||||
|
background: var(--color-surface-alt, #2a2a2a);
|
||||||
|
animation: pulse 1.5s ease-in-out infinite;
|
||||||
|
}
|
||||||
|
@keyframes pulse { 0%, 100% { opacity: 1; } 50% { opacity: 0.5; } }
|
||||||
|
|
||||||
|
/* Workspace: two-column layout */
|
||||||
|
.rsv-workspace {
|
||||||
|
flex: 1;
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 1fr 1fr;
|
||||||
|
gap: var(--space-md, 1rem);
|
||||||
|
min-height: 0;
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
@media (max-width: 900px) {
|
||||||
|
.rsv-workspace {
|
||||||
|
grid-template-columns: 1fr;
|
||||||
|
overflow-y: auto;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Image panel */
|
||||||
|
.rsv-image-panel {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.5rem;
|
||||||
|
min-height: 0;
|
||||||
|
}
|
||||||
|
.rsv-panel-label {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
.modality-badge, .source-badge {
|
||||||
|
font-size: 0.72rem;
|
||||||
|
padding: 2px 8px;
|
||||||
|
border-radius: 10px;
|
||||||
|
background: var(--color-surface-alt, #2a2a2a);
|
||||||
|
color: var(--color-text-muted, #aaa);
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.04em;
|
||||||
|
}
|
||||||
|
.rsv-image-wrap {
|
||||||
|
flex: 1;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
background: var(--color-surface-alt, #111);
|
||||||
|
border-radius: 8px;
|
||||||
|
overflow: hidden;
|
||||||
|
min-height: 200px;
|
||||||
|
}
|
||||||
|
.rsv-image {
|
||||||
|
max-width: 100%;
|
||||||
|
max-height: 100%;
|
||||||
|
object-fit: contain;
|
||||||
|
}
|
||||||
|
.rsv-image-placeholder {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
color: var(--color-text-muted, #666);
|
||||||
|
font-size: 0.85rem;
|
||||||
|
padding: 1rem;
|
||||||
|
text-align: center;
|
||||||
|
}
|
||||||
|
.rsv-path {
|
||||||
|
font-size: 0.7rem;
|
||||||
|
word-break: break-all;
|
||||||
|
opacity: 0.6;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* JSON panel */
|
||||||
|
.rsv-json-panel {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.5rem;
|
||||||
|
min-height: 0;
|
||||||
|
overflow-y: auto;
|
||||||
|
}
|
||||||
|
.rsv-json-block {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.25rem;
|
||||||
|
flex: 1;
|
||||||
|
min-height: 0;
|
||||||
|
}
|
||||||
|
.rsv-json-label {
|
||||||
|
font-size: 0.8rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: var(--color-text-muted, #aaa);
|
||||||
|
margin: 0;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
.label-tag {
|
||||||
|
font-size: 0.68rem;
|
||||||
|
font-weight: 400;
|
||||||
|
padding: 1px 6px;
|
||||||
|
border-radius: 8px;
|
||||||
|
background: var(--color-surface-alt, #2a2a2a);
|
||||||
|
color: var(--color-text-muted, #888);
|
||||||
|
}
|
||||||
|
.label-tag--edit {
|
||||||
|
background: #2a2a00;
|
||||||
|
color: #f2c94c;
|
||||||
|
}
|
||||||
|
.rsv-json {
|
||||||
|
font-family: var(--font-mono, monospace);
|
||||||
|
font-size: 0.75rem;
|
||||||
|
line-height: 1.5;
|
||||||
|
padding: 0.75rem;
|
||||||
|
border-radius: 6px;
|
||||||
|
min-height: 120px;
|
||||||
|
flex: 1;
|
||||||
|
overflow-y: auto;
|
||||||
|
resize: vertical;
|
||||||
|
white-space: pre;
|
||||||
|
}
|
||||||
|
.rsv-json--ground-truth {
|
||||||
|
background: var(--color-surface-alt, #111);
|
||||||
|
color: var(--color-text, #ccc);
|
||||||
|
border: 1px solid var(--color-border, #333);
|
||||||
|
}
|
||||||
|
.rsv-json--edit {
|
||||||
|
background: var(--color-surface, #1a1a1a);
|
||||||
|
color: var(--color-text, #e0e0e0);
|
||||||
|
border: 1px solid var(--color-border, #444);
|
||||||
|
caret-color: var(--color-accent, #4a9eff);
|
||||||
|
outline: none;
|
||||||
|
transition: border-color 0.15s;
|
||||||
|
}
|
||||||
|
.rsv-json--edit:focus {
|
||||||
|
border-color: var(--color-accent, #4a9eff);
|
||||||
|
}
|
||||||
|
.rsv-json--invalid {
|
||||||
|
border-color: var(--color-danger, #eb5757) !important;
|
||||||
|
}
|
||||||
|
.rsv-json-error {
|
||||||
|
font-size: 0.75rem;
|
||||||
|
color: var(--color-danger, #eb5757);
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Action buttons */
|
||||||
|
.rsv-actions {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.5rem;
|
||||||
|
padding-top: 0.25rem;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
.btn-approve, .btn-edit, .btn-reject {
|
||||||
|
flex: 1;
|
||||||
|
min-width: 80px;
|
||||||
|
padding: 0.5rem 0.75rem;
|
||||||
|
border: none;
|
||||||
|
border-radius: 6px;
|
||||||
|
font-size: 0.85rem;
|
||||||
|
font-weight: 600;
|
||||||
|
cursor: pointer;
|
||||||
|
transition: opacity 0.15s;
|
||||||
|
}
|
||||||
|
.btn-approve, .btn-edit, .btn-reject {
|
||||||
|
opacity: 1;
|
||||||
|
}
|
||||||
|
.btn-approve:disabled, .btn-edit:disabled, .btn-reject:disabled {
|
||||||
|
opacity: 0.4;
|
||||||
|
cursor: default;
|
||||||
|
}
|
||||||
|
.btn-approve { background: #1e6e1e; color: #6fcf97; }
|
||||||
|
.btn-approve:hover:not(:disabled) { background: #256325; }
|
||||||
|
.btn-edit { background: #4a4a00; color: #f2c94c; }
|
||||||
|
.btn-edit:hover:not(:disabled) { background: #606000; }
|
||||||
|
.btn-reject { background: #6e1e1e; color: #eb8f8f; }
|
||||||
|
.btn-reject:hover:not(:disabled) { background: #7a2222; }
|
||||||
|
|
||||||
|
/* Toast */
|
||||||
|
.rsv-toast {
|
||||||
|
position: fixed;
|
||||||
|
bottom: 1.5rem;
|
||||||
|
left: 50%;
|
||||||
|
transform: translateX(-50%);
|
||||||
|
background: var(--color-surface, #222);
|
||||||
|
color: var(--color-text, #fff);
|
||||||
|
border: 1px solid var(--color-border, #444);
|
||||||
|
border-radius: 8px;
|
||||||
|
padding: 0.5rem 1.25rem;
|
||||||
|
font-size: 0.85rem;
|
||||||
|
box-shadow: 0 4px 20px rgba(0,0,0,0.4);
|
||||||
|
pointer-events: none;
|
||||||
|
z-index: 100;
|
||||||
|
}
|
||||||
|
.toast-enter-active, .toast-leave-active { transition: opacity 0.2s, transform 0.2s; }
|
||||||
|
.toast-enter-from, .toast-leave-to { opacity: 0; transform: translateX(-50%) translateY(8px); }
|
||||||
|
</style>
|
||||||
Loading…
Reference in a new issue