avocet/CLAUDE.md

# Avocet — Email Classifier Training Tool

## What it is

Shared infrastructure for building and benchmarking email classifiers across the CircuitForge menagerie.
Named for the avocet's sweeping-bill technique — it sweeps through email streams and filters out categories.

**Pipeline:**
```
Scrape (IMAP, wide search, multi-account) → data/email_label_queue.jsonl
                ↓
Label (card-stack UI)                      → data/email_score.jsonl
                ↓
Benchmark (HuggingFace NLI/reranker)       → per-model macro-F1 + latency
```

## Environment

- Python env: `conda run -n job-seeker <cmd>` for basic use (streamlit, yaml, stdlib only)
- Classifier env: `conda run -n job-seeker-classifiers <cmd>` for benchmark (transformers, FlagEmbedding, gliclass)
- Run tests: `/devl/miniconda3/envs/job-seeker/bin/pytest tests/ -v`
  (direct binary — `conda run pytest` can spawn runaway processes)
- Create classifier env: `conda env create -f environment.yml`

## Label Tool (app/label_tool.py)

Card-stack Streamlit UI for manually labeling recruitment emails.

```
conda run -n job-seeker streamlit run app/label_tool.py --server.port 8503
```

- Config: `config/label_tool.yaml` (gitignored — copy from `.example`, or use ⚙️ Settings tab)
- Queue: `data/email_label_queue.jsonl` (gitignored)
- Output: `data/email_score.jsonl` (gitignored)
- Four tabs: 🃏 Label, 📥 Fetch, 📊 Stats, ⚙️ Settings
- Keyboard shortcuts: 1–9 = label, 0 = Other (wildcard, prompts free-text input), S = skip, U = undo
- Dedup: MD5 of `(subject + body[:100])` — cross-account safe

### Settings Tab (⚙️)
- Add / edit / remove IMAP accounts via form UI — no manual YAML editing required
- Per-account fields: display name, host, port, SSL toggle, username, password (masked), folder, days back
- **🔌 Test connection** button per account — connects, logs in, selects folder, reports message count
- Global: max emails per account per fetch
- **💾 Save** writes `config/label_tool.yaml`; **↩ Reload** discards unsaved changes
- `_sync_settings_to_state()` collects widget values before any add/remove to avoid index-key drift

## Benchmark (scripts/benchmark_classifier.py)

```
# List available models
conda run -n job-seeker-classifiers python scripts/benchmark_classifier.py --list-models

# Score against labeled JSONL
conda run -n job-seeker-classifiers python scripts/benchmark_classifier.py --score

# Visual comparison on live IMAP emails
conda run -n job-seeker-classifiers python scripts/benchmark_classifier.py --compare --limit 20

# Include slow/large models
conda run -n job-seeker-classifiers python scripts/benchmark_classifier.py --score --include-slow

# Export DB-labeled emails (⚠️ LLM-generated labels — review first)
conda run -n job-seeker-classifiers python scripts/benchmark_classifier.py --export-db --db /path/to/staging.db
```

## Labels (peregrine defaults — configurable per product)

| Label | Key | Meaning |
|-------|-----|---------|
| `interview_scheduled` | 1 | Phone screen, video call, or on-site invitation |
| `offer_received` | 2 | Formal job offer or offer letter |
| `rejected` | 3 | Application declined or not moving forward |
| `positive_response` | 4 | Recruiter interest or request to connect |
| `survey_received` | 5 | Culture-fit survey or assessment invitation |
| `neutral` | 6 | ATS confirmation (application received, etc.) |
| `event_rescheduled` | 7 | Interview or event moved to a new time |
| `digest` | 8 | Job digest or multi-listing email (scrapeable) |
| `new_lead` | 9 | Unsolicited recruiter outreach or cold contact |
| `hired` | h | Offer accepted, onboarding, welcome email, start date |

## Model Registry (13 models, 7 defaults)

See `scripts/benchmark_classifier.py:MODEL_REGISTRY`.
Default models run without `--include-slow`.
Add `--models deberta-small deberta-small-2pass` to test a specific subset.

## Config Files

- `config/label_tool.yaml` — gitignored; multi-account IMAP config
- `config/label_tool.yaml.example` — committed template

## Data Files

- `data/email_score.jsonl` — gitignored; manually-labeled ground truth
- `data/email_score.jsonl.example` — committed sample for CI
- `data/email_label_queue.jsonl` — gitignored; IMAP fetch queue

## Key Design Notes

- `ZeroShotAdapter.load()` instantiates the pipeline object; `classify()` calls the object.
  Tests patch `scripts.classifier_adapters.pipeline` (the module-level factory) with a
  two-level mock: `mock_factory.return_value = MagicMock(return_value={...})`.
- `two_pass=True` on ZeroShotAdapter: first pass ranks all 6 labels; second pass re-runs
  with only top-2, forcing a binary choice. 2× cost, better confidence.
- `--compare` uses the first account in `label_tool.yaml` for live IMAP emails.
- DB export labels are llama3.1:8b-generated — treat as noisy, not gold truth.

## Vue Label UI (app/api.py + web/)

FastAPI on port 8503 serves both the REST API and the built Vue SPA (`web/dist/`).

```
./manage.sh start-api    # build Vue SPA + start FastAPI (binds 0.0.0.0:8503 — LAN accessible)
./manage.sh stop-api
./manage.sh open-api     # xdg-open http://localhost:8503
```

Logs: `log/api.log`

## Email Field Schema — IMPORTANT

Two schemas exist. The normalization layer in `app/api.py` bridges them automatically.

### JSONL on-disk schema (written by `label_tool.py` and `label_tool.py`'s IMAP fetch)

| Field | Type | Notes |
|-------|------|-------|
| `subject` | str | Email subject line |
| `body` | str | Plain-text body, truncated at 800 chars; HTML stripped by `_strip_html()` |
| `from_addr` | str | Sender address string (`"Name <addr>"`) |
| `date` | str | Raw RFC 2822 date string |
| `account` | str | Display name of the IMAP account that fetched it |
| *(no `id`)* | — | Dedup key is MD5 of `(subject + body[:100])` — never stored on disk |

### Vue API schema (returned by `GET /api/queue`, required by POST endpoints)

| Field | Type | Notes |
|-------|------|-------|
| `id` | str | MD5 content hash, or stored `id` if item has one |
| `subject` | str | Unchanged |
| `body` | str | Unchanged |
| `from` | str | Mapped from `from_addr` (or `from` if already present) |
| `date` | str | Unchanged |
| `source` | str | Mapped from `account` (or `source` if already present) |

### Normalization layer (`_normalize()` in `app/api.py`)

`_normalize(item)` handles the mapping and ID generation. All `GET /api/queue` responses
pass through it. Mutating endpoints (`/api/label`, `/api/skip`, `/api/discard`) look up
items via `_normalize(x)["id"]`, so both real data (no `id`, uses content hash) and test
fixtures (explicit `id` field) work transparently.

### Peregrine integration

Peregrine's `staging.db` uses different field names again:

| staging.db column | Maps to avocet JSONL field |
|-------------------|---------------------------|
| `subject` | `subject` |
| `body` | `body` (may contain HTML — run through `_strip_html()` before queuing) |
| `from_address` | `from_addr` |
| `received_date` | `date` |
| `account` or source context | `account` |

When exporting from Peregrine's DB for avocet labeling, transform to the JSONL schema above
(not the Vue API schema). The `--export-db` flag in `benchmark_classifier.py` does this.
Any new export path should also call `_strip_html()` on the body before writing.

## Relationship to Peregrine

Avocet started as `peregrine/tools/label_tool.py` + `peregrine/scripts/classifier_adapters.py`.
Peregrine retains copies during stabilization; once avocet is proven, peregrine will import from here.