Imitation pipeline: cover letter generation #28

New issue

Open

opened 2026-04-10 22:38:04 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-04-10 22:38:04 -07:00

Owner

Context: Peregrine generates personalized cover letters via LLM as its primary AI feature. A fine-tuned model already exists (alex-cover-writer:latest) but was trained on one user's 62 letters. Avocet is positioned to collect multi-user training data at scale for a generalizable replacement.

What Peregrine uses this for:
Given a job title, company name, and JD excerpt, the model writes a full 3-4 paragraph cover letter in the candidate's established voice. The system prompt injects the candidate's name, career summary, and personality/voice descriptor. Three past cover letters retrieved by TF-IDF cosine similarity are included as few-shot style examples. An optional mission-alignment hint is injected for Para 3 when the company matches a preferred industry (music, animal welfare, education, social impact, healthcare). Recruiter-context framing is injected for Jobgether listings. A refinement path accepts a prior draft plus free-text feedback and asks the model to revise.

Input/output schema:

Input: system context (candidate name + career summary + voice descriptor) + up to 3 style example letters (full text) + optional mission alignment note + optional recruiter context note + role/company/JD excerpt (max 1500 chars) + optional prior draft + optional feedback
Output: full cover letter text ending with a closing salutation and candidate first name; max_tokens=1200 cap; post-processed to trim at first sign-off

Current model/fallback chain:
claude_code → ollama (alex-cover-writer:latest) → vllm → copilot → anthropic
alex-cover-writer:latest is the live fine-tuned model (Llama-3.2-3B-Instruct, QLoRA rank 16, 10 epochs, 62-letter corpus).

Recommended model domain:
Instruction-following 3B-7B; generation task with strong style-imitation requirement. Voice-consistency is the primary quality axis, not factual accuracy. A 3B model fine-tuned on diverse user corpora should outperform a zero-shot 7B.

Can Avocet produce training data for it?
Yes — directly. Avocet's label tool can be extended with a cover letter review card type. Existing Peregrine outputs (user-approved letters that were actually submitted) are high-quality silver labels. The input prompt structure is deterministic and already logged.

Suggested data collection approach:

Export accepted cover letters from Peregrine's staging.db (cover_letter column on applied jobs) as silver-label JSONL using the existing scripts/prepare_training_data.py pattern
Build an Avocet card type that presents the prompt (JD + style examples) alongside the generated letter and asks the labeler to rate voice-match, relevance, and length on a simple rubric (1-5 each)
Human-labeled preference pairs (chosen/rejected) enable DPO (direct preference optimization) fine-tuning in addition to supervised fine-tune
Collect from multiple Peregrine users to generalize beyond a single candidate voice

Related: circuitforge-plans/peregrine/superpowers/; Peregrine scripts/generate_cover_letter.py

**Context:** Peregrine generates personalized cover letters via LLM as its primary AI feature. A fine-tuned model already exists (`alex-cover-writer:latest`) but was trained on one user's 62 letters. Avocet is positioned to collect multi-user training data at scale for a generalizable replacement. **What Peregrine uses this for:** Given a job title, company name, and JD excerpt, the model writes a full 3-4 paragraph cover letter in the candidate's established voice. The system prompt injects the candidate's name, career summary, and personality/voice descriptor. Three past cover letters retrieved by TF-IDF cosine similarity are included as few-shot style examples. An optional mission-alignment hint is injected for Para 3 when the company matches a preferred industry (music, animal welfare, education, social impact, healthcare). Recruiter-context framing is injected for Jobgether listings. A refinement path accepts a prior draft plus free-text feedback and asks the model to revise. **Input/output schema:** - Input: system context (candidate name + career summary + voice descriptor) + up to 3 style example letters (full text) + optional mission alignment note + optional recruiter context note + role/company/JD excerpt (max 1500 chars) + optional prior draft + optional feedback - Output: full cover letter text ending with a closing salutation and candidate first name; `max_tokens=1200` cap; post-processed to trim at first sign-off **Current model/fallback chain:** `claude_code → ollama (alex-cover-writer:latest) → vllm → copilot → anthropic` `alex-cover-writer:latest` is the live fine-tuned model (Llama-3.2-3B-Instruct, QLoRA rank 16, 10 epochs, 62-letter corpus). **Recommended model domain:** Instruction-following 3B-7B; generation task with strong style-imitation requirement. Voice-consistency is the primary quality axis, not factual accuracy. A 3B model fine-tuned on diverse user corpora should outperform a zero-shot 7B. **Can Avocet produce training data for it?** Yes — directly. Avocet's label tool can be extended with a cover letter review card type. Existing Peregrine outputs (user-approved letters that were actually submitted) are high-quality silver labels. The input prompt structure is deterministic and already logged. **Suggested data collection approach:** - Export accepted cover letters from Peregrine's `staging.db` (`cover_letter` column on `applied` jobs) as silver-label JSONL using the existing `scripts/prepare_training_data.py` pattern - Build an Avocet card type that presents the prompt (JD + style examples) alongside the generated letter and asks the labeler to rate voice-match, relevance, and length on a simple rubric (1-5 each) - Human-labeled preference pairs (chosen/rejected) enable DPO (direct preference optimization) fine-tuning in addition to supervised fine-tune - Collect from multiple Peregrine users to generalize beyond a single candidate voice **Related:** `circuitforge-plans/peregrine/superpowers/`; Peregrine `scripts/generate_cover_letter.py`