Per-user LLM config: expose in Settings UI + cloud admin tooling #49

New issue

Open

opened 2026-04-02 09:06:34 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-04-02 09:06:34 -07:00

Owner

Background

Per-user llm.yaml overrides are now wired into the cover letter generation path (task_runner.py → generate() picks up <user_data>/peregrine/config/llm.yaml when it exists, falling back to the global cloud config).

Meghan's file was written manually at /devl/menagerie-data/5b99ca9f-b0ef-46fb-aa21-ee8d5a0ebc21/peregrine/config/llm.yaml with meghan-cover-writer:latest as her ollama cover letter model.

Problem

This is a manual operation right now. There is no:

Settings UI for users to select their custom fine-tuned model
Admin tooling to set/inspect per-user LLM config from outside the container
Coverage in company_research.py or other LLM callers (only generate_cover_letter is wired)

Tasks

Extend per-user config wiring to company_research.py (also calls LLMRouter)
Settings UI: "Custom cover letter model" field in Settings → System (Paid tier+); writes to per-user llm.yaml
Admin script or endpoint to set/inspect per-user LLM config without docker exec
Consider: should the model list come from a live ollama /api/tags call so the UI shows available models?
Document the per-user llm.yaml override pattern in CLAUDE.md

meghan-cover-writer:latest is the fine-tuned Llama 3.2 3B model for Meghan's cover letters
Global cloud default is llama3.1:8b (no fine-tunes for anonymous users)

## Background Per-user `llm.yaml` overrides are now wired into the cover letter generation path (`task_runner.py` → `generate()` picks up `<user_data>/peregrine/config/llm.yaml` when it exists, falling back to the global cloud config). Meghan's file was written manually at `/devl/menagerie-data/5b99ca9f-b0ef-46fb-aa21-ee8d5a0ebc21/peregrine/config/llm.yaml` with `meghan-cover-writer:latest` as her ollama cover letter model. ## Problem This is a manual operation right now. There is no: - Settings UI for users to select their custom fine-tuned model - Admin tooling to set/inspect per-user LLM config from outside the container - Coverage in `company_research.py` or other LLM callers (only `generate_cover_letter` is wired) ## Tasks - [ ] Extend per-user config wiring to `company_research.py` (also calls LLMRouter) - [ ] Settings UI: "Custom cover letter model" field in Settings → System (Paid tier+); writes to per-user `llm.yaml` - [ ] Admin script or endpoint to set/inspect per-user LLM config without docker exec - [ ] Consider: should the model list come from a live ollama `/api/tags` call so the UI shows available models? - [ ] Document the per-user `llm.yaml` override pattern in CLAUDE.md ## Related - `meghan-cover-writer:latest` is the fine-tuned Llama 3.2 3B model for Meghan's cover letters - Global cloud default is `llama3.1:8b` (no fine-tunes for anonymous users)