Commit graph

58 commits

Author SHA1 Message Date
4d66c04d1e feat: containerize fine-tune pipeline (Dockerfile.finetune + make finetune)
- Dockerfile.finetune: PyTorch 2.3/CUDA 12.1 base + unsloth + training stack
- finetune_local.py: auto-register model via Ollama HTTP API after GGUF
  export; path-translate between finetune container mount and Ollama's view;
  update config/llm.yaml automatically; DOCS_DIR env override for Docker
- prepare_training_data.py: DOCS_DIR env override so make prepare-training
  works correctly inside the app container
- compose.yml: add finetune service (cpu/single-gpu/dual-gpu profiles);
  DOCS_DIR=/docs injected into app + finetune containers
- compose.podman-gpu.yml: CDI device override for finetune service
- Makefile: make prepare-training + make finetune targets
2026-02-25 16:22:48 -08:00
6c895b5a9b feat: prompt for model weights directory during install
Interactive prompt lets users with split-drive setups point Ollama and
vLLM model dirs at a dedicated storage drive. Reads current .env value
as default so re-runs are idempotent. Skips prompts in non-interactive
(piped) mode. Creates the target directory immediately and updates .env
in-place via portable awk (Linux + macOS). Also simplifies next-steps
output since model paths are now configured at install time.
2026-02-25 16:08:14 -08:00
1bcbff395d fix: repair beta installer path for Docker-first deployment
- llm.yaml + example: replace localhost URLs with Docker service names
  (ollama:11434, vllm:8000, vision:8002); replace personal model names
  (meghan-cover-writer, llama3.1:8b) with llama3.2:3b
- user.yaml.example: update service hosts to Docker names (ollama, vllm,
  searxng) and searxng port from 8888 (host-mapped) to 8080 (internal)
- wizard step 5: fix hardcoded localhost defaults — wizard runs inside
  Docker, so service name defaults are required for connection tests to pass
- scrapers/companyScraper.py: bundle scraper so Dockerfile COPY succeeds
- setup.sh: remove host Ollama install (conflicts with Docker Ollama on
  port 11434); Docker entrypoint handles model download automatically
- README + setup.sh banner: add Circuit Forge mission statement
2026-02-25 16:03:10 -08:00
f55f7b78fc feat: add Ollama install + service start + model pull to setup.sh 2026-02-25 15:42:56 -08:00
9a4db4e18c feat: Podman support — auto-detect COMPOSE, CDI GPU override, podman-compose in setup.sh 2026-02-25 15:36:36 -08:00
71e30be9b9 docs: fix license server paths — dev under CircuitForge/, live at /devl/ 2026-02-25 15:28:32 -08:00
23b0703485 docs: CircuitForge license server implementation plan (11 tasks) 2026-02-25 15:27:39 -08:00
0bc17a1d84 docs: CircuitForge license server design doc
RS256 JWT, FastAPI + SQLite, multi-product schema, offline-capable
client integration. Covers server, Peregrine client, deployment,
admin workflow, and testing strategy.
2026-02-25 15:21:07 -08:00
a7d9bd075a docs: mark cover letter refinement complete in backlog + changelog 2026-02-25 14:44:50 -08:00
94225c95e7 feat: cover letter iterative refinement — feedback UI + backend params
- generate() accepts previous_result + feedback; appends both to LLM prompt
- task_runner cover_letter handler parses params JSON, passes fields through
- Apply Workspace: "Refine with Feedback" expander with text area + Regenerate
  button; only shown when a draft exists; clears feedback after submitting
- 8 new tests (TestGenerateRefinement + TestTaskRunnerCoverLetterParams)
2026-02-25 14:44:20 -08:00
317d9c661a docs: finalise Circuit Forge product suite naming + product brief 2026-02-25 14:16:56 -08:00
1b1192f8be docs: backlog — Circuit Forge product expansion (heinous tasks platform) 2026-02-25 14:02:07 -08:00
ca90b02db9 docs: mark email sync test checklist complete 2026-02-25 13:56:55 -08:00
ad718893ac test: complete email sync test coverage — 44 new tests across all checklist sections 2026-02-25 13:55:55 -08:00
ba7125799a chore: mkdocs deps, CHANGELOG, remove dead Resume Editor page, backlog gap items 2026-02-25 13:51:13 -08:00
70de34be3a docs: LICENSE-MIT + LICENSE-BSL + updated README for 7-step wizard and current feature set 2026-02-25 12:06:28 -08:00
e6410498af docs: mkdocs wiki — installation, user guide, developer guide, reference
Adds a full MkDocs documentation site under docs/ with Material theme.

Getting Started: installation walkthrough, 7-step first-run wizard guide,
Docker Compose profile reference with GPU memory guidance and preflight.py
description.

User Guide: job discovery (search profiles, custom boards, enrichment),
job review (sorting, match scores, batch actions), apply workspace (cover
letter gen, PDF export, mark applied), interviews (kanban stages, company
research auto-trigger, survey assistant), email sync (IMAP, Gmail App
Password, classification labels, stage auto-updates), integrations (all 13
drivers with tier requirements), settings (every tab documented).

Developer Guide: contributing (dev env setup, code style, branch naming, PR
checklist), architecture (ASCII layer diagram, design decisions), adding
scrapers (full scrape() interface, registration, search profile config,
test patterns), adding integrations (IntegrationBase full interface, auto-
discovery, tier gating, test patterns), testing (patterns, fixtures, what
not to test).

Reference: tier system (full FEATURES table, can_use/tier_label API, dev
override, adding gates), LLM router (backend types, complete() signature,
fallback chains, vision routing, __auto__ resolution, adding backends),
config files (every file with field-level docs and gitignore status).

Also adds CONTRIBUTING.md at repo root pointing to the docs site.
2026-02-25 12:05:49 -08:00
83315de52c docs: backlog — Ultra tier managed applications concept 2026-02-25 11:40:55 -08:00
9e4e9e431a feat: Integrations tab in Settings — connect/test/disconnect all 12 integration drivers 2026-02-25 11:30:44 -08:00
a2473c78d4 refactor: move HF token to Developer tab — hidden from standard user UI 2026-02-25 11:04:13 -08:00
92b13c32ff feat: expanded first-run wizard — complete implementation
13-task implementation covering:
- UserProfile wizard fields (wizard_complete, wizard_step, tier, dev_tier_override,
  dismissed_banners, effective_tier) + params column in background_tasks
- Tier system: FEATURES gate, can_use(), tier_label() (app/wizard/tiers.py)
- Six pure validate() step modules (hardware, tier, identity, resume, inference, search)
- Resume parser: PDF (pdfplumber) + DOCX (python-docx) extraction + LLM structuring
- Integration base class + auto-discovery registry (scripts/integrations/)
- 13 integration drivers (Notion, Google Sheets, Airtable, Google Drive, Dropbox,
  OneDrive, MEGA, Nextcloud, Google Calendar, Apple Calendar, Slack, Discord,
  Home Assistant) + config/integrations/*.yaml.example files
- wizard_generate task type with 8 LLM generation sections + iterative refinement
  (previous_result + feedback support)
- step_integrations module: validate(), get_available(), is_connected()
- Wizard orchestrator rewrite (0_Setup.py): 7 steps, crash recovery, LLM polling
- app.py gate: checks wizard_complete flag in addition to file existence
- Home page: 13 dismissible contextual setup banners (wizard_complete-gated)
- Settings: Developer tab — tier override selectbox + wizard reset button

219 tests passing.
2026-02-25 10:54:24 -08:00
d752139c88 feat: Developer tab in Settings — tier override + wizard reset button 2026-02-25 10:50:14 -08:00
cbd2cd9c78 feat: dismissible setup banners on Home page (13 contextual prompts) 2026-02-25 09:53:34 -08:00
692c68d024 feat: app.py checks wizard_complete flag to gate main app 2026-02-25 09:43:53 -08:00
3eb08dcb92 feat: wizard orchestrator — 7 steps, LLM generation polling, crash recovery
Replaces the old 5-step wizard with a 7-step orchestrator that uses the
step modules built in Tasks 2-8. Steps 1-6 are mandatory (hardware, tier,
identity, resume, inference, search); step 7 (integrations) is optional.
Each Next click validates, writes wizard_step to user.yaml for crash recovery,
and resumes at the correct step on page reload. LLM generation buttons
submit wizard_generate tasks and poll via @st.fragment(run_every=3). Finish
sets wizard_complete=True, removes wizard_step, and calls apply_service_urls.

Adds tests/test_wizard_flow.py (7 tests) covering validate() chain, yaml
persistence helpers, and wizard state inference.
2026-02-25 09:10:51 -08:00
99ae1263d8 feat: step_integrations module with validate() + tier-filtered available list 2026-02-25 08:35:16 -08:00
572bb17d2f docs: backlog — cover letter iterative refinement feedback loop 2026-02-25 08:30:24 -08:00
cfbd065bc0 feat: wizard_generate — feedback + previous_result support for iterative refinement 2026-02-25 08:29:56 -08:00
02ba54ea7a feat: wizard_generate task type — 8 LLM generation sections 2026-02-25 08:25:17 -08:00
442531a13c docs: backlog — Podman support + FastAPI migration path 2026-02-25 08:22:24 -08:00
4114abb761 feat: 13 integration implementations + config examples
Add all 13 integration modules (Notion, Google Drive, Google Sheets,
Airtable, Dropbox, OneDrive, MEGA, Nextcloud, Google Calendar, Apple
Calendar/CalDAV, Slack, Discord, Home Assistant) with fields(), connect(),
and test() implementations. Add config/integrations/*.yaml.example files
and gitignore rules for live config files. Add 5 new registry/schema
tests bringing total to 193 passing.
2026-02-25 08:18:45 -08:00
c87447e017 feat: integration base class + auto-discovery registry 2026-02-25 08:13:14 -08:00
9258a91fd6 feat: resume parser — PDF/DOCX extraction + LLM structuring 2026-02-25 08:04:48 -08:00
69057f6d10 feat: wizard step validate() functions — all six mandatory steps 2026-02-25 08:00:18 -08:00
537172a4ba feat: tier system with FEATURES gate + can_use() + tier_label() 2026-02-25 07:55:47 -08:00
3b83610314 feat: wizard fields in UserProfile + params column in background_tasks
- Add tier, dev_tier_override, wizard_complete, wizard_step, dismissed_banners
  fields to UserProfile with defaults and effective_tier property
- Add params TEXT column to background_tasks table (CREATE + migration)
- Update insert_task() to accept params with params-aware dedup logic
- Update submit_task() and _run_task() to thread params through
- Add test_wizard_defaults, test_effective_tier_override,
  test_effective_tier_no_override, and test_insert_task_with_params
2026-02-25 07:27:14 -08:00
3533b1e4e1 docs: expanded wizard implementation plan — 13 tasks, TDD throughout 2026-02-25 06:29:23 -08:00
6a63c3f55d docs: expanded first-run wizard design
Architecture: wizard module system, mandatory 6-step flow, optional
home banners, tier gating (free/paid/premium + dev_tier_override),
resume upload/parse/builder, LLM generation via background tasks,
integrations registry pattern with 14 v1 services.
2026-02-24 21:30:05 -08:00
ef5ed106f4 chore: add backlog.md + gitignore config/.backup-* dirs 2026-02-24 20:54:12 -08:00
56fb386225 feat: startup preflight — port collision avoidance + resource checks
scripts/preflight.py (stdlib-only, no psutil):
- Port probing: owned services auto-reassign to next free port; external
  services (Ollama) show ✓ reachable / ⚠ not responding
- System resources: CPU cores, RAM (total + available), GPU VRAM via
  nvidia-smi; works on Linux + macOS
- Profile recommendation: remote / cpu / single-gpu / dual-gpu
- vLLM KV cache offload: calculates CPU_OFFLOAD_GB when VRAM < 10 GB
  free and RAM headroom > 4 GB (uses up to 25% of available headroom)
- Writes resolved values to .env for docker compose; single-service mode
  (--service streamlit) for scripted port queries
- Exit 0 unless an owned port genuinely can't be resolved

scripts/manage-ui.sh:
- Calls preflight.py --service streamlit before bind; falls back to
  pure-bash port scan if Python/yaml unavailable

compose.yml:
- vllm command: adds --cpu-offload-gb ${CPU_OFFLOAD_GB:-0}

Makefile:
- start / restart depend on preflight target
- PYTHON variable for env portability
- test target uses PYTHON variable

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 20:36:16 -08:00
81e7699545 feat: migration tool + portable startup scripts
scripts/migrate.py:
- dry-run by default; --apply writes files; --copy-db migrates staging.db
- generates config/user.yaml from source repo's resume + cover letter scripts
- copies gitignored configs (notion, email, adzuna, craigslist, search profiles,
  resume keywords, blocklist, aihawk resume)
- merges fine-tuned model name from source llm.yaml into dest llm.yaml

scripts/manage-ui.sh:
- STREAMLIT_BIN no longer hardcoded; auto-resolves via conda env or PATH;
  override with STREAMLIT_BIN env var

scripts/manage-vllm.sh:
- VLLM_BIN and MODEL_DIR now read from env vars with portable defaults

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 20:25:54 -08:00
35735c3074 feat: LGBTQIA+ focus + Phase 2/3 audit fixes
LGBTQIA+ inclusion section in research briefs:
- user_profile.py: add candidate_lgbtq_focus bool accessor
- user.yaml.example: add candidate_lgbtq_focus flag (default false)
- company_research.py: gate new LGBTQIA+ section behind flag; section
  count now dynamic (7 base + 1 per opt-in section, max 9)
- 2_Settings.py: add "Research Brief Preferences" expander with
  checkboxes for both accessibility and LGBTQIA+ focus flags;
  mission_preferences now round-trips through save (no silent drop)

Phase 2 fixes:
- manage-vllm.sh: MODEL_DIR and VLLM_BIN now read from env vars
  (VLLM_MODELS_DIR, VLLM_BIN) with portable defaults
- search_profiles.yaml: replace personal CS/TAM/Bay Area profiles
  with a documented generic starter profile

Phase 3 fix:
- llm.yaml: rename meghan-cover-writer:latest → llama3.2:3b with
  inline comment for users to substitute their fine-tuned model;
  fix model-exclusion comment

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 20:02:03 -08:00
0cdfb41201 fix: remove hardcoded personal values — Phase 1 audit findings
- 3_Resume_Editor.py: replace "Meghan's" in docstring and caption
- user_profile.py: expose mission_preferences and candidate_accessibility_focus
- user.yaml.example: add mission_preferences section + candidate_accessibility_focus flag
- generate_cover_letter.py: build _MISSION_NOTES from user profile instead of
  hardcoded personal passion notes; falls back to generic defaults when not set
- company_research.py: gate "Inclusion & Accessibility" section behind
  candidate_accessibility_focus flag; section count adjusts (7 or 8) accordingly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 19:57:03 -08:00
02017ea861 feat: add cross-platform dependency installer and Makefile for Linux/macOS
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 19:47:06 -08:00
7ec558e791 feat: complete generalization — smoke tests, README, all personal refs extracted
- UserProfile class drives all personal data
- First-run wizard gates app until user.yaml exists
- Docker Compose stack: remote/cpu/single-gpu/dual-gpu profiles
- Vision service containerized (single-gpu/dual-gpu)
- All Meghan/Library references removed from app and scripts
- Circuit Forge LLC / Peregrine branding throughout

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 19:41:09 -08:00
1ac559df0a feat: add vision service to compose stack and fine-tune wizard tab to Settings
- Add moondream2 vision service to compose.yml (single-gpu + dual-gpu profiles)
- Create scripts/vision_service/Dockerfile for the vision container
- Add VISION_PORT, VISION_MODEL, VISION_REVISION vars to .env.example
- Add Vision Service entry to SERVICES list in Settings (hidden unless gpu profile active)
- Add Fine-Tune Wizard tab (Task 10) to Settings with 3-step upload→preview→train flow
- Tab is always rendered; shows info message when non-GPU profile is active

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 19:37:55 -08:00
105b08b333 feat: services tab uses docker compose commands and SSL-aware health checks
Replace hardcoded systemd/shell-script service commands with docker compose
profile-aware commands. Add inference_profile-based filtering (hidden flag
removes Ollama on remote profile, vLLM unless dual-gpu). Replace TCP socket
health check with HTTP-based _port_open() that accepts host/ssl/verify params
for remote/TLS-terminated service support.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 19:34:44 -08:00
e22f233010 feat: add Docker Compose stack with remote/cpu/single-gpu/dual-gpu profiles
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 19:31:57 -08:00
a90be958f0 fix: persist API keys to .env and write notion.yaml with field_map defaults in wizard
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 19:24:51 -08:00
01cf221bd5 feat: first-run setup wizard gates app until user.yaml is created
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 19:20:35 -08:00