Commit graph

105 commits

Author SHA1 Message Date
dc508d7197 fix: update tests to match refactored scheduler and free-tier Vue SPA
Some checks failed
CI / test (push) Failing after 28s
- task_scheduler: extend LocalScheduler (concrete class), not TaskScheduler
  (Protocol); remove unsupported VRAM kwargs from super().__init__()
- dev-api: lazy import db_migrate inside _startup() to avoid worktree
  scripts cache issue in test_dev_api_settings.py
- test_task_scheduler: update VRAM-attribute tests to match LocalScheduler
  (no _available_vram/_reserved_vram); drop deepest-queue VRAM-gating
  ordering assertion (LocalScheduler is FIFO, not priority-gated);
  suppress PytestUnhandledThreadExceptionWarning on crash test; fix
  budget assertion to not depend on shared pytest tmp dir state
- test_dev_api_settings: patch path functions (_resume_path, _search_prefs_path,
  _license_path, _tokens_path, _config_dir) instead of removed module-level
  constants; mock _TRAINING_JSONL for finetune status idle test
- test_wizard_tiers: Vue SPA is free tier (issue #20), assert True
- test_wizard_api: patch _search_prefs_path() function, not SEARCH_PREFS_PATH
- test_ui_switcher: free-tier vue preference no longer downgrades to streamlit
2026-04-05 07:35:45 -07:00
64554dbef1 feat(#43): numbered SQL migration runner (Rails-style)
Some checks failed
CI / test (push) Failing after 19s
- migrations/001_baseline.sql: full schema baseline (all tables/cols)
- scripts/db_migrate.py: apply sorted *.sql files, track in schema_migrations
- Wired into FastAPI startup and Streamlit app.py startup
- Replaces ad-hoc digest_queue CREATE in _startup()
- 6 tests covering apply, idempotency, partial apply, failure rollback
- docs/developer-guide/contributing.md: migration authoring guide
2026-04-04 22:17:42 -07:00
53b07568d9 feat(vue): accumulated parity work — Q&A, Apply highlights, AppNav switcher, cloud API
API additions (dev-api.py split across this and next commit):
- /api/jobs/{job_id}/qa GET/PATCH/suggest — Interview Prep answer storage + LLM suggestions
- /api/settings/ui-preference POST — persist streamlit/vue preference to user.yaml
- cancel_task() added to scripts/db.py (per-task cancel for Danger Zone)

Vue / UI:
- AppNav: " Classic" button to switch back to Streamlit UI (writes cookie + persists to user.yaml)
- ApplyWorkspace: Resume Highlights panel (collapsible skills/domains/keywords with job-match highlighting)
- SettingsView: hide Data tab in cloud mode (showData guard)
- ResumeProfileView: minor improvements
- useApi.ts: error handling improvements

Infra:
- compose.cloud.yml: add api service (uvicorn dev_api running in cloud container)
- docker/web/nginx.conf: proxy /api/* to api service in cloud mode
- README.md: Vue SPA now listed as Free tier feature
2026-04-04 22:04:51 -07:00
9392ee2979 fix: address code review — drop OLLAMA_RESEARCH_HOST, fix test fidelity, simplify model guard 2026-04-04 19:26:08 -07:00
3f376347d6 feat(preflight): write OLLAMA_HOST to .env when Ollama is adopted from host
When preflight.py adopts a host-running Ollama (or ollama_research) service,
write OLLAMA_HOST (and OLLAMA_RESEARCH_HOST) into .env using host.docker.internal
so LLMRouter's env-var auto-config resolves the correct address from inside the
Docker container without requiring a config/llm.yaml to exist.
2026-04-04 19:02:26 -07:00
f62a9d9901 feat: llm_router shim — tri-level config priority (local > user > env-var) 2026-04-04 18:36:29 -07:00
b06d596d4c feat(vue): open Vue SPA to all tiers; fix cloud nav and feedback button
Some checks failed
CI / test (pull_request) Failing after 1m16s
- Lower vue_ui_beta gate to "free" so all licensed users can access the
  new UI without a paid subscription
- Remove "Paid tier" wording from the Try New UI banner
- Fix Vue SPA navigation in cloud/demo deployments: add VITE_BASE_PATH
  build arg so Vite sets the correct subpath base, and pass
  import.meta.env.BASE_URL to createWebHistory() so router links
  emit /peregrine/... paths that Caddy can match
- Fix feedback button missing on cloud instance by passing
  FORGEJO_API_TOKEN through compose.cloud.yml
- Remove vLLM container from compose.yml (vLLM dropped from stack;
  cf-research service in cfcore covers the use case)
- Fix cloud config path in Apply page (use get_config_dir() so per-user
  cloud data roots resolve correctly for user.yaml and resume YAML)
- Refactor generate_cover_letter._build_system_context and
  _build_mission_notes to accept explicit profile arg (enables
  per-user cover letter generation in cloud multi-tenant mode)
- Add API proxy block to nginx.conf (Vue web container can now call
  /api/ directly without Vite dev proxy)
- Update .env.example: remove vLLM vars, add research model + tuning
  vars for external vLLM deployments
- Update llm.yaml: switch vllm base_url to host.docker.internal
  (vLLM now runs outside Docker stack)

Closes #63 (feedback button)
Related: #8 (Vue SPA), #50–#62 (parity milestone)
2026-04-02 17:41:35 -07:00
66dc42a407 fix(preflight): remove vllm from Docker adoption list
vllm is now managed by cf-orch as a host process — no Docker service
defined in compose.yml. Preflight was detecting port 8000 (llm_server)
and generating a vllm stub in compose.override.yml with no image,
causing `docker compose up` to error on startup.
2026-04-02 16:57:06 -07:00
5b296b3e01 fix(discovery): per-user config dir in cloud mode; normalize job_titles key
Some checks failed
CI / test (push) Failing after 22s
- discover.py: run_discovery() accepts config_dir param; auto-derives it
  from db_path parent (per-user in cloud, falls back to /app/config)
- task_runner.py: passes db_path.parent/config as config_dir to run_discovery
- wizard (0_Setup.py): write 'titles' key not 'job_titles' — matches what
  discover.py and all custom board scrapers read
- adzuna/theladders/craigslist: fall back to 'job_titles' for existing
  profiles written by older wizard versions
- Fixed Sheridan's live config in place (job_titles → titles)
2026-04-01 19:37:29 -07:00
d00d74d994 feat(scheduler): read CF_ORCH_URL env var for coordinator address
Some checks failed
CI / test (push) Failing after 19s
Threads coordinator_url from CF_ORCH_URL env var (default localhost:7700)
into the cfcore TaskScheduler so Docker instances can point at
host.docker.internal:7700 instead of the container's own loopback.

Also adds CF_ORCH_URL to compose.test-cfcore.yml and mounts persistent
patched configs (llm.docker.yaml, user.docker.yaml) for the test instance.
2026-04-01 11:06:38 -07:00
8c42de3f5c feat(merge): merge feature/vue-spa into main
Full Vue 3 SPA merge — closes #8. Major additions:

Backend (dev API):
- dev_api.py → symlink to dev-api.py (importable module alias)
- dev-api.py: full FastAPI backend (settings, jobs, interviews, prep,
  survey, digest, resume optimizer endpoints); cloud session middleware
- scripts/user_profile.py: load_user_profile / save_user_profile helpers
- scripts/discover.py + scripts/imap_sync.py: API-compatible additions

Frontend (web/src/):
- ApplyWorkspace: ATS resume optimizer panel (gap report free, rewrite paid+)
- ResumeOptimizerPanel.vue: new component with task polling + .txt download

Test suite:
- test_dev_api_settings/survey/prep/digest/interviews: full API test coverage
- fix: replace importlib.reload with monkeypatch.setattr(dev_api, "DB_PATH")
  to prevent module global reset breaking isolation across test files

Docs:
- docs/vue-spa-migration.md: migration guide
2026-04-01 07:11:14 -07:00
faa1807e96 feat(api): add job ranker and credential store scripts
- scripts/job_ranker.py: two-stage rank pipeline for /api/jobs/stack
  endpoint; scores pending jobs by match_score + seniority signals
- scripts/credential_store.py: per-user credential management (BYOK
  API keys, email passwords); used by dev_api settings endpoints
2026-04-01 07:10:46 -07:00
02e004ee5c feat(apply): ATS resume optimizer backend — gap report + LLM rewrite
- scripts/resume_optimizer.py: full pipeline (extract_jd_signals →
  prioritize_gaps → rewrite_for_ats → hallucination_check)
- scripts/db.py: add optimized_resume + ats_gap_report columns +
  save_optimized_resume / get_optimized_resume helpers
- tests/test_resume_optimizer.py: 17 unit tests; patches at source
  module (scripts.llm_router.LLMRouter), not consumer

Tier gate: gap report is free; full LLM rewrite is paid+.
2026-04-01 07:09:46 -07:00
931a07d4e0 chore(merge): merge main into feature/vue-spa — resolve ApplyWorkspace conflict
ApplyWorkspace.vue: kept HEAD (vue-spa) version for resume optimizer panel,
cl-error__actions wrapper, and ResumeOptimizerPanel import. main's older
version lacked these additions.
2026-03-31 21:25:15 -07:00
922d91fb91 refactor(scheduler): shim to circuitforge_core.tasks.scheduler
VRAM detection now uses cf-orch free VRAM when coordinator is running,
making the scheduler cooperative with other cf-orch consumers.
Enqueue return value now checked — queue-full tasks are marked failed.
2026-03-31 09:27:43 -07:00
818e46c17e feat: migrate to circuitforge-core for db, llm router, and tiers
Some checks failed
CI / test (push) Failing after 24s
2026-03-25 11:44:19 -07:00
0acde6d199 feat(profile): add ui_preference field (streamlit|vue, default: streamlit) 2026-03-22 15:55:38 -07:00
a380ec33ec fix(settings): task 6 review fixes — credential paths, email security, integrationResults in store
- Anchor CRED_DIR/KEY_PATH to __file__ (not CWD) in credential_store.py
- Fix email PUT: separate password pop from sentinel discard (was fragile or-chain)
- Fix email test: always use stored credential, remove password override path
- Move integrationResults into system store (was view-local — spec violation)
- saveFilePaths/saveDeployConfig write to dedicated error refs, not saveError
2026-03-22 15:46:47 -07:00
f6ddaca14f feat(settings): credential store + fix Task 6 blocking review issues
- add scripts/credential_store.py (keyring/file/env-ref backends, Fernet encryption)
- email password stored via credential store, never returned in GET
- email GET returns password_set flag; PUT accepts new password or ${ENV_VAR} ref
- move integration actions to store (connectIntegration, testIntegration, disconnectIntegration)
- add tier-gating UI with locked state and upgrade prompt
- move subprocess/socket/imaplib/ssl imports to top level
2026-03-22 15:31:45 -07:00
2200d05b5c feat(settings): Search Prefs tab — store, view, API endpoints, remote preference filter 2026-03-21 03:09:51 -07:00
6093275549 fix(settings): final code quality fixes for My Profile tab
- add try/except to sync_identity endpoint
- strip id field from mission_preferences save body
- fix NDA v-for key to use company string (not index), add dedup guard
- move imports out of save_user_profile function body
2026-03-21 02:53:29 -07:00
da7d305588 fix(settings): profile tests assert sync-identity; add load/save_user_profile helpers 2026-03-21 02:31:39 -07:00
347c171e26 fix: prefer HTML body in imap_sync, strip head/style/script, remove 4000-char truncation
- _parse_message now prefers text/html over text/plain so digest emails
  retain href attribute values needed for link extraction
- Strip <head>, <style>, <script> blocks before storing to remove CSS/JS
  garbage while keeping anchor tags intact
- Remove [:4000] truncation — digest emails need full body for URL regex
- Update test: large body should NOT be truncated (assert len == 10_000)
2026-03-20 13:35:30 -07:00
0590a3a12e fix: fix indentation and add try/finally in digest startup 2026-03-20 02:36:23 -07:00
6a1ee3ed28 feat: add digest_queue table to schema and dev-api startup 2026-03-20 02:34:41 -07:00
0758b70306 feat(e2e): add smoke + interaction tests; fix two demo mode errors
- Add tests/e2e/test_smoke.py: page-load error check for all pages
- Add tests/e2e/test_interactions.py: click every interactable, diff
  errors, XFAIL expected demo failures, flag regressions as XPASS
- Fix conftest get_page_errors() to use text_content() instead of
  inner_text() so errors inside collapsed expanders are captured with
  their actual message text (inner_text respects CSS display:none)
- Fix tests/e2e/modes/demo.py base_url to include /peregrine path prefix
  (STREAMLIT_SERVER_BASE_URL_PATH=peregrine set in demo container)

App fixes surfaced by the harness:
- task_runner.py: add DEMO_MODE guard for discovery task — previously
  crashed with FileNotFoundError on search_profiles.yaml before any
  demo guard could fire; now returns friendly error immediately
- 6_Interview_Prep.py: stop auto-triggering LLM session on page load
  in demo mode; show "AI features disabled" info instead, preventing
  a silent st.error() inside the collapsed Practice Q&A expander

Both smoke and interaction tests now pass clean against demo mode.
2026-03-17 07:00:54 -07:00
37d151725e feat: push interview events to connected calendar integrations (#19)
Implements idempotent calendar push for Apple Calendar (CalDAV) and
Google Calendar from the Interviews kanban.

- db: add calendar_event_id column (migration) + set_calendar_event_id helper
- integrations/apple_calendar: create_event / update_event via caldav + icalendar
- integrations/google_calendar: create_event / update_event via google-api-python-client;
  test() now makes a real API call instead of checking file existence
- scripts/calendar_push: orchestrates push/update, builds event title from stage +
  job title + company, attaches job URL and company brief to description,
  defaults to noon UTC / 1hr duration
- app/pages/5_Interviews: "Add to Calendar" / "Update Calendar" button shown
  when interview date is set and a calendar integration is configured
- environment.yml: pin caldav, icalendar, google-api-python-client, google-auth
- tests/test_calendar_push: 9 tests covering create, update, error handling,
  event timing, idempotency, and missing job/date guards
2026-03-16 21:31:22 -07:00
d8549bd356 Merge pull request 'feat: LLM queue optimizer — resource-aware batch scheduler (closes #2)' (#15) from feature/llm-queue-optimizer into main 2026-03-15 16:48:37 -07:00
922ede9258 fix: _trim_to_letter_end matches full name when no profile set
When _profile is None the fallback pattern \w+ only matched the first
word of a two-word sign-off (e.g. 'Alex' from 'Alex Rivera'), silently
dropping the last name. Switch fallback to \w+(?:\s+\w+)? so a full
first+last sign-off is preserved in no-config environments (CI, first run).
2026-03-15 16:43:27 -07:00
9c36c578ef feat: add Jobgether recruiter framing to cover letter generation
When source == "jobgether", build_prompt() injects a recruiter context
note directing the LLM to address the Jobgether recruiter using
"Your client [at {company}] will appreciate..." framing rather than
addressing the employer directly. generate() and task_runner both
thread the is_jobgether flag through automatically.
2026-03-15 09:45:51 -07:00
b3893e9ad9 feat: add Jobgether URL detection and scraper to scrape_url.py 2026-03-15 09:45:50 -07:00
690a1ccf93 feat(task_runner): route LLM tasks through scheduler in submit_task()
Replaces the spawn-per-task model for LLM task types with scheduler
routing: cover_letter, company_research, and wizard_generate are now
enqueued via the TaskScheduler singleton for VRAM-aware batching.
Non-LLM tasks (discovery, email_sync, etc.) continue to spawn daemon
threads directly. Adds autouse clean_scheduler fixture to
test_task_runner.py to prevent singleton cross-test contamination.
2026-03-15 04:52:42 -07:00
3e3c6f1fc5 feat(scheduler): add durability — re-queue surviving LLM tasks on startup 2026-03-15 04:24:11 -07:00
9b96c45b63 feat(scheduler): implement thread-safe singleton get_scheduler/reset_scheduler 2026-03-15 04:19:23 -07:00
a53a03d593 feat(scheduler): implement scheduler loop and batch worker with VRAM-aware scheduling 2026-03-15 04:14:56 -07:00
68d257d278 feat(scheduler): implement enqueue() with depth guard and ghost-row cleanup 2026-03-15 04:05:22 -07:00
46b229094a refactor(scheduler): use module-level _get_gpus directly in __init__ 2026-03-15 04:01:01 -07:00
415e98d401 feat(scheduler): implement TaskScheduler.__init__ with budget loading and VRAM detection 2026-03-15 03:32:11 -07:00
fe8da36e00 feat(scheduler): add task_scheduler.py skeleton with constants and TaskSpec 2026-03-15 03:28:43 -07:00
376e028af5 feat(db): add reset_running_tasks() for durable scheduler restart 2026-03-15 03:22:45 -07:00
2c61d4038f fix(linkedin): update selectors for 2025 public DOM; surface login-wall limitation in UI
LinkedIn's unauthenticated public profile only exposes name, summary (truncated),
current employer name, and certifications. Past roles, education, and skills are
blurred server-side behind a login wall — not a scraper limitation.

- Update selectors: data-section='summary' (was 'about'), .profile-section-card
  for certs, .visible-list for current experience entry
- Strip login-wall noise injected into summary text after 'see more'
- Skip aria-hidden blurred placeholder experience items
- Add info callout in UI directing users to data export zip for full history
2026-03-13 19:47:21 -07:00
3e8b4cd654 fix(cloud): use per-user config dir for wizard gate; redirect on invalid session
- app.py: wizard gate now reads get_config_dir()/user.yaml instead of
  hardcoded repo-level config/ — fixes perpetual onboarding loop in
  cloud mode where per-user wizard_complete was never seen
- app.py: page title corrected to "Peregrine"
- cloud_session.py: add get_config_dir() returning per-user config path
  in cloud mode, repo config/ locally
- cloud_session.py: replace st.error() with JS redirect on missing/invalid
  session token so users land on login page instead of error screen
- Home.py, 4_Apply.py, migrate.py: remove remaining AIHawk UI references
2026-03-13 11:24:42 -07:00
00f0eb4807 feat(linkedin): add staging file parser with re-parse support 2026-03-13 10:18:01 -07:00
e937094884 fix(linkedin): improve scraper error handling, current-job date range, add missing tests 2026-03-13 06:02:03 -07:00
f64ecf81e0 feat(linkedin): add scraper (Playwright + export zip) with URL validation 2026-03-13 01:06:39 -07:00
a43e29e50d feat(linkedin): add HTML parser utils with fixture tests 2026-03-13 01:01:05 -07:00
7a698496f9 feat(cloud): fix backup/restore for cloud mode — SQLCipher encrypt/decrypt
T13: Three fixes:
1. backup.py: _decrypt_db_to_bytes() decrypts SQLCipher DB before archiving
   so the zip is portable to any local Docker install (plain SQLite).
2. backup.py: _encrypt_db_from_bytes() re-encrypts on restore in cloud mode
   so the app can open the restored DB normally.
3. 2_Settings.py: _base_dir uses get_db_path().parent in cloud mode (user's
   per-tenant data dir) instead of the hardcoded app root; db_key wired
   through both create_backup() and restore_backup() calls.

6 new cloud backup tests + 2 unit tests for SQLCipher helpers (pysqlcipher3
mocked — not available in the local conda test env). 419/419 total passing.
2026-03-09 22:41:44 -07:00
96715bdeb6 feat(peregrine): add cloud_session middleware + SQLCipher get_connection()
cloud_session.py: no-op in local mode; in cloud mode resolves Directus JWT
from X-CF-Session header to per-user db_path in st.session_state.

get_connection() in scripts/db.py: transparent SQLCipher/sqlite3 switch —
uses encrypted driver when CLOUD_MODE=true and key provided, vanilla sqlite3
otherwise. libsqlcipher-dev added to Dockerfile for Docker builds.

6 new cloud_session tests + 1 new get_connection test — 34/34 db tests pass.
2026-03-09 19:43:42 -07:00
f60ac07541 test: add missing base_url edge case + clarify 0.0.0.0 marker intent
Document defensive behavior: openai_compat with no base_url returns True
(cloud) because unknown destination is assumed cloud. Add explanatory
comment to LOCAL_URL_MARKERS for the 0.0.0.0 bind-address case.
2026-03-06 14:43:45 -08:00
47d8317d56 feat: byok_guard — cloud backend detection with full test coverage 2026-03-06 14:40:06 -08:00