Commit graph

203 commits

Author SHA1 Message Date
d8549bd356 Merge pull request 'feat: LLM queue optimizer — resource-aware batch scheduler (closes #2)' (#15) from feature/llm-queue-optimizer into main 2026-03-15 16:48:37 -07:00
922ede9258 fix: _trim_to_letter_end matches full name when no profile set
When _profile is None the fallback pattern \w+ only matched the first
word of a two-word sign-off (e.g. 'Alex' from 'Alex Rivera'), silently
dropping the last name. Switch fallback to \w+(?:\s+\w+)? so a full
first+last sign-off is preserved in no-config environments (CI, first run).
2026-03-15 16:43:27 -07:00
062f249ef9 ci: apt-get update before installing libsqlcipher-dev 2026-03-15 16:37:46 -07:00
5527fe9bf8 ci: install libsqlcipher-dev before pip install 2026-03-15 16:36:50 -07:00
722661058a ci: re-trigger after actions enabled 2026-03-15 15:54:27 -07:00
fd02c11441 ci: enable forgejo actions 2026-03-15 15:48:35 -07:00
1400a396ae ci: trigger runner 2026-03-15 15:39:45 -07:00
22696b4e50 docs: add Jobgether non-headless Playwright scraper to backlog
Xvfb-based Playwright can bypass Cloudflare Turnstile on jobgether.com.
Live inspection confirmed selectors; deferred pending Xvfb integration.
2026-03-15 11:59:48 -07:00
9c36c578ef feat: add Jobgether recruiter framing to cover letter generation
When source == "jobgether", build_prompt() injects a recruiter context
note directing the LLM to address the Jobgether recruiter using
"Your client [at {company}] will appreciate..." framing rather than
addressing the employer directly. generate() and task_runner both
thread the is_jobgether flag through automatically.
2026-03-15 09:45:51 -07:00
b3893e9ad9 feat: add Jobgether URL detection and scraper to scrape_url.py 2026-03-15 09:45:50 -07:00
ee054408ea feat: filter Jobgether listings via blocklist 2026-03-15 09:45:50 -07:00
d0987abc0e Merge pull request 'feat: LLM queue optimizer — resource-aware batch scheduler (closes #2)' (#13) from feature/llm-queue-optimizer into main
Reviewed-on: #13
2026-03-15 05:11:29 -07:00
c6fea9b3e2 feat: LLM queue optimizer complete — closes #2
Resource-aware batch scheduler for LLM tasks:
- scripts/task_scheduler.py (new): TaskScheduler singleton with VRAM-aware
  batch scheduling, durability, thread-safe singleton, memory safety
- scripts/task_runner.py: submit_task() routes LLM types through scheduler
- scripts/db.py: reset_running_tasks() for durable restart behavior
- app/app.py: _startup() preserves queued tasks on restart
- config/llm.yaml.example: scheduler VRAM budget config documented
- tests/test_task_scheduler.py (new): 24 tests covering all behaviors

Pre-existing failure: test_generate_calls_llm_router (issue #12, unrelated)
2026-03-15 05:01:24 -07:00
6e0105b0e8 feat(app): use reset_running_tasks() on startup to preserve queued tasks 2026-03-15 04:57:49 -07:00
690a1ccf93 feat(task_runner): route LLM tasks through scheduler in submit_task()
Replaces the spawn-per-task model for LLM task types with scheduler
routing: cover_letter, company_research, and wizard_generate are now
enqueued via the TaskScheduler singleton for VRAM-aware batching.
Non-LLM tasks (discovery, email_sync, etc.) continue to spawn daemon
threads directly. Adds autouse clean_scheduler fixture to
test_task_runner.py to prevent singleton cross-test contamination.
2026-03-15 04:52:42 -07:00
3e3c6f1fc5 feat(scheduler): add durability — re-queue surviving LLM tasks on startup 2026-03-15 04:24:11 -07:00
9b96c45b63 feat(scheduler): implement thread-safe singleton get_scheduler/reset_scheduler 2026-03-15 04:19:23 -07:00
a53a03d593 feat(scheduler): implement scheduler loop and batch worker with VRAM-aware scheduling 2026-03-15 04:14:56 -07:00
68d257d278 feat(scheduler): implement enqueue() with depth guard and ghost-row cleanup 2026-03-15 04:05:22 -07:00
46b229094a refactor(scheduler): use module-level _get_gpus directly in __init__ 2026-03-15 04:01:01 -07:00
415e98d401 feat(scheduler): implement TaskScheduler.__init__ with budget loading and VRAM detection 2026-03-15 03:32:11 -07:00
fe8da36e00 feat(scheduler): add task_scheduler.py skeleton with constants and TaskSpec 2026-03-15 03:28:43 -07:00
e3547cd998 docs(config): add scheduler VRAM budget config to llm.yaml.example 2026-03-15 03:28:26 -07:00
1616858729 refactor(tests): remove unused imports from test_task_scheduler 2026-03-15 03:27:17 -07:00
376e028af5 feat(db): add reset_running_tasks() for durable scheduler restart 2026-03-15 03:22:45 -07:00
12974f030c chore: add .worktrees/ to .gitignore
Prevents worktree directories from being tracked.
2026-03-14 16:30:38 -07:00
2c61d4038f fix(linkedin): update selectors for 2025 public DOM; surface login-wall limitation in UI
LinkedIn's unauthenticated public profile only exposes name, summary (truncated),
current employer name, and certifications. Past roles, education, and skills are
blurred server-side behind a login wall — not a scraper limitation.

- Update selectors: data-section='summary' (was 'about'), .profile-section-card
  for certs, .visible-list for current experience entry
- Strip login-wall noise injected into summary text after 'see more'
- Skip aria-hidden blurred placeholder experience items
- Add info callout in UI directing users to data export zip for full history
2026-03-13 19:47:21 -07:00
e680d9c401 chore: update changelog for v0.4.0 release 2026-03-13 11:28:03 -07:00
208e28b728 docs: update backlog with LinkedIn import follow-up items 2026-03-13 11:24:55 -07:00
3e8b4cd654 fix(cloud): use per-user config dir for wizard gate; redirect on invalid session
- app.py: wizard gate now reads get_config_dir()/user.yaml instead of
  hardcoded repo-level config/ — fixes perpetual onboarding loop in
  cloud mode where per-user wizard_complete was never seen
- app.py: page title corrected to "Peregrine"
- cloud_session.py: add get_config_dir() returning per-user config path
  in cloud mode, repo config/ locally
- cloud_session.py: replace st.error() with JS redirect on missing/invalid
  session token so users land on login page instead of error screen
- Home.py, 4_Apply.py, migrate.py: remove remaining AIHawk UI references
2026-03-13 11:24:42 -07:00
098115b4cc fix(linkedin): conservative settings merge, mkdir guard, split dockerfile playwright layer 2026-03-13 10:58:58 -07:00
a2778cc25d fix(linkedin): move session state pop before tabs; add rerun after settings merge
- Pop _linkedin_extracted before st.tabs() so tab_builder sees the
  freshly populated _parsed_resume in the same render pass (no extra rerun needed)
- Fix tab label capitalisation: "Build Manually" (capital M) per spec
- Add st.rerun() after LinkedIn merge in Settings so form fields
  refresh immediately to show the newly applied data
2026-03-13 10:55:25 -07:00
54d3d44f30 feat(linkedin): install Playwright Chromium in Docker image 2026-03-13 10:44:03 -07:00
ff6dcf0628 feat(linkedin): add LinkedIn import expander to Settings Resume Profile tab 2026-03-13 10:44:02 -07:00
21bd2a5794 feat(linkedin): add LinkedIn tab to wizard resume step 2026-03-13 10:43:53 -07:00
b35e258d48 feat(linkedin): add shared LinkedIn import Streamlit widget 2026-03-13 10:32:23 -07:00
00f0eb4807 feat(linkedin): add staging file parser with re-parse support 2026-03-13 10:18:01 -07:00
e937094884 fix(linkedin): improve scraper error handling, current-job date range, add missing tests 2026-03-13 06:02:03 -07:00
f64ecf81e0 feat(linkedin): add scraper (Playwright + export zip) with URL validation 2026-03-13 01:06:39 -07:00
a43e29e50d feat(linkedin): add HTML parser utils with fixture tests 2026-03-13 01:01:05 -07:00
d703bebb5e feat(cloud): add Heimdall tier resolution to cloud_session
Calls /admin/cloud/resolve after JWT validation to inject the user's
current subscription tier (free/paid/premium/ultra) into session_state
as cloud_tier. Cached 5 minutes via st.cache_data to avoid Heimdall
spam on every Streamlit rerun. Degrades gracefully to free on timeout
or missing token.

New env vars: HEIMDALL_URL, HEIMDALL_ADMIN_TOKEN (added to .env.example
and compose.cloud.yml). HEIMDALL_URL defaults to http://cf-license:8000
for internal Docker network access.

New helper: get_cloud_tier() — returns tier string in cloud mode, "local"
in local-first mode, so pages can distinguish self-hosted from cloud.
2026-03-10 12:31:14 -07:00
04c4efd3e0 fix(cloud): extract cf_session cookie by name from X-CF-Session header 2026-03-10 09:22:08 -07:00
8602107cc1 docs: add cloud architecture + cloud-deployment.md
architecture.md: updated Docker Compose table (3 compose files), database
layer (Postgres platform + SQLite-per-user), cloud session middleware,
telemetry system, and cloud design decisions.

cloud-deployment.md (new): full operational runbook — env vars, data root
layout, GDPR deletion, platform DB queries, telemetry, backup/restore,
Caddy routing, demo instance, and onboarding a new app to the cloud.
2026-03-09 23:02:29 -07:00
7a698496f9 feat(cloud): fix backup/restore for cloud mode — SQLCipher encrypt/decrypt
T13: Three fixes:
1. backup.py: _decrypt_db_to_bytes() decrypts SQLCipher DB before archiving
   so the zip is portable to any local Docker install (plain SQLite).
2. backup.py: _encrypt_db_from_bytes() re-encrypts on restore in cloud mode
   so the app can open the restored DB normally.
3. 2_Settings.py: _base_dir uses get_db_path().parent in cloud mode (user's
   per-tenant data dir) instead of the hardcoded app root; db_key wired
   through both create_backup() and restore_backup() calls.

6 new cloud backup tests + 2 unit tests for SQLCipher helpers (pysqlcipher3
mocked — not available in the local conda test env). 419/419 total passing.
2026-03-09 22:41:44 -07:00
441e4ce4ef feat(cloud): Privacy & Telemetry tab in Settings + update_consent()
T11: Add CLOUD_MODE-gated Privacy tab to Settings with full telemetry
consent UI — hard kill switch, anonymous usage toggle, de-identified
content sharing toggle, and time-limited support access grant. All changes
persist to telemetry_consent table via new update_consent() in telemetry.py.

Tab and all DB calls are completely no-op in local mode (CLOUD_MODE=false).
2026-03-09 22:14:22 -07:00
0e3abb5e63 feat(cloud): add compose.cloud.yml and telemetry consent middleware
T8: compose.cloud.yml — multi-tenant cloud stack on port 8505, CLOUD_MODE=true,
per-user encrypted data at /devl/menagerie-data, joins caddy-proxy_caddy-internal
network; .env.example extended with five cloud-only env vars.

T10: app/telemetry.py — log_usage_event() is the ONLY entry point to usage_events
table; hard kill switch (all_disabled) checked before any DB write; complete no-op
in local mode; swallows all exceptions so telemetry never crashes the app;
psycopg2-binary added to requirements.txt. Event calls wired into 4_Apply.py at
cover_letter_generated and job_applied. 5 tests, 413/413 total passing.
2026-03-09 22:10:18 -07:00
59a6c1ebaf feat(peregrine): wire cloud_session into pages for multi-tenant db path routing
resolve_session() is a no-op in local mode — no behavior change for existing users.
In cloud mode, injects user-scoped db_path into st.session_state at page load.
2026-03-09 20:22:17 -07:00
96715bdeb6 feat(peregrine): add cloud_session middleware + SQLCipher get_connection()
cloud_session.py: no-op in local mode; in cloud mode resolves Directus JWT
from X-CF-Session header to per-user db_path in st.session_state.

get_connection() in scripts/db.py: transparent SQLCipher/sqlite3 switch —
uses encrypted driver when CLOUD_MODE=true and key provided, vanilla sqlite3
otherwise. libsqlcipher-dev added to Dockerfile for Docker builds.

6 new cloud_session tests + 1 new get_connection test — 34/34 db tests pass.
2026-03-09 19:43:42 -07:00
24bb8476ab fix(peregrine): correct port comment in compose.demo.yml, update CLAUDE.md 2026-03-09 15:22:10 -07:00
b9a5bb2afa chore(peregrine): rename compose.menagerie.yml to compose.demo.yml
Public demo instances moving to demo.circuitforge.tech;
menagerie.circuitforge.tech reserved for cloud-hosted managed instances.
2026-03-09 14:55:38 -07:00