Commit graph

9 commits

Author SHA1 Message Date
39ef1320b0 feat(manage): source .env before starting uvicorn
Enables TURNSTONE_MULTI_AGENT_DIAGNOSE and other env vars set in
.env to reach the running process without manual export. Variables
already set in the caller's environment take precedence.
2026-05-25 19:15:33 -07:00
828b69768a refactor: rename ingest → glean throughout codebase
Renames the app/ingest/ package to app/glean/ and updates all
references across Python modules, shell scripts, Vue components,
tests, and documentation.

Intentionally preserved:
- SQLite column name ingest_time (avoids schema migration)
- RetrievedEntry.ingest_time field (maps to the column above)
- Any public-facing JSON keys that reference ingest_time

Changes by category:
- app/ingest/ → app/glean/ (full package move, all parsers)
- app/tasks/ingest_scheduler.py → app/tasks/glean_scheduler.py
- scripts/ingest_corpus.py → scripts/glean_corpus.py
- tests/test_ingest_*.py → tests/test_glean_*.py
- Docstrings, log messages, comments: ingest → glean
- Env var: TURNSTONE_INGEST_INTERVAL → TURNSTONE_GLEAN_INTERVAL
- Shell scripts: glean.log, glean_corpus.py references
- README.md: multi-source ingest → multi-source glean
- .env.example: updated env var name
- patterns/: new diagnostic patterns from 2026-05-20 SSH incident
  (service_crash_loop, pkg_daemon_restart, ssh_forward_conflict)
- SourcesView.vue: pipeline label updated
- All test import paths updated to app.glean.*

285 tests passing.
2026-05-20 23:02:55 -07:00
784a4072b4 feat: SSE streaming diagnose, severity filter pills, per-source-cap search
- diagnose_stream() async generator: status/summary/entries/reasoning/done events
- POST /api/diagnose/stream SSE endpoint wired in rest.py
- entries_in_window() gains per_source_cap to prevent high-volume sources crowding results
- QuickCapture: severity filter pills, filtered entries view, pipeline status spinner
- llm.py: remove overly broad HTTPStatusError re-raise
2026-05-13 15:45:35 -07:00
a3c0962277 feat: qBittorrent log ingestor with 8 diagnostic patterns
Adds app/ingest/qbittorrent.py — auto-detected by the pipeline on the
(YYYY/MM/DD HH:MM:SS) timestamp fingerprint. Handles both slash and dash
date separators, optional [Warning|Critical] bracket levels, and
multi-line continuations (Qt stack traces).

patterns/default.yaml: 8 new qbit_ patterns covering tracker errors,
port bind failures, disk errors, hash check failures, peer bans, download
completion, ratio limits, and session errors.

manage.sh: ingest-qbit [HOST] command mirrors ingest-plex — probes known
default log paths locally or via SSH, ingests, restarts server.

14 tests covering format detection, severity mapping, multiline handling,
and timestamp normalization.
2026-05-10 08:21:16 -07:00
27e33ae85e fix: suppress ANSI colors when stdout is not a terminal
Color variables set to empty strings when stdout is not a tty, preventing
escape codes from being interpreted as commands by background task runners
2026-05-09 10:16:03 -07:00
040428c513 fix: ingest-plex pulls all rotated logs and uses clean source IDs
- Pull all Plex Media Server*.log files (active + .1 through .5)
- Local filenames: cass-plex_media_server.N.log → source_id includes host
- mapfile for clean SSH list; separate ssh per file (no pipeline complexity)
- Auto-restart after ingest instead of prompting
2026-05-09 08:37:04 -07:00
f6b261280c feat: plain-text and Plex log ingestors
- app/ingest/plex.py: Plex Media Server log parser
  Regex-based line parser for 'Mon DD, YYYY HH:MM:SS.mmm [pid] LEVEL - msg'
  format. Handles multi-line entries (stack traces). Detects plex_eae_failure
  and all other patterns via shared pattern library.
- app/ingest/plaintext.py: generic fallback parser for unrecognized formats
  Extracts timestamps (ISO 8601, syslog, common log) and severity via regex.
- pipeline.py: detect plex format via is_plex_log(); fall back to plaintext
  instead of skipping; process *.log files alongside *.jsonl; add ingest_file()
  for single-file ingestion.
- scripts/ingest_corpus.py: accept single file or directory as target
- manage.sh: ingest-plex command SSHes to Cass (or HOST arg), pulls
  Plex Media Server.log, and ingests it directly
2026-05-08 17:50:01 -07:00
9e46cd4c7f fix: serve Vue SPA from FastAPI, drop separate port 8535
Python http.server can't do SPA routing and Caddy was forwarding
/turnstone/* paths that the static server couldn't resolve.

- app/rest.py: mount web/dist/assets as StaticFiles; add SPA catch-all
  route that serves index.html for any unmatched path
- manage.sh: start/stop/status simplified to single process on :8534;
  remove UI_PORT / UI_PID_FILE; drop http.server invocation
- Caddyfile: replace split API/:8534 + SPA/:8535 block with a single
  strip_prefix + reverse_proxy to :8534
2026-05-08 17:27:46 -07:00
1f5854e90b chore: standardize manage.sh, remove start_dev.sh
- manage.sh: start/stop/restart/status/logs/open/dev/ingest/build-fts/test
  following avocet pattern (PID files, colored output, native processes)
- start mode: builds Vue SPA, uvicorn on :8534, python http.server on :8535
- dev mode: uvicorn --reload + Vite HMR, trap cleanup on exit
- scripts/start_dev.sh: removed (superseded by manage.sh dev)
2026-05-08 16:58:12 -07:00