turnstone/app/glean
pyr0ball 5b151c2509 fix: split incidents tables to dedicated turnstone-incidents.db (#60)
FTS5 bulk-insert write locks starved the incident API and bundle endpoints
during log bursts (sonarr/radarr, high-volume docker sources). Fix mirrors
the context_facts split (context -> turnstone-context.db):

- Add INCIDENTS_DB_PATH / TURNSTONE_INCIDENTS_DB env var in rest.py
- Add _INCIDENTS_SCHEMA, ensure_incidents_schema(), and
  migrate_incidents_to_dedicated_db() in glean/pipeline.py
- Stub out incidents/received_bundles/sent_bundles in _SCHEMA (no-op
  CREATE IF NOT EXISTS) so legacy single-file deployments still open
- Thread incidents_db_path through diagnose_stream -> run_pipeline ->
  FalsePositiveSuppressor.suppress -> _fetch_resolved_incidents
- One-shot migration on startup: copy existing rows from main DB to
  incidents DB via INSERT OR IGNORE (idempotent, safe to re-run)
- Fix test_blocklist_endpoints fixtures to patch CONTEXT_DB_PATH and
  INCIDENTS_DB_PATH alongside DB_PATH (worktree has no data/ dir)

372 tests passing.

Closes: #60
2026-06-01 15:54:23 -07:00
..
__init__.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
base.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
caddy.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
dmesg_log.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
doc_upload.py fix(db): add timeout=30s to all sqlite3.connect() calls across app 2026-05-26 23:12:48 -07:00
docker_log.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
journald.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
mqtt_subscriber.py fix(db): add timeout=30s to all sqlite3.connect() calls across app 2026-05-26 23:12:48 -07:00
pipeline.py fix: split incidents tables to dedicated turnstone-incidents.db (#60) 2026-06-01 15:54:23 -07:00
plaintext.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
plex.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
qbittorrent.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
servarr.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
ssh.py feat: SSH remote host glean — transport layer and pipeline integration (closes #22, backend) 2026-05-20 23:03:13 -07:00
syslog.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
tautulli.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
wazuh.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00