turnstone/scripts
pyr0ball e543ab70f7 feat: dual-backend SQLite/Postgres + multi-tenant source namespacing
- Add app/db/ abstraction layer: Backend enum, DbConn wrapper,
  dialect helper (q() for ? vs %s paramstyle), get_conn(), tenant_id()
- Auto-detect backend from DATABASE_URL; SQLite remains default when
  unset — no config change for local deployments
- Add tenant_id column to all three logical DBs (main, context, incidents);
  idempotent ALTER TABLE migration runs before schema scripts on existing DBs
- All INSERTs inject tenant_id; SELECTs use (tenant_id = ? OR tenant_id = '')
  for backward compat with pre-namespacing rows
- Add docker-compose.yml with named volume turnstone_pgdata (survives rebuilds)
  and optional external Postgres support via DATABASE_URL override
- Add scripts/migrate_sqlite_to_postgres.py — one-shot idempotent migration
  for existing SQLite data; ON CONFLICT DO NOTHING for safe re-runs
- Fix SSH glean path in pipeline.py to use ensure_schema + get_conn
  (was still using raw sqlite3.connect + old _SCHEMA without tenant_id)
- Fix FTS5 JOIN ambiguity: qualify repeat_count as f.repeat_count in search
- Update all tests to use ensure_*_schema fixtures; add row_factory where needed
- 394/394 tests passing

Closes: #42
Closes: #50
2026-06-08 08:37:54 -07:00
..
manifests feat(diagnose): tech-level post-processor, offline mode, API auth, context harvest 2026-05-28 08:51:05 -07:00
build_fts_index.py refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
collect_cluster_logs.sh fix(db): add timeout=30s to all sqlite3.connect() calls across app 2026-05-26 23:12:48 -07:00
docker-cluster.sh fix: correct cf-orch port to 7700; fix relative time parsing in diagnose; fix syslog PRI prefix 2026-05-13 05:33:41 -07:00
export_corpus.py feat: periodic corpus export — push ERROR/CRITICAL entries and incidents to Avocet 2026-05-11 17:08:35 -07:00
export_journal.sh refactor: rename ingest → glean throughout codebase 2026-05-20 23:02:55 -07:00
glean_corpus.py fix(glean): add timeout=30s to all pipeline DB connections; add --force flag; new patterns 2026-05-26 22:36:45 -07:00
harvest_docs.py feat(diagnose): tech-level post-processor, offline mode, API auth, context harvest 2026-05-28 08:51:05 -07:00
migrate_sqlite_to_postgres.py feat: dual-backend SQLite/Postgres + multi-tenant source namespacing 2026-06-08 08:37:54 -07:00
syslog_receiver.py feat: add UDP syslog receiver for network device log collection 2026-05-13 04:58:51 -07:00
turnstone-cluster-collect.service fix: run collect service as alan user; call ingest directly without Docker 2026-05-13 05:17:43 -07:00
turnstone-cluster-collect.timer refactor: use live watcher + systemd timer instead of cron for cluster ingest 2026-05-13 04:55:25 -07:00
turnstone-cluster.service feat: source-scoped diagnose; multi-node Docker log collection 2026-05-13 08:10:42 -07:00
turnstone-syslog-receiver.service feat: add UDP syslog receiver for network device log collection 2026-05-13 04:58:51 -07:00
update.sh chore: add update.sh deploy script; gitignore patterns/watch.yaml 2026-05-11 16:07:07 -07:00
watch_plex.py feat: plex EAE watchdog and plex_eae_failure pattern 2026-05-08 13:41:34 -07:00