Commit graph

6 commits

Author SHA1 Message Date
0311d72e53 feat: dual-backend SQLite/Postgres + multi-tenant source namespacing
- Add app/db/ abstraction layer: Backend enum, DbConn wrapper,
  dialect helper (q() for ? vs %s paramstyle), get_conn(), tenant_id()
- Auto-detect backend from DATABASE_URL; SQLite remains default when
  unset — no config change for local deployments
- Add tenant_id column to all three logical DBs (main, context, incidents);
  idempotent ALTER TABLE migration runs before schema scripts on existing DBs
- All INSERTs inject tenant_id; SELECTs use (tenant_id = ? OR tenant_id = '')
  for backward compat with pre-namespacing rows
- Add docker-compose.yml with named volume turnstone_pgdata (survives rebuilds)
  and optional external Postgres support via DATABASE_URL override
- Add scripts/migrate_sqlite_to_postgres.py — one-shot idempotent migration
  for existing SQLite data; ON CONFLICT DO NOTHING for safe re-runs
- Fix SSH glean path in pipeline.py to use ensure_schema + get_conn
  (was still using raw sqlite3.connect + old _SCHEMA without tenant_id)
- Fix FTS5 JOIN ambiguity: qualify repeat_count as f.repeat_count in search
- Update all tests to use ensure_*_schema fixtures; add row_factory where needed
- 394/394 tests passing

Closes: #42
Closes: #50
2026-06-08 08:37:54 -07:00
2a4a5a5152 feat(container): multi-agent env vars, HF cache mount, and ML deps
podman-standalone.sh:
- Add HF_CACHE_DIR=/opt/turnstone/hf-cache with mkdir guard
- Mount HF_HOME=/hf-cache so model weights persist across restarts
- Forward all multi-agent env vars (TURNSTONE_MULTI_AGENT_DIAGNOSE,
  GPU_SERVER_URL, TURNSTONE_CLASSIFIER_MODEL, TURNSTONE_EMBED_*)
- Add documentation comments for Contributor/Contributor2 remote instance setup

requirements.txt:
- Add torch (CPU-only), transformers, sentence-transformers for the
  5-stage multi-agent diagnose pipeline (classifier + suppressor stages)
- Use --extra-index-url for cpu wheel to keep image ~2GB lighter
- Both modules keep ImportError guards so server starts without them,
  but container images should ship fully capable
2026-05-26 13:20:26 -07:00
828b69768a refactor: rename ingest → glean throughout codebase
Renames the app/ingest/ package to app/glean/ and updates all
references across Python modules, shell scripts, Vue components,
tests, and documentation.

Intentionally preserved:
- SQLite column name ingest_time (avoids schema migration)
- RetrievedEntry.ingest_time field (maps to the column above)
- Any public-facing JSON keys that reference ingest_time

Changes by category:
- app/ingest/ → app/glean/ (full package move, all parsers)
- app/tasks/ingest_scheduler.py → app/tasks/glean_scheduler.py
- scripts/ingest_corpus.py → scripts/glean_corpus.py
- tests/test_ingest_*.py → tests/test_glean_*.py
- Docstrings, log messages, comments: ingest → glean
- Env var: TURNSTONE_INGEST_INTERVAL → TURNSTONE_GLEAN_INTERVAL
- Shell scripts: glean.log, glean_corpus.py references
- README.md: multi-source ingest → multi-source glean
- .env.example: updated env var name
- patterns/: new diagnostic patterns from 2026-05-20 SSH incident
  (service_crash_loop, pkg_daemon_restart, ssh_forward_conflict)
- SourcesView.vue: pipeline label updated
- All test import paths updated to app.glean.*

285 tests passing.
2026-05-20 23:02:55 -07:00
0882083755 feat: LLM reasoning layer — Ollama summarization on diagnose results 2026-05-11 11:35:07 -07:00
abd142addf feat: add diagnose service with NL time extraction via dateparser
Adds app/services/diagnose.py with parse_time_window() (dateparser-backed
NL time phrase extraction with 60-min fallback) and diagnose() (layered
FTS + window search returning severity/source summary). Includes 5 TDD tests.
2026-05-11 09:04:50 -07:00
df87f22416 feat: Podman container deployment for Contributor2's system
- Dockerfile: multi-stage build (node:22-alpine builds Vue SPA, python:3.12-slim
  runs uvicorn) — no Node.js required on the host
- requirements.txt: minimal runtime deps (fastapi, uvicorn, pyyaml, aiofiles,
  pydantic, python-multipart)
- podman-standalone.sh: mirrors Peregrine's deployment pattern; binds
  /opt/turnstone/data + /opt/turnstone/patterns + /opt/qbittorrent/config/data/logs
  (ro); includes cron and Caddy config instructions in comments
- .gitignore: add log/, .turnstone-api.pid (generated by manage.sh dev mode)
2026-05-11 05:13:58 -07:00