turnstone/app
pyr0ball ccc9a9eecf fix(watcher): remove per-flush FTS sync to eliminate SQLite write lock contention
Each WatchSource was calling build_fts_index() every 3 flushes (~30s).
With 70+ active sources, this produced a near-continuous stream of FTS
INSERT operations, each holding the SQLite write lock for several seconds
while scanning the 5.4GB log_entries table. Every other writer (other
watcher flushes, cybersec scorer) timed out with 'database is locked'.

FTS index is now only updated by the glean scheduler (every 900s) and
the manual `build-fts` command — both already call build_fts_index()
through glean_dir(). Real-time freshness of watcher-ingested entries
in FTS was ~30s before; it's now up to ~15min, which is acceptable.

This is the root cause of the persistent 'database is locked' errors
blocking the cybersec scorer (issue #9).

Closes: #9
2026-06-10 12:42:24 -07:00
..
api feat: initial Turnstone POC — ingest, FTS search, MCP server 2026-05-08 12:12:34 -07:00
context feat: dual-backend SQLite/Postgres + multi-tenant source namespacing 2026-06-08 08:37:54 -07:00
db feat: cybersec zero-shot scoring pipeline (#9) 2026-06-10 01:03:25 -07:00
glean feat: dual-backend SQLite/Postgres + multi-tenant source namespacing 2026-06-08 08:37:54 -07:00
services feat: cybersec zero-shot scoring pipeline (#9) 2026-06-10 01:03:25 -07:00
tasks feat: cybersec zero-shot scoring pipeline (#9) 2026-06-10 01:03:25 -07:00
watch fix(watcher): remove per-flush FTS sync to eliminate SQLite write lock contention 2026-06-10 12:42:24 -07:00
__init__.py feat: initial Turnstone POC — ingest, FTS search, MCP server 2026-05-08 12:12:34 -07:00
mcp_server.py feat: dual-backend SQLite/Postgres + multi-tenant source namespacing 2026-06-08 08:37:54 -07:00
rest.py feat: cybersec zero-shot scoring pipeline (#9) 2026-06-10 01:03:25 -07:00