Commit graph

8 commits

Author SHA1 Message Date
aafb4e2cad fix: separate context KB into own SQLite file to eliminate write-lock contention
context_facts, context_documents, and context_chunks now live in
turnstone-context.db (sibling of turnstone.db).  The glean scheduler
held write locks on the main DB long enough to cause 5-second timeout
failures on context fact inserts; separate files have independent WAL
write locks so they never contend.

Changes:
- pipeline.py: extract _CONTEXT_SCHEMA + ensure_context_schema()
- rest.py: CONTEXT_DB_PATH (TURNSTONE_CONTEXT_DB env var, defaults to
  sibling file); init via ensure_context_schema(); all context routes
  pass CONTEXT_DB_PATH; diagnose_stream receives context_db_path kwarg
- diagnose/__init__.py: diagnose_stream() accepts context_db_path
  (falls back to db_path for backward compat); retrieve_context uses it
- store.py: sqlite3.connect() timeout=30.0 — Python driver retry loop
  is independent of PRAGMA busy_timeout; needed for any remaining
  contention during test or single-file deployments

Closes: #42
2026-05-25 21:19:32 -07:00
3e7a1fa064 refactor: extract embeddings service layer — decouple context embedder from Ollama
- New app/services/embeddings.py: TURNSTONE_EMBED_* env vars, multi-backend support
- embedder.py delegates to service layer; re-exports EMBEDDING_AVAILABLE for compat
- retriever.py updated to use service layer
- Test coverage updated in tests/context/test_embedder.py
2026-05-25 11:01:25 -07:00
3362eb1fb6 fix: final review fixes — port guard, network error handling, wizard back nav, tablist arrow keys, dialog focus trap
- wizard.py: wrap syslog_port int() in try/except to default 514 on non-numeric input
- ContextView: add try/catch to doDelete, doDeleteFact, addFact for network errors
- ContextView: arrow-key navigation for tablist (ArrowLeft/ArrowRight)
- DiagnoseView: arrow-key navigation for tablist (ArrowLeft/ArrowRight)
- WizardOverlay: reset current_step to last schema step when clicking 'Go back and edit'
- WizardOverlay: focus trap on Tab/Shift+Tab within dialog element
2026-05-13 17:40:40 -07:00
f361c86019 feat: optional sqlite-vec embedding pipeline for Paid-tier RAG 2026-05-13 16:32:57 -07:00
9c8c60e461 feat: wizard state machine — structured Q&A writes context facts and source config 2026-05-13 16:25:52 -07:00
9a4931b0ba feat: context retriever — keyword fact lookup and chunk search 2026-05-13 16:23:54 -07:00
c62b0bb12a feat: context chunker — type detection, YAML extraction, text chunking
- Implement document type detection for yaml/json/markdown/text
- Extract service facts from docker-compose YAML (names, images, ports)
- Split text into overlapping word chunks (300-word default with 50-word overlap)
- Enforce 5 MB file size limit
- Comprehensive TDD test suite: 15 tests passing
2026-05-13 15:54:51 -07:00
dd977f0bf1 feat: context store — fact and document CRUD 2026-05-13 15:53:03 -07:00