turnstone/tests
pyr0ball 63c742a708 feat: periodic ingest scheduler + Orchard submission pipeline
Adds asyncio-native background scheduler (TURNSTONE_INGEST_INTERVAL,
default 900s) that runs batch ingest then pushes pattern-matched entries
to a remote CF harvest endpoint (TURNSTONE_SUBMIT_ENDPOINT).

- app/tasks/ingest_scheduler.py: IngestState, scheduler_loop, run_once,
  submit_matched, _query_matched_since — asyncio.Lock prevents concurrent runs
- app/rest.py: POST /api/ingest/batch (pre-parsed entry receiver),
  GET /api/tasks/ingest/status, POST /api/tasks/ingest (manual trigger),
  TURNSTONE_INGEST_INTERVAL + TURNSTONE_SUBMIT_ENDPOINT env wiring in lifespan
- docker-compose.submissions.yml: segregated contrib1 (8536) + contrib2 (8537)
  receiving instances on Heimdall, isolated DBs under
  /devl/docker/turnstone-submissions/<node>/
- podman-standalone.sh: pass-through for TURNSTONE_SUBMIT_ENDPOINT +
  TURNSTONE_SOURCE_HOST
- app/ingest/mqtt_subscriber.py: MQTT log source adapter
- app/ingest/wazuh.py: Wazuh alert JSON adapter
- tests/test_ingest_wazuh.py: Wazuh adapter test suite
2026-05-20 08:57:25 -07:00
..
context feat: optional sqlite-vec embedding pipeline for Paid-tier RAG 2026-05-13 16:32:57 -07:00
__init__.py feat: initial Turnstone POC — ingest, FTS search, MCP server 2026-05-08 12:12:34 -07:00
test_blocklist_endpoints.py feat(blocklist): 6 REST endpoints + Pi-hole settings fields 2026-05-15 21:15:09 -07:00
test_export_corpus.py feat: periodic corpus export — push ERROR/CRITICAL entries and incidents to Avocet 2026-05-11 17:08:35 -07:00
test_ingest_dmesg.py feat: syslog and dmesg parsers with graceful journald fallback 2026-05-11 06:57:38 -07:00
test_ingest_qbittorrent.py fix: ingestors treat naive log timestamps as local time, not UTC 2026-05-13 18:16:33 -07:00
test_ingest_syslog.py fix: ingestors treat naive log timestamps as local time, not UTC 2026-05-13 18:16:33 -07:00
test_ingest_tautulli.py fix: tautulli — hmac token compare, public pattern loader, startup cache, endpoint tests 2026-05-13 19:08:49 -07:00
test_ingest_wazuh.py feat: periodic ingest scheduler + Orchard submission pipeline 2026-05-20 08:57:25 -07:00
test_service_blocklist.py feat(blocklist): extraction scan + candidate CRUD + full test suite 2026-05-15 21:05:49 -07:00
test_service_pihole.py fix(blocklist): validate _v6_auth session JSON, add auth-failure test 2026-05-15 21:03:03 -07:00
test_services_diagnose.py fix: correct time_detected logic, immutable sort pattern, add diagnose() test 2026-05-11 09:08:24 -07:00
test_services_llm.py feat: switch LLM backend to OpenAI-compat; add cf-orch remote inference support 2026-05-12 12:58:38 -07:00
test_watch_watcher.py feat: add file tail source type; configure example-node watchers 2026-05-11 15:44:10 -07:00