turnstone/app
pyr0ball 63c742a708 feat: periodic ingest scheduler + Orchard submission pipeline
Adds asyncio-native background scheduler (TURNSTONE_INGEST_INTERVAL,
default 900s) that runs batch ingest then pushes pattern-matched entries
to a remote CF harvest endpoint (TURNSTONE_SUBMIT_ENDPOINT).

- app/tasks/ingest_scheduler.py: IngestState, scheduler_loop, run_once,
  submit_matched, _query_matched_since — asyncio.Lock prevents concurrent runs
- app/rest.py: POST /api/ingest/batch (pre-parsed entry receiver),
  GET /api/tasks/ingest/status, POST /api/tasks/ingest (manual trigger),
  TURNSTONE_INGEST_INTERVAL + TURNSTONE_SUBMIT_ENDPOINT env wiring in lifespan
- docker-compose.submissions.yml: segregated contrib1 (8536) + contrib2 (8537)
  receiving instances on Heimdall, isolated DBs under
  /devl/docker/turnstone-submissions/<node>/
- podman-standalone.sh: pass-through for TURNSTONE_SUBMIT_ENDPOINT +
  TURNSTONE_SOURCE_HOST
- app/ingest/mqtt_subscriber.py: MQTT log source adapter
- app/ingest/wazuh.py: Wazuh alert JSON adapter
- tests/test_ingest_wazuh.py: Wazuh adapter test suite
2026-05-20 08:57:25 -07:00
..
api feat: initial Turnstone POC — ingest, FTS search, MCP server 2026-05-08 12:12:34 -07:00
context fix: final review fixes — port guard, network error handling, wizard back nav, tablist arrow keys, dialog focus trap 2026-05-13 17:40:40 -07:00
ingest feat: periodic ingest scheduler + Orchard submission pipeline 2026-05-20 08:57:25 -07:00
services fix(blocklist): get_candidate for O(1) push/unblock, 400 on malformed device_names JSON 2026-05-15 21:19:02 -07:00
tasks feat: periodic ingest scheduler + Orchard submission pipeline 2026-05-20 08:57:25 -07:00
watch feat: add file tail source type; configure example-node watchers 2026-05-11 15:44:10 -07:00
__init__.py feat: initial Turnstone POC — ingest, FTS search, MCP server 2026-05-08 12:12:34 -07:00
mcp_server.py feat: initial Turnstone POC — ingest, FTS search, MCP server 2026-05-08 12:12:34 -07:00
rest.py feat: periodic ingest scheduler + Orchard submission pipeline 2026-05-20 08:57:25 -07:00