pyr0ball
0f86d35062
Adds asyncio-native background scheduler (TURNSTONE_INGEST_INTERVAL, default 900s) that runs batch ingest then pushes pattern-matched entries to a remote CF harvest endpoint (TURNSTONE_SUBMIT_ENDPOINT). - app/tasks/ingest_scheduler.py: IngestState, scheduler_loop, run_once, submit_matched, _query_matched_since — asyncio.Lock prevents concurrent runs - app/rest.py: POST /api/ingest/batch (pre-parsed entry receiver), GET /api/tasks/ingest/status, POST /api/tasks/ingest (manual trigger), TURNSTONE_INGEST_INTERVAL + TURNSTONE_SUBMIT_ENDPOINT env wiring in lifespan - docker-compose.submissions.yml: segregated daniel (8536) + xander (8537) receiving instances on Heimdall, isolated DBs under /devl/docker/turnstone-submissions/<node>/ - podman-standalone.sh: pass-through for TURNSTONE_SUBMIT_ENDPOINT + TURNSTONE_SOURCE_HOST - app/ingest/mqtt_subscriber.py: MQTT log source adapter - app/ingest/wazuh.py: Wazuh alert JSON adapter - tests/test_ingest_wazuh.py: Wazuh adapter test suite
28 lines
1.2 KiB
Text
28 lines
1.2 KiB
Text
# Turnstone environment variables
|
|
# Copy to .env and adjust for your setup. All variables are optional unless noted.
|
|
|
|
# --- Database & paths ---
|
|
# TURNSTONE_DB=/data/turnstone.db
|
|
# TURNSTONE_PATTERNS=/patterns
|
|
# TURNSTONE_SOURCE_HOST=my-server
|
|
|
|
# --- GPU / LLM inference ---
|
|
# GPU_SERVER_URL — URL of your GPU inference server (Ollama, vLLM, or cf-orch coordinator).
|
|
# Paid+ users: leave unset to auto-default to https://orch.circuitforge.tech via CF_LICENSE_KEY.
|
|
# Local Ollama (default if unset): http://localhost:11434
|
|
# Local cf-orch coordinator: http://<YOUR_HOST_IP>:7700
|
|
# CF_ORCH_URL is also accepted as a backward-compatible alias.
|
|
# GPU_SERVER_URL=http://localhost:11434
|
|
|
|
# --- CircuitForge license (Paid+) ---
|
|
# Enables cloud GPU inference and premium features.
|
|
# When set, GPU_SERVER_URL defaults to https://orch.circuitforge.tech automatically.
|
|
# CF_LICENSE_KEY=CFG-TRSN-XXXX-XXXX-XXXX
|
|
|
|
# --- Bundle endpoint (optional) ---
|
|
# Remote endpoint to push diagnostic bundles for escalation.
|
|
# TURNSTONE_BUNDLE_ENDPOINT=https://example.com/api/bundles
|
|
|
|
# --- Periodic batch ingest ---
|
|
# Seconds between automatic ingest runs from sources.yaml. Set to 0 to disable.
|
|
# TURNSTONE_INGEST_INTERVAL=900
|