Commit graph

5 commits

Author SHA1 Message Date
b0e4d8d5f4 chore: sanitize internal hostnames and IP references
- Rename patterns/sources-example-node.yaml → patterns/sources-example.yaml
  and update header/comments to be host-agnostic
- Replace internal node names in gen_corpus.py _HOSTS with generic names
- Replace example-node hostname in syslog test fixtures with testhost
- Replace example-node example in mcp_server.py doc with myserver
- Replace private LAN IP (<YOUR_HOST_IP>) in docker-standalone.sh with
  <HEIMDALL_LAN_IP> placeholder
- Replace private IPs in sources-cluster.yaml comments with <YOUR_HOST_IP>
- Remove instance-specific hostname from llm.py fallback comment
- Replace Caddy example domain in podman-standalone.sh with placeholder
2026-06-13 10:02:46 -07:00
ca559b82f2 feat: cybersec zero-shot scoring pipeline (#9)
Second-pass cybersec classifier using DeBERTa-v3-base-mnli (already
cached — no download required). Runs after each anomaly scoring pass on
entries flagged by the anomaly scorer or with pattern matches.

Architecture:
- app/services/cybersec.py: zero-shot-classification pipeline with 5
  cybersec candidate labels (auth failure, privilege escalation, network
  intrusion, malware, data exfiltration). Writes ml_score/ml_label/
  ml_scored_at to log_entries; inserts high-confidence hits into
  detections with scorer='cybersec'.
- app/tasks/cybersec_scorer.py: async background task (same shape as
  anomaly_scorer.py).
- REST: GET/POST /turnstone/api/cybersec/status|run|detections.
  GET /turnstone/api/anomaly/detections now accepts scorer= filter.

Schema: ml_score, ml_label, ml_scored_at added to log_entries; scorer
column added to detections (idempotent migrations + DDL for both SQLite
and Postgres).

UI: Security Alerts view gains Source dropdown (All / Anomaly / Cybersec)
and cybersec scorer status badge. Label dropdown split into optgroups.

Deployment: TURNSTONE_CYBERSEC_MODEL/DEVICE/THRESHOLD vars added to
.env.example, docker-compose.yml, docker-standalone.sh.

Tests: 10 new tests — no model, no eligible entries, scoring, detection
creation, normal label suppression, threshold filtering, pattern-tag
filtering, idempotency, list filtering, scorer column filter.
416/416 passing.

Closes: #9
2026-06-10 01:03:25 -07:00
26a413b093 chore: wire anomaly scoring pipeline into deployment config
Add TURNSTONE_ANOMALY_* env vars to docker-compose.yml, docker-standalone.sh,
and .env.example. Mount shared HF model cache (/Library/Assets/LLM on Heimdall)
as read-only bind in both compose and standalone — avoids re-downloading models
that are already cached by the diagnose pipeline.

Heimdall: byviz/bylastic_classification_logs already cached, threshold 0.80,
glean-triggered only (TURNSTONE_ANOMALY_INTERVAL=0).
2026-06-09 23:01:48 -07:00
0d43b23f0c docs(container): fix GPU_SERVER_URL for Xander — use public orch.circuitforge.tech
Xander's example-node.tv has no WireGuard route to Heimdall's LAN (10.1.10.x),
so the <YOUR_HOST_IP>:7700 private address is unreachable from there.

Use the public cf-orch endpoint instead:
  GPU_SERVER_URL=https://orch.circuitforge.tech

Daniel's Huginn has WireGuard to Heimdall LAN — <YOUR_HOST_IP>:7700 stays correct.
Added both options to docker-standalone.sh for clarity.
2026-05-26 13:39:38 -07:00
d57d44e285 feat(container): add docker-standalone.sh for Docker hosts (Daniel/Huginn)
Mirrors podman-standalone.sh for Docker-native setups. Key differences:
- Uses ~/turnstone as default REPO_DIR (no /opt assumption)
- -p 8534:8534 port mapping instead of --net=host
- No systemd unit generation (Docker --restart=unless-stopped handles reboots)
- Volume mounts without :Z (Docker SELinux labeling differs from Podman)

Documents the multi-agent setup steps for Huginn:
  export GPU_SERVER_URL=http://<YOUR_HOST_IP>:7700
  export TURNSTONE_MULTI_AGENT_DIAGNOSE=true
  bash ~/turnstone/docker-standalone.sh
2026-05-26 13:21:54 -07:00