Circuit-Forge/turnstone
5
0
Fork 0
Code Issues 11 Pull requests 1 Projects Releases 6 Packages Wiki Activity Actions
turnstone/podman-standalone.sh
pyr0ball b6b69e2150 feat(incidents): auto-incident detection + example-node Podman setup 
Auto-incident detector:
- New app/tasks/incident_detector.py: post-glean error cluster detector
  - Sliding window algorithm: source + N errors within window_s seconds
  - Deduplication via issue_type='auto:{source_id}' + interval overlap check
  - Respects TURNSTONE_AUTO_INCIDENT_THRESHOLD (default 5) and
    TURNSTONE_AUTO_INCIDENT_WINDOW (default 600s) env vars
  - 20 tests all passing
- Wired into glean_scheduler.run_once() and scheduler_loop()
- TURNSTONE_AUTO_INCIDENT env var to disable (default enabled)

Podman standalone improvements:
- REPO_DIR auto-detected from script location (no longer hardcoded to /opt/turnstone)
- DATA_DIR/PATTERNS_DIR/HF_CACHE_DIR configurable via env vars
- Bootstrap step copies host-specific sources-<hostname>.yaml on first run
- Auto-incident env vars passed through

example-node sources:
- patterns/sources-example-node.yaml: Sonarr, Radarr, Bazarr, Prowlarr,
  Tautulli, autoscan, organizr, nextcloud, journal export
2026-06-11 18:37:53 -07:00

201 lines
9.7 KiB
Bash
Executable file
Raw Blame History

#!/usr/bin/env bash
# podman-standalone.sh — Turnstone rootful Podman setup (no Compose)
#
# For hosts running system Podman (non-rootless) with systemd.
# Turnstone is a diagnostic log intelligence layer — glean service logs,
# search by symptom, and view incidents in a lightweight web UI.
#
# ── Prerequisites ────────────────────────────────────────────────────────────
# 1. Clone the repo:
# sudo git clone https://git.opensourcesolarpunk.com/Circuit-Forge/turnstone.git /opt/turnstone
# sudo chown -R x:x /opt/turnstone
#
# 2. Build the image (requires Docker or Podman with BuildKit/multi-stage support):
# cd /opt/turnstone && podman build -t localhost/turnstone:latest .
#
# 3. Create data and patterns directories, then copy config files:
# mkdir -p /opt/turnstone/{data,patterns}
# cp /opt/turnstone/patterns/default.yaml /opt/turnstone/patterns/
# cp /opt/turnstone/patterns/sources.yaml /opt/turnstone/patterns/
# # Edit sources.yaml if any paths differ on this host.
#
# 4. Run this script:
# bash /opt/turnstone/podman-standalone.sh
#
# ── After setup — generate systemd unit file ─────────────────────────────────
# sudo podman generate systemd --new --name turnstone \
# | sudo tee /etc/systemd/system/turnstone.service
# sudo systemctl daemon-reload
# sudo systemctl enable --now turnstone
#
# ── Gleaning logs ─────────────────────────────────────────────────────────────
# All service logs under /opt are accessible inside the container.
# Sources are configured in patterns/sources.yaml (bind-mounted at /patterns/).
#
# To glean all sources (run manually or via cron):
#
# sudo podman exec turnstone python scripts/glean_corpus.py \
# --sources /patterns/sources.yaml --db /data/turnstone.db
#
# Example cron (every 15 minutes, add to root's crontab with: sudo crontab -e):
# */15 * * * * podman exec turnstone python scripts/glean_corpus.py \
# --sources /patterns/sources.yaml --db /data/turnstone.db >> /var/log/turnstone-glean.log 2>&1
#
# To add a new log source: edit /opt/turnstone/patterns/sources.yaml — no restart needed.
#
# ── Adding Caddy reverse proxy ────────────────────────────────────────────────
# Add to /etc/caddy/Caddyfile:
#
# turnstone.example-node.tv {
# import protected
# reverse_proxy 10.0.0.10:8534
# import cloudflare
# }
#
# Then: sudo systemctl reload caddy
#
# ── Ports ────────────────────────────────────────────────────────────────────
# Turnstone UI → http://localhost:8534/turnstone/
#
set -euo pipefail
# Auto-detect repo from script location — works whether cloned to /opt/turnstone
# or to /Library/Development/CircuitForge/turnstone or any other path.
REPO_DIR="${TURNSTONE_REPO_DIR:-$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)}"
# Data and patterns live OUTSIDE the repo so they survive git pulls.
DATA_DIR="${TURNSTONE_DATA_DIR:-/opt/turnstone-data}"
PATTERNS_DIR="${TURNSTONE_PATTERNS_DIR:-${DATA_DIR}/patterns}"
HF_CACHE_DIR="${TURNSTONE_HF_CACHE:-${DATA_DIR}/hf-cache}"
TZ="${TZ:-America/Los_Angeles}"
# ── Bundle push configuration ────────────────────────────────────────────────
# Set TURNSTONE_BUNDLE_ENDPOINT before running this script to enable the
# "Send Bundle" button in the Incidents UI:
#
# export TURNSTONE_BUNDLE_ENDPOINT=https://turnstone.circuitforge.tech/turnstone/api/bundles
# bash /opt/turnstone/podman-standalone.sh
#
# ── Orchard submission (opt-in telemetry) ────────────────────────────────────
# Set TURNSTONE_SUBMIT_ENDPOINT to push pattern-matched log entries to a CF
# receiving instance after each glean run. Only matched entries are sent —
# no raw log content. Used to build Avocet training data.
#
# export TURNSTONE_SUBMIT_ENDPOINT=https://harvest.circuitforge.tech/contrib2
# bash /opt/turnstone/podman-standalone.sh
#
# TURNSTONE_SOURCE_HOST is auto-detected from `hostname` — override if needed.
#
# ── Multi-agent diagnose pipeline ────────────────────────────────────────────
# The 5-stage ML pipeline requires three env vars and a writable HF cache dir:
#
# TURNSTONE_MULTI_AGENT_DIAGNOSE=true — enable the pipeline
# GPU_SERVER_URL=http://<orch-host>:7700 — cf-orch coordinator or Ollama base URL
#
# ML models are downloaded on first diagnose run and cached in HF_CACHE_DIR.
# On a CPU-only host (no GPU) set TURNSTONE_EMBED_DEVICE=cpu (default).
#
# For Contributor2's instance (example-node.tv) — no WireGuard to Heimdall LAN,
# use the public cf-orch endpoint instead:
# export GPU_SERVER_URL=https://orch.circuitforge.tech
# export TURNSTONE_MULTI_AGENT_DIAGNOSE=true
# sudo bash /opt/turnstone/podman-standalone.sh
#
# For Contributor's instance (Huginn) — WireGuard reaches Heimdall LAN directly,
# use docker-standalone.sh (not this script — Docker host):
# export GPU_SERVER_URL=http://<YOUR_HOST_IP>:7700
# export TURNSTONE_MULTI_AGENT_DIAGNOSE=true
# bash ~/turnstone/docker-standalone.sh
# ── Turnstone container ───────────────────────────────────────────────────────
# Image is built locally — no registry auto-update label.
# Run this script after every `git pull` to rebuild and redeploy.
#
# /opt is mounted read-only so all service logs under /opt/*/config/logs/ are
# accessible without per-service mounts. Add new sources to patterns/sources.yaml
# — no container restart needed.
#
# Must be run as root (sudo bash podman-standalone.sh) — rootful Podman only.
#
# Bootstrap data and patterns dirs if this is a first run
mkdir -p "${DATA_DIR}" "${PATTERNS_DIR}" "${HF_CACHE_DIR}"
# Copy default patterns if the dir is empty (first run only)
if [ -z "$(ls -A "${PATTERNS_DIR}")" ]; then
cp "${REPO_DIR}/patterns/default.yaml" "${PATTERNS_DIR}/"
# Copy host-specific sources if present, otherwise copy the generic template
HOST_SOURCES="${REPO_DIR}/patterns/sources-$(hostname).yaml"
if [ -f "${HOST_SOURCES}" ]; then
cp "${HOST_SOURCES}" "${PATTERNS_DIR}/sources.yaml"
echo "==> Installed host-specific sources: ${HOST_SOURCES}"
else
cp "${REPO_DIR}/patterns/sources.yaml" "${PATTERNS_DIR}/"
echo "==> Installed default sources.yaml — edit ${PATTERNS_DIR}/sources.yaml for this host"
fi
fi
# Build image from current source (bakes app/ code into the image)
echo "Building Turnstone image..."
podman build -t localhost/turnstone:latest "${REPO_DIR}"
# Remove existing container if present (safe re-run)
podman rm -f turnstone 2>/dev/null || true
podman run -d \
--name=turnstone \
--restart=unless-stopped \
--net=host \
-v "${DATA_DIR}:/data:Z" \
-v "${PATTERNS_DIR}:/patterns:Z" \
-v "${HF_CACHE_DIR}:/hf-cache:Z" \
-v /opt:/opt:ro \
-v /var/log:/var/log:ro \
-e TURNSTONE_DB=/data/turnstone.db \
-e TURNSTONE_SOURCE_HOST="$(hostname)" \
-e TURNSTONE_BUNDLE_ENDPOINT="${TURNSTONE_BUNDLE_ENDPOINT:-}" \
-e TURNSTONE_SUBMIT_ENDPOINT="${TURNSTONE_SUBMIT_ENDPOINT:-}" \
-e PYTHONUNBUFFERED=1 \
-e TZ="${TZ}" \
-e TURNSTONE_MULTI_AGENT_DIAGNOSE="${TURNSTONE_MULTI_AGENT_DIAGNOSE:-false}" \
-e GPU_SERVER_URL="${GPU_SERVER_URL:-}" \
-e HF_HOME=/hf-cache \
-e TURNSTONE_AUTO_INCIDENT="${TURNSTONE_AUTO_INCIDENT:-true}" \
-e TURNSTONE_AUTO_INCIDENT_THRESHOLD="${TURNSTONE_AUTO_INCIDENT_THRESHOLD:-5}" \
-e TURNSTONE_AUTO_INCIDENT_WINDOW="${TURNSTONE_AUTO_INCIDENT_WINDOW:-600}" \
-e TURNSTONE_CLASSIFIER_MODEL="${TURNSTONE_CLASSIFIER_MODEL:-byviz/bylastic_classification_logs}" \
-e TURNSTONE_EMBED_BACKEND="${TURNSTONE_EMBED_BACKEND:-sentence_transformers}" \
-e TURNSTONE_EMBED_MODEL="${TURNSTONE_EMBED_MODEL:-sentence-transformers/all-MiniLM-L6-v2}" \
-e TURNSTONE_EMBED_DEVICE="${TURNSTONE_EMBED_DEVICE:-cpu}" \
--health-cmd="curl -f http://localhost:8534/turnstone/health || exit 1" \
--health-interval=30s \
--health-timeout=10s \
--health-start-period=20s \
--health-retries=3 \
localhost/turnstone:latest
echo ""
echo "Turnstone is starting up."
echo " UI: http://localhost:8534/turnstone/"
echo ""
# Regenerate systemd unit so it references the freshly-built image.
# The --new flag means systemd re-creates the container on each start
# rather than binding to a specific container ID.
if [ -d /etc/systemd/system ]; then
echo "Regenerating systemd unit..."
podman generate systemd --new --name turnstone \
| tee /etc/systemd/system/turnstone.service > /dev/null
systemctl daemon-reload
systemctl enable turnstone.service 2>/dev/null || true
echo " systemd unit updated — run: sudo systemctl restart turnstone.service"
echo ""
fi
echo "Check container health with:"
echo " sudo podman ps"
echo " sudo podman logs turnstone"
echo ""
echo "To glean all sources now:"
echo " sudo podman exec turnstone python scripts/glean_corpus.py \\"
echo " --sources /patterns/sources.yaml --db /data/turnstone.db"
echo ""
echo "To add a new source: edit /opt/turnstone/patterns/sources.yaml — no restart needed."
Reference in a new issue View git blame Copy permalink