pyr0ball
8d5324f1fe
Turnstone incidents now carry an issue_type tag (free-text with datalist
suggestions) used to categorize patterns for signature building.
Backend:
- Incident model gains issue_type; additive ALTER TABLE migration keeps
existing DBs working without a full schema rebuild
- New received_bundles table stores incoming JSON bundles with indexes on
bundled_at and issue_type
- build_bundle() assembles incident + related log entries into a versioned
bundle dict; store_bundle()/list_bundles()/get_bundle() for the receiver
- POST /api/incidents/{id}/send — pushes bundle to TURNSTONE_BUNDLE_ENDPOINT
- GET /api/incidents/{id}/bundle — export without sending
- POST /api/bundles — receive and store an incoming bundle
- GET /api/bundles — list all received bundles
- TURNSTONE_SOURCE_HOST and TURNSTONE_BUNDLE_ENDPOINT env vars; auto-set
source host from hostname in podman-standalone.sh
Frontend:
- Incidents form: issue_type field with datalist suggestions; Type column
in the table; Send Bundle button + status feedback in the detail drawer
- New BundlesView: collapsible bundle rows, inline JSON parse (no extra
round-trip), Export JSON download button
- Router and nav updated with /bundles route
120 lines
5.4 KiB
Bash
Executable file
120 lines
5.4 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# podman-standalone.sh — Turnstone rootful Podman setup (no Compose)
|
|
#
|
|
# For hosts running system Podman (non-rootless) with systemd.
|
|
# Turnstone is a diagnostic log intelligence layer — ingest service logs,
|
|
# search by symptom, and view incidents in a lightweight web UI.
|
|
#
|
|
# ── Prerequisites ────────────────────────────────────────────────────────────
|
|
# 1. Clone the repo:
|
|
# sudo git clone https://git.opensourcesolarpunk.com/Circuit-Forge/turnstone.git /opt/turnstone
|
|
# sudo chown -R x:x /opt/turnstone
|
|
#
|
|
# 2. Build the image (requires Docker or Podman with BuildKit/multi-stage support):
|
|
# cd /opt/turnstone && podman build -t localhost/turnstone:latest .
|
|
#
|
|
# 3. Create data and custom patterns directories:
|
|
# mkdir -p /opt/turnstone/{data,patterns}
|
|
# # Optionally copy default patterns as a starting point:
|
|
# cp /opt/turnstone/patterns/default.yaml /opt/turnstone/patterns/
|
|
#
|
|
# 4. Run this script:
|
|
# bash /opt/turnstone/podman-standalone.sh
|
|
#
|
|
# ── After setup — generate systemd unit file ─────────────────────────────────
|
|
# sudo podman generate systemd --new --name turnstone \
|
|
# | sudo tee /etc/systemd/system/turnstone.service
|
|
# sudo systemctl daemon-reload
|
|
# sudo systemctl enable --now turnstone
|
|
#
|
|
# ── Ingesting logs ────────────────────────────────────────────────────────────
|
|
# Log files on the host are bind-mounted read-only under /logs/ in the
|
|
# container. To ingest (run manually or via cron):
|
|
#
|
|
# podman exec turnstone python scripts/ingest_corpus.py \
|
|
# /logs/qbittorrent/qbittorrent.log /data/turnstone.db
|
|
#
|
|
# Example cron (every 15 minutes):
|
|
# */15 * * * * podman exec turnstone python scripts/ingest_corpus.py \
|
|
# /logs/qbittorrent/qbittorrent.log /data/turnstone.db >> /var/log/turnstone-ingest.log 2>&1
|
|
#
|
|
# ── Adding Caddy reverse proxy ────────────────────────────────────────────────
|
|
# Add to /etc/caddy/Caddyfile:
|
|
#
|
|
# turnstone.example-node.tv {
|
|
# import protected
|
|
# reverse_proxy 10.0.0.10:8534
|
|
# import cloudflare
|
|
# }
|
|
#
|
|
# Then: sudo systemctl reload caddy
|
|
#
|
|
# ── Ports ────────────────────────────────────────────────────────────────────
|
|
# Turnstone UI → http://localhost:8534/turnstone/
|
|
#
|
|
set -euo pipefail
|
|
|
|
REPO_DIR=/opt/turnstone
|
|
DATA_DIR=/opt/turnstone/data
|
|
PATTERNS_DIR=/opt/turnstone/patterns
|
|
TZ=America/Los_Angeles
|
|
|
|
# ── Bundle push configuration ────────────────────────────────────────────────
|
|
# Set TURNSTONE_BUNDLE_ENDPOINT before running this script to enable the
|
|
# "Send Bundle" button in the Incidents UI:
|
|
#
|
|
# export TURNSTONE_BUNDLE_ENDPOINT=https://turnstone.circuitforge.tech/turnstone/api/bundles
|
|
# bash /opt/turnstone/podman-standalone.sh
|
|
#
|
|
# TURNSTONE_SOURCE_HOST is auto-detected from `hostname` — override if needed.
|
|
|
|
# ── Log source bind mounts ────────────────────────────────────────────────────
|
|
# Add or remove mount flags below for each service whose logs you want to ingest.
|
|
# Inside the container, paths appear under /logs/<service>/
|
|
#
|
|
QBIT_LOGS=/opt/qbittorrent/config/data/logs
|
|
|
|
# ── Turnstone container ───────────────────────────────────────────────────────
|
|
# Image is built locally — no registry auto-update label.
|
|
# To update: podman build -t localhost/turnstone:latest /opt/turnstone
|
|
# podman restart turnstone
|
|
#
|
|
# Remove existing container if present (safe re-run)
|
|
podman rm -f turnstone 2>/dev/null || true
|
|
|
|
podman run -d \
|
|
--name=turnstone \
|
|
--restart=unless-stopped \
|
|
--net=host \
|
|
-v "${DATA_DIR}:/data:Z" \
|
|
-v "${PATTERNS_DIR}:/patterns:Z" \
|
|
-v "${QBIT_LOGS}:/logs/qbittorrent:ro" \
|
|
-e TURNSTONE_DB=/data/turnstone.db \
|
|
-e TURNSTONE_SOURCE_HOST="$(hostname)" \
|
|
-e TURNSTONE_BUNDLE_ENDPOINT="${TURNSTONE_BUNDLE_ENDPOINT:-}" \
|
|
-e PYTHONUNBUFFERED=1 \
|
|
-e TZ="${TZ}" \
|
|
--health-cmd="curl -f http://localhost:8534/turnstone/health || exit 1" \
|
|
--health-interval=30s \
|
|
--health-timeout=10s \
|
|
--health-start-period=20s \
|
|
--health-retries=3 \
|
|
localhost/turnstone:latest
|
|
|
|
echo ""
|
|
echo "Turnstone is starting up."
|
|
echo " UI: http://localhost:8534/turnstone/"
|
|
echo ""
|
|
echo "Check container health with:"
|
|
echo " podman ps"
|
|
echo " podman logs turnstone"
|
|
echo ""
|
|
echo "To register as a systemd service:"
|
|
echo " sudo podman generate systemd --new --name turnstone \\"
|
|
echo " | sudo tee /etc/systemd/system/turnstone.service"
|
|
echo " sudo systemctl daemon-reload"
|
|
echo " sudo systemctl enable --now turnstone"
|
|
echo ""
|
|
echo "To ingest qBittorrent logs now:"
|
|
echo " podman exec turnstone python scripts/ingest_corpus.py \\"
|
|
echo " /logs/qbittorrent/qbittorrent.log /data/turnstone.db"
|