pyr0ball
aa80f307fe
Renames the app/ingest/ package to app/glean/ and updates all references across Python modules, shell scripts, Vue components, tests, and documentation. Intentionally preserved: - SQLite column name ingest_time (avoids schema migration) - RetrievedEntry.ingest_time field (maps to the column above) - Any public-facing JSON keys that reference ingest_time Changes by category: - app/ingest/ → app/glean/ (full package move, all parsers) - app/tasks/ingest_scheduler.py → app/tasks/glean_scheduler.py - scripts/ingest_corpus.py → scripts/glean_corpus.py - tests/test_ingest_*.py → tests/test_glean_*.py - Docstrings, log messages, comments: ingest → glean - Env var: TURNSTONE_INGEST_INTERVAL → TURNSTONE_GLEAN_INTERVAL - Shell scripts: glean.log, glean_corpus.py references - README.md: multi-source ingest → multi-source glean - .env.example: updated env var name - patterns/: new diagnostic patterns from 2026-05-20 SSH incident (service_crash_loop, pkg_daemon_restart, ssh_forward_conflict) - SourcesView.vue: pipeline label updated - All test import paths updated to app.glean.* 285 tests passing.
96 lines
3.7 KiB
YAML
96 lines
3.7 KiB
YAML
# Turnstone log sources — edit this file to add or remove services.
|
|
# NOTE: the system-journal entry requires export_journal.sh to run on the HOST
|
|
# before the container glean step. See crontab setup instructions in the README.
|
|
# Run glean manually:
|
|
# sudo podman exec turnstone python scripts/glean_corpus.py \
|
|
# --sources /patterns/sources.yaml --db /data/turnstone.db
|
|
#
|
|
# Paths here are container-side paths under the /opt bind mount.
|
|
# Missing paths are skipped with a warning — safe to leave entries for
|
|
# services that are temporarily down.
|
|
|
|
sources:
|
|
# ── System (exported by export_journal.sh on the host) ───────────────────
|
|
# journal-export.jsonl and dmesg-export.txt are written to /opt/turnstone/data/
|
|
# by the export script before each glean run.
|
|
- id: system-journal
|
|
path: /data/journal-export.jsonl
|
|
|
|
- id: dmesg
|
|
path: /data/dmesg-export.txt
|
|
|
|
# ── Syslog / rsyslog (direct file reads via /var/log bind mount) ──────────
|
|
# Uncomment the file(s) present on your system.
|
|
# Debian/Ubuntu:
|
|
# - id: syslog
|
|
# path: /var/log/syslog
|
|
# - id: auth-log
|
|
# path: /var/log/auth.log
|
|
# - id: kern-log
|
|
# path: /var/log/kern.log
|
|
# RHEL/Fedora/Rocky:
|
|
# - id: messages
|
|
# path: /var/log/messages
|
|
# - id: secure
|
|
# path: /var/log/secure
|
|
|
|
# ── Download ─────────────────────────────────────────────────────────────
|
|
- id: qbittorrent
|
|
path: /opt/qbittorrent/config/data/logs/qbittorrent.log
|
|
|
|
# ── Servarr stack ─────────────────────────────────────────────────────────
|
|
- id: sonarr
|
|
path: /opt/sonarr/config/logs/sonarr.0.txt
|
|
|
|
- id: radarr
|
|
path: /opt/radarr/config/logs/radarr.0.txt
|
|
|
|
- id: lidarr
|
|
path: /opt/lidarr/config/logs/Lidarr.0.txt
|
|
|
|
- id: readarr
|
|
path: /opt/readarr/config/logs/readarr.0.txt
|
|
|
|
- id: whisparr
|
|
path: /opt/whisparr/config/logs/whisparr.0.txt
|
|
|
|
- id: prowlarr
|
|
path: /opt/prowlarr/config/logs/prowlarr.0.txt
|
|
|
|
- id: bazarr
|
|
path: /opt/bazarr/config/log/bazarr.log
|
|
|
|
# ── Usenet ────────────────────────────────────────────────────────────────
|
|
- id: nzbget
|
|
path: /opt/nzbget/config/nzbget.log
|
|
|
|
# ── Media / Requests ─────────────────────────────────────────────────────
|
|
- id: tautulli
|
|
path: /opt/tautulli/config/logs/tautulli.log
|
|
|
|
- id: jellyseerr
|
|
path: /opt/jellyseerr/config/logs/jellyseerr.log
|
|
|
|
# ── MQTT / IoT (live — subscribe mode, no path needed) ───────────────────
|
|
# Requires: pip install circuitforge-core[mqtt]
|
|
# These sources are handled by the live MQTT subscriber task (not batch glean).
|
|
# Uncomment and configure to enable.
|
|
#
|
|
# Meshtastic MQTT bridge (node must have MQTT uplink enabled):
|
|
# - id: meshtastic-home
|
|
# type: mqtt
|
|
# broker_host: 10.1.10.5 # IP of your local MQTT broker (e.g. Mosquitto on Huginn)
|
|
# broker_port: 1883
|
|
# topics:
|
|
# - msh/# # all Meshtastic regions; use msh/us-east/# to narrow
|
|
#
|
|
# Generic IoT sensors:
|
|
# - id: iot-home
|
|
# type: mqtt
|
|
# broker_host: localhost
|
|
# broker_port: 1883
|
|
# topics:
|
|
# - home/+/temperature
|
|
# - home/+/humidity
|
|
# - home/+/motion
|
|
# severity: INFO
|