turnstone/patterns/sources.yaml
pyr0ball f46aba8165 feat: multi-source ingest via sources.yaml + servarr parser
- Add servarr.py parser for all *arr services (sonarr/radarr/lidarr/
  prowlarr/readarr/whisparr/bazarr) — pipe-delimited format with
  component prefix prepended for searchability
- Add ingest_sources() to pipeline.py; reads sources.yaml, skips
  missing paths with a warning so cron keeps running if a service
  is down
- Add --sources mode to ingest_corpus.py CLI; legacy positional args
  unchanged for backward compat
- Add patterns/sources.yaml with all of Contributor2's discovered service
  log paths (qbit, 7 servarr services, nzbget, tautulli, jellyseerr)
- Replace per-service volume mounts in podman-standalone.sh with
  /opt:/opt:ro + /var/log:/var/log:ro; adding a new source now
  requires only editing sources.yaml — no container restart
2026-05-11 06:26:32 -07:00

46 lines
1.9 KiB
YAML

# Turnstone log sources — edit this file to add or remove services.
# Run ingest manually:
# sudo podman exec turnstone python scripts/ingest_corpus.py \
# --sources /patterns/sources.yaml --db /data/turnstone.db
#
# Paths here are container-side paths under the /opt bind mount.
# Missing paths are skipped with a warning — safe to leave entries for
# services that are temporarily down.
sources:
# ── Download ─────────────────────────────────────────────────────────────
- id: qbittorrent
path: /opt/qbittorrent/config/data/logs/qbittorrent.log
# ── Servarr stack ─────────────────────────────────────────────────────────
- id: sonarr
path: /opt/sonarr/config/logs/sonarr.0.txt
- id: radarr
path: /opt/radarr/config/logs/radarr.0.txt
- id: lidarr
path: /opt/lidarr/config/logs/Lidarr.0.txt
- id: readarr
path: /opt/readarr/config/logs/readarr.0.txt
- id: whisparr
path: /opt/whisparr/config/logs/whisparr.0.txt
- id: prowlarr
path: /opt/prowlarr/config/logs/prowlarr.0.txt
- id: bazarr
path: /opt/bazarr/config/log/bazarr.log
# ── Usenet ────────────────────────────────────────────────────────────────
- id: nzbget
path: /opt/nzbget/config/nzbget.log
# ── Media / Requests ─────────────────────────────────────────────────────
- id: tautulli
path: /opt/tautulli/config/logs/tautulli.log
- id: jellyseerr
path: /opt/jellyseerr/config/logs/jellyseerr.log