POST /turnstone/api/ingest/tautulli accepts Tautulli notification agent
payloads and stores them as log_entries under source 'tautulli'. Severity
maps error->CRITICAL, buffer->WARN, all others->None. Optional bearer token
auth via X-Tautulli-Token header + tautulli_token pref. FTS index rebuilt
as a background task after each write. 28 new tests, all passing.
All five parsers (plex, syslog, servarr, qbittorrent, plaintext) were
using .replace(tzinfo=timezone.utc) on naive datetimes parsed from log
files, which slaps a UTC label on what is actually local-time data.
On a UTC-7 system a 2pm entry was stored as 14:00Z instead of 21:00Z,
causing time-window searches to return zero results.
Fix: use .astimezone(timezone.utc) instead, which treats the naive
datetime as local time and converts correctly.
Tests updated to round-trip back to local time for assertion so they
pass on any timezone, not just UTC.
scripts/syslog_receiver.py: asyncio UDP server listening on port 5140,
appends raw syslog lines to network-syslog.txt for the Turnstone live
watcher to tail. Requires no root — port 5140 is non-privileged.
scripts/turnstone-syslog-receiver.service: systemd unit for auto-start.
app/ingest/syslog.py: strip optional RFC 3164 <PRI> prefix before
parsing so network-forwarded syslog (OpenWRT logd, Arista EOS, etc.)
is handled correctly without the PRI value breaking the regex.
- Add servarr.py parser for all *arr services (sonarr/radarr/lidarr/
prowlarr/readarr/whisparr/bazarr) — pipe-delimited format with
component prefix prepended for searchability
- Add ingest_sources() to pipeline.py; reads sources.yaml, skips
missing paths with a warning so cron keeps running if a service
is down
- Add --sources mode to ingest_corpus.py CLI; legacy positional args
unchanged for backward compat
- Add patterns/sources.yaml with all of Xander's discovered service
log paths (qbit, 7 servarr services, nzbget, tautulli, jellyseerr)
- Replace per-service volume mounts in podman-standalone.sh with
/opt:/opt:ro + /var/log:/var/log:ro; adding a new source now
requires only editing sources.yaml — no container restart
Xander's ghcr.io/hotio/qbittorrent:latest container uses a different format
than the classic GUI build: `(N) 2026-04-26T03:32:59 - message` with a
single-char level code before an ISO timestamp, not inside parens.
Added _HOTIO_RE alongside _CLASSIC_RE; unified via _match_line() helper so
parse() loop is unchanged. 28 tests passing, both formats covered.
Turnstone incidents now carry an issue_type tag (free-text with datalist
suggestions) used to categorize patterns for signature building.
Backend:
- Incident model gains issue_type; additive ALTER TABLE migration keeps
existing DBs working without a full schema rebuild
- New received_bundles table stores incoming JSON bundles with indexes on
bundled_at and issue_type
- build_bundle() assembles incident + related log entries into a versioned
bundle dict; store_bundle()/list_bundles()/get_bundle() for the receiver
- POST /api/incidents/{id}/send — pushes bundle to TURNSTONE_BUNDLE_ENDPOINT
- GET /api/incidents/{id}/bundle — export without sending
- POST /api/bundles — receive and store an incoming bundle
- GET /api/bundles — list all received bundles
- TURNSTONE_SOURCE_HOST and TURNSTONE_BUNDLE_ENDPOINT env vars; auto-set
source host from hostname in podman-standalone.sh
Frontend:
- Incidents form: issue_type field with datalist suggestions; Type column
in the table; Send Bundle button + status feedback in the detail drawer
- New BundlesView: collapsible bundle rows, inline JSON parse (no extra
round-trip), Export JSON download button
- Router and nav updated with /bundles route
- Add GET /api/stats endpoint with 24h windowed aggregation (criticals,
errors, per-source health, recent criticals list)
- Fix timestamp format bug: strftime('%Y-%m-%dT%H:%M:%S', ...) to match
stored ISO-8601 T-separated timestamps (datetime('now') uses space)
- Add composite index idx_ts_repeat(timestamp_iso, repeat_count) — drops
stats query from 3.5 s to <1 ms by resolving both WHERE conditions
from the index without table row fetches
- New DashboardView: 3 stat cards, source health table with health dots,
diagnose-per-source button, recent criticals panel, zero-state card
- Router default / → /dashboard; Dashboard first in nav
- DiagnoseView: reads ?q= query param on mount and auto-runs; shows
formatted LLM summary block
- LogEntryRow: expand/collapse for long entries (>200 chars or multiline)
Adds app/ingest/qbittorrent.py — auto-detected by the pipeline on the
(YYYY/MM/DD HH:MM:SS) timestamp fingerprint. Handles both slash and dash
date separators, optional [Warning|Critical] bracket levels, and
multi-line continuations (Qt stack traces).
patterns/default.yaml: 8 new qbit_ patterns covering tracker errors,
port bind failures, disk errors, hash check failures, peer bans, download
completion, ratio limits, and session errors.
manage.sh: ingest-qbit [HOST] command mirrors ingest-plex — probes known
default log paths locally or via SSH, ingests, restarts server.
14 tests covering format detection, severity mapping, multiline handling,
and timestamp normalization.
- Add `incidents` table to SQLite schema (id, label, started_at, ended_at,
notes, created_at, severity)
- Extract `ensure_schema()` from ingest pipeline so tables are always
created at startup, not only during ingest
- New `app/services/incidents.py`: create/list/get/delete + time-window
entry association (FTS keyword search + raw window fallback)
- New `entries_in_window()` in search.py: plain SQL scan for incident
detail when keyword FTS returns nothing
- REST endpoints: POST/GET /api/incidents, GET/DELETE /api/incidents/{id}
- Incident detail returns up to 100 associated log entries sorted by
timestamp, prioritising FTS keyword hits then ERROR/CRITICAL then all
- app/ingest/plex.py: Plex Media Server log parser
Regex-based line parser for 'Mon DD, YYYY HH:MM:SS.mmm [pid] LEVEL - msg'
format. Handles multi-line entries (stack traces). Detects plex_eae_failure
and all other patterns via shared pattern library.
- app/ingest/plaintext.py: generic fallback parser for unrecognized formats
Extracts timestamps (ISO 8601, syslog, common log) and severity via regex.
- pipeline.py: detect plex format via is_plex_log(); fall back to plaintext
instead of skipping; process *.log files alongside *.jsonl; add ingest_file()
for single-file ingestion.
- scripts/ingest_corpus.py: accept single file or directory as target
- manage.sh: ingest-plex command SSHes to Cass (or HOST arg), pulls
Plex Media Server.log, and ingests it directly
Ingest pipeline (journald / Caddy / Docker-wrapped formats) with
per-source state tracking (repeat dedup, out-of-order detection),
named pattern tagging at ingest time, and idempotent SHA1-keyed writes.
FTS5 search layer with porter stemmer, severity/source/pattern/time
filters, and BM25 ranking. MCP server (FastMCP stdio) with three tools:
search_logs, diagnose, list_log_sources — compatible with both
Claude Code and Copilot CLI.
WAL mode enabled on all connections. FTS index auto-built after ingest.
MCP configs included for Claude Code (.mcp.json) and Copilot CLI
(.github/copilot/mcp.json).
