turnstone

169 commits 12 branches 9 tags 4.9 MiB

Author	SHA1	Message	Date
pyr0ball	5a9281a686	fix(glean): add timeout=30s to all pipeline DB connections; add --force flag; new patterns pipeline.py: - Add timeout=30.0 to all sqlite3.connect() calls (5 total). Previously only ensure_context_schema() had it. The main glean writers would fail immediately under lock contention from the live watcher or concurrent manual glean runs. glean_corpus.py: - Add --force flag (passed through to glean_sources/glean_file/glean_dir). Without it, unchanged-fingerprint files were silently skipped even after pattern updates. Use after editing patterns/default.yaml. patterns/default.yaml: - Add 9 new patterns for Muninn / cluster-wide coverage: vpn_tunnel_fail WireGuard/tunnel service failures vpn_handshake WireGuard peer handshake events dns_degraded systemd-resolved DNS fallback/degradation nvidia_api_mismatch NVIDIA kernel module vs userspace mismatch nvidia_xid NVIDIA Xid GPU hardware faults nvidia_gpu_reset NVIDIA GPU reset / NVLink faults acpi_error ACPI firmware _DSM evaluation failures thermal_throttle CPU/GPU thermal throttling / RAPL unavailable undervoltage PSU undervoltage / brownout events - Sync from /devl/turnstone-cluster/patterns/default.yaml (authoritative live copy updated first; repo copy was stale)	2026-05-26 22:36:45 -07:00
pyr0ball	828b69768a	refactor: rename ingest → glean throughout codebase Renames the app/ingest/ package to app/glean/ and updates all references across Python modules, shell scripts, Vue components, tests, and documentation. Intentionally preserved: - SQLite column name ingest_time (avoids schema migration) - RetrievedEntry.ingest_time field (maps to the column above) - Any public-facing JSON keys that reference ingest_time Changes by category: - app/ingest/ → app/glean/ (full package move, all parsers) - app/tasks/ingest_scheduler.py → app/tasks/glean_scheduler.py - scripts/ingest_corpus.py → scripts/glean_corpus.py - tests/test_ingest_.py → tests/test_glean_.py - Docstrings, log messages, comments: ingest → glean - Env var: TURNSTONE_INGEST_INTERVAL → TURNSTONE_GLEAN_INTERVAL - Shell scripts: glean.log, glean_corpus.py references - README.md: multi-source ingest → multi-source glean - .env.example: updated env var name - patterns/: new diagnostic patterns from 2026-05-20 SSH incident (service_crash_loop, pkg_daemon_restart, ssh_forward_conflict) - SourcesView.vue: pipeline label updated - All test import paths updated to app.glean.* 285 tests passing.	2026-05-20 23:02:55 -07:00

Author

SHA1

Message

Date

pyr0ball

5a9281a686

fix(glean): add timeout=30s to all pipeline DB connections; add --force flag; new patterns

pipeline.py:
- Add timeout=30.0 to all sqlite3.connect() calls (5 total).
  Previously only ensure_context_schema() had it. The main glean
  writers would fail immediately under lock contention from the live
  watcher or concurrent manual glean runs.

glean_corpus.py:
- Add --force flag (passed through to glean_sources/glean_file/glean_dir).
  Without it, unchanged-fingerprint files were silently skipped even
  after pattern updates. Use after editing patterns/default.yaml.

patterns/default.yaml:
- Add 9 new patterns for Muninn / cluster-wide coverage:
    vpn_tunnel_fail     WireGuard/tunnel service failures
    vpn_handshake       WireGuard peer handshake events
    dns_degraded        systemd-resolved DNS fallback/degradation
    nvidia_api_mismatch NVIDIA kernel module vs userspace mismatch
    nvidia_xid          NVIDIA Xid GPU hardware faults
    nvidia_gpu_reset    NVIDIA GPU reset / NVLink faults
    acpi_error          ACPI firmware _DSM evaluation failures
    thermal_throttle    CPU/GPU thermal throttling / RAPL unavailable
    undervoltage        PSU undervoltage / brownout events
- Sync from /devl/turnstone-cluster/patterns/default.yaml (authoritative
  live copy updated first; repo copy was stale)

2026-05-26 22:36:45 -07:00

pyr0ball

828b69768a

refactor: rename ingest → glean throughout codebase

Renames the app/ingest/ package to app/glean/ and updates all
references across Python modules, shell scripts, Vue components,
tests, and documentation.

Intentionally preserved:
- SQLite column name ingest_time (avoids schema migration)
- RetrievedEntry.ingest_time field (maps to the column above)
- Any public-facing JSON keys that reference ingest_time

Changes by category:
- app/ingest/ → app/glean/ (full package move, all parsers)
- app/tasks/ingest_scheduler.py → app/tasks/glean_scheduler.py
- scripts/ingest_corpus.py → scripts/glean_corpus.py
- tests/test_ingest_*.py → tests/test_glean_*.py
- Docstrings, log messages, comments: ingest → glean
- Env var: TURNSTONE_INGEST_INTERVAL → TURNSTONE_GLEAN_INTERVAL
- Shell scripts: glean.log, glean_corpus.py references
- README.md: multi-source ingest → multi-source glean
- .env.example: updated env var name
- patterns/: new diagnostic patterns from 2026-05-20 SSH incident
  (service_crash_loop, pkg_daemon_restart, ssh_forward_conflict)
- SourcesView.vue: pipeline label updated
- All test import paths updated to app.glean.*

285 tests passing.

2026-05-20 23:02:55 -07:00

Renamed from scripts/ingest_corpus.py (Browse further)

2 commits