pyr0ball
a6ea0b9c58
Implements the full signal detection pipeline: Backend: - app/services/lemmy/client.py: async Lemmy API v3 client, community@instance addressing, integer cursor dedup, normalised post dicts - app/services/scraper.py: platform-agnostic scraper; Reddit (.json API, fullname cursor) + Lemmy (integer ID cursor); keyword/regex/all match modes, min_score gate, NormalizedPost shape, upsert dedup via UNIQUE post_id - app/api/endpoints/signals.py: CRUD for signal_rules + signals queue; POST /signals/scrape manual trigger; scrape-state viewer - migrations 010-012: signal_rules, signals, signal_scrape_state tables - scheduler: interval job every 30 min (scraper_enabled=True in config) - Fixed migration collision: 007_signal_rules.sql → 010, 008 → 011, 009 → 012 Frontend: - SignalsView.vue: signal feed with status filter (new/saved/dismissed), keyword chips, score/comment counts, save/dismiss actions, rules editor panel - api.ts: SignalRule, Signal types + signalRules/signals API methods - Nav: Signals as default landing route (replaces /campaigns default) Closes #7 (signal extraction), closes #10 (Lemmy JSON crawler)
17 lines
1 KiB
SQL
17 lines
1 KiB
SQL
-- Signal monitoring rules: what to watch for across communities
|
|
CREATE TABLE IF NOT EXISTS signal_rules (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
name TEXT NOT NULL, -- human label ("CF pain points", "Kiwi mentions")
|
|
platform TEXT NOT NULL DEFAULT 'reddit',
|
|
sub TEXT, -- NULL = apply to all monitored subs
|
|
keywords TEXT NOT NULL DEFAULT '[]', -- JSON array of strings
|
|
match_mode TEXT NOT NULL DEFAULT 'any', -- 'any' | 'all' | 'regex'
|
|
min_score INTEGER NOT NULL DEFAULT 0, -- ignore posts below this karma score
|
|
label TEXT, -- signal category: pain-point|feedback|mention|trust
|
|
active INTEGER NOT NULL DEFAULT 1,
|
|
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
notes TEXT
|
|
);
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_signal_rules_platform_sub ON signal_rules(platform, sub);
|
|
CREATE INDEX IF NOT EXISTS idx_signal_rules_active ON signal_rules(active);
|