After a search, the API now returns a session_id. If any trust scores are
partial (pending seller age or category data), the frontend opens a
Server-Sent Events stream to /api/updates/{session_id}. As the background
BTF (account age) and category enrichment threads complete, they re-score
affected listings and push updated TrustScore payloads over SSE. The
frontend patches the trustScores and sellers maps reactively so signal
dots light up without requiring a manual re-search.
Backend:
- _update_queues registry maps session_id -> SimpleQueue (thread-safe bridge)
- _trigger_scraper_enrichment accepts session_id/user_db/query, builds a
seller->listings map, calls _push_updates() after each enrichment pass
which re-scores, saves trust scores, and puts events on the queue
- New GET /api/updates/{session_id} SSE endpoint: polls queue every 500ms,
emits heartbeats every 15s, closes on sentinel None or 90s timeout
- search endpoint generates session_id and returns it in response
Frontend:
- search store adds enriching state and _openUpdates() / closeUpdates()
- On search completion, if partial scores exist, opens EventSource stream
- onmessage: patches trustScores and sellers maps (new Map() to trigger
Vue reactivity), updates marketPrice if included
- on 'done' event or error: closes stream, enriching = false
- SearchView: pulsing 'Updating scores...' badge in toolbar while enriching
SQLite's executescript() auto-commits each DDL statement, so a partial
migration failure leaves columns in the DB without marking the migration
done. On the next startup the runner retries and hits duplicate column errors.
Use ADD COLUMN IF NOT EXISTS (SQLite 3.35+, shipped in Python 3.11+)
so migrations 004 and 005 are safe to re-run in any partial state.
001_init.sql already defines first_seen_at in the CREATE TABLE statement.
On fresh installs, migration 004 failed with 'duplicate column name: first_seen_at'.
Remove the redundant ALTER TABLE; last_seen_at/times_seen/price_at_first_seen
are still added by 004 as before.
- aggregator: also check listing.condition against damage keywords so listings
with eBay condition "for parts or not working" flag scratch_dent_mentioned
even when the title looks clean
- aggregator: add "parts/repair" (slash) + "parts or not working" to keyword set
- trust/__init__.py: pass listing.condition into aggregate()
- 3 new regression tests (synthetic fixtures, 17 total passing)
- SearchView: extract DEFAULT_FILTERS const + resetFilters(); add "Clear filters"
button that shows only when activeFilterCount > 0 with count badge
- .env.example: document LLM inference env vars (ANTHROPIC/OPENAI/OLLAMA/CF_ORCH_URL)
and cf-core wiring notes; closes#17
- _affiliate_url() helper appends EPN params when EBAY_AFFILIATE_CAMPAIGN_ID set
- Clean /itm/ URLs by default (no affiliate tracking without explicit opt-in)
- affiliate_active flag in search response drives frontend disclosure
- SearchView shows 'Links may include an affiliate code' when active
- .env.example documents EBAY_AFFILIATE_CAMPAIGN_ID with EPN registration link
- Closes#19
- api/main.py: GET /api/feedback/status + POST /api/feedback — creates
Forgejo issues; disabled (503) when FORGEJO_API_TOKEN unset, 403 in
demo mode; includes view, version, platform context in issue body
- FeedbackButton.vue: 2-step modal (type → review → submit); probes
/api/feedback/status on mount, stays hidden until confirmed enabled
- App.vue: mount FeedbackButton with current route name as view context;
import useRoute for reactive route name tracking
- .env.example: document FORGEJO_API_TOKEN / FORGEJO_REPO / FORGEJO_API_URL
- Rename 002_background_tasks.sql → 007_background_tasks.sql to avoid
collision with existing 002_add_listing_format.sql migration
- Add CREATE UNIQUE INDEX on trust_scores(listing_id) in same migration
so save_trust_scores() can use ON CONFLICT upsert semantics
- Add Store.save_trust_scores() — upserts scores keyed by listing_id;
preserves photo_analysis_json so runner writes are never clobbered
- runner.py: replace raw sqlite3.connect() with get_connection() throughout
(timeout=30 + WAL mode); fix connection leak in insert_task via try/finally
- _run_trust_photo_analysis: read 'user_db' from params to write results to
the correct per-user DB in cloud mode (was silently writing to wrong DB)
- main.py lifespan: use _shared_db_path() in cloud mode so background_tasks
queue lives in shared DB, not _LOCAL_SNIPE_DB
- Add _enqueue_vision_tasks() and call it after score_batch() — this is the
missing enqueue call site; gated by features.photo_analysis (Paid tier)
- Test fixture: add missing 'stage' column to background_tasks schema
Directus 11.x JWT payload uses 'id' (not 'sub') for the user UUID.
Our validate_session_jwt required 'sub' → MissingRequiredClaimError on
every request → persistent 401 on all cloud endpoints.
BTF enrichment (_fetch_item_html) was constructing invalid URLs like
https://www.ebay.com/itm/v1|123456789|0 when listings came from the API
adapter (Browse API itemId format). Extract the numeric segment from
compound IDs before appending to EBAY_ITEM_URL — scraper IDs are already
plain numeric so the split is a no-op for that adapter.
Large retailers like Newegg legitimately reuse manufacturer stock photos
across listings. Duplicate photo hash is not a scam signal for sellers
with 1000+ feedback — suppress the red flag for them.
- compose.cloud.yml: snipe-cloud project, proper Docker bridge network
(api is internal-only, no host port), port 8514 for nginx
- docker/web/Dockerfile: VITE_BASE_URL + VITE_API_BASE build args so
Vite bakes the /snipe path prefix into the bundle at cloud build time
- docker/web/nginx.cloud.conf: upstream api:8510 via Docker network
(vs 172.17.0.1:8510 in dev which uses host networking)
- manage.sh: cloud-start/stop/restart/status/logs/build commands
- stores/search.ts: VITE_API_BASE prefix on all /api fetch calls
Gate: Caddy basicauth (username: cf) — temporary gate while proper
Heimdall license validation UI is built. Password stored at
/devl/snipe-cloud-data/.beta-password (host-only, not in repo).
Note: Caddyfile updated separately (caddy-proxy volume, not this repo).
- Two sidebar fields: 'Must include' and 'Must exclude' (comma-separated)
- Must-exclude terms forwarded to eBay _nkw as -term prefixes (native eBay
support) so exclusions reduce the eBay result set at the source — improves
market comp quality as a side effect
- Must-include applied client-side only (substring, case-insensitive)
- Both applied client-side via passesFilter() for instant response without
re-fetching (cache-friendly)
- Exclude input has subtle red border tint (color-mix) to signal intent
- Hint text: 're-search to apply to eBay' reminds user negatives need a
new search to take effect at the eBay level
- Parallel execution: search() and get_completed_sales() now run
concurrently via ThreadPoolExecutor — each gets its own Store/SQLite
connection for thread safety. First cold search time ~halved.
- Pagination: SearchFilters.pages (default 1) controls how many eBay
result pages are fetched. Both search and sold-comps support up to 3
parallel Playwright sessions per call (capped to avoid Xvfb overload).
UI: segmented 1/2/3/5 pages selector in filter sidebar with cost hint.
- True median: get_completed_sales() now averages the two middle values
for even-length price lists instead of always taking the lower bound.
- Fix suspicious_price false positive: aggregator now checks
signal_scores.get("price_vs_market") == 0 (pre-None-substitution)
so listings without market data are never flagged as suspicious.
- Fix title pollution: scraper strips eBay's hidden screen-reader span
("Opens in a new window or tab") from listing titles via regex.
Lazy-imports playwright/playwright_stealth inside _get() so pure
parsing functions are importable without the full browser stack.
- Tests: 48 pass on host (scraper tests now runnable without Docker),
new regression guards for all three bug fixes.
Scraper can't fetch seller profile age without following each listing's
seller link. Using 0 as sentinel caused every scraped seller to trigger
new_account and account_under_30_days red flags erroneously.
- Seller.account_age_days: int → Optional[int] (None = not yet fetched)
- Migration 003: recreate sellers table without NOT NULL constraint
- MetadataScorer: return None for unknown age → score_is_partial=True
- Aggregator: gate age flags on is not None
- Scraper: account_age_days=None instead of 0
