snipe/CLAUDE.md

4.3 KiB
Raw Blame History

Snipe — Developer Context

eBay listing monitor with seller trust scoring and auction sniping.

Stack

Layer Tech Port
Frontend Vue 3 + Pinia + UnoCSS + Vite (nginx) 8509
API FastAPI (uvicorn, network_mode: host) 8510
Scraper Playwright + playwright-stealth + Xvfb
DB SQLite (data/snipe.db)
Core circuitforge-core (editable install)

CLI

./manage.sh start|stop|restart|status|logs|open|build|test

Docker

docker compose up -d          # start
docker compose build api web  # rebuild after Python/Vue changes
docker compose logs -f api    # tail API logs

compose.override.yml bind-mounts ./tests and ./app for hot reload.

nginx proxies /api/172.17.0.1:8510 (Docker bridge IP — api uses host networking).

Critical Gotchas

Kasada bot protection: eBay blocks requests, curl_cffi, headless Playwright, and all /usr/ and /fdbk/ seller profile pages. Only headed Chromium via Xvfb passes. /itm/ listing pages DO load and contain a BTF (below the fold) seller card with "Joined {Mon} {Year}" — use this for account age enrichment.

Xvfb display counter: Module-level itertools.cycle(range(200, 300)) issues unique display numbers (:200:299) per _get() call to prevent lock file collisions when multiple Playwright sessions run in parallel.

HTML cache: 5-minute in-memory cache keyed by full URL. Prevents duplicate 15s Playwright scrapes within one session. Cleared on restart.

SQLite thread safety: Each concurrent thread (search + comps run in parallel) must have its own Store instance — sqlite3.connect() is not thread-safe across threads. See api/main.py.

nginx rebuild gotcha: nginx config is baked into the image at build time. After editing docker/web/nginx.conf, always docker compose build web.

Playwright imports are lazy: sync_playwright and Stealth import inside _get() — not at module level — so the pure parsing functions (scrape_listings, scrape_sellers) can be imported on the host without Docker's browser stack installed.

DB Migrations

Auto-applied by Store.__init__() via circuitforge_core.db.run_migrations. Migration files: app/db/migrations/001_init.sql, 002_buying_format.sql, 003_nullable_account_age.sql

Tests

# Host (no Docker needed — pure parsing tests)
conda run -n job-seeker python -m pytest tests/ -v --ignore=tests/test_integration.py

# In container
./manage.sh test

48 tests. Scraper tests run on host thanks to lazy Playwright imports.

Trust Scoring Architecture

TrustScorer
  ├── MetadataScorer   → 5 signals × 020 = 0100 composite
  │     account_age, feedback_count, feedback_ratio,
  │     price_vs_market (vs sold comps), category_history
  ├── PhotoScorer      → phash dedup (free); vision analysis (paid stub)
  └── Aggregator       → composite score, red flags, hard filters

Red flag sentinel gotcha: signal_scores uses None for missing data; clean dict substitutes None → 0 for arithmetic. Always check signal_scores.get("key") (not clean["key"]) when gating hard-filter flags — otherwise absent data fires false positives.

Key Files

File Purpose
api/main.py FastAPI endpoint — parallel search+comps, serialization
app/platforms/ebay/scraper.py Playwright scraper, HTML cache, page parser
app/trust/aggregator.py Composite score, red flags, hard filters
app/trust/metadata.py 5 metadata signals
app/db/store.py SQLite read/write (batch methods)
web/src/views/SearchView.vue Filter sidebar + results layout
web/src/stores/search.ts Pinia store — API calls, result state
web/src/components/ListingCard.vue Listing card + auction dim style
web/src/assets/theme.css Central theme (CSS custom properties)

Pending Work

  • Keyword filtering — must-include / must-exclude; negatives forwarded to eBay _nkw and applied client-side
  • Seller enrichment — background scrape of /itm/ BTF seller card for account age (account_age_days)
  • Bulk report UI — multi-select listings → eBay report deep-link + CF community blocklist
  • Snipe scheduling — configurable bid-time offset, human approval gate