From 4d9a945dbd7b27dfa19de4d2ac9caecb9f2e3d09 Mon Sep 17 00:00:00 2001 From: pyr0ball Date: Wed, 25 Mar 2026 22:30:14 -0700 Subject: [PATCH] docs: add CLAUDE.md with architecture, gotchas, and pending work --- CLAUDE.md | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..0987ee2 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,96 @@ +# Snipe — Developer Context + +> eBay listing monitor with seller trust scoring and auction sniping. + +## Stack + +| Layer | Tech | Port | +|-------|------|------| +| Frontend | Vue 3 + Pinia + UnoCSS + Vite (nginx) | 8509 | +| API | FastAPI (uvicorn, `network_mode: host`) | 8510 | +| Scraper | Playwright + playwright-stealth + Xvfb | — | +| DB | SQLite (`data/snipe.db`) | — | +| Core | circuitforge-core (editable install) | — | + +## CLI + +```bash +./manage.sh start|stop|restart|status|logs|open|build|test +``` + +## Docker + +```bash +docker compose up -d # start +docker compose build api web # rebuild after Python/Vue changes +docker compose logs -f api # tail API logs +``` + +`compose.override.yml` bind-mounts `./tests` and `./app` for hot reload. + +nginx proxies `/api/` → `172.17.0.1:8510` (Docker bridge IP — api uses host networking). + +## Critical Gotchas + +**Kasada bot protection:** eBay blocks `requests`, `curl_cffi`, headless Playwright, and all `/usr/` and `/fdbk/` seller profile pages. Only headed Chromium via Xvfb passes. `/itm/` listing pages DO load and contain a BTF (below the fold) seller card with "Joined {Mon} {Year}" — use this for account age enrichment. + +**Xvfb display counter:** Module-level `itertools.cycle(range(200, 300))` issues unique display numbers (`:200`–`:299`) per `_get()` call to prevent lock file collisions when multiple Playwright sessions run in parallel. + +**HTML cache:** 5-minute in-memory cache keyed by full URL. Prevents duplicate 15s Playwright scrapes within one session. Cleared on restart. + +**SQLite thread safety:** Each concurrent thread (search + comps run in parallel) must have its own `Store` instance — `sqlite3.connect()` is not thread-safe across threads. See `api/main.py`. + +**nginx rebuild gotcha:** nginx config is baked into the image at build time. After editing `docker/web/nginx.conf`, always `docker compose build web`. + +**Playwright imports are lazy:** `sync_playwright` and `Stealth` import inside `_get()` — not at module level — so the pure parsing functions (`scrape_listings`, `scrape_sellers`) can be imported on the host without Docker's browser stack installed. + +## DB Migrations + +Auto-applied by `Store.__init__()` via `circuitforge_core.db.run_migrations`. +Migration files: `app/db/migrations/001_init.sql`, `002_buying_format.sql`, `003_nullable_account_age.sql` + +## Tests + +```bash +# Host (no Docker needed — pure parsing tests) +conda run -n job-seeker python -m pytest tests/ -v --ignore=tests/test_integration.py + +# In container +./manage.sh test +``` + +48 tests. Scraper tests run on host thanks to lazy Playwright imports. + +## Trust Scoring Architecture + +``` +TrustScorer + ├── MetadataScorer → 5 signals × 0–20 = 0–100 composite + │ account_age, feedback_count, feedback_ratio, + │ price_vs_market (vs sold comps), category_history + ├── PhotoScorer → phash dedup (free); vision analysis (paid stub) + └── Aggregator → composite score, red flags, hard filters +``` + +**Red flag sentinel gotcha:** `signal_scores` uses `None` for missing data; `clean` dict substitutes `None → 0` for arithmetic. Always check `signal_scores.get("key")` (not `clean["key"]`) when gating hard-filter flags — otherwise absent data fires false positives. + +## Key Files + +| File | Purpose | +|------|---------| +| `api/main.py` | FastAPI endpoint — parallel search+comps, serialization | +| `app/platforms/ebay/scraper.py` | Playwright scraper, HTML cache, page parser | +| `app/trust/aggregator.py` | Composite score, red flags, hard filters | +| `app/trust/metadata.py` | 5 metadata signals | +| `app/db/store.py` | SQLite read/write (batch methods) | +| `web/src/views/SearchView.vue` | Filter sidebar + results layout | +| `web/src/stores/search.ts` | Pinia store — API calls, result state | +| `web/src/components/ListingCard.vue` | Listing card + auction dim style | +| `web/src/assets/theme.css` | Central theme (CSS custom properties) | + +## Pending Work + +- **Keyword filtering** — must-include / must-exclude; negatives forwarded to eBay `_nkw` and applied client-side +- **Seller enrichment** — background scrape of `/itm/` BTF seller card for account age (`account_age_days`) +- **Bulk report UI** — multi-select listings → eBay report deep-link + CF community blocklist +- **Snipe scheduling** — configurable bid-time offset, human approval gate