Commit graph

34 commits

Author SHA1 Message Date
0a93b7386a docs: update README to reflect MVP feature set (affiliate links, feedback FAB, vision scheduling) 2026-04-04 19:15:40 -07:00
860276420e refactor: replace _affiliate_url() with circuitforge-core wrap_url() (cf-core #21) 2026-04-04 18:31:02 -07:00
0430454dad feat: eBay affiliate link builder (Option B — user-configurable, CF fallback)
- _affiliate_url() helper appends EPN params when EBAY_AFFILIATE_CAMPAIGN_ID set
- Clean /itm/ URLs by default (no affiliate tracking without explicit opt-in)
- affiliate_active flag in search response drives frontend disclosure
- SearchView shows 'Links may include an affiliate code' when active
- .env.example documents EBAY_AFFILIATE_CAMPAIGN_ID with EPN registration link
- Closes #19
2026-04-03 22:06:41 -07:00
0617fc8256 feat: add in-app feedback FAB
- api/main.py: GET /api/feedback/status + POST /api/feedback — creates
  Forgejo issues; disabled (503) when FORGEJO_API_TOKEN unset, 403 in
  demo mode; includes view, version, platform context in issue body
- FeedbackButton.vue: 2-step modal (type → review → submit); probes
  /api/feedback/status on mount, stays hidden until confirmed enabled
- App.vue: mount FeedbackButton with current route name as view context;
  import useRoute for reactive route name tracking
- .env.example: document FORGEJO_API_TOKEN / FORGEJO_REPO / FORGEJO_API_URL
2026-04-03 21:42:26 -07:00
d5419d2b1b Merge pull request 'feat(tasks): add vision task scheduler for trust photo analysis' (#14) from feature/shared-task-scheduler into main 2026-04-03 21:41:11 -07:00
7c720db644 fix(tests): update saved_searches tier test to match intentional ungating 2026-04-03 21:35:49 -07:00
e93e3de207 feat: scammer blocklist, search/listing UI overhaul, tier refactor
**Scammer blocklist**
- migration 006: scammer_blocklist table (platform + seller_id unique key,
  source: manual|csv_import|community)
- ScammerEntry dataclass + Store.add/remove/list_blocklist methods
- blocklist.ts Pinia store — CRUD, export CSV, import CSV with validation
- BlocklistView.vue — list with search, export/import, bulk-remove; sellers
  show on ListingCard with force-score-0 badge
- API: GET/POST/DELETE /api/blocklist + CSV export/import endpoints
- Router: /blocklist route added; AppNav link

**Migration renumber**
- 002_background_tasks.sql → 007_background_tasks.sql (correct sequence
  after blocklist; idempotent CREATE IF NOT EXISTS safe for existing DBs)

**Search + listing UI overhaul**
- SearchView.vue: keyword expansion preview, filter chips for condition/
  format/price, saved-search quick-run button, paginated results
- ListingCard.vue: trust tier badge, scammer flag overlay, photo count
  chip, quick-block button, save-to-search action
- savedSearches store: optimistic update on run, last-run timestamp

**Tier refactor**
- tiers.py: full rewrite with docstring ladder, BYOK LOCAL_VISION_UNLOCKABLE
  flag, intentionally-free list with rationale (scammer_db, saved_searches,
  market_comps free to maximise adoption)

**Trust aggregator + scraper**
- aggregator.py: blocklist check short-circuits scoring to 0/BAD_ACTOR
- scraper.py: listing format detection, photo count, improved title parsing

**Theme**
- theme.css: trust tier color tokens, badge variants, blocklist badge
2026-04-03 19:08:54 -07:00
d9660093b1 fix(tasks): address code review — cloud DB path, migration number, connection handling, enqueue site
- Rename 002_background_tasks.sql → 007_background_tasks.sql to avoid
  collision with existing 002_add_listing_format.sql migration
- Add CREATE UNIQUE INDEX on trust_scores(listing_id) in same migration
  so save_trust_scores() can use ON CONFLICT upsert semantics
- Add Store.save_trust_scores() — upserts scores keyed by listing_id;
  preserves photo_analysis_json so runner writes are never clobbered
- runner.py: replace raw sqlite3.connect() with get_connection() throughout
  (timeout=30 + WAL mode); fix connection leak in insert_task via try/finally
- _run_trust_photo_analysis: read 'user_db' from params to write results to
  the correct per-user DB in cloud mode (was silently writing to wrong DB)
- main.py lifespan: use _shared_db_path() in cloud mode so background_tasks
  queue lives in shared DB, not _LOCAL_SNIPE_DB
- Add _enqueue_vision_tasks() and call it after score_batch() — this is the
  missing enqueue call site; gated by features.photo_analysis (Paid tier)
- Test fixture: add missing 'stage' column to background_tasks schema
2026-03-31 17:00:01 -07:00
f7c5e8dc17 feat(tasks): add vision task scheduler for trust_photo_analysis
Wires circuitforge_core.tasks.scheduler into Snipe. Adds trust_photo_analysis
background task: downloads primary listing photo, calls LLMRouter with vision
capability, writes result to trust_scores.photo_analysis_json (Paid tier).
photo_analysis_json column already existed in 001_init.sql migration.
2026-03-31 09:27:55 -07:00
f26020cf7f fix: use Directus 11 'id' claim instead of 'sub' for JWT user_id
Directus 11.x JWT payload uses 'id' (not 'sub') for the user UUID.
Our validate_session_jwt required 'sub' → MissingRequiredClaimError on
every request → persistent 401 on all cloud endpoints.
2026-03-27 08:34:06 -07:00
ee1e72992b fix: strip v1|...|0 Browse API item ID prefix before building /itm/ URL
BTF enrichment (_fetch_item_html) was constructing invalid URLs like
https://www.ebay.com/itm/v1|123456789|0 when listings came from the API
adapter (Browse API itemId format). Extract the numeric segment from
compound IDs before appending to EBAY_ITEM_URL — scraper IDs are already
plain numeric so the split is a no-op for that adapter.
2026-03-27 08:07:05 -07:00
9e20759dbe feat: wire cloud session, Heimdall licensing, and split-store DB isolation
- api/cloud_session.py: new module — JWT validation (Directus HS256),
  Heimdall provision+tier-resolve, CloudUser+SessionFeatures dataclasses,
  compute_features() tier→feature-flag mapping, require_tier() dependency
  factory, get_session() FastAPI dependency (local-mode transparent passthrough)
- api/main.py: remove _DB_PATH singleton; all endpoints receive session via
  Depends(get_session); shared_store (sellers/comps) and user_store (listings/
  saved_searches) created per-request from session.shared_db / session.user_db;
  pages capped to features.max_pages; saved_searches limit enforced for free tier;
  /api/session endpoint exposes tier+features to frontend; _trigger_scraper_enrichment
  receives shared_db Path (background thread creates its own Store)
- app/platforms/ebay/adapter.py, scraper.py: rename store→shared_store parameter
  (adapters only touch sellers+comps, never listings — naming reflects this)
- app/trust/__init__.py: rename store→shared_store (TrustScorer reads
  sellers+comps from shared DB; listing staging fields come from caller)
- app/db/store.py: refresh_seller_categories gains listing_store param for
  split-DB mode (reads listings from user_store, writes categories to self)
- web/src/stores/session.ts: new Pinia store — bootstrap() fetches /api/session,
  exposes tier+features reactively; falls back to full-access local defaults
- web/src/App.vue: call session.bootstrap() on mount
- web/src/views/SearchView.vue: import session store; pages buttons disabled+greyed
  above features.max_pages with upgrade tooltip
- compose.cloud.yml: add CLOUD_MODE=true + CLOUD_DATA_ROOT env; fix volume mount
- docker/web/nginx.cloud.conf: forward X-CF-Session header from Caddy to API
- .env.example: document cloud env vars (CLOUD_MODE, DIRECTUS_JWT_SECRET, etc.)
2026-03-27 02:07:06 -07:00
a61166f48a fix(trust): suppress duplicate_photo for established retailers (1000+ feedback)
Large retailers like Newegg legitimately reuse manufacturer stock photos
across listings. Duplicate photo hash is not a scam signal for sellers
with 1000+ feedback — suppress the red flag for them.
2026-03-27 01:07:42 -07:00
98695b00f0 feat(snipe): eBay trust scoring MVP — search, filters, enrichment, comps
Core trust scoring:
- Five metadata signals (account age, feedback count/ratio, price vs market,
  category history), composited 0–100
- CV-based price signal suppression for heterogeneous search results
  (e.g. mixed laptop generations won't false-positive suspicious_price)
- Expanded scratch/dent title detection: evasive redirects, functional problem
  phrases, DIY/repair indicators
- Hard filters: new_account, established_bad_actor
- Soft flags: low_feedback, suspicious_price, duplicate_photo, scratch_dent,
  long_on_market, significant_price_drop

Search & filtering:
- Browse API adapter (up to 200 items/page) + Playwright scraper fallback
- OR-group query expansion for comprehensive variant coverage
- Must-include (AND/ANY/groups), must-exclude, category, price range filters
- Saved searches with full filter round-trip via URL params

Seller enrichment:
- Background BTF /itm/ scraping for account age (Kasada-safe headed Chromium)
- On-demand enrichment: POST /api/enrich + ListingCard ↻ button
- Category history derived from Browse API categories field (free, no extra calls)
- Shopping API GetUserProfile inline enrichment for API adapter

Market comps:
- eBay Marketplace Insights API with Browse API fallback (catches 403 + 404)
- Comps prioritised in ThreadPoolExecutor (submitted first)

Infrastructure:
- Staging DB fields: times_seen, first_seen_at, price_at_first_seen, category_name
- Migrations 004 (staging tracking) + 005 (listing category)
- eBay webhook handler stub
- Cloud compose stack (compose.cloud.yml)
- Vue frontend: search store, saved searches store, ListingCard, filter sidebar

Docs:
- README fully rewritten to reflect MVP status + full feature documentation
- Roadmap table linked to all 13 Forgejo issues
2026-03-26 23:37:09 -07:00
a8add8e96b feat(snipe): cloud deployment under menagerie.circuitforge.tech/snipe
- compose.cloud.yml: snipe-cloud project, proper Docker bridge network
  (api is internal-only, no host port), port 8514 for nginx
- docker/web/Dockerfile: VITE_BASE_URL + VITE_API_BASE build args so
  Vite bakes the /snipe path prefix into the bundle at cloud build time
- docker/web/nginx.cloud.conf: upstream api:8510 via Docker network
  (vs 172.17.0.1:8510 in dev which uses host networking)
- manage.sh: cloud-start/stop/restart/status/logs/build commands
- stores/search.ts: VITE_API_BASE prefix on all /api fetch calls

Gate: Caddy basicauth (username: cf) — temporary gate while proper
Heimdall license validation UI is built. Password stored at
/devl/snipe-cloud-data/.beta-password (host-only, not in repo).

Note: Caddyfile updated separately (caddy-proxy volume, not this repo).
2026-03-26 08:14:01 -07:00
11f2a3c2b3 feat(snipe): keyword must-include/must-exclude filtering
- Two sidebar fields: 'Must include' and 'Must exclude' (comma-separated)
- Must-exclude terms forwarded to eBay _nkw as -term prefixes (native eBay
  support) so exclusions reduce the eBay result set at the source — improves
  market comp quality as a side effect
- Must-include applied client-side only (substring, case-insensitive)
- Both applied client-side via passesFilter() for instant response without
  re-fetching (cache-friendly)
- Exclude input has subtle red border tint (color-mix) to signal intent
- Hint text: 're-search to apply to eBay' reminds user negatives need a
  new search to take effect at the eBay level
2026-03-25 22:54:24 -07:00
ea78b9c2cd feat(snipe): parallel search+comps, pagination, title fix, price flag fix
- Parallel execution: search() and get_completed_sales() now run
  concurrently via ThreadPoolExecutor — each gets its own Store/SQLite
  connection for thread safety. First cold search time ~halved.

- Pagination: SearchFilters.pages (default 1) controls how many eBay
  result pages are fetched. Both search and sold-comps support up to 3
  parallel Playwright sessions per call (capped to avoid Xvfb overload).
  UI: segmented 1/2/3/5 pages selector in filter sidebar with cost hint.

- True median: get_completed_sales() now averages the two middle values
  for even-length price lists instead of always taking the lower bound.

- Fix suspicious_price false positive: aggregator now checks
  signal_scores.get("price_vs_market") == 0 (pre-None-substitution)
  so listings without market data are never flagged as suspicious.

- Fix title pollution: scraper strips eBay's hidden screen-reader span
  ("Opens in a new window or tab") from listing titles via regex.
  Lazy-imports playwright/playwright_stealth inside _get() so pure
  parsing functions are importable without the full browser stack.

- Tests: 48 pass on host (scraper tests now runnable without Docker),
  new regression guards for all three bug fixes.
2026-03-25 22:16:08 -07:00
2ab41219f8 fix: account_age_days=None for scraper tier, stop false new_account flags
Scraper can't fetch seller profile age without following each listing's
seller link. Using 0 as sentinel caused every scraped seller to trigger
new_account and account_under_30_days red flags erroneously.

- Seller.account_age_days: int → Optional[int] (None = not yet fetched)
- Migration 003: recreate sellers table without NOT NULL constraint
- MetadataScorer: return None for unknown age → score_is_partial=True
- Aggregator: gate age flags on is not None
- Scraper: account_age_days=None instead of 0
2026-03-25 20:36:43 -07:00
58263d814a feat(snipe): FastAPI layer, Playwright+Xvfb scraper, caching, tests
- FastAPI service (port 8510) wrapping scraper + trust scorer
- Playwright+Xvfb+stealth transport to bypass eBay Kasada bot protection
- li.s-card selector migration (eBay markup change from li.s-item)
- Three-layer caching: HTML (5min), phash (permanent), market comp (6h SQLite)
- Batch DB writes (executemany + single commit) — warm requests <1s
- Unique Xvfb display counter (:200–:299) prevents lock file collisions
- Vue 3 nginx web service (port 8509) proxying /api/ to FastAPI
- Auction card de-emphasis: opacity 0.72 for listings with >1h remaining
- 35 scraper unit tests updated for new li.s-card fixture markup
- tests/ volume-mounted in compose.override.yml for live test editing
2026-03-25 20:09:30 -07:00
720744f75e chore: remove node_modules from tracking 2026-03-25 15:13:06 -07:00
c787ed751c chore: gitignore web/node_modules and web/dist 2026-03-25 15:12:57 -07:00
7a704441a6 feat(snipe): Vue 3 frontend scaffold + Docker web service
- web/: Vue 3 + Vite + UnoCSS + Pinia, dark tactical theme (amber/#0d1117)
- AppNav, ListingCard, SearchView with filters/sort, composables
  (useSnipeMode, useKonamiCode, useMotion), Pinia search store
- Steal shimmer, auction countdown, Snipe Mode easter egg all native in Vue
- docker/web/: nginx + multi-stage Dockerfile (node build → nginx serve)
- compose.yml: api (8510) + web (8509) services
- Dockerfile CMD updated to uvicorn for upcoming FastAPI layer
- Clean build: 0 TS errors, 380 modules
2026-03-25 15:11:35 -07:00
07794ee163 fix: rename app/app.py → streamlit_app.py to resolve package shadowing 2026-03-25 15:05:12 -07:00
6ec0f957b9 feat(snipe): auction support + easter eggs (Konami, The Steal, de-emphasis)
Auction metadata:
- Listing model gains buying_format + ends_at fields
- Migration 002 adds columns to existing databases
- scraper.py: parse s-item__time-left → absolute ends_at ISO timestamp
- normaliser.py: extract buyingOptions + itemEndDate from Browse API
- store.py: save/get updated for new fields

Easter eggs (app/ui/components/easter_eggs.py):
- Konami code detector (JS → URL param → Streamlit rerun)
- Web Audio API snipe call synthesis, gated behind sidebar checkbox
  (disabled by default for safety/accessibility)
- "The Steal" gold shimmer: trust ≥ 90, price 15–30% below market,
  no suspicious_price flag
- Auction de-emphasis: soft caption when > 1h remaining

UI updates:
- listing_row: steal banner + auction notice per row
- Search: inject CSS, check snipe mode, "Ending soon" sort option,
  pass market_price from comp cache to row renderer
- app.py: Konami detector + audio enable/disable sidebar toggle

Tests: 22 new tests (72 total, all green)
2026-03-25 14:27:02 -07:00
68a9879191 feat: add scraper adapter with auto-detect fallback and partial score logging 2026-03-25 14:12:29 -07:00
4977e517fe feat: Snipe MVP v0.1 — eBay trust scorer with faceted filter UI 2026-03-25 13:09:49 -07:00
59791fd163 feat: add search UI with dynamic filter sidebar and listing rows 2026-03-25 13:08:55 -07:00
95ccd8f1b3 feat: add snipe tier gates with LOCAL_VISION_UNLOCKABLE 2026-03-25 13:08:55 -07:00
ee3c85bfb0 feat: add metadata scorer, photo hash dedup, and trust aggregator 2026-03-25 13:08:55 -07:00
1672e215b2 feat: add eBay adapter with Browse API, Seller API, and market comps 2026-03-25 13:08:55 -07:00
a8eb11dc46 feat: add PlatformAdapter base and eBay token manager 2026-03-25 13:08:55 -07:00
675146ff1a feat: add data models, migrations, and store 2026-03-25 13:08:55 -07:00
ac114da5e7 feat: scaffold snipe repo 2026-03-25 13:08:54 -07:00
3053285ba5 Initial commit 2026-03-10 20:52:28 -07:00