feat(snipe): eBay trust scoring MVP — search, filters, enrichment, comps
Core trust scoring: - Five metadata signals (account age, feedback count/ratio, price vs market, category history), composited 0–100 - CV-based price signal suppression for heterogeneous search results (e.g. mixed laptop generations won't false-positive suspicious_price) - Expanded scratch/dent title detection: evasive redirects, functional problem phrases, DIY/repair indicators - Hard filters: new_account, established_bad_actor - Soft flags: low_feedback, suspicious_price, duplicate_photo, scratch_dent, long_on_market, significant_price_drop Search & filtering: - Browse API adapter (up to 200 items/page) + Playwright scraper fallback - OR-group query expansion for comprehensive variant coverage - Must-include (AND/ANY/groups), must-exclude, category, price range filters - Saved searches with full filter round-trip via URL params Seller enrichment: - Background BTF /itm/ scraping for account age (Kasada-safe headed Chromium) - On-demand enrichment: POST /api/enrich + ListingCard ↻ button - Category history derived from Browse API categories field (free, no extra calls) - Shopping API GetUserProfile inline enrichment for API adapter Market comps: - eBay Marketplace Insights API with Browse API fallback (catches 403 + 404) - Comps prioritised in ThreadPoolExecutor (submitted first) Infrastructure: - Staging DB fields: times_seen, first_seen_at, price_at_first_seen, category_name - Migrations 004 (staging tracking) + 005 (listing category) - eBay webhook handler stub - Cloud compose stack (compose.cloud.yml) - Vue frontend: search store, saved searches store, ListingCard, filter sidebar Docs: - README fully rewritten to reflect MVP status + full feature documentation - Roadmap table linked to all 13 Forgejo issues
This commit is contained in:
parent
a8add8e96b
commit
98695b00f0
27 changed files with 2487 additions and 228 deletions
36
.env.example
36
.env.example
|
|
@ -1,11 +1,33 @@
|
||||||
# Snipe works out of the box with the scraper (no credentials needed).
|
# Snipe works out of the box with the scraper (no credentials needed).
|
||||||
# Set EBAY_CLIENT_ID + EBAY_CLIENT_SECRET to unlock full trust scores
|
# Set eBay API credentials to unlock full trust scores —
|
||||||
# (account age and category history signals require the eBay Browse API).
|
# account age and category history signals require the eBay Browse API.
|
||||||
# Without credentials the app logs a warning and uses the scraper automatically.
|
# Without credentials the app logs a warning and falls back to the scraper.
|
||||||
|
|
||||||
# Optional — eBay API credentials (self-hosters / paid CF cloud tier)
|
# ── eBay Developer Keys — Production ──────────────────────────────────────────
|
||||||
# EBAY_CLIENT_ID=your-client-id-here
|
# From https://developer.ebay.com/my/keys (Production tab)
|
||||||
# EBAY_CLIENT_SECRET=your-client-secret-here
|
EBAY_APP_ID=
|
||||||
# EBAY_ENV=production # or: sandbox
|
EBAY_DEV_ID=
|
||||||
|
EBAY_CERT_ID=
|
||||||
|
|
||||||
|
# ── eBay Developer Keys — Sandbox ─────────────────────────────────────────────
|
||||||
|
# From https://developer.ebay.com/my/keys (Sandbox tab)
|
||||||
|
EBAY_SANDBOX_APP_ID=
|
||||||
|
EBAY_SANDBOX_DEV_ID=
|
||||||
|
EBAY_SANDBOX_CERT_ID=
|
||||||
|
|
||||||
|
# ── Active environment ─────────────────────────────────────────────────────────
|
||||||
|
# production | sandbox
|
||||||
|
EBAY_ENV=production
|
||||||
|
|
||||||
|
# ── eBay Account Deletion Webhook ──────────────────────────────────────────────
|
||||||
|
# Register endpoint at https://developer.ebay.com/my/notification — required for
|
||||||
|
# production key activation. Set EBAY_NOTIFICATION_ENDPOINT to the public HTTPS
|
||||||
|
# URL eBay will POST to (e.g. https://snipe.circuitforge.tech/api/ebay/account-deletion).
|
||||||
|
EBAY_NOTIFICATION_TOKEN=
|
||||||
|
EBAY_NOTIFICATION_ENDPOINT=
|
||||||
|
# Set to false during sandbox/registration (no production token available yet).
|
||||||
|
# Set to true once production credentials are active — enforces ECDSA verification.
|
||||||
|
EBAY_WEBHOOK_VERIFY_SIGNATURES=true
|
||||||
|
|
||||||
|
# ── Database ───────────────────────────────────────────────────────────────────
|
||||||
SNIPE_DB=data/snipe.db
|
SNIPE_DB=data/snipe.db
|
||||||
|
|
|
||||||
177
README.md
177
README.md
|
|
@ -1,49 +1,170 @@
|
||||||
# Snipe — Auction Sniping & Bid Management
|
# Snipe — Auction Sniping & Listing Intelligence
|
||||||
|
|
||||||
> *Part of the Circuit Forge LLC "AI for the tasks you hate most" suite.*
|
> *Part of the Circuit Forge LLC "AI for the tasks you hate most" suite.*
|
||||||
|
|
||||||
**Status:** Backlog — not yet started. Peregrine must prove the model first.
|
**Status:** Active — eBay listing search + seller trust scoring MVP complete. Auction sniping engine and multi-platform support are next.
|
||||||
|
|
||||||
## What it does
|
## What it does
|
||||||
|
|
||||||
Snipe manages online auction participation: monitoring listings across platforms, scheduling last-second bids, tracking price history to avoid overpaying, and managing the post-win logistics (payment, shipping coordination, provenance documentation for antiques).
|
Snipe has two layers that work together:
|
||||||
|
|
||||||
|
**Layer 1 — Listing intelligence (MVP, implemented)**
|
||||||
|
Before you bid, Snipe tells you whether a listing is worth your time. It fetches eBay listings, scores each seller's trustworthiness across five signals, flags suspicious pricing relative to completed sales, and surfaces red flags like new accounts, cosmetic damage buried in titles, and listings that have been sitting unsold for weeks.
|
||||||
|
|
||||||
|
**Layer 2 — Auction sniping (roadmap)**
|
||||||
|
Snipe manages the bid itself: monitors listings across platforms, schedules last-second bids, handles soft-close extensions, and guides you through the post-win logistics (payment routing, shipping coordination, provenance documentation for antiques).
|
||||||
|
|
||||||
The name is the origin of the word "sniping" — common snipes are notoriously elusive birds, secretive and camouflaged, that flush suddenly from cover. Shooting one required extreme patience, stillness, and a precise last-second shot. That's the auction strategy.
|
The name is the origin of the word "sniping" — common snipes are notoriously elusive birds, secretive and camouflaged, that flush suddenly from cover. Shooting one required extreme patience, stillness, and a precise last-second shot. That's the auction strategy.
|
||||||
|
|
||||||
## Primary platforms
|
---
|
||||||
|
|
||||||
|
## Implemented: eBay Listing Intelligence
|
||||||
|
|
||||||
|
### Search & filtering
|
||||||
|
- Full-text eBay search via Browse API (with Playwright scraper fallback when no API credentials configured)
|
||||||
|
- Price range, must-include keywords (AND / ANY / OR-groups mode), must-exclude terms, eBay category filter
|
||||||
|
- OR-group mode expands keyword combinations into multiple targeted queries and deduplicates results — eBay relevance won't silently drop variants
|
||||||
|
- Pages-to-fetch control: each Browse API page returns up to 200 listings
|
||||||
|
- Saved searches with one-click re-run that restores all filter settings
|
||||||
|
|
||||||
|
### Seller trust scoring
|
||||||
|
Five signals, each scored 0–20, composited to 0–100:
|
||||||
|
|
||||||
|
| Signal | What it measures |
|
||||||
|
|--------|-----------------|
|
||||||
|
| `account_age` | Days since eBay account registration |
|
||||||
|
| `feedback_count` | Total feedback received |
|
||||||
|
| `feedback_ratio` | Positive feedback percentage |
|
||||||
|
| `price_vs_market` | Listing price vs. median of recent completed sales |
|
||||||
|
| `category_history` | Whether seller has history selling in this category |
|
||||||
|
|
||||||
|
Scores are marked **partial** when signals are unavailable (e.g. account age not yet enriched). Partial scores are displayed with a visual indicator rather than penalizing the seller for missing data.
|
||||||
|
|
||||||
|
### Red flags
|
||||||
|
Hard filters that override the composite score:
|
||||||
|
- `new_account` — account registered within 7 days
|
||||||
|
- `established_bad_actor` — feedback ratio < 80% with 20+ reviews
|
||||||
|
|
||||||
|
Soft flags surfaced as warnings:
|
||||||
|
- `account_under_30_days` — account under 30 days old
|
||||||
|
- `low_feedback_count` — fewer than 10 reviews
|
||||||
|
- `suspicious_price` — listing price below 50% of market median *(suppressed automatically when the search returns a heterogeneous price distribution — e.g. mixed laptop generations — to prevent false positives)*
|
||||||
|
- `duplicate_photo` — same image found on another listing (perceptual hash)
|
||||||
|
- `scratch_dent_mentioned` — title keywords indicating cosmetic damage, functional problems, or evasive language (see below)
|
||||||
|
- `long_on_market` — listing has been seen 5+ times over 14+ days without selling
|
||||||
|
- `significant_price_drop` — current price more than 20% below first-seen price
|
||||||
|
|
||||||
|
### Scratch & dent title detection
|
||||||
|
Scans listing titles for signals the item may have undisclosed damage or problems:
|
||||||
|
- **Explicit damage**: scratch, scuff, dent, crack, chip, blemish, worn
|
||||||
|
- **Condition catch-alls**: as is, for parts, parts only, spares or repair
|
||||||
|
- **Evasive redirects**: "see description", "read description", "see photos for" (seller hiding damage detail in listing body)
|
||||||
|
- **Functional problems**: "not working", "stopped working", "no power", "dead on arrival", "powers on but", "faulty", "broken screen/hinge/port"
|
||||||
|
- **DIY/repair listings**: "needs repair", "needs tlc", "project laptop", "for repair", "sold as is"
|
||||||
|
|
||||||
|
### Seller enrichment
|
||||||
|
- **Inline (API adapter)**: account age filled from Browse API `registrationDate` field
|
||||||
|
- **Background (scraper)**: `/itm/` listing pages scraped for seller "Joined" date via Playwright + Xvfb (Kasada-safe headed Chromium)
|
||||||
|
- **On-demand**: ↻ button on any listing card triggers `POST /api/enrich` — runs enrichment and re-scores without waiting for a second search
|
||||||
|
- **Category history**: derived from the seller's accumulated listing data (Browse API `categories` field); improves with every search, no extra API calls
|
||||||
|
|
||||||
|
### Market price comparison
|
||||||
|
Completed sales fetched via eBay Marketplace Insights API (with Browse API fallback for app tiers that don't have Insights access). Median stored per query hash, used to score `price_vs_market` across all listings in a search.
|
||||||
|
|
||||||
|
### Adapters
|
||||||
|
| Adapter | When used | Signals available |
|
||||||
|
|---------|-----------|-------------------|
|
||||||
|
| Browse API (`api`) | eBay API credentials configured | All signals; account age inline |
|
||||||
|
| Playwright scraper (`scraper`) | No credentials / forced | All signals except account age (async BTF enrichment) |
|
||||||
|
| `auto` (default) | — | API if credentials present, scraper otherwise |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Stack
|
||||||
|
|
||||||
|
| Layer | Tech | Port |
|
||||||
|
|-------|------|------|
|
||||||
|
| Frontend | Vue 3 + Pinia + UnoCSS + Vite (nginx) | 8509 |
|
||||||
|
| API | FastAPI (uvicorn) | 8510 |
|
||||||
|
| Scraper | Playwright + playwright-stealth + Xvfb | — |
|
||||||
|
| DB | SQLite (`data/snipe.db`) | — |
|
||||||
|
| Core | circuitforge-core (editable install) | — |
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./manage.sh start # start all services
|
||||||
|
./manage.sh stop # stop
|
||||||
|
./manage.sh logs # tail logs
|
||||||
|
./manage.sh open # open in browser
|
||||||
|
```
|
||||||
|
|
||||||
|
Cloud stack (shared DB, multi-user):
|
||||||
|
```bash
|
||||||
|
docker compose -f compose.cloud.yml -p snipe-cloud up -d
|
||||||
|
docker compose -f compose.cloud.yml -p snipe-cloud build api # after Python changes
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Roadmap
|
||||||
|
|
||||||
|
### Near-term (eBay)
|
||||||
|
|
||||||
|
| Issue | Feature |
|
||||||
|
|-------|---------|
|
||||||
|
| [#1](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/1) | SSE/WebSocket live score push — enriched data appears without re-search |
|
||||||
|
| [#2](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/2) | eBay OAuth (Connect eBay Account) for full trust score access via Trading API |
|
||||||
|
| [#4](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/4) | Scammer database: community blocklist + batch eBay Trust & Safety reporting |
|
||||||
|
| [#5](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/5) | UPC/product lookup → LLM-crafted search terms (paid tier) |
|
||||||
|
| [#8](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/8) | "Triple Red" easter egg: CSS animation when all hard flags fire simultaneously |
|
||||||
|
| [#11](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/11) | Vision-based photo condition assessment — moondream2 (local) / Claude vision (cloud, paid) |
|
||||||
|
| [#12](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/12) | Background saved-search monitoring with configurable alerts |
|
||||||
|
|
||||||
|
### Cloud / infrastructure
|
||||||
|
|
||||||
|
| Issue | Feature |
|
||||||
|
|-------|---------|
|
||||||
|
| [#6](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/6) | Shared seller/scammer/comps DB across cloud users (public data, no re-scraping) |
|
||||||
|
| [#7](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/7) | Shared image hash DB — requires explicit opt-in consent (CF privacy-by-architecture) |
|
||||||
|
|
||||||
|
### Auction sniping engine
|
||||||
|
|
||||||
|
| Issue | Feature |
|
||||||
|
|-------|---------|
|
||||||
|
| [#9](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/9) | Bid scheduling + snipe execution (NTP-synchronized, soft-close handling, human approval gate) |
|
||||||
|
| [#13](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/13) | Post-win workflow: payment routing, shipping coordination, provenance documentation |
|
||||||
|
|
||||||
|
### Multi-platform expansion
|
||||||
|
|
||||||
|
| Issue | Feature |
|
||||||
|
|-------|---------|
|
||||||
|
| [#10](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/10) | CT Bids, HiBid, AuctionZip, Invaluable, GovPlanet, Bidsquare, Proxibid |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Primary platforms (full vision)
|
||||||
|
|
||||||
|
- **eBay** — general + collectibles *(search + trust scoring: implemented)*
|
||||||
- **CT Bids** — Connecticut state surplus and municipal auctions
|
- **CT Bids** — Connecticut state surplus and municipal auctions
|
||||||
- **GovPlanet / IronPlanet** — government surplus equipment
|
- **GovPlanet / IronPlanet** — government surplus equipment
|
||||||
- **AuctionZip** — antique auction house aggregator (1,000+ houses)
|
- **AuctionZip** — antique auction house aggregator (1,000+ houses)
|
||||||
- **Invaluable / LiveAuctioneers** — fine art and antiques
|
- **Invaluable / LiveAuctioneers** — fine art and antiques
|
||||||
- **Bidsquare** — antiques and collectibles
|
- **Bidsquare** — antiques and collectibles
|
||||||
- **eBay** — general + collectibles
|
|
||||||
- **HiBid** — estate auctions
|
- **HiBid** — estate auctions
|
||||||
- **Proxibid** — industrial and collector auctions
|
- **Proxibid** — industrial and collector auctions
|
||||||
|
|
||||||
## Why it's hard
|
## Why auctions are hard
|
||||||
|
|
||||||
Online auctions are frustrating because:
|
Online auctions are frustrating because:
|
||||||
- Winning requires being present at the exact closing moment — sometimes 2 AM
|
- Winning requires being present at the exact closing moment — sometimes 2 AM
|
||||||
- Platforms vary wildly: some allow proxy bids, some don't; closing times extend on activity
|
- Platforms vary wildly: some allow proxy bids, some don't; closing times extend on activity
|
||||||
- Price history is hidden — you don't know if an item is underpriced or a trap
|
- Price history is hidden — you don't know if an item is underpriced or a trap
|
||||||
- Shipping logistics for large / fragile antiques require coordination with auction house
|
- Sellers hide damage in descriptions rather than titles to avoid automated filters
|
||||||
|
- Shipping logistics for large / fragile antiques require coordination with the auction house
|
||||||
- Provenance documentation is inconsistent across auction houses
|
- Provenance documentation is inconsistent across auction houses
|
||||||
|
|
||||||
## Core pipeline
|
## Bidding strategy engine (planned)
|
||||||
|
|
||||||
```
|
|
||||||
Configure search (categories, keywords, platforms, max price, location)
|
|
||||||
→ Monitor listings → Alert on matching items
|
|
||||||
→ Human review: approve or skip
|
|
||||||
→ Price research: comparable sales history, condition assessment via photos
|
|
||||||
→ Schedule snipe bid (configurable: X seconds before close, Y% above current)
|
|
||||||
→ Execute bid → Monitor for counter-bid (soft-close extension handling)
|
|
||||||
→ Win notification → Payment + shipping coordination workflow
|
|
||||||
→ Provenance documentation for antiques
|
|
||||||
```
|
|
||||||
|
|
||||||
## Bidding strategy engine
|
|
||||||
|
|
||||||
- **Hard snipe**: submit bid N seconds before close (default: 8s)
|
- **Hard snipe**: submit bid N seconds before close (default: 8s)
|
||||||
- **Soft-close handling**: detect if platform extends on last-minute bids; adjust strategy
|
- **Soft-close handling**: detect if platform extends on last-minute bids; adjust strategy
|
||||||
|
|
@ -51,10 +172,10 @@ Configure search (categories, keywords, platforms, max price, location)
|
||||||
- **Reserve detection**: identify likely reserve price from bid history patterns
|
- **Reserve detection**: identify likely reserve price from bid history patterns
|
||||||
- **Comparable sales**: pull recent auction results for same/similar items across platforms
|
- **Comparable sales**: pull recent auction results for same/similar items across platforms
|
||||||
|
|
||||||
## Post-win workflow
|
## Post-win workflow (planned)
|
||||||
|
|
||||||
1. Payment method routing (platform-specific: CC, wire, check)
|
1. Payment method routing (platform-specific: CC, wire, check)
|
||||||
2. Shipping quote requests to approved carriers (for freight / large items)
|
2. Shipping quote requests to approved carriers (freight / large items via uShip; parcel via FedEx/UPS)
|
||||||
3. Condition report request from auction house
|
3. Condition report request from auction house
|
||||||
4. Provenance packet generation (for antiques / fine art resale or insurance)
|
4. Provenance packet generation (for antiques / fine art resale or insurance)
|
||||||
5. Add to inventory (for dealers / collectors tracking portfolio value)
|
5. Add to inventory (for dealers / collectors tracking portfolio value)
|
||||||
|
|
@ -65,10 +186,10 @@ Configure search (categories, keywords, platforms, max price, location)
|
||||||
|
|
||||||
## Tech notes
|
## Tech notes
|
||||||
|
|
||||||
- Shared `circuitforge-core` scaffold
|
- Shared `circuitforge-core` scaffold (DB, LLM router, tier system, config)
|
||||||
- Platform adapters: AuctionZip, Invaluable, HiBid, eBay, CT Bids (Playwright + API where available)
|
- Platform adapters: currently eBay only; AuctionZip, Invaluable, HiBid, CT Bids planned (Playwright + API where available)
|
||||||
- Bid execution: Playwright automation with precise timing (NTP-synchronized)
|
- Bid execution: Playwright automation with precise timing (NTP-synchronized)
|
||||||
- Soft-close detection: platform-specific rules engine
|
- Soft-close detection: platform-specific rules engine
|
||||||
- Comparable sales: scrape completed auctions, normalize by condition/provenance
|
- Comparable sales: eBay completed listings via Marketplace Insights API + Browse API fallback
|
||||||
- Vision module: condition assessment from listing photos (moondream2 / Claude vision)
|
- Vision module: condition assessment from listing photos — moondream2 / Claude vision (paid tier stub in `app/trust/photo.py`)
|
||||||
- Shipping quote integration: uShip API for freight, FedEx / UPS for parcel
|
- **Kasada bypass**: headed Chromium via Xvfb; all scraping uses this path — headless and `requests`-based approaches are blocked by eBay
|
||||||
|
|
|
||||||
149
api/ebay_webhook.py
Normal file
149
api/ebay_webhook.py
Normal file
|
|
@ -0,0 +1,149 @@
|
||||||
|
"""eBay Marketplace Account Deletion webhook.
|
||||||
|
|
||||||
|
Required to activate eBay production API credentials.
|
||||||
|
|
||||||
|
Protocol (https://developer.ebay.com/develop/guides-v2/marketplace-user-account-deletion):
|
||||||
|
|
||||||
|
GET /api/ebay/account-deletion?challenge_code=<hex>
|
||||||
|
→ {"challengeResponse": SHA256(code + token + endpoint_url)}
|
||||||
|
|
||||||
|
POST /api/ebay/account-deletion
|
||||||
|
Header: X-EBAY-SIGNATURE: <base64-JSON {"kid": "...", "signature": "<b64>"}>
|
||||||
|
Body: JSON notification payload
|
||||||
|
→ 200 on valid + deleted, 412 on bad signature
|
||||||
|
|
||||||
|
Public keys are fetched from the eBay Notification API and cached for 1 hour.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from fastapi import APIRouter, Header, HTTPException, Request
|
||||||
|
from cryptography.exceptions import InvalidSignature
|
||||||
|
from cryptography.hazmat.primitives.asymmetric.ec import ECDSA
|
||||||
|
from cryptography.hazmat.primitives.hashes import SHA1
|
||||||
|
from cryptography.hazmat.primitives.serialization import load_pem_public_key
|
||||||
|
|
||||||
|
from app.db.store import Store
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
_DB_PATH = Path(os.environ.get("SNIPE_DB", "data/snipe.db"))
|
||||||
|
|
||||||
|
# ── Public-key cache ──────────────────────────────────────────────────────────
|
||||||
|
# eBay key rotation is rare; 1-hour TTL is appropriate.
|
||||||
|
_KEY_CACHE_TTL = 3600
|
||||||
|
_key_cache: dict[str, tuple[bytes, float]] = {} # kid → (pem_bytes, expiry)
|
||||||
|
|
||||||
|
# The eBay Notification service is a unified production-side system — signing keys
|
||||||
|
# always live at api.ebay.com regardless of whether the app uses sandbox or production
|
||||||
|
# Browse API credentials.
|
||||||
|
_EBAY_KEY_URL = "https://api.ebay.com/commerce/notification/v1/public_key/{kid}"
|
||||||
|
|
||||||
|
|
||||||
|
def _fetch_public_key(kid: str) -> bytes:
|
||||||
|
"""Return PEM public key bytes for the given kid, using a 1-hour cache."""
|
||||||
|
cached = _key_cache.get(kid)
|
||||||
|
if cached and time.time() < cached[1]:
|
||||||
|
return cached[0]
|
||||||
|
|
||||||
|
key_url = _EBAY_KEY_URL.format(kid=kid)
|
||||||
|
resp = requests.get(key_url, timeout=10)
|
||||||
|
if not resp.ok:
|
||||||
|
log.error("public key fetch failed: %s %s — body: %s", resp.status_code, key_url, resp.text[:500])
|
||||||
|
resp.raise_for_status()
|
||||||
|
pem_str: str = resp.json()["key"]
|
||||||
|
pem_bytes = pem_str.encode()
|
||||||
|
_key_cache[kid] = (pem_bytes, time.time() + _KEY_CACHE_TTL)
|
||||||
|
return pem_bytes
|
||||||
|
|
||||||
|
|
||||||
|
# ── GET — challenge verification ──────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.get("/api/ebay/account-deletion")
|
||||||
|
def ebay_challenge(challenge_code: str):
|
||||||
|
"""Respond to eBay's endpoint verification challenge.
|
||||||
|
|
||||||
|
eBay sends this GET once when you register the endpoint URL.
|
||||||
|
Response must be the SHA-256 hex digest of (code + token + endpoint).
|
||||||
|
"""
|
||||||
|
token = os.environ.get("EBAY_NOTIFICATION_TOKEN", "")
|
||||||
|
endpoint = os.environ.get("EBAY_NOTIFICATION_ENDPOINT", "")
|
||||||
|
if not token or not endpoint:
|
||||||
|
log.error("EBAY_NOTIFICATION_TOKEN or EBAY_NOTIFICATION_ENDPOINT not set")
|
||||||
|
raise HTTPException(status_code=500, detail="Webhook not configured")
|
||||||
|
|
||||||
|
digest = hashlib.sha256(
|
||||||
|
(challenge_code + token + endpoint).encode()
|
||||||
|
).hexdigest()
|
||||||
|
return {"challengeResponse": digest}
|
||||||
|
|
||||||
|
|
||||||
|
# ── POST — deletion notification ──────────────────────────────────────────────
|
||||||
|
|
||||||
|
@router.post("/api/ebay/account-deletion", status_code=200)
|
||||||
|
async def ebay_account_deletion(
|
||||||
|
request: Request,
|
||||||
|
x_ebay_signature: Optional[str] = Header(default=None),
|
||||||
|
):
|
||||||
|
"""Process an eBay Marketplace Account Deletion notification.
|
||||||
|
|
||||||
|
Verifies the ECDSA/SHA1 signature, then permanently deletes all stored
|
||||||
|
data (sellers + listings) for the named eBay user.
|
||||||
|
"""
|
||||||
|
body_bytes = await request.body()
|
||||||
|
|
||||||
|
# 1. Parse and verify signature header
|
||||||
|
if not x_ebay_signature:
|
||||||
|
log.warning("ebay_account_deletion: missing X-EBAY-SIGNATURE header")
|
||||||
|
raise HTTPException(status_code=412, detail="Missing signature")
|
||||||
|
|
||||||
|
try:
|
||||||
|
sig_json = json.loads(base64.b64decode(x_ebay_signature))
|
||||||
|
kid: str = sig_json["kid"]
|
||||||
|
sig_b64: str = sig_json["signature"]
|
||||||
|
sig_bytes = base64.b64decode(sig_b64)
|
||||||
|
except Exception as exc:
|
||||||
|
log.warning("ebay_account_deletion: malformed signature header — %s", exc)
|
||||||
|
raise HTTPException(status_code=412, detail="Malformed signature header")
|
||||||
|
|
||||||
|
# 2. Fetch and verify with eBay public key
|
||||||
|
# EBAY_WEBHOOK_VERIFY_SIGNATURES=false skips ECDSA during sandbox/registration phase.
|
||||||
|
# Set to true (default) once production credentials are active.
|
||||||
|
skip_verify = os.environ.get("EBAY_WEBHOOK_VERIFY_SIGNATURES", "true").lower() == "false"
|
||||||
|
if skip_verify:
|
||||||
|
log.warning("ebay_account_deletion: signature verification DISABLED — enable before production")
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
pem_bytes = _fetch_public_key(kid)
|
||||||
|
pub_key = load_pem_public_key(pem_bytes)
|
||||||
|
pub_key.verify(sig_bytes, body_bytes, ECDSA(SHA1()))
|
||||||
|
except InvalidSignature:
|
||||||
|
log.warning("ebay_account_deletion: ECDSA signature verification failed (kid=%s)", kid)
|
||||||
|
raise HTTPException(status_code=412, detail="Signature verification failed")
|
||||||
|
except Exception as exc:
|
||||||
|
log.error("ebay_account_deletion: unexpected error during verification — %s", exc)
|
||||||
|
raise HTTPException(status_code=412, detail="Verification error")
|
||||||
|
|
||||||
|
# 3. Extract username from notification payload and delete data
|
||||||
|
try:
|
||||||
|
payload = json.loads(body_bytes)
|
||||||
|
username: str = payload["notification"]["data"]["username"]
|
||||||
|
except (KeyError, json.JSONDecodeError) as exc:
|
||||||
|
log.error("ebay_account_deletion: could not parse payload — %s", exc)
|
||||||
|
raise HTTPException(status_code=400, detail="Unrecognisable payload")
|
||||||
|
|
||||||
|
store = Store(_DB_PATH)
|
||||||
|
store.delete_seller_data("ebay", username)
|
||||||
|
log.info("ebay_account_deletion: deleted data for eBay user %r", username)
|
||||||
|
return {}
|
||||||
323
api/main.py
323
api/main.py
|
|
@ -9,13 +9,19 @@ from concurrent.futures import ThreadPoolExecutor
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from fastapi import FastAPI, HTTPException
|
from fastapi import FastAPI, HTTPException
|
||||||
|
from pydantic import BaseModel
|
||||||
from fastapi.middleware.cors import CORSMiddleware
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
|
||||||
from circuitforge_core.config import load_env
|
from circuitforge_core.config import load_env
|
||||||
from app.db.store import Store
|
from app.db.store import Store
|
||||||
|
from app.db.models import SavedSearch as SavedSearchModel
|
||||||
from app.platforms import SearchFilters
|
from app.platforms import SearchFilters
|
||||||
from app.platforms.ebay.scraper import ScrapedEbayAdapter
|
from app.platforms.ebay.scraper import ScrapedEbayAdapter
|
||||||
|
from app.platforms.ebay.adapter import EbayAdapter
|
||||||
|
from app.platforms.ebay.auth import EbayTokenManager
|
||||||
|
from app.platforms.ebay.query_builder import expand_queries, parse_groups
|
||||||
from app.trust import TrustScorer
|
from app.trust import TrustScorer
|
||||||
|
from api.ebay_webhook import router as ebay_webhook_router
|
||||||
|
|
||||||
load_env(Path(".env"))
|
load_env(Path(".env"))
|
||||||
log = logging.getLogger(__name__)
|
log = logging.getLogger(__name__)
|
||||||
|
|
@ -23,7 +29,24 @@ log = logging.getLogger(__name__)
|
||||||
_DB_PATH = Path(os.environ.get("SNIPE_DB", "data/snipe.db"))
|
_DB_PATH = Path(os.environ.get("SNIPE_DB", "data/snipe.db"))
|
||||||
_DB_PATH.parent.mkdir(exist_ok=True)
|
_DB_PATH.parent.mkdir(exist_ok=True)
|
||||||
|
|
||||||
|
|
||||||
|
def _ebay_creds() -> tuple[str, str, str]:
|
||||||
|
"""Return (client_id, client_secret, env) from env vars.
|
||||||
|
|
||||||
|
New names: EBAY_APP_ID / EBAY_CERT_ID (sandbox: EBAY_SANDBOX_APP_ID / EBAY_SANDBOX_CERT_ID)
|
||||||
|
Legacy fallback: EBAY_CLIENT_ID / EBAY_CLIENT_SECRET
|
||||||
|
"""
|
||||||
|
env = os.environ.get("EBAY_ENV", "production").strip()
|
||||||
|
if env == "sandbox":
|
||||||
|
client_id = os.environ.get("EBAY_SANDBOX_APP_ID", "").strip()
|
||||||
|
client_secret = os.environ.get("EBAY_SANDBOX_CERT_ID", "").strip()
|
||||||
|
else:
|
||||||
|
client_id = (os.environ.get("EBAY_APP_ID") or os.environ.get("EBAY_CLIENT_ID", "")).strip()
|
||||||
|
client_secret = (os.environ.get("EBAY_CERT_ID") or os.environ.get("EBAY_CLIENT_SECRET", "")).strip()
|
||||||
|
return client_id, client_secret, env
|
||||||
|
|
||||||
app = FastAPI(title="Snipe API", version="0.1.0")
|
app = FastAPI(title="Snipe API", version="0.1.0")
|
||||||
|
app.include_router(ebay_webhook_router)
|
||||||
|
|
||||||
app.add_middleware(
|
app.add_middleware(
|
||||||
CORSMiddleware,
|
CORSMiddleware,
|
||||||
|
|
@ -38,59 +61,202 @@ def health():
|
||||||
return {"status": "ok"}
|
return {"status": "ok"}
|
||||||
|
|
||||||
|
|
||||||
|
def _trigger_scraper_enrichment(listings: list, store: Store) -> None:
|
||||||
|
"""Fire-and-forget background enrichment for missing seller signals.
|
||||||
|
|
||||||
|
Two enrichment passes run concurrently in the same daemon thread:
|
||||||
|
1. BTF (/itm/ pages) — fills account_age_days for sellers where it is None.
|
||||||
|
2. _ssn search pages — fills category_history_json for sellers with no history.
|
||||||
|
|
||||||
|
The main response returns immediately; enriched data lands in the DB for
|
||||||
|
future searches. Uses ScrapedEbayAdapter's Playwright stack regardless of
|
||||||
|
which adapter was used for the main search (Shopping API handles age for
|
||||||
|
the API adapter inline; BTF is the fallback for no-creds / scraper mode).
|
||||||
|
"""
|
||||||
|
# Caps per search: limits Playwright sessions launched in the background so we
|
||||||
|
# don't hammer Kasada or spin up dozens of Xvfb instances after a large search.
|
||||||
|
# Remaining sellers get enriched incrementally on subsequent searches.
|
||||||
|
_BTF_MAX_PER_SEARCH = 3
|
||||||
|
_CAT_MAX_PER_SEARCH = 3
|
||||||
|
|
||||||
|
needs_btf: dict[str, str] = {}
|
||||||
|
needs_categories: list[str] = []
|
||||||
|
|
||||||
|
for listing in listings:
|
||||||
|
sid = listing.seller_platform_id
|
||||||
|
if not sid:
|
||||||
|
continue
|
||||||
|
seller = store.get_seller("ebay", sid)
|
||||||
|
if not seller:
|
||||||
|
continue
|
||||||
|
if (seller.account_age_days is None
|
||||||
|
and sid not in needs_btf
|
||||||
|
and len(needs_btf) < _BTF_MAX_PER_SEARCH):
|
||||||
|
needs_btf[sid] = listing.platform_listing_id
|
||||||
|
if (seller.category_history_json in ("{}", "", None)
|
||||||
|
and sid not in needs_categories
|
||||||
|
and len(needs_categories) < _CAT_MAX_PER_SEARCH):
|
||||||
|
needs_categories.append(sid)
|
||||||
|
|
||||||
|
if not needs_btf and not needs_categories:
|
||||||
|
return
|
||||||
|
|
||||||
|
log.info(
|
||||||
|
"Scraper enrichment: %d BTF age + %d category pages queued",
|
||||||
|
len(needs_btf), len(needs_categories),
|
||||||
|
)
|
||||||
|
|
||||||
|
def _run():
|
||||||
|
try:
|
||||||
|
enricher = ScrapedEbayAdapter(Store(_DB_PATH))
|
||||||
|
if needs_btf:
|
||||||
|
enricher.enrich_sellers_btf(needs_btf, max_workers=2)
|
||||||
|
log.info("BTF enrichment complete for %d sellers", len(needs_btf))
|
||||||
|
if needs_categories:
|
||||||
|
enricher.enrich_sellers_categories(needs_categories, max_workers=2)
|
||||||
|
log.info("Category enrichment complete for %d sellers", len(needs_categories))
|
||||||
|
except Exception as e:
|
||||||
|
log.warning("Scraper enrichment failed: %s", e)
|
||||||
|
|
||||||
|
import threading
|
||||||
|
t = threading.Thread(target=_run, daemon=True)
|
||||||
|
t.start()
|
||||||
|
|
||||||
|
|
||||||
def _parse_terms(raw: str) -> list[str]:
|
def _parse_terms(raw: str) -> list[str]:
|
||||||
"""Split a comma-separated keyword string into non-empty, stripped terms."""
|
"""Split a comma-separated keyword string into non-empty, stripped terms."""
|
||||||
return [t.strip() for t in raw.split(",") if t.strip()]
|
return [t.strip() for t in raw.split(",") if t.strip()]
|
||||||
|
|
||||||
|
|
||||||
|
def _make_adapter(store: Store, force: str = "auto"):
|
||||||
|
"""Return the appropriate adapter.
|
||||||
|
|
||||||
|
force: "auto" | "api" | "scraper"
|
||||||
|
auto — API if creds present, else scraper
|
||||||
|
api — Browse API (raises if no creds)
|
||||||
|
scraper — Playwright scraper regardless of creds
|
||||||
|
"""
|
||||||
|
client_id, client_secret, env = _ebay_creds()
|
||||||
|
has_creds = bool(client_id and client_secret)
|
||||||
|
|
||||||
|
if force == "scraper":
|
||||||
|
return ScrapedEbayAdapter(store)
|
||||||
|
if force == "api":
|
||||||
|
if not has_creds:
|
||||||
|
raise ValueError("adapter=api requested but no eBay API credentials configured")
|
||||||
|
return EbayAdapter(EbayTokenManager(client_id, client_secret, env), store, env=env)
|
||||||
|
# auto
|
||||||
|
if has_creds:
|
||||||
|
return EbayAdapter(EbayTokenManager(client_id, client_secret, env), store, env=env)
|
||||||
|
log.debug("No eBay API credentials — using scraper adapter (partial trust scores)")
|
||||||
|
return ScrapedEbayAdapter(store)
|
||||||
|
|
||||||
|
|
||||||
|
def _adapter_name(force: str = "auto") -> str:
|
||||||
|
"""Return the name of the adapter that would be used — without creating it."""
|
||||||
|
client_id, client_secret, _ = _ebay_creds()
|
||||||
|
if force == "scraper":
|
||||||
|
return "scraper"
|
||||||
|
if force == "api" or (force == "auto" and client_id and client_secret):
|
||||||
|
return "api"
|
||||||
|
return "scraper"
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/search")
|
@app.get("/api/search")
|
||||||
def search(
|
def search(
|
||||||
q: str = "",
|
q: str = "",
|
||||||
max_price: float = 0,
|
max_price: float = 0,
|
||||||
min_price: float = 0,
|
min_price: float = 0,
|
||||||
pages: int = 1,
|
pages: int = 1,
|
||||||
must_include: str = "", # comma-separated; applied client-side only
|
must_include: str = "", # raw filter string; client-side always applied
|
||||||
must_exclude: str = "", # comma-separated; forwarded to eBay AND applied client-side
|
must_include_mode: str = "all", # "all" | "any" | "groups" — drives eBay expansion
|
||||||
|
must_exclude: str = "", # comma-separated; forwarded to eBay -term + client-side
|
||||||
|
category_id: str = "", # eBay category ID — forwarded to Browse API / scraper _sacat
|
||||||
|
adapter: str = "auto", # "auto" | "api" | "scraper" — override adapter selection
|
||||||
):
|
):
|
||||||
if not q.strip():
|
if not q.strip():
|
||||||
return {"listings": [], "trust_scores": {}, "sellers": {}, "market_price": None}
|
return {"listings": [], "trust_scores": {}, "sellers": {}, "market_price": None, "adapter_used": _adapter_name(adapter)}
|
||||||
|
|
||||||
filters = SearchFilters(
|
must_exclude_terms = _parse_terms(must_exclude)
|
||||||
|
|
||||||
|
# In Groups mode, expand OR groups into multiple targeted eBay queries to
|
||||||
|
# guarantee comprehensive result coverage — eBay relevance won't silently drop variants.
|
||||||
|
if must_include_mode == "groups" and must_include.strip():
|
||||||
|
or_groups = parse_groups(must_include)
|
||||||
|
ebay_queries = expand_queries(q, or_groups)
|
||||||
|
else:
|
||||||
|
ebay_queries = [q]
|
||||||
|
|
||||||
|
base_filters = SearchFilters(
|
||||||
max_price=max_price if max_price > 0 else None,
|
max_price=max_price if max_price > 0 else None,
|
||||||
min_price=min_price if min_price > 0 else None,
|
min_price=min_price if min_price > 0 else None,
|
||||||
pages=max(1, pages),
|
pages=max(1, pages),
|
||||||
must_include=_parse_terms(must_include),
|
must_exclude=must_exclude_terms, # forwarded to eBay -term by the scraper
|
||||||
must_exclude=_parse_terms(must_exclude),
|
category_id=category_id.strip() or None,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Each adapter gets its own Store (SQLite connection) — required for thread safety.
|
adapter_used = _adapter_name(adapter)
|
||||||
# search() and get_completed_sales() run concurrently; they write to different tables
|
|
||||||
# so SQLite file-level locking is the only contention point.
|
# Each thread creates its own Store — sqlite3 check_same_thread=True.
|
||||||
search_adapter = ScrapedEbayAdapter(Store(_DB_PATH))
|
def _run_search(ebay_query: str) -> list:
|
||||||
comps_adapter = ScrapedEbayAdapter(Store(_DB_PATH))
|
return _make_adapter(Store(_DB_PATH), adapter).search(ebay_query, base_filters)
|
||||||
|
|
||||||
|
def _run_comps() -> None:
|
||||||
|
try:
|
||||||
|
_make_adapter(Store(_DB_PATH), adapter).get_completed_sales(q, pages)
|
||||||
|
except Exception:
|
||||||
|
log.warning("comps: unhandled exception for %r", q, exc_info=True)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
with ThreadPoolExecutor(max_workers=2) as ex:
|
# Comps submitted first — guarantees an immediate worker slot even at max concurrency.
|
||||||
listings_future = ex.submit(search_adapter.search, q, filters)
|
# Seller enrichment runs after the executor exits (background thread), so comps are
|
||||||
comps_future = ex.submit(comps_adapter.get_completed_sales, q, pages)
|
# always prioritised over tracking seller age / category history.
|
||||||
listings = listings_future.result()
|
max_workers = min(len(ebay_queries) + 1, 5)
|
||||||
comps_future.result() # wait; side-effect is saving market comp to DB
|
with ThreadPoolExecutor(max_workers=max_workers) as ex:
|
||||||
|
comps_future = ex.submit(_run_comps)
|
||||||
|
search_futures = [ex.submit(_run_search, eq) for eq in ebay_queries]
|
||||||
|
|
||||||
|
# Merge and deduplicate across all search queries
|
||||||
|
seen_ids: set[str] = set()
|
||||||
|
listings: list = []
|
||||||
|
for fut in search_futures:
|
||||||
|
for listing in fut.result():
|
||||||
|
if listing.platform_listing_id not in seen_ids:
|
||||||
|
seen_ids.add(listing.platform_listing_id)
|
||||||
|
listings.append(listing)
|
||||||
|
comps_future.result() # side-effect: market comp written to DB
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
log.warning("eBay scrape failed: %s", e)
|
log.warning("eBay scrape failed: %s", e)
|
||||||
raise HTTPException(status_code=502, detail=f"eBay search failed: {e}")
|
raise HTTPException(status_code=502, detail=f"eBay search failed: {e}")
|
||||||
|
|
||||||
# Use search_adapter's store for post-processing — it has the sellers already written
|
log.info("Multi-search: %d queries → %d unique listings", len(ebay_queries), len(listings))
|
||||||
store = search_adapter._store
|
|
||||||
|
# Main-thread store for all post-search reads/writes — fresh connection, same thread.
|
||||||
|
store = Store(_DB_PATH)
|
||||||
store.save_listings(listings)
|
store.save_listings(listings)
|
||||||
|
|
||||||
|
# Derive category_history from accumulated listing data — free for API adapter
|
||||||
|
# (category_name comes from Browse API response), no-op for scraper listings (category_name=None).
|
||||||
|
seller_ids = list({l.seller_platform_id for l in listings if l.seller_platform_id})
|
||||||
|
n_cat = store.refresh_seller_categories("ebay", seller_ids)
|
||||||
|
if n_cat:
|
||||||
|
log.info("Category history derived for %d sellers from listing data", n_cat)
|
||||||
|
|
||||||
|
# Re-fetch to hydrate staging fields (times_seen, first_seen_at, id, price_at_first_seen)
|
||||||
|
# that are only available from the DB after the upsert.
|
||||||
|
staged = store.get_listings_staged("ebay", [l.platform_listing_id for l in listings])
|
||||||
|
listings = [staged.get(l.platform_listing_id, l) for l in listings]
|
||||||
|
|
||||||
|
# BTF enrichment: scrape /itm/ pages for sellers missing account_age_days.
|
||||||
|
# Runs in the background so it doesn't delay the response; next search of
|
||||||
|
# the same sellers will have full scores.
|
||||||
|
_trigger_scraper_enrichment(listings, store)
|
||||||
|
|
||||||
scorer = TrustScorer(store)
|
scorer = TrustScorer(store)
|
||||||
trust_scores_list = scorer.score_batch(listings, q)
|
trust_scores_list = scorer.score_batch(listings, q)
|
||||||
|
|
||||||
# Market comp written by comps_adapter — read from a fresh connection to avoid
|
|
||||||
# cross-thread connection reuse
|
|
||||||
comp_store = Store(_DB_PATH)
|
|
||||||
query_hash = hashlib.md5(q.encode()).hexdigest()
|
query_hash = hashlib.md5(q.encode()).hexdigest()
|
||||||
comp = comp_store.get_market_comp("ebay", query_hash)
|
comp = store.get_market_comp("ebay", query_hash)
|
||||||
market_price = comp.median_price if comp else None
|
market_price = comp.median_price if comp else None
|
||||||
|
|
||||||
# Serialize — keyed by platform_listing_id for easy Vue lookup
|
# Serialize — keyed by platform_listing_id for easy Vue lookup
|
||||||
|
|
@ -113,4 +279,117 @@ def search(
|
||||||
"trust_scores": trust_map,
|
"trust_scores": trust_map,
|
||||||
"sellers": seller_map,
|
"sellers": seller_map,
|
||||||
"market_price": market_price,
|
"market_price": market_price,
|
||||||
|
"adapter_used": adapter_used,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── On-demand enrichment ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@app.post("/api/enrich")
|
||||||
|
def enrich_seller(seller: str, listing_id: str, query: str = ""):
|
||||||
|
"""Synchronous on-demand enrichment for a single seller + re-score.
|
||||||
|
|
||||||
|
Runs enrichment paths in parallel:
|
||||||
|
- Shopping API GetUserProfile (fast, ~500ms) — account_age_days if API creds present
|
||||||
|
- BTF /itm/ Playwright scrape (~20s) — account_age_days fallback
|
||||||
|
- _ssn Playwright scrape (~20s) — category_history_json
|
||||||
|
|
||||||
|
BTF and _ssn run concurrently; total wall time ~20s when Playwright needed.
|
||||||
|
Returns the updated trust_score and seller so the frontend can patch in-place.
|
||||||
|
"""
|
||||||
|
import threading
|
||||||
|
store = Store(_DB_PATH)
|
||||||
|
|
||||||
|
seller_obj = store.get_seller("ebay", seller)
|
||||||
|
if not seller_obj:
|
||||||
|
raise HTTPException(status_code=404, detail=f"Seller '{seller}' not found")
|
||||||
|
|
||||||
|
# Fast path: Shopping API for account age (inline, no Playwright)
|
||||||
|
try:
|
||||||
|
api_adapter = _make_adapter(store, "api")
|
||||||
|
if hasattr(api_adapter, "enrich_sellers_shopping_api"):
|
||||||
|
api_adapter.enrich_sellers_shopping_api([seller])
|
||||||
|
except Exception:
|
||||||
|
pass # no API creds — fall through to BTF
|
||||||
|
|
||||||
|
seller_obj = store.get_seller("ebay", seller)
|
||||||
|
needs_btf = seller_obj is not None and seller_obj.account_age_days is None
|
||||||
|
needs_categories = seller_obj is None or seller_obj.category_history_json in ("{}", "", None)
|
||||||
|
|
||||||
|
# Slow path: Playwright for remaining gaps (BTF + _ssn in parallel threads)
|
||||||
|
if needs_btf or needs_categories:
|
||||||
|
scraper = ScrapedEbayAdapter(Store(_DB_PATH))
|
||||||
|
errors: list[Exception] = []
|
||||||
|
|
||||||
|
def _btf():
|
||||||
|
try:
|
||||||
|
scraper.enrich_sellers_btf({seller: listing_id}, max_workers=1)
|
||||||
|
except Exception as e:
|
||||||
|
errors.append(e)
|
||||||
|
|
||||||
|
def _ssn():
|
||||||
|
try:
|
||||||
|
ScrapedEbayAdapter(Store(_DB_PATH)).enrich_sellers_categories([seller], max_workers=1)
|
||||||
|
except Exception as e:
|
||||||
|
errors.append(e)
|
||||||
|
|
||||||
|
threads = []
|
||||||
|
if needs_btf:
|
||||||
|
threads.append(threading.Thread(target=_btf, daemon=True))
|
||||||
|
if needs_categories:
|
||||||
|
threads.append(threading.Thread(target=_ssn, daemon=True))
|
||||||
|
for t in threads:
|
||||||
|
t.start()
|
||||||
|
for t in threads:
|
||||||
|
t.join(timeout=60)
|
||||||
|
|
||||||
|
if errors:
|
||||||
|
log.warning("enrich_seller: %d scrape error(s): %s", len(errors), errors[0])
|
||||||
|
|
||||||
|
# Re-fetch listing with staging fields, re-score
|
||||||
|
staged = store.get_listings_staged("ebay", [listing_id])
|
||||||
|
listing = staged.get(listing_id)
|
||||||
|
if not listing:
|
||||||
|
raise HTTPException(status_code=404, detail=f"Listing '{listing_id}' not found")
|
||||||
|
|
||||||
|
scorer = TrustScorer(store)
|
||||||
|
trust_list = scorer.score_batch([listing], query or listing.title)
|
||||||
|
trust = trust_list[0] if trust_list else None
|
||||||
|
|
||||||
|
seller_final = store.get_seller("ebay", seller)
|
||||||
|
return {
|
||||||
|
"trust_score": dataclasses.asdict(trust) if trust else None,
|
||||||
|
"seller": dataclasses.asdict(seller_final) if seller_final else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── Saved Searches ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
class SavedSearchCreate(BaseModel):
|
||||||
|
name: str
|
||||||
|
query: str
|
||||||
|
filters_json: str = "{}"
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/api/saved-searches")
|
||||||
|
def list_saved_searches():
|
||||||
|
return {"saved_searches": [dataclasses.asdict(s) for s in Store(_DB_PATH).list_saved_searches()]}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/api/saved-searches", status_code=201)
|
||||||
|
def create_saved_search(body: SavedSearchCreate):
|
||||||
|
created = Store(_DB_PATH).save_saved_search(
|
||||||
|
SavedSearchModel(name=body.name, query=body.query, platform="ebay", filters_json=body.filters_json)
|
||||||
|
)
|
||||||
|
return dataclasses.asdict(created)
|
||||||
|
|
||||||
|
|
||||||
|
@app.delete("/api/saved-searches/{saved_id}", status_code=204)
|
||||||
|
def delete_saved_search(saved_id: int):
|
||||||
|
Store(_DB_PATH).delete_saved_search(saved_id)
|
||||||
|
|
||||||
|
|
||||||
|
@app.patch("/api/saved-searches/{saved_id}/run")
|
||||||
|
def mark_saved_search_run(saved_id: int):
|
||||||
|
Store(_DB_PATH).update_saved_search_last_run(saved_id)
|
||||||
|
return {"ok": True}
|
||||||
|
|
|
||||||
24
app/db/migrations/004_staging_tracking.sql
Normal file
24
app/db/migrations/004_staging_tracking.sql
Normal file
|
|
@ -0,0 +1,24 @@
|
||||||
|
-- Staging DB: persistent listing tracking across searches.
|
||||||
|
-- Adds temporal metadata to listings so we can detect stale/repriced/recurring items.
|
||||||
|
|
||||||
|
ALTER TABLE listings ADD COLUMN first_seen_at TEXT;
|
||||||
|
ALTER TABLE listings ADD COLUMN last_seen_at TEXT;
|
||||||
|
ALTER TABLE listings ADD COLUMN times_seen INTEGER NOT NULL DEFAULT 1;
|
||||||
|
ALTER TABLE listings ADD COLUMN price_at_first_seen REAL;
|
||||||
|
|
||||||
|
-- Backfill existing rows so columns are non-null where we have data
|
||||||
|
UPDATE listings SET
|
||||||
|
first_seen_at = fetched_at,
|
||||||
|
last_seen_at = fetched_at,
|
||||||
|
price_at_first_seen = price
|
||||||
|
WHERE first_seen_at IS NULL;
|
||||||
|
|
||||||
|
-- Price history: append-only snapshots; one row per (listing, price) change.
|
||||||
|
-- Duplicate prices are ignored (INSERT OR IGNORE) so only transitions are recorded.
|
||||||
|
CREATE TABLE IF NOT EXISTS listing_price_history (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
listing_id INTEGER NOT NULL REFERENCES listings(id),
|
||||||
|
price REAL NOT NULL,
|
||||||
|
captured_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
UNIQUE(listing_id, price)
|
||||||
|
);
|
||||||
3
app/db/migrations/005_listing_category.sql
Normal file
3
app/db/migrations/005_listing_category.sql
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
-- Add per-listing category name, extracted from eBay API response.
|
||||||
|
-- Used to derive seller category_history_json without _ssn scraping.
|
||||||
|
ALTER TABLE listings ADD COLUMN category_name TEXT;
|
||||||
|
|
@ -34,6 +34,12 @@ class Listing:
|
||||||
id: Optional[int] = None
|
id: Optional[int] = None
|
||||||
fetched_at: Optional[str] = None
|
fetched_at: Optional[str] = None
|
||||||
trust_score_id: Optional[int] = None
|
trust_score_id: Optional[int] = None
|
||||||
|
category_name: Optional[str] = None # leaf category from eBay API (e.g. "Graphics/Video Cards")
|
||||||
|
# Staging DB fields — populated from DB after upsert
|
||||||
|
first_seen_at: Optional[str] = None
|
||||||
|
last_seen_at: Optional[str] = None
|
||||||
|
times_seen: int = 1
|
||||||
|
price_at_first_seen: Optional[float] = None
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
|
|
|
||||||
178
app/db/store.py
178
app/db/store.py
|
|
@ -7,7 +7,7 @@ from typing import Optional
|
||||||
|
|
||||||
from circuitforge_core.db import get_connection, run_migrations
|
from circuitforge_core.db import get_connection, run_migrations
|
||||||
|
|
||||||
from .models import Listing, Seller, TrustScore, MarketComp
|
from .models import Listing, Seller, TrustScore, MarketComp, SavedSearch
|
||||||
|
|
||||||
MIGRATIONS_DIR = Path(__file__).parent / "migrations"
|
MIGRATIONS_DIR = Path(__file__).parent / "migrations"
|
||||||
|
|
||||||
|
|
@ -19,6 +19,18 @@ class Store:
|
||||||
|
|
||||||
# --- Seller ---
|
# --- Seller ---
|
||||||
|
|
||||||
|
def delete_seller_data(self, platform: str, platform_seller_id: str) -> None:
|
||||||
|
"""Permanently erase a seller and all their listings — GDPR/eBay deletion compliance."""
|
||||||
|
self._conn.execute(
|
||||||
|
"DELETE FROM sellers WHERE platform=? AND platform_seller_id=?",
|
||||||
|
(platform, platform_seller_id),
|
||||||
|
)
|
||||||
|
self._conn.execute(
|
||||||
|
"DELETE FROM listings WHERE platform=? AND seller_platform_id=?",
|
||||||
|
(platform, platform_seller_id),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
def save_seller(self, seller: Seller) -> None:
|
def save_seller(self, seller: Seller) -> None:
|
||||||
self.save_sellers([seller])
|
self.save_sellers([seller])
|
||||||
|
|
||||||
|
|
@ -47,31 +59,141 @@ class Store:
|
||||||
return None
|
return None
|
||||||
return Seller(*row[:7], id=row[7], fetched_at=row[8])
|
return Seller(*row[:7], id=row[7], fetched_at=row[8])
|
||||||
|
|
||||||
|
def refresh_seller_categories(self, platform: str, seller_ids: list[str]) -> int:
|
||||||
|
"""Derive category_history_json for sellers that lack it by aggregating
|
||||||
|
their stored listings' category_name values.
|
||||||
|
|
||||||
|
Returns the count of sellers updated.
|
||||||
|
"""
|
||||||
|
from app.platforms.ebay.scraper import _classify_category_label # lazy to avoid circular
|
||||||
|
|
||||||
|
if not seller_ids:
|
||||||
|
return 0
|
||||||
|
updated = 0
|
||||||
|
for sid in seller_ids:
|
||||||
|
seller = self.get_seller(platform, sid)
|
||||||
|
if not seller or seller.category_history_json not in ("{}", "", None):
|
||||||
|
continue # already enriched
|
||||||
|
rows = self._conn.execute(
|
||||||
|
"SELECT category_name, COUNT(*) FROM listings "
|
||||||
|
"WHERE platform=? AND seller_platform_id=? AND category_name IS NOT NULL "
|
||||||
|
"GROUP BY category_name",
|
||||||
|
(platform, sid),
|
||||||
|
).fetchall()
|
||||||
|
if not rows:
|
||||||
|
continue
|
||||||
|
counts: dict[str, int] = {}
|
||||||
|
for cat_name, cnt in rows:
|
||||||
|
key = _classify_category_label(cat_name)
|
||||||
|
if key:
|
||||||
|
counts[key] = counts.get(key, 0) + cnt
|
||||||
|
if counts:
|
||||||
|
from dataclasses import replace
|
||||||
|
updated_seller = replace(seller, category_history_json=json.dumps(counts))
|
||||||
|
self.save_seller(updated_seller)
|
||||||
|
updated += 1
|
||||||
|
return updated
|
||||||
|
|
||||||
# --- Listing ---
|
# --- Listing ---
|
||||||
|
|
||||||
def save_listing(self, listing: Listing) -> None:
|
def save_listing(self, listing: Listing) -> None:
|
||||||
self.save_listings([listing])
|
self.save_listings([listing])
|
||||||
|
|
||||||
def save_listings(self, listings: list[Listing]) -> None:
|
def save_listings(self, listings: list[Listing]) -> None:
|
||||||
|
"""Upsert listings, preserving first_seen_at and price_at_first_seen on conflict.
|
||||||
|
|
||||||
|
Uses INSERT ... ON CONFLICT DO UPDATE (SQLite 3.24+) so row IDs are stable
|
||||||
|
across searches — trust_score FK references survive re-indexing.
|
||||||
|
times_seen and last_seen_at accumulate on every sighting.
|
||||||
|
"""
|
||||||
|
now = datetime.now(timezone.utc).isoformat()
|
||||||
self._conn.executemany(
|
self._conn.executemany(
|
||||||
"INSERT OR REPLACE INTO listings "
|
"""
|
||||||
"(platform, platform_listing_id, title, price, currency, condition, "
|
INSERT INTO listings
|
||||||
"seller_platform_id, url, photo_urls, listing_age_days, buying_format, ends_at) "
|
(platform, platform_listing_id, title, price, currency, condition,
|
||||||
"VALUES (?,?,?,?,?,?,?,?,?,?,?,?)",
|
seller_platform_id, url, photo_urls, listing_age_days, buying_format,
|
||||||
|
ends_at, first_seen_at, last_seen_at, times_seen, price_at_first_seen,
|
||||||
|
category_name)
|
||||||
|
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,1,?,?)
|
||||||
|
ON CONFLICT(platform, platform_listing_id) DO UPDATE SET
|
||||||
|
title = excluded.title,
|
||||||
|
price = excluded.price,
|
||||||
|
condition = excluded.condition,
|
||||||
|
seller_platform_id = excluded.seller_platform_id,
|
||||||
|
url = excluded.url,
|
||||||
|
photo_urls = excluded.photo_urls,
|
||||||
|
listing_age_days = excluded.listing_age_days,
|
||||||
|
buying_format = excluded.buying_format,
|
||||||
|
ends_at = excluded.ends_at,
|
||||||
|
last_seen_at = excluded.last_seen_at,
|
||||||
|
times_seen = times_seen + 1,
|
||||||
|
category_name = COALESCE(excluded.category_name, category_name)
|
||||||
|
-- first_seen_at and price_at_first_seen intentionally preserved
|
||||||
|
""",
|
||||||
[
|
[
|
||||||
(l.platform, l.platform_listing_id, l.title, l.price, l.currency,
|
(l.platform, l.platform_listing_id, l.title, l.price, l.currency,
|
||||||
l.condition, l.seller_platform_id, l.url,
|
l.condition, l.seller_platform_id, l.url,
|
||||||
json.dumps(l.photo_urls), l.listing_age_days, l.buying_format, l.ends_at)
|
json.dumps(l.photo_urls), l.listing_age_days, l.buying_format, l.ends_at,
|
||||||
|
now, now, l.price, l.category_name)
|
||||||
|
for l in listings
|
||||||
|
],
|
||||||
|
)
|
||||||
|
# Record price snapshots — INSERT OR IGNORE means only price changes land
|
||||||
|
self._conn.executemany(
|
||||||
|
"""
|
||||||
|
INSERT OR IGNORE INTO listing_price_history (listing_id, price, captured_at)
|
||||||
|
SELECT id, ?, ? FROM listings
|
||||||
|
WHERE platform=? AND platform_listing_id=?
|
||||||
|
""",
|
||||||
|
[
|
||||||
|
(l.price, now, l.platform, l.platform_listing_id)
|
||||||
for l in listings
|
for l in listings
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
self._conn.commit()
|
self._conn.commit()
|
||||||
|
|
||||||
|
def get_listings_staged(self, platform: str, platform_listing_ids: list[str]) -> dict[str, "Listing"]:
|
||||||
|
"""Bulk fetch listings by platform_listing_id, returning staging fields.
|
||||||
|
|
||||||
|
Returns a dict keyed by platform_listing_id. Used to hydrate freshly-normalised
|
||||||
|
listing objects after save_listings() so trust scoring sees times_seen,
|
||||||
|
first_seen_at, price_at_first_seen, and the DB-assigned id.
|
||||||
|
"""
|
||||||
|
if not platform_listing_ids:
|
||||||
|
return {}
|
||||||
|
placeholders = ",".join("?" * len(platform_listing_ids))
|
||||||
|
rows = self._conn.execute(
|
||||||
|
f"SELECT platform, platform_listing_id, title, price, currency, condition, "
|
||||||
|
f"seller_platform_id, url, photo_urls, listing_age_days, id, fetched_at, "
|
||||||
|
f"buying_format, ends_at, first_seen_at, last_seen_at, times_seen, price_at_first_seen, "
|
||||||
|
f"category_name "
|
||||||
|
f"FROM listings WHERE platform=? AND platform_listing_id IN ({placeholders})",
|
||||||
|
[platform] + list(platform_listing_ids),
|
||||||
|
).fetchall()
|
||||||
|
result: dict[str, Listing] = {}
|
||||||
|
for row in rows:
|
||||||
|
pid = row[1]
|
||||||
|
result[pid] = Listing(
|
||||||
|
*row[:8],
|
||||||
|
photo_urls=json.loads(row[8]),
|
||||||
|
listing_age_days=row[9],
|
||||||
|
id=row[10],
|
||||||
|
fetched_at=row[11],
|
||||||
|
buying_format=row[12] or "fixed_price",
|
||||||
|
ends_at=row[13],
|
||||||
|
first_seen_at=row[14],
|
||||||
|
last_seen_at=row[15],
|
||||||
|
times_seen=row[16] or 1,
|
||||||
|
price_at_first_seen=row[17],
|
||||||
|
category_name=row[18],
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
|
||||||
def get_listing(self, platform: str, platform_listing_id: str) -> Optional[Listing]:
|
def get_listing(self, platform: str, platform_listing_id: str) -> Optional[Listing]:
|
||||||
row = self._conn.execute(
|
row = self._conn.execute(
|
||||||
"SELECT platform, platform_listing_id, title, price, currency, condition, "
|
"SELECT platform, platform_listing_id, title, price, currency, condition, "
|
||||||
"seller_platform_id, url, photo_urls, listing_age_days, id, fetched_at, "
|
"seller_platform_id, url, photo_urls, listing_age_days, id, fetched_at, "
|
||||||
"buying_format, ends_at "
|
"buying_format, ends_at, first_seen_at, last_seen_at, times_seen, price_at_first_seen "
|
||||||
"FROM listings WHERE platform=? AND platform_listing_id=?",
|
"FROM listings WHERE platform=? AND platform_listing_id=?",
|
||||||
(platform, platform_listing_id),
|
(platform, platform_listing_id),
|
||||||
).fetchone()
|
).fetchone()
|
||||||
|
|
@ -85,6 +207,10 @@ class Store:
|
||||||
fetched_at=row[11],
|
fetched_at=row[11],
|
||||||
buying_format=row[12] or "fixed_price",
|
buying_format=row[12] or "fixed_price",
|
||||||
ends_at=row[13],
|
ends_at=row[13],
|
||||||
|
first_seen_at=row[14],
|
||||||
|
last_seen_at=row[15],
|
||||||
|
times_seen=row[16] or 1,
|
||||||
|
price_at_first_seen=row[17],
|
||||||
)
|
)
|
||||||
|
|
||||||
# --- MarketComp ---
|
# --- MarketComp ---
|
||||||
|
|
@ -99,6 +225,44 @@ class Store:
|
||||||
)
|
)
|
||||||
self._conn.commit()
|
self._conn.commit()
|
||||||
|
|
||||||
|
# --- SavedSearch ---
|
||||||
|
|
||||||
|
def save_saved_search(self, s: SavedSearch) -> SavedSearch:
|
||||||
|
cur = self._conn.execute(
|
||||||
|
"INSERT INTO saved_searches (name, query, platform, filters_json) VALUES (?,?,?,?)",
|
||||||
|
(s.name, s.query, s.platform, s.filters_json),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
row = self._conn.execute(
|
||||||
|
"SELECT id, created_at FROM saved_searches WHERE id=?", (cur.lastrowid,)
|
||||||
|
).fetchone()
|
||||||
|
return SavedSearch(
|
||||||
|
name=s.name, query=s.query, platform=s.platform,
|
||||||
|
filters_json=s.filters_json, id=row[0], created_at=row[1],
|
||||||
|
)
|
||||||
|
|
||||||
|
def list_saved_searches(self) -> list[SavedSearch]:
|
||||||
|
rows = self._conn.execute(
|
||||||
|
"SELECT name, query, platform, filters_json, id, created_at, last_run_at "
|
||||||
|
"FROM saved_searches ORDER BY created_at DESC"
|
||||||
|
).fetchall()
|
||||||
|
return [
|
||||||
|
SavedSearch(name=r[0], query=r[1], platform=r[2], filters_json=r[3],
|
||||||
|
id=r[4], created_at=r[5], last_run_at=r[6])
|
||||||
|
for r in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
def delete_saved_search(self, saved_id: int) -> None:
|
||||||
|
self._conn.execute("DELETE FROM saved_searches WHERE id=?", (saved_id,))
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
def update_saved_search_last_run(self, saved_id: int) -> None:
|
||||||
|
self._conn.execute(
|
||||||
|
"UPDATE saved_searches SET last_run_at=? WHERE id=?",
|
||||||
|
(datetime.now(timezone.utc).isoformat(), saved_id),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
def get_market_comp(self, platform: str, query_hash: str) -> Optional[MarketComp]:
|
def get_market_comp(self, platform: str, query_hash: str) -> Optional[MarketComp]:
|
||||||
row = self._conn.execute(
|
row = self._conn.execute(
|
||||||
"SELECT platform, query_hash, median_price, sample_count, expires_at, id, fetched_at "
|
"SELECT platform, query_hash, median_price, sample_count, expires_at, id, fetched_at "
|
||||||
|
|
|
||||||
|
|
@ -15,6 +15,7 @@ class SearchFilters:
|
||||||
pages: int = 1 # number of result pages to fetch (48 listings/page)
|
pages: int = 1 # number of result pages to fetch (48 listings/page)
|
||||||
must_include: list[str] = field(default_factory=list) # client-side title filter
|
must_include: list[str] = field(default_factory=list) # client-side title filter
|
||||||
must_exclude: list[str] = field(default_factory=list) # forwarded to eBay -term AND client-side
|
must_exclude: list[str] = field(default_factory=list) # forwarded to eBay -term AND client-side
|
||||||
|
category_id: Optional[str] = None # eBay category ID (e.g. "27386" = GPUs)
|
||||||
|
|
||||||
|
|
||||||
class PlatformAdapter(ABC):
|
class PlatformAdapter(ABC):
|
||||||
|
|
|
||||||
|
|
@ -1,16 +1,58 @@
|
||||||
"""eBay Browse API adapter."""
|
"""eBay Browse API adapter."""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
import hashlib
|
import hashlib
|
||||||
|
import logging
|
||||||
|
from dataclasses import replace
|
||||||
from datetime import datetime, timedelta, timezone
|
from datetime import datetime, timedelta, timezone
|
||||||
from typing import Optional
|
from typing import Optional
|
||||||
import requests
|
import requests
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_SHOPPING_BASE = "https://open.api.ebay.com/shopping"
|
||||||
|
|
||||||
|
# Rate limiting for Shopping API GetUserProfile calls.
|
||||||
|
# Enrichment is incremental — these caps spread API calls across multiple
|
||||||
|
# searches rather than bursting on first encounter with a new seller batch.
|
||||||
|
_SHOPPING_API_MAX_PER_SEARCH = 5 # sellers enriched per search call
|
||||||
|
_SHOPPING_API_INTER_REQUEST_DELAY = 0.5 # seconds between successive calls
|
||||||
|
_SELLER_ENRICH_TTL_HOURS = 24 # skip re-enrichment within this window
|
||||||
|
|
||||||
from app.db.models import Listing, Seller, MarketComp
|
from app.db.models import Listing, Seller, MarketComp
|
||||||
from app.db.store import Store
|
from app.db.store import Store
|
||||||
from app.platforms import PlatformAdapter, SearchFilters
|
from app.platforms import PlatformAdapter, SearchFilters
|
||||||
from app.platforms.ebay.auth import EbayTokenManager
|
from app.platforms.ebay.auth import EbayTokenManager
|
||||||
from app.platforms.ebay.normaliser import normalise_listing, normalise_seller
|
from app.platforms.ebay.normaliser import normalise_listing, normalise_seller
|
||||||
|
|
||||||
|
_BROWSE_LIMIT = 200 # max items per Browse API page
|
||||||
|
_INSIGHTS_BASE = {
|
||||||
|
"production": "https://api.ebay.com/buy/marketplace_insights/v1_beta",
|
||||||
|
"sandbox": "https://api.sandbox.ebay.com/buy/marketplace_insights/v1_beta",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _build_browse_query(base_query: str, or_groups: list[list[str]], must_exclude: list[str]) -> str:
|
||||||
|
"""Convert OR groups + exclusions into Browse API boolean query syntax.
|
||||||
|
|
||||||
|
Browse API uses SQL-like boolean: AND (implicit), OR (keyword), NOT (keyword).
|
||||||
|
Parentheses work as grouping operators.
|
||||||
|
Example: 'GPU (16gb OR 24gb OR 48gb) (nvidia OR rtx OR geforce) NOT "parts only"'
|
||||||
|
"""
|
||||||
|
parts = [base_query.strip()]
|
||||||
|
for group in or_groups:
|
||||||
|
clean = [t.strip() for t in group if t.strip()]
|
||||||
|
if len(clean) == 1:
|
||||||
|
parts.append(clean[0])
|
||||||
|
elif len(clean) > 1:
|
||||||
|
parts.append(f"({' OR '.join(clean)})")
|
||||||
|
for term in must_exclude:
|
||||||
|
term = term.strip()
|
||||||
|
if term:
|
||||||
|
# Use minus syntax (-term / -"phrase") — Browse API's NOT keyword
|
||||||
|
# over-filters dramatically in practice; minus works like web search negatives.
|
||||||
|
parts.append(f'-"{term}"' if " " in term else f"-{term}")
|
||||||
|
return " ".join(p for p in parts if p)
|
||||||
|
|
||||||
BROWSE_BASE = {
|
BROWSE_BASE = {
|
||||||
"production": "https://api.ebay.com/buy/browse/v1",
|
"production": "https://api.ebay.com/buy/browse/v1",
|
||||||
"sandbox": "https://api.sandbox.ebay.com/buy/browse/v1",
|
"sandbox": "https://api.sandbox.ebay.com/buy/browse/v1",
|
||||||
|
|
@ -25,29 +67,146 @@ class EbayAdapter(PlatformAdapter):
|
||||||
def __init__(self, token_manager: EbayTokenManager, store: Store, env: str = "production"):
|
def __init__(self, token_manager: EbayTokenManager, store: Store, env: str = "production"):
|
||||||
self._tokens = token_manager
|
self._tokens = token_manager
|
||||||
self._store = store
|
self._store = store
|
||||||
|
self._env = env
|
||||||
self._browse_base = BROWSE_BASE[env]
|
self._browse_base = BROWSE_BASE[env]
|
||||||
|
|
||||||
def _headers(self) -> dict:
|
def _headers(self) -> dict:
|
||||||
return {"Authorization": f"Bearer {self._tokens.get_token()}"}
|
return {"Authorization": f"Bearer {self._tokens.get_token()}"}
|
||||||
|
|
||||||
def search(self, query: str, filters: SearchFilters) -> list[Listing]:
|
def search(self, query: str, filters: SearchFilters) -> list[Listing]:
|
||||||
params: dict = {"q": query, "limit": 50}
|
# Build Browse API boolean query from OR groups + exclusions
|
||||||
filter_parts = []
|
browse_q = _build_browse_query(query, getattr(filters, "or_groups", []), filters.must_exclude)
|
||||||
|
|
||||||
|
filter_parts: list[str] = []
|
||||||
if filters.max_price:
|
if filters.max_price:
|
||||||
filter_parts.append(f"price:[..{filters.max_price}],priceCurrency:USD")
|
filter_parts.append(f"price:[..{filters.max_price}],priceCurrency:USD")
|
||||||
|
if filters.min_price:
|
||||||
|
filter_parts.append(f"price:[{filters.min_price}..],priceCurrency:USD")
|
||||||
if filters.condition:
|
if filters.condition:
|
||||||
cond_map = {"new": "NEW", "used": "USED", "open box": "OPEN_BOX", "for parts": "FOR_PARTS_NOT_WORKING"}
|
cond_map = {
|
||||||
|
"new": "NEW", "used": "USED",
|
||||||
|
"open box": "OPEN_BOX", "for parts": "FOR_PARTS_NOT_WORKING",
|
||||||
|
}
|
||||||
ebay_conds = [cond_map[c] for c in filters.condition if c in cond_map]
|
ebay_conds = [cond_map[c] for c in filters.condition if c in cond_map]
|
||||||
if ebay_conds:
|
if ebay_conds:
|
||||||
filter_parts.append(f"conditions:{{{','.join(ebay_conds)}}}")
|
filter_parts.append(f"conditions:{{{','.join(ebay_conds)}}}")
|
||||||
if filter_parts:
|
|
||||||
params["filter"] = ",".join(filter_parts)
|
|
||||||
|
|
||||||
resp = requests.get(f"{self._browse_base}/item_summary/search",
|
base_params: dict = {"q": browse_q, "limit": _BROWSE_LIMIT}
|
||||||
headers=self._headers(), params=params)
|
if filter_parts:
|
||||||
resp.raise_for_status()
|
base_params["filter"] = ",".join(filter_parts)
|
||||||
items = resp.json().get("itemSummaries", [])
|
if filters.category_id:
|
||||||
return [normalise_listing(item) for item in items]
|
base_params["category_ids"] = filters.category_id
|
||||||
|
|
||||||
|
pages = max(1, filters.pages)
|
||||||
|
seen_ids: set[str] = set()
|
||||||
|
listings: list[Listing] = []
|
||||||
|
sellers_to_save: dict[str, Seller] = {}
|
||||||
|
|
||||||
|
for page in range(pages):
|
||||||
|
params = {**base_params, "offset": page * _BROWSE_LIMIT}
|
||||||
|
resp = requests.get(
|
||||||
|
f"{self._browse_base}/item_summary/search",
|
||||||
|
headers=self._headers(),
|
||||||
|
params=params,
|
||||||
|
)
|
||||||
|
resp.raise_for_status()
|
||||||
|
data = resp.json()
|
||||||
|
items = data.get("itemSummaries", [])
|
||||||
|
if not items:
|
||||||
|
break # no more results
|
||||||
|
|
||||||
|
for item in items:
|
||||||
|
listing = normalise_listing(item)
|
||||||
|
if listing.platform_listing_id not in seen_ids:
|
||||||
|
seen_ids.add(listing.platform_listing_id)
|
||||||
|
listings.append(listing)
|
||||||
|
# Extract inline seller data available in item_summary
|
||||||
|
seller_raw = item.get("seller", {})
|
||||||
|
if seller_raw.get("username") and seller_raw["username"] not in sellers_to_save:
|
||||||
|
sellers_to_save[seller_raw["username"]] = normalise_seller(seller_raw)
|
||||||
|
|
||||||
|
if not data.get("next"):
|
||||||
|
break # Browse API paginates via "next" href; absence = last page
|
||||||
|
|
||||||
|
if sellers_to_save:
|
||||||
|
self._store.save_sellers(list(sellers_to_save.values()))
|
||||||
|
|
||||||
|
# Enrich sellers missing account_age_days via Shopping API (fast HTTP, no Playwright).
|
||||||
|
# Capped at _SHOPPING_API_MAX_PER_SEARCH to avoid bursting the daily quota when
|
||||||
|
# many new sellers appear in a single search batch.
|
||||||
|
needs_age = [s.platform_seller_id for s in sellers_to_save.values()
|
||||||
|
if s.account_age_days is None]
|
||||||
|
if needs_age:
|
||||||
|
self.enrich_sellers_shopping_api(needs_age[:_SHOPPING_API_MAX_PER_SEARCH])
|
||||||
|
|
||||||
|
return listings
|
||||||
|
|
||||||
|
def enrich_sellers_shopping_api(self, usernames: list[str]) -> None:
|
||||||
|
"""Fetch RegistrationDate for sellers via Shopping API GetUserProfile.
|
||||||
|
|
||||||
|
Uses app-level Bearer token — no user OAuth required. Silently skips
|
||||||
|
on rate limit (error 1.21) or any other failure so the search response
|
||||||
|
is never blocked. BTF scraping remains the fallback for the scraper adapter.
|
||||||
|
|
||||||
|
Rate limiting: _SHOPPING_API_INTER_REQUEST_DELAY between calls; sellers
|
||||||
|
enriched within _SELLER_ENRICH_TTL_HOURS are skipped (account age doesn't
|
||||||
|
change day to day). Callers should already cap the list length.
|
||||||
|
"""
|
||||||
|
token = self._tokens.get_token()
|
||||||
|
headers = {
|
||||||
|
"X-EBAY-API-IAF-TOKEN": f"Bearer {token}",
|
||||||
|
"User-Agent": "Mozilla/5.0",
|
||||||
|
}
|
||||||
|
cutoff = datetime.now(timezone.utc) - timedelta(hours=_SELLER_ENRICH_TTL_HOURS)
|
||||||
|
first = True
|
||||||
|
for username in usernames:
|
||||||
|
try:
|
||||||
|
# Skip recently enriched sellers — account age doesn't change daily.
|
||||||
|
seller = self._store.get_seller("ebay", username)
|
||||||
|
if seller and seller.fetched_at:
|
||||||
|
try:
|
||||||
|
ft = datetime.fromisoformat(seller.fetched_at.replace("Z", "+00:00"))
|
||||||
|
if ft.tzinfo is None:
|
||||||
|
ft = ft.replace(tzinfo=timezone.utc)
|
||||||
|
if ft > cutoff and seller.account_age_days is not None:
|
||||||
|
continue
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
if not first:
|
||||||
|
import time as _time
|
||||||
|
_time.sleep(_SHOPPING_API_INTER_REQUEST_DELAY)
|
||||||
|
first = False
|
||||||
|
|
||||||
|
resp = requests.get(
|
||||||
|
_SHOPPING_BASE,
|
||||||
|
headers=headers,
|
||||||
|
params={
|
||||||
|
"callname": "GetUserProfile",
|
||||||
|
"appid": self._tokens.client_id,
|
||||||
|
"siteid": "0",
|
||||||
|
"version": "967",
|
||||||
|
"UserID": username,
|
||||||
|
"responseencoding": "JSON",
|
||||||
|
},
|
||||||
|
timeout=10,
|
||||||
|
)
|
||||||
|
data = resp.json()
|
||||||
|
if data.get("Ack") != "Success":
|
||||||
|
errors = data.get("Errors", [])
|
||||||
|
if any(e.get("ErrorCode") == "1.21" for e in errors):
|
||||||
|
log.debug("Shopping API rate-limited for %s — BTF fallback", username)
|
||||||
|
continue
|
||||||
|
reg_date = data.get("User", {}).get("RegistrationDate")
|
||||||
|
if reg_date:
|
||||||
|
dt = datetime.fromisoformat(reg_date.replace("Z", "+00:00"))
|
||||||
|
age_days = (datetime.now(timezone.utc) - dt).days
|
||||||
|
seller = self._store.get_seller("ebay", username)
|
||||||
|
if seller:
|
||||||
|
self._store.save_seller(replace(seller, account_age_days=age_days))
|
||||||
|
log.debug("Shopping API: %s registered %d days ago", username, age_days)
|
||||||
|
except Exception as e:
|
||||||
|
log.debug("Shopping API enrich failed for %s: %s", username, e)
|
||||||
|
|
||||||
def get_seller(self, seller_platform_id: str) -> Optional[Seller]:
|
def get_seller(self, seller_platform_id: str) -> Optional[Seller]:
|
||||||
cached = self._store.get_seller("ebay", seller_platform_id)
|
cached = self._store.get_seller("ebay", seller_platform_id)
|
||||||
|
|
@ -69,30 +228,62 @@ class EbayAdapter(PlatformAdapter):
|
||||||
except Exception:
|
except Exception:
|
||||||
return None # Caller handles None gracefully (partial score)
|
return None # Caller handles None gracefully (partial score)
|
||||||
|
|
||||||
def get_completed_sales(self, query: str) -> list[Listing]:
|
def get_completed_sales(self, query: str, pages: int = 1) -> list[Listing]:
|
||||||
query_hash = hashlib.md5(query.encode()).hexdigest()
|
query_hash = hashlib.md5(query.encode()).hexdigest()
|
||||||
cached = self._store.get_market_comp("ebay", query_hash)
|
if self._store.get_market_comp("ebay", query_hash):
|
||||||
if cached:
|
return [] # cache hit
|
||||||
return [] # Comp data is used directly; return empty to signal cache hit
|
|
||||||
|
|
||||||
params = {"q": query, "limit": 20, "filter": "buyingOptions:{FIXED_PRICE}"}
|
prices: list[float] = []
|
||||||
try:
|
try:
|
||||||
resp = requests.get(f"{self._browse_base}/item_summary/search",
|
# Marketplace Insights API returns sold/completed items — best source for comps.
|
||||||
headers=self._headers(), params=params)
|
# Falls back gracefully to Browse API active listings if the endpoint is
|
||||||
|
# unavailable (requires buy.marketplace.insights scope).
|
||||||
|
insights_base = _INSIGHTS_BASE.get(self._env, _INSIGHTS_BASE["production"])
|
||||||
|
resp = requests.get(
|
||||||
|
f"{insights_base}/item_summary/search",
|
||||||
|
headers=self._headers(),
|
||||||
|
params={"q": query, "limit": 50, "filter": "buyingOptions:{FIXED_PRICE}"},
|
||||||
|
)
|
||||||
|
if resp.status_code in (403, 404):
|
||||||
|
# 403 = scope not granted; 404 = endpoint not available for this app tier.
|
||||||
|
# Both mean: fall back to active listing prices via Browse API.
|
||||||
|
log.info("comps api: Marketplace Insights unavailable (%d), falling back to Browse API", resp.status_code)
|
||||||
|
raise PermissionError("Marketplace Insights not available")
|
||||||
resp.raise_for_status()
|
resp.raise_for_status()
|
||||||
items = resp.json().get("itemSummaries", [])
|
items = resp.json().get("itemSummaries", [])
|
||||||
listings = [normalise_listing(item) for item in items]
|
prices = [float(i["lastSoldPrice"]["value"]) for i in items if "lastSoldPrice" in i]
|
||||||
if listings:
|
log.info("comps api: Marketplace Insights returned %d items, %d with lastSoldPrice", len(items), len(prices))
|
||||||
prices = sorted(l.price for l in listings)
|
except PermissionError:
|
||||||
median = prices[len(prices) // 2]
|
# Fallback: use active listing prices (less accurate but always available)
|
||||||
comp = MarketComp(
|
try:
|
||||||
platform="ebay",
|
resp = requests.get(
|
||||||
query_hash=query_hash,
|
f"{self._browse_base}/item_summary/search",
|
||||||
median_price=median,
|
headers=self._headers(),
|
||||||
sample_count=len(prices),
|
params={"q": query, "limit": 50, "filter": "buyingOptions:{FIXED_PRICE}"},
|
||||||
expires_at=(datetime.now(timezone.utc) + timedelta(hours=6)).isoformat(),
|
|
||||||
)
|
)
|
||||||
self._store.save_market_comp(comp)
|
resp.raise_for_status()
|
||||||
return listings
|
items = resp.json().get("itemSummaries", [])
|
||||||
|
prices = [float(i["price"]["value"]) for i in items if "price" in i]
|
||||||
|
log.info("comps api: Browse API fallback returned %d items, %d with price", len(items), len(prices))
|
||||||
|
except Exception:
|
||||||
|
log.warning("comps api: Browse API fallback failed for %r", query, exc_info=True)
|
||||||
|
return []
|
||||||
except Exception:
|
except Exception:
|
||||||
|
log.warning("comps api: unexpected error for %r", query, exc_info=True)
|
||||||
return []
|
return []
|
||||||
|
|
||||||
|
if not prices:
|
||||||
|
log.warning("comps api: 0 valid prices extracted — no comp saved for %r", query)
|
||||||
|
return []
|
||||||
|
|
||||||
|
prices.sort()
|
||||||
|
n = len(prices)
|
||||||
|
median = (prices[n // 2 - 1] + prices[n // 2]) / 2 if n % 2 == 0 else prices[n // 2]
|
||||||
|
self._store.save_market_comp(MarketComp(
|
||||||
|
platform="ebay",
|
||||||
|
query_hash=query_hash,
|
||||||
|
median_price=median,
|
||||||
|
sample_count=n,
|
||||||
|
expires_at=(datetime.now(timezone.utc) + timedelta(hours=6)).isoformat(),
|
||||||
|
))
|
||||||
|
return []
|
||||||
|
|
|
||||||
|
|
@ -21,6 +21,10 @@ class EbayTokenManager:
|
||||||
self._token: Optional[str] = None
|
self._token: Optional[str] = None
|
||||||
self._expires_at: float = 0.0
|
self._expires_at: float = 0.0
|
||||||
|
|
||||||
|
@property
|
||||||
|
def client_id(self) -> str:
|
||||||
|
return self._client_id
|
||||||
|
|
||||||
def get_token(self) -> str:
|
def get_token(self) -> str:
|
||||||
"""Return a valid access token, fetching or refreshing as needed."""
|
"""Return a valid access token, fetching or refreshing as needed."""
|
||||||
if self._token and time.time() < self._expires_at - 60:
|
if self._token and time.time() < self._expires_at - 60:
|
||||||
|
|
|
||||||
|
|
@ -2,6 +2,7 @@
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
import json
|
import json
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
|
from typing import Optional
|
||||||
from app.db.models import Listing, Seller
|
from app.db.models import Listing, Seller
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -41,6 +42,10 @@ def normalise_listing(raw: dict) -> Listing:
|
||||||
except ValueError:
|
except ValueError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
# Leaf category is categories[0] (most specific); parent path follows.
|
||||||
|
categories = raw.get("categories", [])
|
||||||
|
category_name: Optional[str] = categories[0]["categoryName"] if categories else None
|
||||||
|
|
||||||
seller = raw.get("seller", {})
|
seller = raw.get("seller", {})
|
||||||
return Listing(
|
return Listing(
|
||||||
platform="ebay",
|
platform="ebay",
|
||||||
|
|
@ -55,13 +60,14 @@ def normalise_listing(raw: dict) -> Listing:
|
||||||
listing_age_days=listing_age_days,
|
listing_age_days=listing_age_days,
|
||||||
buying_format=buying_format,
|
buying_format=buying_format,
|
||||||
ends_at=ends_at,
|
ends_at=ends_at,
|
||||||
|
category_name=category_name,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def normalise_seller(raw: dict) -> Seller:
|
def normalise_seller(raw: dict) -> Seller:
|
||||||
feedback_pct = float(raw.get("feedbackPercentage", "0").strip("%")) / 100.0
|
feedback_pct = float(raw.get("feedbackPercentage", "0").strip("%")) / 100.0
|
||||||
|
|
||||||
account_age_days = 0
|
account_age_days: Optional[int] = None # None = registrationDate not in API response
|
||||||
reg_date_raw = raw.get("registrationDate", "")
|
reg_date_raw = raw.get("registrationDate", "")
|
||||||
if reg_date_raw:
|
if reg_date_raw:
|
||||||
try:
|
try:
|
||||||
|
|
|
||||||
85
app/platforms/ebay/query_builder.py
Normal file
85
app/platforms/ebay/query_builder.py
Normal file
|
|
@ -0,0 +1,85 @@
|
||||||
|
"""
|
||||||
|
Build eBay-compatible boolean search queries from OR groups.
|
||||||
|
|
||||||
|
eBay honors parenthetical OR groups in the _nkw search parameter:
|
||||||
|
(term1,term2,term3) → must contain at least one of these terms
|
||||||
|
-term / -"phrase" → must NOT contain this term / phrase
|
||||||
|
space between groups → implicit AND
|
||||||
|
|
||||||
|
expand_queries() generates one eBay query per term in the smallest OR group,
|
||||||
|
using eBay's OR syntax for all remaining groups. This guarantees coverage even
|
||||||
|
if eBay's relevance ranking would suppress some matches in a single combined query.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
base = "GPU"
|
||||||
|
or_groups = [["16gb","24gb","40gb","48gb"], ["nvidia","quadro","rtx","geforce","titan"]]
|
||||||
|
→ 4 queries (one per memory size, brand group as eBay OR):
|
||||||
|
"GPU 16gb (nvidia,quadro,rtx,geforce,titan)"
|
||||||
|
"GPU 24gb (nvidia,quadro,rtx,geforce,titan)"
|
||||||
|
"GPU 40gb (nvidia,quadro,rtx,geforce,titan)"
|
||||||
|
"GPU 48gb (nvidia,quadro,rtx,geforce,titan)"
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
def _group_to_ebay(group: list[str]) -> str:
|
||||||
|
"""Convert a list of alternatives to an eBay OR clause."""
|
||||||
|
clean = [t.strip() for t in group if t.strip()]
|
||||||
|
if not clean:
|
||||||
|
return ""
|
||||||
|
if len(clean) == 1:
|
||||||
|
return clean[0]
|
||||||
|
return f"({','.join(clean)})"
|
||||||
|
|
||||||
|
|
||||||
|
def build_ebay_query(base_query: str, or_groups: list[list[str]]) -> str:
|
||||||
|
"""
|
||||||
|
Build a single eBay _nkw query string using eBay's parenthetical OR syntax.
|
||||||
|
Exclusions are handled separately via SearchFilters.must_exclude.
|
||||||
|
"""
|
||||||
|
parts = [base_query.strip()]
|
||||||
|
for group in or_groups:
|
||||||
|
clause = _group_to_ebay(group)
|
||||||
|
if clause:
|
||||||
|
parts.append(clause)
|
||||||
|
return " ".join(p for p in parts if p)
|
||||||
|
|
||||||
|
|
||||||
|
def expand_queries(base_query: str, or_groups: list[list[str]]) -> list[str]:
|
||||||
|
"""
|
||||||
|
Expand OR groups into one eBay query per term in the smallest group,
|
||||||
|
using eBay's OR syntax for all remaining groups.
|
||||||
|
|
||||||
|
This guarantees every term in the pivot group is explicitly searched,
|
||||||
|
which prevents eBay's relevance engine from silently skipping rare variants.
|
||||||
|
Falls back to a single query when there are no OR groups.
|
||||||
|
"""
|
||||||
|
if not or_groups:
|
||||||
|
return [base_query.strip()]
|
||||||
|
|
||||||
|
# Pivot on the smallest group to minimise the number of Playwright calls
|
||||||
|
smallest_idx = min(range(len(or_groups)), key=lambda i: len(or_groups[i]))
|
||||||
|
pivot = or_groups[smallest_idx]
|
||||||
|
rest = [g for i, g in enumerate(or_groups) if i != smallest_idx]
|
||||||
|
|
||||||
|
queries = []
|
||||||
|
for term in pivot:
|
||||||
|
q = build_ebay_query(base_query, [[term]] + rest)
|
||||||
|
queries.append(q)
|
||||||
|
return queries
|
||||||
|
|
||||||
|
|
||||||
|
def parse_groups(raw: str) -> list[list[str]]:
|
||||||
|
"""
|
||||||
|
Parse a Groups-mode must_include string into nested OR groups.
|
||||||
|
|
||||||
|
Format: comma separates groups (AND), pipe separates alternatives within a group (OR).
|
||||||
|
"16gb|24gb|48gb, nvidia|rtx|geforce"
|
||||||
|
→ [["16gb","24gb","48gb"], ["nvidia","rtx","geforce"]]
|
||||||
|
"""
|
||||||
|
groups = []
|
||||||
|
for chunk in raw.split(","):
|
||||||
|
alts = [t.strip().lower() for t in chunk.split("|") if t.strip()]
|
||||||
|
if alts:
|
||||||
|
groups.append(alts)
|
||||||
|
return groups
|
||||||
|
|
@ -3,8 +3,8 @@
|
||||||
Data available from search results HTML (single page load):
|
Data available from search results HTML (single page load):
|
||||||
✅ title, price, condition, photos, URL
|
✅ title, price, condition, photos, URL
|
||||||
✅ seller username, feedback count, feedback ratio
|
✅ seller username, feedback count, feedback ratio
|
||||||
❌ account registration date → account_age_score = None (score_is_partial)
|
❌ account registration date → enriched async via BTF /itm/ scrape
|
||||||
❌ category history → category_history_score = None (score_is_partial)
|
❌ category history → enriched async via _ssn seller search page
|
||||||
|
|
||||||
This is the MIT discovery layer. EbayAdapter (paid/CF proxy) unlocks full trust scores.
|
This is the MIT discovery layer. EbayAdapter (paid/CF proxy) unlocks full trust scores.
|
||||||
"""
|
"""
|
||||||
|
|
@ -12,12 +12,16 @@ from __future__ import annotations
|
||||||
|
|
||||||
import hashlib
|
import hashlib
|
||||||
import itertools
|
import itertools
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
import re
|
import re
|
||||||
import time
|
import time
|
||||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||||
from datetime import datetime, timedelta, timezone
|
from datetime import datetime, timedelta, timezone
|
||||||
from typing import Optional
|
from typing import Optional
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
from bs4 import BeautifulSoup
|
from bs4 import BeautifulSoup
|
||||||
|
|
||||||
from app.db.models import Listing, MarketComp, Seller
|
from app.db.models import Listing, MarketComp, Seller
|
||||||
|
|
@ -25,7 +29,12 @@ from app.db.store import Store
|
||||||
from app.platforms import PlatformAdapter, SearchFilters
|
from app.platforms import PlatformAdapter, SearchFilters
|
||||||
|
|
||||||
EBAY_SEARCH_URL = "https://www.ebay.com/sch/i.html"
|
EBAY_SEARCH_URL = "https://www.ebay.com/sch/i.html"
|
||||||
|
EBAY_ITEM_URL = "https://www.ebay.com/itm/"
|
||||||
_HTML_CACHE_TTL = 300 # seconds — 5 minutes
|
_HTML_CACHE_TTL = 300 # seconds — 5 minutes
|
||||||
|
_JOINED_RE = re.compile(r"Joined\s+(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\w*\s+(\d{4})", re.I)
|
||||||
|
_MONTH_MAP = {m: i+1 for i, m in enumerate(
|
||||||
|
["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"]
|
||||||
|
)}
|
||||||
|
|
||||||
# Module-level cache persists across per-request adapter instantiations.
|
# Module-level cache persists across per-request adapter instantiations.
|
||||||
# Keyed by URL; value is (html, expiry_timestamp).
|
# Keyed by URL; value is (html, expiry_timestamp).
|
||||||
|
|
@ -53,6 +62,25 @@ _FEEDBACK_RE = re.compile(r"([\d.]+)%\s+positive\s+\(([0-9,]+)\)", re.I)
|
||||||
_PRICE_RE = re.compile(r"[\d,]+\.?\d*")
|
_PRICE_RE = re.compile(r"[\d,]+\.?\d*")
|
||||||
_ITEM_ID_RE = re.compile(r"/itm/(\d+)")
|
_ITEM_ID_RE = re.compile(r"/itm/(\d+)")
|
||||||
_TIME_LEFT_RE = re.compile(r"(?:(\d+)d\s*)?(?:(\d+)h\s*)?(?:(\d+)m\s*)?(?:(\d+)s\s*)?left", re.I)
|
_TIME_LEFT_RE = re.compile(r"(?:(\d+)d\s*)?(?:(\d+)h\s*)?(?:(\d+)m\s*)?(?:(\d+)s\s*)?left", re.I)
|
||||||
|
_PARENS_COUNT_RE = re.compile(r"\((\d{1,6})\)")
|
||||||
|
|
||||||
|
# Maps title-keyword fragments → internal MetadataScorer category keys.
|
||||||
|
# Checked in order — first match wins. Broader terms intentionally listed last.
|
||||||
|
_CATEGORY_KEYWORDS: list[tuple[frozenset[str], str]] = [
|
||||||
|
(frozenset(["cell phone", "smartphone", "mobile phone"]), "CELL_PHONES"),
|
||||||
|
(frozenset(["video game", "gaming", "console", "playstation", "xbox", "nintendo"]), "VIDEO_GAMES"),
|
||||||
|
(frozenset(["computer", "tablet", "laptop", "notebook", "chromebook"]), "COMPUTERS_TABLETS"),
|
||||||
|
(frozenset(["electronic"]), "ELECTRONICS"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _classify_category_label(text: str) -> Optional[str]:
|
||||||
|
"""Map an eBay category label to an internal MetadataScorer key, or None."""
|
||||||
|
lower = text.lower()
|
||||||
|
for keywords, key in _CATEGORY_KEYWORDS:
|
||||||
|
if any(kw in lower for kw in keywords):
|
||||||
|
return key
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
@ -215,6 +243,33 @@ def scrape_sellers(html: str) -> dict[str, Seller]:
|
||||||
return sellers
|
return sellers
|
||||||
|
|
||||||
|
|
||||||
|
def scrape_seller_categories(html: str) -> dict[str, int]:
|
||||||
|
"""Parse category distribution from a seller's _ssn search page.
|
||||||
|
|
||||||
|
eBay renders category refinements in the left sidebar. We scan all
|
||||||
|
anchor-text blocks for recognisable category labels and accumulate
|
||||||
|
listing counts from the adjacent parenthetical "(N)" strings.
|
||||||
|
|
||||||
|
Returns a dict like {"ELECTRONICS": 45, "CELL_PHONES": 23}.
|
||||||
|
Empty dict = no recognisable categories found (score stays None).
|
||||||
|
"""
|
||||||
|
soup = BeautifulSoup(html, "lxml")
|
||||||
|
counts: dict[str, int] = {}
|
||||||
|
|
||||||
|
# eBay sidebar refinement links contain the category label and a count.
|
||||||
|
# Multiple layout variants exist — scan broadly and classify by keyword.
|
||||||
|
for el in soup.select("a[href*='_sacat='], li.x-refine__main__list--value a"):
|
||||||
|
text = el.get_text(separator=" ", strip=True)
|
||||||
|
key = _classify_category_label(text)
|
||||||
|
if not key:
|
||||||
|
continue
|
||||||
|
m = _PARENS_COUNT_RE.search(text)
|
||||||
|
count = int(m.group(1)) if m else 1
|
||||||
|
counts[key] = counts.get(key, 0) + count
|
||||||
|
|
||||||
|
return counts
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Adapter
|
# Adapter
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
@ -232,17 +287,12 @@ class ScrapedEbayAdapter(PlatformAdapter):
|
||||||
self._store = store
|
self._store = store
|
||||||
self._delay = delay
|
self._delay = delay
|
||||||
|
|
||||||
def _get(self, params: dict) -> str:
|
def _fetch_url(self, url: str) -> str:
|
||||||
"""Fetch eBay search HTML via a stealthed Playwright Chromium instance.
|
"""Core Playwright fetch — stealthed headed Chromium via Xvfb.
|
||||||
|
|
||||||
Uses Xvfb virtual display (headless=False) to avoid Kasada's headless
|
Shared by both search (_get) and BTF item-page enrichment (_fetch_item_html).
|
||||||
detection — same pattern as other CF scrapers that face JS challenges.
|
Results cached for _HTML_CACHE_TTL seconds.
|
||||||
|
|
||||||
Results are cached for _HTML_CACHE_TTL seconds so repeated searches
|
|
||||||
for the same query return immediately without re-scraping.
|
|
||||||
"""
|
"""
|
||||||
url = EBAY_SEARCH_URL + "?" + "&".join(f"{k}={v}" for k, v in params.items())
|
|
||||||
|
|
||||||
cached = _html_cache.get(url)
|
cached = _html_cache.get(url)
|
||||||
if cached and time.time() < cached[1]:
|
if cached and time.time() < cached[1]:
|
||||||
return cached[0]
|
return cached[0]
|
||||||
|
|
@ -286,8 +336,100 @@ class ScrapedEbayAdapter(PlatformAdapter):
|
||||||
_html_cache[url] = (html, time.time() + _HTML_CACHE_TTL)
|
_html_cache[url] = (html, time.time() + _HTML_CACHE_TTL)
|
||||||
return html
|
return html
|
||||||
|
|
||||||
|
def _get(self, params: dict) -> str:
|
||||||
|
"""Fetch eBay search results HTML. params → query string appended to EBAY_SEARCH_URL."""
|
||||||
|
url = EBAY_SEARCH_URL + "?" + "&".join(f"{k}={v}" for k, v in params.items())
|
||||||
|
return self._fetch_url(url)
|
||||||
|
|
||||||
|
def _fetch_item_html(self, item_id: str) -> str:
|
||||||
|
"""Fetch a single eBay listing page. /itm/ pages pass Kasada; /usr/ pages do not."""
|
||||||
|
return self._fetch_url(f"{EBAY_ITEM_URL}{item_id}")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _parse_joined_date(html: str) -> Optional[int]:
|
||||||
|
"""Parse 'Joined {Mon} {Year}' from a listing page BTF seller card.
|
||||||
|
|
||||||
|
Returns account_age_days (int) or None if the date is not found.
|
||||||
|
eBay renders this as a span.ux-textspans inside the seller section.
|
||||||
|
"""
|
||||||
|
m = _JOINED_RE.search(html)
|
||||||
|
if not m:
|
||||||
|
return None
|
||||||
|
month_str, year_str = m.group(1)[:3].capitalize(), m.group(2)
|
||||||
|
month = _MONTH_MAP.get(month_str)
|
||||||
|
if not month:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
reg_date = datetime(int(year_str), month, 1, tzinfo=timezone.utc)
|
||||||
|
return (datetime.now(timezone.utc) - reg_date).days
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def enrich_sellers_btf(
|
||||||
|
self,
|
||||||
|
seller_to_listing: dict[str, str],
|
||||||
|
max_workers: int = 2,
|
||||||
|
) -> None:
|
||||||
|
"""Background BTF enrichment — scrape /itm/ pages to fill in account_age_days.
|
||||||
|
|
||||||
|
seller_to_listing: {seller_platform_id -> platform_listing_id}
|
||||||
|
Only pass sellers whose account_age_days is None (unknown from API batch).
|
||||||
|
Caller limits the dict to new/stale sellers to avoid redundant scrapes.
|
||||||
|
|
||||||
|
Runs Playwright fetches in a thread pool (max_workers=2 by default to
|
||||||
|
avoid hammering Kasada). Updates seller records in the DB in-place.
|
||||||
|
Does not raise — failures per-seller are silently skipped so the main
|
||||||
|
search response is never blocked.
|
||||||
|
"""
|
||||||
|
def _enrich_one(item: tuple[str, str]) -> None:
|
||||||
|
seller_id, listing_id = item
|
||||||
|
try:
|
||||||
|
html = self._fetch_item_html(listing_id)
|
||||||
|
age_days = self._parse_joined_date(html)
|
||||||
|
if age_days is not None:
|
||||||
|
seller = self._store.get_seller("ebay", seller_id)
|
||||||
|
if seller:
|
||||||
|
from dataclasses import replace
|
||||||
|
updated = replace(seller, account_age_days=age_days)
|
||||||
|
self._store.save_seller(updated)
|
||||||
|
except Exception:
|
||||||
|
pass # non-fatal: partial score is better than a crashed enrichment
|
||||||
|
|
||||||
|
with ThreadPoolExecutor(max_workers=max_workers) as ex:
|
||||||
|
list(ex.map(_enrich_one, seller_to_listing.items()))
|
||||||
|
|
||||||
|
def enrich_sellers_categories(
|
||||||
|
self,
|
||||||
|
seller_platform_ids: list[str],
|
||||||
|
max_workers: int = 2,
|
||||||
|
) -> None:
|
||||||
|
"""Scrape _ssn seller pages to populate category_history_json.
|
||||||
|
|
||||||
|
Uses the same headed Playwright stack as search() — the _ssn=USERNAME
|
||||||
|
filter is just a query param on the standard search template, so it
|
||||||
|
passes Kasada identically. Silently skips on failure so the main
|
||||||
|
search response is never affected.
|
||||||
|
"""
|
||||||
|
def _enrich_one(seller_id: str) -> None:
|
||||||
|
try:
|
||||||
|
html = self._get({"_ssn": seller_id, "_sop": "12", "_ipg": "48"})
|
||||||
|
categories = scrape_seller_categories(html)
|
||||||
|
if categories:
|
||||||
|
seller = self._store.get_seller("ebay", seller_id)
|
||||||
|
if seller:
|
||||||
|
from dataclasses import replace
|
||||||
|
updated = replace(seller, category_history_json=json.dumps(categories))
|
||||||
|
self._store.save_seller(updated)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
with ThreadPoolExecutor(max_workers=max_workers) as ex:
|
||||||
|
list(ex.map(_enrich_one, seller_platform_ids))
|
||||||
|
|
||||||
def search(self, query: str, filters: SearchFilters) -> list[Listing]:
|
def search(self, query: str, filters: SearchFilters) -> list[Listing]:
|
||||||
base_params: dict = {"_nkw": query, "_sop": "15", "_ipg": "48"}
|
base_params: dict = {"_nkw": query, "_sop": "15", "_ipg": "48"}
|
||||||
|
if filters.category_id:
|
||||||
|
base_params["_sacat"] = filters.category_id
|
||||||
|
|
||||||
if filters.max_price:
|
if filters.max_price:
|
||||||
base_params["_udhi"] = str(filters.max_price)
|
base_params["_udhi"] = str(filters.max_price)
|
||||||
|
|
@ -303,10 +445,15 @@ class ScrapedEbayAdapter(PlatformAdapter):
|
||||||
base_params["LH_ItemCondition"] = "|".join(codes)
|
base_params["LH_ItemCondition"] = "|".join(codes)
|
||||||
|
|
||||||
# Append negative keywords to the eBay query — eBay supports "-term" in _nkw natively.
|
# Append negative keywords to the eBay query — eBay supports "-term" in _nkw natively.
|
||||||
# This reduces junk results at the source and improves market comp quality.
|
# Multi-word phrases must be quoted: -"parts only" not -parts only (which splits the words).
|
||||||
if filters.must_exclude:
|
if filters.must_exclude:
|
||||||
excludes = " ".join(f"-{t.strip()}" for t in filters.must_exclude if t.strip())
|
parts = []
|
||||||
base_params["_nkw"] = f"{base_params['_nkw']} {excludes}"
|
for t in filters.must_exclude:
|
||||||
|
t = t.strip()
|
||||||
|
if not t:
|
||||||
|
continue
|
||||||
|
parts.append(f'-"{t}"' if " " in t else f"-{t}")
|
||||||
|
base_params["_nkw"] = f"{base_params['_nkw']} {' '.join(parts)}"
|
||||||
|
|
||||||
pages = max(1, filters.pages)
|
pages = max(1, filters.pages)
|
||||||
page_params = [{**base_params, "_pgn": str(p)} for p in range(1, pages + 1)]
|
page_params = [{**base_params, "_pgn": str(p)} for p in range(1, pages + 1)]
|
||||||
|
|
@ -346,6 +493,7 @@ class ScrapedEbayAdapter(PlatformAdapter):
|
||||||
pages = max(1, pages)
|
pages = max(1, pages)
|
||||||
page_params = [{**base_params, "_pgn": str(p)} for p in range(1, pages + 1)]
|
page_params = [{**base_params, "_pgn": str(p)} for p in range(1, pages + 1)]
|
||||||
|
|
||||||
|
log.info("comps scrape: fetching %d page(s) of sold listings for %r", pages, query)
|
||||||
try:
|
try:
|
||||||
with ThreadPoolExecutor(max_workers=min(pages, 3)) as ex:
|
with ThreadPoolExecutor(max_workers=min(pages, 3)) as ex:
|
||||||
htmls = list(ex.map(self._get, page_params))
|
htmls = list(ex.map(self._get, page_params))
|
||||||
|
|
@ -369,6 +517,10 @@ class ScrapedEbayAdapter(PlatformAdapter):
|
||||||
sample_count=len(prices),
|
sample_count=len(prices),
|
||||||
expires_at=(datetime.now(timezone.utc) + timedelta(hours=6)).isoformat(),
|
expires_at=(datetime.now(timezone.utc) + timedelta(hours=6)).isoformat(),
|
||||||
))
|
))
|
||||||
|
log.info("comps scrape: saved market comp median=$%.2f from %d prices", median, len(prices))
|
||||||
|
else:
|
||||||
|
log.warning("comps scrape: %d listings parsed but 0 valid prices — no comp saved", len(all_listings))
|
||||||
return all_listings
|
return all_listings
|
||||||
except Exception:
|
except Exception:
|
||||||
|
log.warning("comps scrape: failed for %r", query, exc_info=True)
|
||||||
return []
|
return []
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,7 @@ from .aggregator import Aggregator
|
||||||
from app.db.models import Seller, Listing, TrustScore
|
from app.db.models import Seller, Listing, TrustScore
|
||||||
from app.db.store import Store
|
from app.db.store import Store
|
||||||
import hashlib
|
import hashlib
|
||||||
|
import math
|
||||||
|
|
||||||
|
|
||||||
class TrustScorer:
|
class TrustScorer:
|
||||||
|
|
@ -24,6 +25,16 @@ class TrustScorer:
|
||||||
comp = self._store.get_market_comp("ebay", query_hash)
|
comp = self._store.get_market_comp("ebay", query_hash)
|
||||||
market_median = comp.median_price if comp else None
|
market_median = comp.median_price if comp else None
|
||||||
|
|
||||||
|
# Coefficient of variation: stddev/mean across batch prices.
|
||||||
|
# None when fewer than 2 priced listings (can't compute variance).
|
||||||
|
_prices = [l.price for l in listings if l.price > 0]
|
||||||
|
if len(_prices) >= 2:
|
||||||
|
_mean = sum(_prices) / len(_prices)
|
||||||
|
_stddev = math.sqrt(sum((p - _mean) ** 2 for p in _prices) / len(_prices))
|
||||||
|
price_cv: float | None = _stddev / _mean if _mean > 0 else None
|
||||||
|
else:
|
||||||
|
price_cv = None
|
||||||
|
|
||||||
photo_url_sets = [l.photo_urls for l in listings]
|
photo_url_sets = [l.photo_urls for l in listings]
|
||||||
duplicates = self._photo.check_duplicates(photo_url_sets)
|
duplicates = self._photo.check_duplicates(photo_url_sets)
|
||||||
|
|
||||||
|
|
@ -31,11 +42,19 @@ class TrustScorer:
|
||||||
for listing, is_dup in zip(listings, duplicates):
|
for listing, is_dup in zip(listings, duplicates):
|
||||||
seller = self._store.get_seller("ebay", listing.seller_platform_id)
|
seller = self._store.get_seller("ebay", listing.seller_platform_id)
|
||||||
if seller:
|
if seller:
|
||||||
signal_scores = self._meta.score(seller, market_median, listing.price)
|
signal_scores = self._meta.score(seller, market_median, listing.price, price_cv)
|
||||||
else:
|
else:
|
||||||
signal_scores = {k: None for k in
|
signal_scores = {k: None for k in
|
||||||
["account_age", "feedback_count", "feedback_ratio",
|
["account_age", "feedback_count", "feedback_ratio",
|
||||||
"price_vs_market", "category_history"]}
|
"price_vs_market", "category_history"]}
|
||||||
trust = self._agg.aggregate(signal_scores, is_dup, seller, listing.id or 0)
|
trust = self._agg.aggregate(
|
||||||
|
signal_scores, is_dup, seller,
|
||||||
|
listing_id=listing.id or 0,
|
||||||
|
listing_title=listing.title,
|
||||||
|
times_seen=listing.times_seen,
|
||||||
|
first_seen_at=listing.first_seen_at,
|
||||||
|
price=listing.price,
|
||||||
|
price_at_first_seen=listing.price_at_first_seen,
|
||||||
|
)
|
||||||
scores.append(trust)
|
scores.append(trust)
|
||||||
return scores
|
return scores
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,7 @@
|
||||||
"""Composite score and red flag extraction."""
|
"""Composite score and red flag extraction."""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
import json
|
import json
|
||||||
|
from datetime import datetime, timezone
|
||||||
from typing import Optional
|
from typing import Optional
|
||||||
from app.db.models import Seller, TrustScore
|
from app.db.models import Seller, TrustScore
|
||||||
|
|
||||||
|
|
@ -8,6 +9,55 @@ HARD_FILTER_AGE_DAYS = 7
|
||||||
HARD_FILTER_BAD_RATIO_MIN_COUNT = 20
|
HARD_FILTER_BAD_RATIO_MIN_COUNT = 20
|
||||||
HARD_FILTER_BAD_RATIO_THRESHOLD = 0.80
|
HARD_FILTER_BAD_RATIO_THRESHOLD = 0.80
|
||||||
|
|
||||||
|
# Title keywords that suggest cosmetic damage or wear (free-tier title scan).
|
||||||
|
# Description-body scan (paid BSL feature) runs via BTF enrichment — not implemented yet.
|
||||||
|
_SCRATCH_DENT_KEYWORDS = frozenset([
|
||||||
|
# Explicit cosmetic damage
|
||||||
|
"scratch", "scratched", "scratches", "scuff", "scuffed",
|
||||||
|
"dent", "dented", "ding", "dinged",
|
||||||
|
"crack", "cracked", "chip", "chipped",
|
||||||
|
"damage", "damaged", "cosmetic damage",
|
||||||
|
"blemish", "wear", "worn", "worn in",
|
||||||
|
# Parts / condition catch-alls
|
||||||
|
"as is", "for parts", "parts only", "spares or repair", "parts or repair",
|
||||||
|
# Evasive redirects — seller hiding damage detail in listing body
|
||||||
|
"see description", "read description", "read listing", "see listing",
|
||||||
|
"see photos for", "see pics for", "see images for",
|
||||||
|
# Functional problem phrases (phrases > single words to avoid false positives)
|
||||||
|
"issue with", "issues with", "problem with", "problems with",
|
||||||
|
"not working", "stopped working", "doesn't work", "does not work",
|
||||||
|
"no power", "dead on arrival", "powers on but", "turns on but", "boots but",
|
||||||
|
"faulty", "broken screen", "broken hinge", "broken port",
|
||||||
|
# DIY / project / repair listings
|
||||||
|
"needs repair", "needs work", "needs tlc",
|
||||||
|
"project unit", "project item", "project laptop", "project phone",
|
||||||
|
"for repair", "sold as is",
|
||||||
|
])
|
||||||
|
|
||||||
|
|
||||||
|
def _has_damage_keywords(title: str) -> bool:
|
||||||
|
lower = title.lower()
|
||||||
|
return any(kw in lower for kw in _SCRATCH_DENT_KEYWORDS)
|
||||||
|
|
||||||
|
|
||||||
|
_LONG_ON_MARKET_MIN_SIGHTINGS = 5
|
||||||
|
_LONG_ON_MARKET_MIN_DAYS = 14
|
||||||
|
_PRICE_DROP_THRESHOLD = 0.20 # 20% below first-seen price
|
||||||
|
|
||||||
|
|
||||||
|
def _days_since(iso: Optional[str]) -> Optional[int]:
|
||||||
|
if not iso:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
dt = datetime.fromisoformat(iso.replace("Z", "+00:00"))
|
||||||
|
# Normalize to naive UTC so both paths (timezone-aware ISO and SQLite
|
||||||
|
# CURRENT_TIMESTAMP naive strings) compare correctly.
|
||||||
|
if dt.tzinfo is not None:
|
||||||
|
dt = dt.replace(tzinfo=None)
|
||||||
|
return (datetime.utcnow() - dt).days
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
class Aggregator:
|
class Aggregator:
|
||||||
def aggregate(
|
def aggregate(
|
||||||
|
|
@ -16,10 +66,24 @@ class Aggregator:
|
||||||
photo_hash_duplicate: bool,
|
photo_hash_duplicate: bool,
|
||||||
seller: Optional[Seller],
|
seller: Optional[Seller],
|
||||||
listing_id: int = 0,
|
listing_id: int = 0,
|
||||||
|
listing_title: str = "",
|
||||||
|
times_seen: int = 1,
|
||||||
|
first_seen_at: Optional[str] = None,
|
||||||
|
price: float = 0.0,
|
||||||
|
price_at_first_seen: Optional[float] = None,
|
||||||
) -> TrustScore:
|
) -> TrustScore:
|
||||||
is_partial = any(v is None for v in signal_scores.values())
|
is_partial = any(v is None for v in signal_scores.values())
|
||||||
clean = {k: (v if v is not None else 0) for k, v in signal_scores.items()}
|
clean = {k: (v if v is not None else 0) for k, v in signal_scores.items()}
|
||||||
composite = sum(clean.values())
|
|
||||||
|
# Score only against signals that returned real data — treating "no data"
|
||||||
|
# as 0 conflates "bad signal" with "missing signal" and drags scores down
|
||||||
|
# unfairly when the API doesn't expose a field (e.g. registrationDate).
|
||||||
|
available = [v for v in signal_scores.values() if v is not None]
|
||||||
|
available_max = len(available) * 20
|
||||||
|
if available_max > 0:
|
||||||
|
composite = round((sum(available) / available_max) * 100)
|
||||||
|
else:
|
||||||
|
composite = 0
|
||||||
|
|
||||||
red_flags: list[str] = []
|
red_flags: list[str] = []
|
||||||
|
|
||||||
|
|
@ -41,6 +105,18 @@ class Aggregator:
|
||||||
red_flags.append("suspicious_price")
|
red_flags.append("suspicious_price")
|
||||||
if photo_hash_duplicate:
|
if photo_hash_duplicate:
|
||||||
red_flags.append("duplicate_photo")
|
red_flags.append("duplicate_photo")
|
||||||
|
if listing_title and _has_damage_keywords(listing_title):
|
||||||
|
red_flags.append("scratch_dent_mentioned")
|
||||||
|
|
||||||
|
# Staging DB signals
|
||||||
|
days_in_index = _days_since(first_seen_at)
|
||||||
|
if (times_seen >= _LONG_ON_MARKET_MIN_SIGHTINGS
|
||||||
|
and days_in_index is not None
|
||||||
|
and days_in_index >= _LONG_ON_MARKET_MIN_DAYS):
|
||||||
|
red_flags.append("long_on_market")
|
||||||
|
if (price_at_first_seen and price_at_first_seen > 0
|
||||||
|
and price < price_at_first_seen * (1 - _PRICE_DROP_THRESHOLD)):
|
||||||
|
red_flags.append("significant_price_drop")
|
||||||
|
|
||||||
return TrustScore(
|
return TrustScore(
|
||||||
listing_id=listing_id,
|
listing_id=listing_id,
|
||||||
|
|
|
||||||
|
|
@ -6,6 +6,11 @@ from app.db.models import Seller
|
||||||
|
|
||||||
ELECTRONICS_CATEGORIES = {"ELECTRONICS", "COMPUTERS_TABLETS", "VIDEO_GAMES", "CELL_PHONES"}
|
ELECTRONICS_CATEGORIES = {"ELECTRONICS", "COMPUTERS_TABLETS", "VIDEO_GAMES", "CELL_PHONES"}
|
||||||
|
|
||||||
|
# Coefficient of variation (stddev/mean) above which the price distribution is
|
||||||
|
# considered too heterogeneous to trust the market median for scam detection.
|
||||||
|
# e.g. "Lenovo RTX intel" mixes $200 old ThinkPads with $2000 Legions → CV ~1.0+
|
||||||
|
_HETEROGENEOUS_CV_THRESHOLD = 0.6
|
||||||
|
|
||||||
|
|
||||||
class MetadataScorer:
|
class MetadataScorer:
|
||||||
def score(
|
def score(
|
||||||
|
|
@ -13,12 +18,13 @@ class MetadataScorer:
|
||||||
seller: Seller,
|
seller: Seller,
|
||||||
market_median: Optional[float],
|
market_median: Optional[float],
|
||||||
listing_price: float,
|
listing_price: float,
|
||||||
|
price_cv: Optional[float] = None,
|
||||||
) -> dict[str, Optional[int]]:
|
) -> dict[str, Optional[int]]:
|
||||||
return {
|
return {
|
||||||
"account_age": self._account_age(seller.account_age_days) if seller.account_age_days is not None else None,
|
"account_age": self._account_age(seller.account_age_days) if seller.account_age_days is not None else None,
|
||||||
"feedback_count": self._feedback_count(seller.feedback_count),
|
"feedback_count": self._feedback_count(seller.feedback_count),
|
||||||
"feedback_ratio": self._feedback_ratio(seller.feedback_ratio, seller.feedback_count),
|
"feedback_ratio": self._feedback_ratio(seller.feedback_ratio, seller.feedback_count),
|
||||||
"price_vs_market": self._price_vs_market(listing_price, market_median),
|
"price_vs_market": self._price_vs_market(listing_price, market_median, price_cv),
|
||||||
"category_history": self._category_history(seller.category_history_json),
|
"category_history": self._category_history(seller.category_history_json),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -43,9 +49,11 @@ class MetadataScorer:
|
||||||
if ratio < 0.98: return 15
|
if ratio < 0.98: return 15
|
||||||
return 20
|
return 20
|
||||||
|
|
||||||
def _price_vs_market(self, price: float, median: Optional[float]) -> Optional[int]:
|
def _price_vs_market(self, price: float, median: Optional[float], price_cv: Optional[float] = None) -> Optional[int]:
|
||||||
if median is None: return None # data unavailable → aggregator sets score_is_partial
|
if median is None: return None # data unavailable → aggregator sets score_is_partial
|
||||||
if median <= 0: return None
|
if median <= 0: return None
|
||||||
|
if price_cv is not None and price_cv > _HETEROGENEOUS_CV_THRESHOLD:
|
||||||
|
return None # mixed model/generation search — median is unreliable
|
||||||
ratio = price / median
|
ratio = price / median
|
||||||
if ratio < 0.50: return 0 # >50% below = scam
|
if ratio < 0.50: return 0 # >50% below = scam
|
||||||
if ratio < 0.70: return 5 # >30% below = suspicious
|
if ratio < 0.70: return 5 # >30% below = suspicious
|
||||||
|
|
@ -53,11 +61,13 @@ class MetadataScorer:
|
||||||
if ratio <= 1.20: return 20
|
if ratio <= 1.20: return 20
|
||||||
return 15 # above market = still ok, just expensive
|
return 15 # above market = still ok, just expensive
|
||||||
|
|
||||||
def _category_history(self, category_history_json: str) -> int:
|
def _category_history(self, category_history_json: str) -> Optional[int]:
|
||||||
try:
|
try:
|
||||||
history = json.loads(category_history_json)
|
history = json.loads(category_history_json)
|
||||||
except (ValueError, TypeError):
|
except (ValueError, TypeError):
|
||||||
return 0
|
return None # unparseable → data unavailable
|
||||||
|
if not history:
|
||||||
|
return None # empty dict → no category data from this source
|
||||||
electronics_sales = sum(
|
electronics_sales = sum(
|
||||||
v for k, v in history.items() if k in ELECTRONICS_CATEGORIES
|
v for k, v in history.items() if k in ELECTRONICS_CATEGORIES
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -48,8 +48,41 @@ def _get_adapter(store: Store) -> PlatformAdapter:
|
||||||
return ScrapedEbayAdapter(store)
|
return ScrapedEbayAdapter(store)
|
||||||
|
|
||||||
|
|
||||||
|
def _keyword_passes(title_lower: str, state: FilterState) -> bool:
|
||||||
|
"""Apply must_include / must_exclude keyword filtering against a lowercased title."""
|
||||||
|
include_raw = state.must_include.strip()
|
||||||
|
if include_raw:
|
||||||
|
mode = state.must_include_mode
|
||||||
|
if mode == "groups":
|
||||||
|
groups = [
|
||||||
|
[alt.strip().lower() for alt in g.split("|") if alt.strip()]
|
||||||
|
for g in include_raw.split(",")
|
||||||
|
if any(alt.strip() for alt in g.split("|"))
|
||||||
|
]
|
||||||
|
if not all(any(alt in title_lower for alt in group) for group in groups):
|
||||||
|
return False
|
||||||
|
elif mode == "any":
|
||||||
|
terms = [t.strip().lower() for t in include_raw.split(",") if t.strip()]
|
||||||
|
if not any(t in title_lower for t in terms):
|
||||||
|
return False
|
||||||
|
else: # "all"
|
||||||
|
terms = [t.strip().lower() for t in include_raw.split(",") if t.strip()]
|
||||||
|
if not all(t in title_lower for t in terms):
|
||||||
|
return False
|
||||||
|
|
||||||
|
exclude_raw = state.must_exclude.strip()
|
||||||
|
if exclude_raw:
|
||||||
|
terms = [t.strip().lower() for t in exclude_raw.split(",") if t.strip()]
|
||||||
|
if any(t in title_lower for t in terms):
|
||||||
|
return False
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
def _passes_filter(listing, trust, seller, state: FilterState) -> bool:
|
def _passes_filter(listing, trust, seller, state: FilterState) -> bool:
|
||||||
import json
|
import json
|
||||||
|
if not _keyword_passes(listing.title.lower(), state):
|
||||||
|
return False
|
||||||
if trust and trust.composite_score < state.min_trust_score:
|
if trust and trust.composite_score < state.min_trust_score:
|
||||||
return False
|
return False
|
||||||
if state.min_price and listing.price < state.min_price:
|
if state.min_price and listing.price < state.min_price:
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,9 @@ class FilterState:
|
||||||
hide_marketing_photos: bool = False
|
hide_marketing_photos: bool = False
|
||||||
hide_suspicious_price: bool = False
|
hide_suspicious_price: bool = False
|
||||||
hide_duplicate_photos: bool = False
|
hide_duplicate_photos: bool = False
|
||||||
|
must_include: str = ""
|
||||||
|
must_include_mode: str = "all" # "all" | "any" | "groups"
|
||||||
|
must_exclude: str = ""
|
||||||
|
|
||||||
|
|
||||||
def build_filter_options(
|
def build_filter_options(
|
||||||
|
|
@ -78,6 +81,29 @@ def render_filter_sidebar(
|
||||||
st.sidebar.markdown("### Filters")
|
st.sidebar.markdown("### Filters")
|
||||||
st.sidebar.caption(f"{len(pairs)} results")
|
st.sidebar.caption(f"{len(pairs)} results")
|
||||||
|
|
||||||
|
st.sidebar.markdown("**Keywords**")
|
||||||
|
state.must_include_mode = st.sidebar.radio(
|
||||||
|
"Must include mode",
|
||||||
|
options=["all", "any", "groups"],
|
||||||
|
format_func=lambda m: {"all": "All (AND)", "any": "Any (OR)", "groups": "Groups (CNF)"}[m],
|
||||||
|
horizontal=True,
|
||||||
|
key="include_mode",
|
||||||
|
label_visibility="collapsed",
|
||||||
|
)
|
||||||
|
hint = {
|
||||||
|
"all": "Every term must appear",
|
||||||
|
"any": "At least one term must appear",
|
||||||
|
"groups": "Comma = AND · pipe | = OR within group",
|
||||||
|
}[state.must_include_mode]
|
||||||
|
state.must_include = st.sidebar.text_input(
|
||||||
|
"Must include", value="", placeholder="16gb, founders…" if state.must_include_mode != "groups" else "founders|fe, 16gb…",
|
||||||
|
key="must_include",
|
||||||
|
)
|
||||||
|
st.sidebar.caption(hint)
|
||||||
|
state.must_exclude = st.sidebar.text_input(
|
||||||
|
"Must exclude", value="", placeholder="broken, parts…", key="must_exclude",
|
||||||
|
)
|
||||||
|
|
||||||
state.min_trust_score = st.sidebar.slider("Min trust score", 0, 100, 0, key="min_trust")
|
state.min_trust_score = st.sidebar.slider("Min trust score", 0, 100, 0, key="min_trust")
|
||||||
st.sidebar.caption(
|
st.sidebar.caption(
|
||||||
f"🟢 Safe (80+): {opts.score_bands['safe']} "
|
f"🟢 Safe (80+): {opts.score_bands['safe']} "
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,7 @@ services:
|
||||||
context: ..
|
context: ..
|
||||||
dockerfile: snipe/Dockerfile
|
dockerfile: snipe/Dockerfile
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
env_file: .env
|
||||||
# No network_mode: host — isolated on snipe-cloud-net; nginx reaches it via 'api:8510'
|
# No network_mode: host — isolated on snipe-cloud-net; nginx reaches it via 'api:8510'
|
||||||
volumes:
|
volumes:
|
||||||
- /devl/snipe-cloud-data:/app/snipe/data
|
- /devl/snipe-cloud-data:/app/snipe/data
|
||||||
|
|
|
||||||
|
|
@ -20,6 +20,7 @@ dependencies = [
|
||||||
"uvicorn[standard]>=0.29",
|
"uvicorn[standard]>=0.29",
|
||||||
"playwright>=1.44",
|
"playwright>=1.44",
|
||||||
"playwright-stealth>=1.0",
|
"playwright-stealth>=1.0",
|
||||||
|
"cryptography>=42.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
[tool.setuptools.packages.find]
|
[tool.setuptools.packages.find]
|
||||||
|
|
|
||||||
|
|
@ -61,8 +61,8 @@
|
||||||
{{ flagLabel(flag) }}
|
{{ flagLabel(flag) }}
|
||||||
</span>
|
</span>
|
||||||
</div>
|
</div>
|
||||||
<p v-if="trust?.score_is_partial" class="card__partial-warning">
|
<p v-if="pendingSignalNames.length" class="card__score-pending">
|
||||||
⚠ Partial score — some data unavailable
|
↻ Updating: {{ pendingSignalNames.join(', ') }}
|
||||||
</p>
|
</p>
|
||||||
<p v-if="!trust" class="card__partial-warning">
|
<p v-if="!trust" class="card__partial-warning">
|
||||||
⚠ Could not score this listing
|
⚠ Could not score this listing
|
||||||
|
|
@ -72,9 +72,32 @@
|
||||||
<!-- Score + price column -->
|
<!-- Score + price column -->
|
||||||
<div class="card__score-col">
|
<div class="card__score-col">
|
||||||
<!-- Trust score badge -->
|
<!-- Trust score badge -->
|
||||||
<div class="card__trust" :class="trustClass" :title="`Trust score: ${trust?.composite_score ?? '?'}/100`">
|
<div
|
||||||
|
class="card__trust"
|
||||||
|
:class="[trustClass, { 'card__trust--partial': trust?.score_is_partial }]"
|
||||||
|
:title="trustBadgeTitle"
|
||||||
|
>
|
||||||
<span class="card__trust-num">{{ trust?.composite_score ?? '?' }}</span>
|
<span class="card__trust-num">{{ trust?.composite_score ?? '?' }}</span>
|
||||||
<span class="card__trust-label">Trust</span>
|
<span class="card__trust-label">Trust</span>
|
||||||
|
<!-- Signal dots: one per scoring signal, grey = pending -->
|
||||||
|
<span v-if="trust" class="card__signal-dots" aria-hidden="true">
|
||||||
|
<span
|
||||||
|
v-for="dot in signalDots"
|
||||||
|
:key="dot.key"
|
||||||
|
class="card__signal-dot"
|
||||||
|
:class="dot.pending ? 'card__signal-dot--pending' : 'card__signal-dot--present'"
|
||||||
|
:title="dot.label"
|
||||||
|
/>
|
||||||
|
</span>
|
||||||
|
<!-- Jump the queue: force enrichment for this seller -->
|
||||||
|
<button
|
||||||
|
v-if="pendingSignalNames.length"
|
||||||
|
class="card__enrich-btn"
|
||||||
|
:class="{ 'card__enrich-btn--spinning': enriching, 'card__enrich-btn--error': enrichError }"
|
||||||
|
:title="enrichError ? 'Enrichment failed — try again' : 'Refresh score now'"
|
||||||
|
:disabled="enriching"
|
||||||
|
@click.stop="onEnrich"
|
||||||
|
>{{ enrichError ? '✗' : '↻' }}</button>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- Price -->
|
<!-- Price -->
|
||||||
|
|
@ -99,6 +122,7 @@
|
||||||
<script setup lang="ts">
|
<script setup lang="ts">
|
||||||
import { computed, ref } from 'vue'
|
import { computed, ref } from 'vue'
|
||||||
import type { Listing, TrustScore, Seller } from '../stores/search'
|
import type { Listing, TrustScore, Seller } from '../stores/search'
|
||||||
|
import { useSearchStore } from '../stores/search'
|
||||||
|
|
||||||
const props = defineProps<{
|
const props = defineProps<{
|
||||||
listing: Listing
|
listing: Listing
|
||||||
|
|
@ -107,6 +131,23 @@ const props = defineProps<{
|
||||||
marketPrice: number | null
|
marketPrice: number | null
|
||||||
}>()
|
}>()
|
||||||
|
|
||||||
|
const store = useSearchStore()
|
||||||
|
const enriching = ref(false)
|
||||||
|
const enrichError = ref(false)
|
||||||
|
|
||||||
|
async function onEnrich() {
|
||||||
|
if (enriching.value) return
|
||||||
|
enriching.value = true
|
||||||
|
enrichError.value = false
|
||||||
|
try {
|
||||||
|
await store.enrichSeller(props.listing.seller_platform_id, props.listing.platform_listing_id)
|
||||||
|
} catch {
|
||||||
|
enrichError.value = true
|
||||||
|
} finally {
|
||||||
|
enriching.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
const imgFailed = ref(false)
|
const imgFailed = ref(false)
|
||||||
|
|
||||||
// ── Computed helpers ─────────────────────────────────────────────────────────
|
// ── Computed helpers ─────────────────────────────────────────────────────────
|
||||||
|
|
@ -149,6 +190,7 @@ const conditionLabel = computed(() => {
|
||||||
const accountAgeLabel = computed(() => {
|
const accountAgeLabel = computed(() => {
|
||||||
if (!props.seller) return ''
|
if (!props.seller) return ''
|
||||||
const days = props.seller.account_age_days
|
const days = props.seller.account_age_days
|
||||||
|
if (days == null) return 'member'
|
||||||
if (days >= 365) return `${Math.floor(days / 365)}yr member`
|
if (days >= 365) return `${Math.floor(days / 365)}yr member`
|
||||||
return `${days}d member`
|
return `${days}d member`
|
||||||
})
|
})
|
||||||
|
|
@ -182,6 +224,32 @@ const trustClass = computed(() => {
|
||||||
return 'card__trust--low'
|
return 'card__trust--low'
|
||||||
})
|
})
|
||||||
|
|
||||||
|
interface SignalDot { key: string; label: string; pending: boolean }
|
||||||
|
|
||||||
|
const signalDots = computed<SignalDot[]>(() => {
|
||||||
|
const agePending = props.seller?.account_age_days == null
|
||||||
|
const catPending = !props.seller || props.seller.category_history_json === '{}'
|
||||||
|
const mktPending = props.marketPrice == null
|
||||||
|
return [
|
||||||
|
{ key: 'feedback_count', label: 'Feedback count', pending: false },
|
||||||
|
{ key: 'feedback_ratio', label: 'Feedback ratio', pending: false },
|
||||||
|
{ key: 'account_age', label: agePending ? 'Account age: pending' : 'Account age', pending: agePending },
|
||||||
|
{ key: 'price_vs_market', label: mktPending ? 'Market price: pending' : 'vs market price', pending: mktPending },
|
||||||
|
{ key: 'category_history', label: catPending ? 'Category history: pending' : 'Category history', pending: catPending },
|
||||||
|
]
|
||||||
|
})
|
||||||
|
|
||||||
|
const pendingSignalNames = computed<string[]>(() => {
|
||||||
|
if (!props.trust?.score_is_partial) return []
|
||||||
|
return signalDots.value.filter(d => d.pending).map(d => d.label.replace(': pending', ''))
|
||||||
|
})
|
||||||
|
|
||||||
|
const trustBadgeTitle = computed(() => {
|
||||||
|
const base = `Trust score: ${props.trust?.composite_score ?? '?'}/100`
|
||||||
|
if (!pendingSignalNames.value.length) return base
|
||||||
|
return `${base} · pending: ${pendingSignalNames.value.join(', ')} (search again to update)`
|
||||||
|
})
|
||||||
|
|
||||||
const isSteal = computed(() => {
|
const isSteal = computed(() => {
|
||||||
const s = props.trust?.composite_score
|
const s = props.trust?.composite_score
|
||||||
if (!s || s < 80) return false
|
if (!s || s < 80) return false
|
||||||
|
|
@ -311,6 +379,13 @@ const formattedMarket = computed(() => {
|
||||||
margin: 0;
|
margin: 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.card__score-pending {
|
||||||
|
font-size: 0.7rem;
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
margin: 0;
|
||||||
|
font-style: italic;
|
||||||
|
}
|
||||||
|
|
||||||
/* Score + price column */
|
/* Score + price column */
|
||||||
.card__score-col {
|
.card__score-col {
|
||||||
display: flex;
|
display: flex;
|
||||||
|
|
@ -350,6 +425,51 @@ const formattedMarket = computed(() => {
|
||||||
.card__trust--low { color: var(--trust-low); }
|
.card__trust--low { color: var(--trust-low); }
|
||||||
.card__trust--unknown { color: var(--color-text-muted); }
|
.card__trust--unknown { color: var(--color-text-muted); }
|
||||||
|
|
||||||
|
.card__trust--partial {
|
||||||
|
animation: trust-pulse 2.5s ease-in-out infinite;
|
||||||
|
}
|
||||||
|
@keyframes trust-pulse {
|
||||||
|
0%, 100% { opacity: 1; }
|
||||||
|
50% { opacity: 0.55; }
|
||||||
|
}
|
||||||
|
|
||||||
|
.card__signal-dots {
|
||||||
|
display: flex;
|
||||||
|
gap: 3px;
|
||||||
|
margin-top: 4px;
|
||||||
|
justify-content: center;
|
||||||
|
}
|
||||||
|
.card__signal-dot {
|
||||||
|
width: 5px;
|
||||||
|
height: 5px;
|
||||||
|
border-radius: 50%;
|
||||||
|
flex-shrink: 0;
|
||||||
|
}
|
||||||
|
.card__signal-dot--present { background: currentColor; opacity: 0.7; }
|
||||||
|
.card__signal-dot--pending { background: var(--color-border); opacity: 1; }
|
||||||
|
|
||||||
|
.card__enrich-btn {
|
||||||
|
margin-top: 4px;
|
||||||
|
background: none;
|
||||||
|
border: 1px solid currentColor;
|
||||||
|
border-radius: var(--radius-sm);
|
||||||
|
color: currentColor;
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 0.65rem;
|
||||||
|
line-height: 1;
|
||||||
|
opacity: 0.6;
|
||||||
|
padding: 1px 4px;
|
||||||
|
transition: opacity 150ms ease;
|
||||||
|
}
|
||||||
|
.card__enrich-btn:hover:not(:disabled) { opacity: 1; }
|
||||||
|
.card__enrich-btn:disabled { cursor: default; }
|
||||||
|
.card__enrich-btn--spinning { animation: enrich-spin 0.8s linear infinite; }
|
||||||
|
.card__enrich-btn--error { color: var(--color-error); opacity: 1; }
|
||||||
|
@keyframes enrich-spin {
|
||||||
|
from { transform: rotate(0deg); }
|
||||||
|
to { transform: rotate(360deg); }
|
||||||
|
}
|
||||||
|
|
||||||
.card__price-wrap {
|
.card__price-wrap {
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
|
|
|
||||||
56
web/src/stores/savedSearches.ts
Normal file
56
web/src/stores/savedSearches.ts
Normal file
|
|
@ -0,0 +1,56 @@
|
||||||
|
import { defineStore } from 'pinia'
|
||||||
|
import { ref } from 'vue'
|
||||||
|
import type { SavedSearch, SearchFilters } from './search'
|
||||||
|
|
||||||
|
export type { SavedSearch }
|
||||||
|
|
||||||
|
const apiBase = (import.meta.env.VITE_API_BASE as string) ?? ''
|
||||||
|
|
||||||
|
export const useSavedSearchesStore = defineStore('savedSearches', () => {
|
||||||
|
const items = ref<SavedSearch[]>([])
|
||||||
|
const loading = ref(false)
|
||||||
|
const error = ref<string | null>(null)
|
||||||
|
|
||||||
|
async function fetchAll() {
|
||||||
|
loading.value = true
|
||||||
|
error.value = null
|
||||||
|
try {
|
||||||
|
const res = await fetch(`${apiBase}/api/saved-searches`)
|
||||||
|
if (!res.ok) throw new Error(`${res.status} ${res.statusText}`)
|
||||||
|
const data = await res.json() as { saved_searches: SavedSearch[] }
|
||||||
|
items.value = data.saved_searches
|
||||||
|
} catch (e) {
|
||||||
|
error.value = e instanceof Error ? e.message : 'Failed to load saved searches'
|
||||||
|
} finally {
|
||||||
|
loading.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function create(name: string, query: string, filters: SearchFilters): Promise<SavedSearch> {
|
||||||
|
// Strip per-run fields before persisting
|
||||||
|
const { pages: _pages, ...persistable } = filters
|
||||||
|
const res = await fetch(`${apiBase}/api/saved-searches`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ name, query, filters_json: JSON.stringify(persistable) }),
|
||||||
|
})
|
||||||
|
if (!res.ok) throw new Error(`Save failed: ${res.status} ${res.statusText}`)
|
||||||
|
const created = await res.json() as SavedSearch
|
||||||
|
items.value = [created, ...items.value]
|
||||||
|
return created
|
||||||
|
}
|
||||||
|
|
||||||
|
async function remove(id: number) {
|
||||||
|
await fetch(`${apiBase}/api/saved-searches/${id}`, { method: 'DELETE' })
|
||||||
|
items.value = items.value.filter(s => s.id !== id)
|
||||||
|
}
|
||||||
|
|
||||||
|
async function markRun(id: number) {
|
||||||
|
// Fire-and-forget — don't block navigation on this
|
||||||
|
fetch(`${apiBase}/api/saved-searches/${id}/run`, { method: 'PATCH' }).catch(() => {})
|
||||||
|
const item = items.value.find(s => s.id === id)
|
||||||
|
if (item) item.last_run_at = new Date().toISOString()
|
||||||
|
}
|
||||||
|
|
||||||
|
return { items, loading, error, fetchAll, create, remove, markRun }
|
||||||
|
})
|
||||||
|
|
@ -42,13 +42,25 @@ export interface Seller {
|
||||||
platform: string
|
platform: string
|
||||||
platform_seller_id: string
|
platform_seller_id: string
|
||||||
username: string
|
username: string
|
||||||
account_age_days: number
|
account_age_days: number | null
|
||||||
feedback_count: number
|
feedback_count: number
|
||||||
feedback_ratio: number // 0.0–1.0
|
feedback_ratio: number // 0.0–1.0
|
||||||
category_history_json: string
|
category_history_json: string
|
||||||
fetched_at: string | null
|
fetched_at: string | null
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export type MustIncludeMode = 'all' | 'any' | 'groups'
|
||||||
|
|
||||||
|
export interface SavedSearch {
|
||||||
|
id: number
|
||||||
|
name: string
|
||||||
|
query: string
|
||||||
|
platform: string
|
||||||
|
filters_json: string // JSON blob of SearchFilters subset
|
||||||
|
created_at: string | null
|
||||||
|
last_run_at: string | null
|
||||||
|
}
|
||||||
|
|
||||||
export interface SearchFilters {
|
export interface SearchFilters {
|
||||||
minTrustScore?: number
|
minTrustScore?: number
|
||||||
minPrice?: number
|
minPrice?: number
|
||||||
|
|
@ -60,9 +72,15 @@ export interface SearchFilters {
|
||||||
hideNewAccounts?: boolean
|
hideNewAccounts?: boolean
|
||||||
hideSuspiciousPrice?: boolean
|
hideSuspiciousPrice?: boolean
|
||||||
hideDuplicatePhotos?: boolean
|
hideDuplicatePhotos?: boolean
|
||||||
pages?: number // number of eBay result pages to fetch (48 listings/page, default 1)
|
hideScratchDent?: boolean
|
||||||
mustInclude?: string // comma-separated; client-side title filter
|
hideLongOnMarket?: boolean
|
||||||
mustExclude?: string // comma-separated; forwarded to eBay -term AND client-side
|
hidePriceDrop?: boolean
|
||||||
|
pages?: number // number of eBay result pages to fetch (48 listings/page, default 1)
|
||||||
|
mustInclude?: string // term string; client-side title filter; semantics set by mustIncludeMode
|
||||||
|
mustIncludeMode?: MustIncludeMode // 'all' = AND, 'any' = OR, 'groups' = CNF (pipe = OR within group)
|
||||||
|
mustExclude?: string // comma-separated; forwarded to eBay -term AND client-side
|
||||||
|
categoryId?: string // eBay category ID (e.g. "27386" = Graphics/Video Cards)
|
||||||
|
adapter?: 'auto' | 'api' | 'scraper' // override adapter selection
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Store ────────────────────────────────────────────────────────────────────
|
// ── Store ────────────────────────────────────────────────────────────────────
|
||||||
|
|
@ -73,10 +91,24 @@ export const useSearchStore = defineStore('search', () => {
|
||||||
const trustScores = ref<Map<string, TrustScore>>(new Map()) // key: platform_listing_id
|
const trustScores = ref<Map<string, TrustScore>>(new Map()) // key: platform_listing_id
|
||||||
const sellers = ref<Map<string, Seller>>(new Map()) // key: platform_seller_id
|
const sellers = ref<Map<string, Seller>>(new Map()) // key: platform_seller_id
|
||||||
const marketPrice = ref<number | null>(null)
|
const marketPrice = ref<number | null>(null)
|
||||||
|
const adapterUsed = ref<'api' | 'scraper' | null>(null)
|
||||||
const loading = ref(false)
|
const loading = ref(false)
|
||||||
const error = ref<string | null>(null)
|
const error = ref<string | null>(null)
|
||||||
|
|
||||||
|
let _abort: AbortController | null = null
|
||||||
|
|
||||||
|
function cancelSearch() {
|
||||||
|
_abort?.abort()
|
||||||
|
_abort = null
|
||||||
|
loading.value = false
|
||||||
|
}
|
||||||
|
|
||||||
async function search(q: string, filters: SearchFilters = {}) {
|
async function search(q: string, filters: SearchFilters = {}) {
|
||||||
|
// Cancel any in-flight search before starting a new one
|
||||||
|
_abort?.abort()
|
||||||
|
_abort = new AbortController()
|
||||||
|
const signal = _abort.signal
|
||||||
|
|
||||||
query.value = q
|
query.value = q
|
||||||
loading.value = true
|
loading.value = true
|
||||||
error.value = null
|
error.value = null
|
||||||
|
|
@ -91,8 +123,11 @@ export const useSearchStore = defineStore('search', () => {
|
||||||
if (filters.minPrice != null) params.set('min_price', String(filters.minPrice))
|
if (filters.minPrice != null) params.set('min_price', String(filters.minPrice))
|
||||||
if (filters.pages != null && filters.pages > 1) params.set('pages', String(filters.pages))
|
if (filters.pages != null && filters.pages > 1) params.set('pages', String(filters.pages))
|
||||||
if (filters.mustInclude?.trim()) params.set('must_include', filters.mustInclude.trim())
|
if (filters.mustInclude?.trim()) params.set('must_include', filters.mustInclude.trim())
|
||||||
|
if (filters.mustIncludeMode) params.set('must_include_mode', filters.mustIncludeMode)
|
||||||
if (filters.mustExclude?.trim()) params.set('must_exclude', filters.mustExclude.trim())
|
if (filters.mustExclude?.trim()) params.set('must_exclude', filters.mustExclude.trim())
|
||||||
const res = await fetch(`${apiBase}/api/search?${params}`)
|
if (filters.categoryId?.trim()) params.set('category_id', filters.categoryId.trim())
|
||||||
|
if (filters.adapter && filters.adapter !== 'auto') params.set('adapter', filters.adapter)
|
||||||
|
const res = await fetch(`${apiBase}/api/search?${params}`, { signal })
|
||||||
if (!res.ok) throw new Error(`Search failed: ${res.status} ${res.statusText}`)
|
if (!res.ok) throw new Error(`Search failed: ${res.status} ${res.statusText}`)
|
||||||
|
|
||||||
const data = await res.json() as {
|
const data = await res.json() as {
|
||||||
|
|
@ -100,20 +135,45 @@ export const useSearchStore = defineStore('search', () => {
|
||||||
trust_scores: Record<string, TrustScore>
|
trust_scores: Record<string, TrustScore>
|
||||||
sellers: Record<string, Seller>
|
sellers: Record<string, Seller>
|
||||||
market_price: number | null
|
market_price: number | null
|
||||||
|
adapter_used: 'api' | 'scraper'
|
||||||
}
|
}
|
||||||
|
|
||||||
results.value = data.listings ?? []
|
results.value = data.listings ?? []
|
||||||
trustScores.value = new Map(Object.entries(data.trust_scores ?? {}))
|
trustScores.value = new Map(Object.entries(data.trust_scores ?? {}))
|
||||||
sellers.value = new Map(Object.entries(data.sellers ?? {}))
|
sellers.value = new Map(Object.entries(data.sellers ?? {}))
|
||||||
marketPrice.value = data.market_price ?? null
|
marketPrice.value = data.market_price ?? null
|
||||||
|
adapterUsed.value = data.adapter_used ?? null
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
error.value = e instanceof Error ? e.message : 'Unknown error'
|
if (e instanceof DOMException && e.name === 'AbortError') {
|
||||||
results.value = []
|
// User cancelled — clear loading but don't surface as an error
|
||||||
|
results.value = []
|
||||||
|
} else {
|
||||||
|
error.value = e instanceof Error ? e.message : 'Unknown error'
|
||||||
|
results.value = []
|
||||||
|
}
|
||||||
} finally {
|
} finally {
|
||||||
loading.value = false
|
loading.value = false
|
||||||
|
_abort = null
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function enrichSeller(sellerUsername: string, listingId: string): Promise<void> {
|
||||||
|
const apiBase = (import.meta.env.VITE_API_BASE as string) ?? ''
|
||||||
|
const params = new URLSearchParams({
|
||||||
|
seller: sellerUsername,
|
||||||
|
listing_id: listingId,
|
||||||
|
query: query.value,
|
||||||
|
})
|
||||||
|
const res = await fetch(`${apiBase}/api/enrich?${params}`, { method: 'POST' })
|
||||||
|
if (!res.ok) throw new Error(`Enrich failed: ${res.status} ${res.statusText}`)
|
||||||
|
const data = await res.json() as {
|
||||||
|
trust_score: TrustScore | null
|
||||||
|
seller: Seller | null
|
||||||
|
}
|
||||||
|
if (data.trust_score) trustScores.value.set(listingId, data.trust_score)
|
||||||
|
if (data.seller) sellers.value.set(sellerUsername, data.seller)
|
||||||
|
}
|
||||||
|
|
||||||
function clearResults() {
|
function clearResults() {
|
||||||
results.value = []
|
results.value = []
|
||||||
trustScores.value = new Map()
|
trustScores.value = new Map()
|
||||||
|
|
@ -128,9 +188,12 @@ export const useSearchStore = defineStore('search', () => {
|
||||||
trustScores,
|
trustScores,
|
||||||
sellers,
|
sellers,
|
||||||
marketPrice,
|
marketPrice,
|
||||||
|
adapterUsed,
|
||||||
loading,
|
loading,
|
||||||
error,
|
error,
|
||||||
search,
|
search,
|
||||||
|
cancelSearch,
|
||||||
|
enrichSeller,
|
||||||
clearResults,
|
clearResults,
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
|
||||||
|
|
@ -1,55 +1,218 @@
|
||||||
<template>
|
<template>
|
||||||
<div class="saved-view">
|
<div class="saved-view">
|
||||||
<div class="placeholder">
|
<header class="saved-header">
|
||||||
<span class="placeholder__icon" aria-hidden="true">🔖</span>
|
<h1 class="saved-title">Saved Searches</h1>
|
||||||
<h1 class="placeholder__title">Saved Searches</h1>
|
</header>
|
||||||
<p class="placeholder__body">Coming soon — save searches and get background monitoring alerts when new matching listings appear.</p>
|
|
||||||
<RouterLink to="/" class="placeholder__back">← Back to Search</RouterLink>
|
<div v-if="store.loading" class="saved-state">
|
||||||
|
<p class="saved-state-text">Loading…</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div v-else-if="store.error" class="saved-state saved-state--error" role="alert">
|
||||||
|
{{ store.error }}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-else-if="!store.items.length" class="saved-state">
|
||||||
|
<span class="saved-state-icon" aria-hidden="true">🔖</span>
|
||||||
|
<p class="saved-state-text">No saved searches yet.</p>
|
||||||
|
<p class="saved-state-hint">Run a search and click <strong>Save</strong> to bookmark it here.</p>
|
||||||
|
<RouterLink to="/" class="saved-back">← Go to Search</RouterLink>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<ul v-else class="saved-list" role="list">
|
||||||
|
<li v-for="item in store.items" :key="item.id" class="saved-card">
|
||||||
|
<div class="saved-card-body">
|
||||||
|
<p class="saved-card-name">{{ item.name }}</p>
|
||||||
|
<p class="saved-card-query">
|
||||||
|
<span class="saved-card-q-label">q:</span>
|
||||||
|
{{ item.query }}
|
||||||
|
</p>
|
||||||
|
<p class="saved-card-meta">
|
||||||
|
<span v-if="item.last_run_at">Last run {{ formatDate(item.last_run_at) }}</span>
|
||||||
|
<span v-else>Never run</span>
|
||||||
|
· Saved {{ formatDate(item.created_at) }}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<div class="saved-card-actions">
|
||||||
|
<button class="saved-run-btn" type="button" @click="onRun(item)">
|
||||||
|
Run
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
class="saved-delete-btn"
|
||||||
|
type="button"
|
||||||
|
:aria-label="`Delete saved search: ${item.name}`"
|
||||||
|
@click="onDelete(item.id)"
|
||||||
|
>
|
||||||
|
✕
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
</div>
|
</div>
|
||||||
</template>
|
</template>
|
||||||
|
|
||||||
<script setup lang="ts">
|
<script setup lang="ts">
|
||||||
import { RouterLink } from 'vue-router'
|
import { onMounted } from 'vue'
|
||||||
|
import { useRouter, RouterLink } from 'vue-router'
|
||||||
|
import { useSavedSearchesStore } from '../stores/savedSearches'
|
||||||
|
import type { SavedSearch } from '../stores/savedSearches'
|
||||||
|
|
||||||
|
const store = useSavedSearchesStore()
|
||||||
|
const router = useRouter()
|
||||||
|
|
||||||
|
onMounted(() => store.fetchAll())
|
||||||
|
|
||||||
|
function formatDate(iso: string | null): string {
|
||||||
|
if (!iso) return '—'
|
||||||
|
const d = new Date(iso)
|
||||||
|
return d.toLocaleDateString(undefined, { month: 'short', day: 'numeric', year: 'numeric' })
|
||||||
|
}
|
||||||
|
|
||||||
|
async function onRun(item: SavedSearch) {
|
||||||
|
store.markRun(item.id)
|
||||||
|
const query: Record<string, string> = { q: item.query }
|
||||||
|
if (item.filters_json && item.filters_json !== '{}') query.filters = item.filters_json
|
||||||
|
router.push({ path: '/', query })
|
||||||
|
}
|
||||||
|
|
||||||
|
async function onDelete(id: number) {
|
||||||
|
await store.remove(id)
|
||||||
|
}
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
<style scoped>
|
<style scoped>
|
||||||
.saved-view {
|
.saved-view {
|
||||||
display: flex;
|
display: flex;
|
||||||
align-items: center;
|
flex-direction: column;
|
||||||
justify-content: center;
|
min-height: 100dvh;
|
||||||
min-height: 60dvh;
|
|
||||||
padding: var(--space-8);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
.placeholder {
|
.saved-header {
|
||||||
|
padding: var(--space-6);
|
||||||
|
border-bottom: 1px solid var(--color-border);
|
||||||
|
background: var(--color-surface-2);
|
||||||
|
}
|
||||||
|
|
||||||
|
.saved-title {
|
||||||
|
font-family: var(--font-display);
|
||||||
|
font-size: 1.25rem;
|
||||||
|
color: var(--color-text);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Empty / loading / error state */
|
||||||
|
.saved-state {
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
align-items: center;
|
align-items: center;
|
||||||
gap: var(--space-4);
|
gap: var(--space-4);
|
||||||
|
padding: var(--space-16) var(--space-4);
|
||||||
text-align: center;
|
text-align: center;
|
||||||
max-width: 480px;
|
|
||||||
}
|
}
|
||||||
|
.saved-state--error { color: var(--color-error); }
|
||||||
.placeholder__icon { font-size: 3rem; }
|
.saved-state-icon { font-size: 2.5rem; }
|
||||||
|
.saved-state-text { color: var(--color-text-muted); font-size: 0.9375rem; margin: 0; }
|
||||||
.placeholder__title {
|
.saved-state-hint { color: var(--color-text-muted); font-size: 0.875rem; margin: 0; }
|
||||||
font-family: var(--font-display);
|
.saved-back {
|
||||||
font-size: 1.5rem;
|
|
||||||
color: var(--app-primary);
|
|
||||||
}
|
|
||||||
|
|
||||||
.placeholder__body {
|
|
||||||
color: var(--color-text-muted);
|
|
||||||
line-height: 1.6;
|
|
||||||
}
|
|
||||||
|
|
||||||
.placeholder__back {
|
|
||||||
color: var(--app-primary);
|
color: var(--app-primary);
|
||||||
text-decoration: none;
|
text-decoration: none;
|
||||||
font-weight: 600;
|
font-weight: 600;
|
||||||
transition: opacity 150ms ease;
|
font-size: 0.875rem;
|
||||||
|
}
|
||||||
|
.saved-back:hover { opacity: 0.75; }
|
||||||
|
|
||||||
|
/* Card list */
|
||||||
|
.saved-list {
|
||||||
|
list-style: none;
|
||||||
|
padding: var(--space-6);
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: var(--space-3);
|
||||||
|
max-width: 720px;
|
||||||
}
|
}
|
||||||
|
|
||||||
.placeholder__back:hover { opacity: 0.75; }
|
.saved-card {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: var(--space-4);
|
||||||
|
padding: var(--space-4) var(--space-5);
|
||||||
|
background: var(--color-surface-2);
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: var(--radius-md);
|
||||||
|
transition: border-color 150ms ease;
|
||||||
|
}
|
||||||
|
.saved-card:hover { border-color: var(--app-primary); }
|
||||||
|
|
||||||
|
.saved-card-body { flex: 1; min-width: 0; }
|
||||||
|
|
||||||
|
.saved-card-name {
|
||||||
|
font-weight: 600;
|
||||||
|
font-size: 0.9375rem;
|
||||||
|
color: var(--color-text);
|
||||||
|
margin: 0 0 var(--space-1);
|
||||||
|
white-space: nowrap;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
}
|
||||||
|
|
||||||
|
.saved-card-query {
|
||||||
|
font-family: var(--font-mono);
|
||||||
|
font-size: 0.75rem;
|
||||||
|
color: var(--app-primary);
|
||||||
|
margin: 0 0 var(--space-1);
|
||||||
|
white-space: nowrap;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
}
|
||||||
|
.saved-card-q-label {
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
margin-right: var(--space-1);
|
||||||
|
}
|
||||||
|
|
||||||
|
.saved-card-meta {
|
||||||
|
font-size: 0.75rem;
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.saved-card-actions {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: var(--space-2);
|
||||||
|
flex-shrink: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.saved-run-btn {
|
||||||
|
padding: var(--space-2) var(--space-4);
|
||||||
|
background: var(--app-primary);
|
||||||
|
border: none;
|
||||||
|
border-radius: var(--radius-md);
|
||||||
|
color: var(--color-text-inverse);
|
||||||
|
font-family: var(--font-body);
|
||||||
|
font-size: 0.875rem;
|
||||||
|
font-weight: 600;
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background 150ms ease;
|
||||||
|
}
|
||||||
|
.saved-run-btn:hover { background: var(--app-primary-hover); }
|
||||||
|
|
||||||
|
.saved-delete-btn {
|
||||||
|
padding: var(--space-2);
|
||||||
|
background: transparent;
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: var(--radius-md);
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
font-size: 0.75rem;
|
||||||
|
line-height: 1;
|
||||||
|
cursor: pointer;
|
||||||
|
transition: border-color 150ms ease, color 150ms ease;
|
||||||
|
min-width: 28px;
|
||||||
|
}
|
||||||
|
.saved-delete-btn:hover { border-color: var(--color-error); color: var(--color-error); }
|
||||||
|
|
||||||
|
@media (max-width: 767px) {
|
||||||
|
.saved-header { padding: var(--space-4); }
|
||||||
|
.saved-list { padding: var(--space-4); }
|
||||||
|
.saved-card { flex-direction: column; align-items: flex-start; gap: var(--space-3); }
|
||||||
|
.saved-card-actions { width: 100%; justify-content: flex-end; }
|
||||||
|
}
|
||||||
</style>
|
</style>
|
||||||
|
|
|
||||||
|
|
@ -3,6 +3,22 @@
|
||||||
<!-- Search bar -->
|
<!-- Search bar -->
|
||||||
<header class="search-header">
|
<header class="search-header">
|
||||||
<form class="search-form" @submit.prevent="onSearch" role="search">
|
<form class="search-form" @submit.prevent="onSearch" role="search">
|
||||||
|
<label for="cat-select" class="sr-only">Category</label>
|
||||||
|
<select
|
||||||
|
id="cat-select"
|
||||||
|
v-model="filters.categoryId"
|
||||||
|
class="search-category-select"
|
||||||
|
:class="{ 'search-category-select--active': filters.categoryId }"
|
||||||
|
:disabled="store.loading"
|
||||||
|
title="Filter by category"
|
||||||
|
>
|
||||||
|
<option value="">All</option>
|
||||||
|
<optgroup v-for="group in CATEGORY_GROUPS" :key="group.label" :label="group.label">
|
||||||
|
<option v-for="cat in group.cats" :key="cat.id" :value="cat.id">
|
||||||
|
{{ cat.name }}
|
||||||
|
</option>
|
||||||
|
</optgroup>
|
||||||
|
</select>
|
||||||
<label for="search-input" class="sr-only">Search listings</label>
|
<label for="search-input" class="sr-only">Search listings</label>
|
||||||
<input
|
<input
|
||||||
id="search-input"
|
id="search-input"
|
||||||
|
|
@ -17,13 +33,143 @@
|
||||||
<MagnifyingGlassIcon class="search-btn-icon" aria-hidden="true" />
|
<MagnifyingGlassIcon class="search-btn-icon" aria-hidden="true" />
|
||||||
<span>{{ store.loading ? 'Searching…' : 'Search' }}</span>
|
<span>{{ store.loading ? 'Searching…' : 'Search' }}</span>
|
||||||
</button>
|
</button>
|
||||||
|
<button
|
||||||
|
v-if="store.loading"
|
||||||
|
type="button"
|
||||||
|
class="cancel-btn"
|
||||||
|
@click="store.cancelSearch()"
|
||||||
|
title="Cancel search"
|
||||||
|
>✕ Cancel</button>
|
||||||
|
<button
|
||||||
|
v-else
|
||||||
|
type="button"
|
||||||
|
class="save-bookmark-btn"
|
||||||
|
:disabled="!queryInput.trim()"
|
||||||
|
:title="showSaveForm ? 'Cancel' : 'Save this search'"
|
||||||
|
@click="showSaveForm = !showSaveForm; if (showSaveForm) saveName = queryInput.trim()"
|
||||||
|
>
|
||||||
|
<BookmarkIcon class="search-btn-icon" aria-hidden="true" />
|
||||||
|
</button>
|
||||||
|
</form>
|
||||||
|
<form v-if="showSaveForm" class="save-inline-form" @submit.prevent="onSave">
|
||||||
|
<input
|
||||||
|
v-model="saveName"
|
||||||
|
class="save-name-input"
|
||||||
|
placeholder="Name this search…"
|
||||||
|
autocomplete="off"
|
||||||
|
autofocus
|
||||||
|
/>
|
||||||
|
<button type="submit" class="save-confirm-btn">Save</button>
|
||||||
|
<button type="button" class="save-cancel-btn" @click="showSaveForm = false">✕</button>
|
||||||
|
<span v-if="saveSuccess" class="save-success">Saved!</span>
|
||||||
|
<span v-if="saveError" class="save-error">{{ saveError }}</span>
|
||||||
</form>
|
</form>
|
||||||
</header>
|
</header>
|
||||||
|
|
||||||
<div class="search-body">
|
<div class="search-body">
|
||||||
<!-- Filter sidebar -->
|
<!-- Filter sidebar -->
|
||||||
<aside class="filter-sidebar" aria-label="Search filters">
|
<aside class="filter-sidebar" aria-label="Search filters">
|
||||||
<h2 class="filter-heading">Filters</h2>
|
|
||||||
|
<!-- ── eBay Search Parameters ─────────────────────────────────────── -->
|
||||||
|
<!-- These are sent to eBay. Changes require a new search to take effect. -->
|
||||||
|
<h2 class="filter-section-heading filter-section-heading--search">
|
||||||
|
eBay Search
|
||||||
|
</h2>
|
||||||
|
<p class="filter-section-hint">Re-search to apply changes below</p>
|
||||||
|
|
||||||
|
<fieldset class="filter-group">
|
||||||
|
<legend class="filter-label">
|
||||||
|
Data source
|
||||||
|
<span
|
||||||
|
v-if="store.adapterUsed"
|
||||||
|
class="adapter-badge"
|
||||||
|
:class="store.adapterUsed === 'api' ? 'adapter-badge--api' : 'adapter-badge--scraper'"
|
||||||
|
>{{ store.adapterUsed === 'api' ? 'eBay API' : 'Scraper' }}</span>
|
||||||
|
</legend>
|
||||||
|
<div class="filter-pages" role="group" aria-label="Data source adapter">
|
||||||
|
<button
|
||||||
|
v-for="m in ADAPTER_MODES"
|
||||||
|
:key="m.value"
|
||||||
|
type="button"
|
||||||
|
class="filter-pages-btn"
|
||||||
|
:class="{ 'filter-pages-btn--active': filters.adapter === m.value }"
|
||||||
|
@click="filters.adapter = m.value"
|
||||||
|
>{{ m.label }}</button>
|
||||||
|
</div>
|
||||||
|
<p class="filter-pages-hint">Auto uses API when credentials are set</p>
|
||||||
|
</fieldset>
|
||||||
|
|
||||||
|
<fieldset class="filter-group">
|
||||||
|
<legend class="filter-label">Pages to fetch</legend>
|
||||||
|
<div class="filter-pages" role="group" aria-label="Number of result pages">
|
||||||
|
<button
|
||||||
|
v-for="p in [1, 2, 3, 5]"
|
||||||
|
:key="p"
|
||||||
|
type="button"
|
||||||
|
class="filter-pages-btn"
|
||||||
|
:class="{ 'filter-pages-btn--active': filters.pages === p }"
|
||||||
|
@click="filters.pages = p"
|
||||||
|
>{{ p }}</button>
|
||||||
|
</div>
|
||||||
|
<p class="filter-pages-hint">{{ pagesHint }}</p>
|
||||||
|
</fieldset>
|
||||||
|
|
||||||
|
<fieldset class="filter-group">
|
||||||
|
<legend class="filter-label">Price range</legend>
|
||||||
|
<div class="filter-row">
|
||||||
|
<input v-model.number="filters.minPrice" type="number" min="0" class="filter-input" placeholder="Min $" />
|
||||||
|
<input v-model.number="filters.maxPrice" type="number" min="0" class="filter-input" placeholder="Max $" />
|
||||||
|
</div>
|
||||||
|
<p class="filter-pages-hint">Forwarded to eBay API</p>
|
||||||
|
</fieldset>
|
||||||
|
|
||||||
|
<fieldset class="filter-group">
|
||||||
|
<legend class="filter-label">Keywords</legend>
|
||||||
|
<div class="filter-row">
|
||||||
|
<label class="filter-label-sm" for="f-include">Must include</label>
|
||||||
|
<div class="filter-mode-row">
|
||||||
|
<button
|
||||||
|
v-for="m in INCLUDE_MODES"
|
||||||
|
:key="m.value"
|
||||||
|
type="button"
|
||||||
|
class="filter-pages-btn"
|
||||||
|
:class="{ 'filter-pages-btn--active': filters.mustIncludeMode === m.value }"
|
||||||
|
@click="filters.mustIncludeMode = m.value"
|
||||||
|
>{{ m.label }}</button>
|
||||||
|
</div>
|
||||||
|
<input
|
||||||
|
id="f-include"
|
||||||
|
v-model="filters.mustInclude"
|
||||||
|
type="text"
|
||||||
|
class="filter-input filter-input--keyword"
|
||||||
|
:placeholder="filters.mustIncludeMode === 'groups' ? 'founders|fe, 16gb\u2026' : '16gb, founders\u2026'"
|
||||||
|
autocomplete="off"
|
||||||
|
spellcheck="false"
|
||||||
|
/>
|
||||||
|
<p class="filter-pages-hint">{{ includeHint }}</p>
|
||||||
|
</div>
|
||||||
|
<div class="filter-row">
|
||||||
|
<label class="filter-label-sm" for="f-exclude">Must exclude</label>
|
||||||
|
<input
|
||||||
|
id="f-exclude"
|
||||||
|
v-model="filters.mustExclude"
|
||||||
|
type="text"
|
||||||
|
class="filter-input filter-input--keyword filter-input--exclude"
|
||||||
|
placeholder="broken, parts\u2026"
|
||||||
|
autocomplete="off"
|
||||||
|
spellcheck="false"
|
||||||
|
/>
|
||||||
|
<p class="filter-pages-hint">Excludes forwarded to eBay on re-search</p>
|
||||||
|
</div>
|
||||||
|
</fieldset>
|
||||||
|
|
||||||
|
<!-- ── Post-search Filters ────────────────────────────────────────── -->
|
||||||
|
<!-- Applied locally to current results — no re-search needed. -->
|
||||||
|
<div class="filter-section-divider" role="separator"></div>
|
||||||
|
<h2 class="filter-section-heading filter-section-heading--filter">
|
||||||
|
Filter Results
|
||||||
|
</h2>
|
||||||
|
<p class="filter-section-hint">Applied instantly to current results</p>
|
||||||
|
|
||||||
<fieldset class="filter-group">
|
<fieldset class="filter-group">
|
||||||
<legend class="filter-label">Min Trust Score</legend>
|
<legend class="filter-label">Min Trust Score</legend>
|
||||||
|
|
@ -41,69 +187,19 @@
|
||||||
<span class="filter-range-val">{{ filters.minTrustScore ?? 0 }}</span>
|
<span class="filter-range-val">{{ filters.minTrustScore ?? 0 }}</span>
|
||||||
</fieldset>
|
</fieldset>
|
||||||
|
|
||||||
<fieldset class="filter-group">
|
<details class="filter-group filter-collapsible">
|
||||||
<legend class="filter-label">Pages to fetch</legend>
|
<summary class="filter-collapsible-summary">Condition</summary>
|
||||||
<div class="filter-pages" role="group" aria-label="Number of result pages">
|
<div class="filter-collapsible-body">
|
||||||
<button
|
<label v-for="cond in CONDITIONS" :key="cond.value" class="filter-check">
|
||||||
v-for="p in [1, 2, 3, 5]"
|
<input
|
||||||
:key="p"
|
type="checkbox"
|
||||||
type="button"
|
:value="cond.value"
|
||||||
class="filter-pages-btn"
|
v-model="filters.conditions"
|
||||||
:class="{ 'filter-pages-btn--active': filters.pages === p }"
|
/>
|
||||||
@click="filters.pages = p"
|
{{ cond.label }}
|
||||||
>{{ p }}</button>
|
</label>
|
||||||
</div>
|
</div>
|
||||||
<p class="filter-pages-hint">{{ (filters.pages ?? 1) * 48 }} listings · {{ (filters.pages ?? 1) * 2 }} Playwright calls</p>
|
</details>
|
||||||
</fieldset>
|
|
||||||
|
|
||||||
<fieldset class="filter-group">
|
|
||||||
<legend class="filter-label">Keywords</legend>
|
|
||||||
<div class="filter-row">
|
|
||||||
<label class="filter-label-sm" for="f-include">Must include</label>
|
|
||||||
<input
|
|
||||||
id="f-include"
|
|
||||||
v-model="filters.mustInclude"
|
|
||||||
type="text"
|
|
||||||
class="filter-input filter-input--keyword"
|
|
||||||
placeholder="16gb, founders…"
|
|
||||||
autocomplete="off"
|
|
||||||
spellcheck="false"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
<div class="filter-row">
|
|
||||||
<label class="filter-label-sm" for="f-exclude">Must exclude</label>
|
|
||||||
<input
|
|
||||||
id="f-exclude"
|
|
||||||
v-model="filters.mustExclude"
|
|
||||||
type="text"
|
|
||||||
class="filter-input filter-input--keyword filter-input--exclude"
|
|
||||||
placeholder="broken, parts…"
|
|
||||||
autocomplete="off"
|
|
||||||
spellcheck="false"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
<p class="filter-pages-hint">Comma-separated · re-search to apply to eBay</p>
|
|
||||||
</fieldset>
|
|
||||||
|
|
||||||
<fieldset class="filter-group">
|
|
||||||
<legend class="filter-label">Price</legend>
|
|
||||||
<div class="filter-row">
|
|
||||||
<input v-model.number="filters.minPrice" type="number" min="0" class="filter-input" placeholder="Min $" />
|
|
||||||
<input v-model.number="filters.maxPrice" type="number" min="0" class="filter-input" placeholder="Max $" />
|
|
||||||
</div>
|
|
||||||
</fieldset>
|
|
||||||
|
|
||||||
<fieldset class="filter-group">
|
|
||||||
<legend class="filter-label">Condition</legend>
|
|
||||||
<label v-for="cond in CONDITIONS" :key="cond.value" class="filter-check">
|
|
||||||
<input
|
|
||||||
type="checkbox"
|
|
||||||
:value="cond.value"
|
|
||||||
v-model="filters.conditions"
|
|
||||||
/>
|
|
||||||
{{ cond.label }}
|
|
||||||
</label>
|
|
||||||
</fieldset>
|
|
||||||
|
|
||||||
<fieldset class="filter-group">
|
<fieldset class="filter-group">
|
||||||
<legend class="filter-label">Seller</legend>
|
<legend class="filter-label">Seller</legend>
|
||||||
|
|
@ -131,6 +227,18 @@
|
||||||
<input type="checkbox" v-model="filters.hideDuplicatePhotos" />
|
<input type="checkbox" v-model="filters.hideDuplicatePhotos" />
|
||||||
Duplicate photos
|
Duplicate photos
|
||||||
</label>
|
</label>
|
||||||
|
<label class="filter-check">
|
||||||
|
<input type="checkbox" v-model="filters.hideScratchDent" />
|
||||||
|
Scratch / dent mentioned
|
||||||
|
</label>
|
||||||
|
<label class="filter-check">
|
||||||
|
<input type="checkbox" v-model="filters.hideLongOnMarket" />
|
||||||
|
Long on market (≥5 sightings, 14d+)
|
||||||
|
</label>
|
||||||
|
<label class="filter-check">
|
||||||
|
<input type="checkbox" v-model="filters.hidePriceDrop" />
|
||||||
|
Significant price drop (≥20%)
|
||||||
|
</label>
|
||||||
</fieldset>
|
</fieldset>
|
||||||
</aside>
|
</aside>
|
||||||
|
|
||||||
|
|
@ -163,12 +271,14 @@
|
||||||
· {{ hiddenCount }} hidden by filters
|
· {{ hiddenCount }} hidden by filters
|
||||||
</span>
|
</span>
|
||||||
</p>
|
</p>
|
||||||
<label for="sort-select" class="sr-only">Sort by</label>
|
<div class="toolbar-actions">
|
||||||
<select id="sort-select" v-model="sortBy" class="sort-select">
|
<label for="sort-select" class="sr-only">Sort by</label>
|
||||||
<option v-for="opt in SORT_OPTIONS" :key="opt.value" :value="opt.value">
|
<select id="sort-select" v-model="sortBy" class="sort-select">
|
||||||
{{ opt.label }}
|
<option v-for="opt in SORT_OPTIONS" :key="opt.value" :value="opt.value">
|
||||||
</option>
|
{{ opt.label }}
|
||||||
</select>
|
</option>
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- Cards -->
|
<!-- Cards -->
|
||||||
|
|
@ -189,15 +299,56 @@
|
||||||
</template>
|
</template>
|
||||||
|
|
||||||
<script setup lang="ts">
|
<script setup lang="ts">
|
||||||
import { ref, computed, reactive } from 'vue'
|
import { ref, computed, reactive, onMounted } from 'vue'
|
||||||
import { MagnifyingGlassIcon, ExclamationTriangleIcon } from '@heroicons/vue/24/outline'
|
import { useRoute } from 'vue-router'
|
||||||
|
import { MagnifyingGlassIcon, ExclamationTriangleIcon, BookmarkIcon } from '@heroicons/vue/24/outline'
|
||||||
import { useSearchStore } from '../stores/search'
|
import { useSearchStore } from '../stores/search'
|
||||||
import type { Listing, TrustScore, SearchFilters } from '../stores/search'
|
import type { Listing, TrustScore, SearchFilters, MustIncludeMode } from '../stores/search'
|
||||||
|
import { useSavedSearchesStore } from '../stores/savedSearches'
|
||||||
import ListingCard from '../components/ListingCard.vue'
|
import ListingCard from '../components/ListingCard.vue'
|
||||||
|
|
||||||
|
const route = useRoute()
|
||||||
const store = useSearchStore()
|
const store = useSearchStore()
|
||||||
|
const savedStore = useSavedSearchesStore()
|
||||||
const queryInput = ref('')
|
const queryInput = ref('')
|
||||||
|
|
||||||
|
// Save search UI state
|
||||||
|
const showSaveForm = ref(false)
|
||||||
|
const saveName = ref('')
|
||||||
|
const saveError = ref<string | null>(null)
|
||||||
|
const saveSuccess = ref(false)
|
||||||
|
|
||||||
|
async function onSave() {
|
||||||
|
if (!saveName.value.trim()) return
|
||||||
|
saveError.value = null
|
||||||
|
try {
|
||||||
|
await savedStore.create(saveName.value.trim(), store.query, { ...filters })
|
||||||
|
saveSuccess.value = true
|
||||||
|
showSaveForm.value = false
|
||||||
|
saveName.value = ''
|
||||||
|
setTimeout(() => { saveSuccess.value = false }, 2500)
|
||||||
|
} catch (e) {
|
||||||
|
saveError.value = e instanceof Error ? e.message : 'Save failed'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Auto-run if ?q= param present (e.g. launched from Saved Searches)
|
||||||
|
onMounted(() => {
|
||||||
|
const q = route.query.q
|
||||||
|
if (typeof q === 'string' && q.trim()) {
|
||||||
|
queryInput.value = q.trim()
|
||||||
|
// Restore saved filters (e.g. category, price range, trust threshold)
|
||||||
|
const f = route.query.filters
|
||||||
|
if (typeof f === 'string') {
|
||||||
|
try {
|
||||||
|
const restored = JSON.parse(f) as Partial<SearchFilters>
|
||||||
|
Object.assign(filters, restored)
|
||||||
|
} catch { /* malformed — ignore */ }
|
||||||
|
}
|
||||||
|
onSearch()
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
// ── Filters ──────────────────────────────────────────────────────────────────
|
// ── Filters ──────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
const filters = reactive<SearchFilters>({
|
const filters = reactive<SearchFilters>({
|
||||||
|
|
@ -210,9 +361,15 @@ const filters = reactive<SearchFilters>({
|
||||||
hideNewAccounts: false,
|
hideNewAccounts: false,
|
||||||
hideSuspiciousPrice: false,
|
hideSuspiciousPrice: false,
|
||||||
hideDuplicatePhotos: false,
|
hideDuplicatePhotos: false,
|
||||||
|
hideScratchDent: false,
|
||||||
|
hideLongOnMarket: false,
|
||||||
|
hidePriceDrop: false,
|
||||||
pages: 1,
|
pages: 1,
|
||||||
mustInclude: '',
|
mustInclude: '',
|
||||||
|
mustIncludeMode: 'all',
|
||||||
mustExclude: '',
|
mustExclude: '',
|
||||||
|
categoryId: '',
|
||||||
|
adapter: 'auto' as 'auto' | 'api' | 'scraper',
|
||||||
})
|
})
|
||||||
|
|
||||||
// Parse comma-separated keyword strings into trimmed, lowercase, non-empty term arrays
|
// Parse comma-separated keyword strings into trimmed, lowercase, non-empty term arrays
|
||||||
|
|
@ -222,6 +379,82 @@ const parsedMustInclude = computed(() =>
|
||||||
const parsedMustExclude = computed(() =>
|
const parsedMustExclude = computed(() =>
|
||||||
(filters.mustExclude ?? '').split(',').map(t => t.trim().toLowerCase()).filter(Boolean)
|
(filters.mustExclude ?? '').split(',').map(t => t.trim().toLowerCase()).filter(Boolean)
|
||||||
)
|
)
|
||||||
|
// Groups mode: comma = group separator, pipe = OR within group → string[][]
|
||||||
|
// e.g. "founders|fe, 16gb" → [["founders","fe"], ["16gb"]]
|
||||||
|
const parsedMustIncludeGroups = computed(() =>
|
||||||
|
(filters.mustInclude ?? '').split(',')
|
||||||
|
.map(group => group.split('|').map(t => t.trim().toLowerCase()).filter(Boolean))
|
||||||
|
.filter(g => g.length > 0)
|
||||||
|
)
|
||||||
|
|
||||||
|
const INCLUDE_MODES: { value: MustIncludeMode; label: string }[] = [
|
||||||
|
{ value: 'all', label: 'All' },
|
||||||
|
{ value: 'any', label: 'Any' },
|
||||||
|
{ value: 'groups', label: 'Groups' },
|
||||||
|
]
|
||||||
|
|
||||||
|
const includeHint = computed(() => {
|
||||||
|
switch (filters.mustIncludeMode) {
|
||||||
|
case 'any': return 'At least one term must appear'
|
||||||
|
case 'groups': return 'Comma = AND · pipe | = OR within group'
|
||||||
|
default: return 'Every term must appear'
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
const ADAPTER_MODES: { value: 'auto' | 'api' | 'scraper'; label: string }[] = [
|
||||||
|
{ value: 'auto', label: 'Auto' },
|
||||||
|
{ value: 'api', label: 'API' },
|
||||||
|
{ value: 'scraper', label: 'Scraper' },
|
||||||
|
]
|
||||||
|
|
||||||
|
const pagesHint = computed(() => {
|
||||||
|
const p = filters.pages ?? 1
|
||||||
|
const effective = filters.adapter === 'scraper' ? 'scraper'
|
||||||
|
: filters.adapter === 'api' ? 'api'
|
||||||
|
: store.adapterUsed ?? 'api' // assume API until first search
|
||||||
|
if (effective === 'scraper') {
|
||||||
|
return `${p * 48} listings · ${p} Playwright calls`
|
||||||
|
}
|
||||||
|
return `Up to ${p * 200} listings · ${p} Browse API call${p > 1 ? 's' : ''}`
|
||||||
|
})
|
||||||
|
|
||||||
|
const CATEGORY_GROUPS = [
|
||||||
|
{ label: 'Computers', cats: [
|
||||||
|
{ id: '175673', name: 'Computer Components & Parts' },
|
||||||
|
{ id: '27386', name: 'Graphics / Video Cards' },
|
||||||
|
{ id: '164', name: 'CPUs / Processors' },
|
||||||
|
{ id: '1244', name: 'Motherboards' },
|
||||||
|
{ id: '170083', name: 'Memory (RAM)' },
|
||||||
|
{ id: '56083', name: 'Hard Drives & SSDs' },
|
||||||
|
{ id: '42017', name: 'Power Supplies' },
|
||||||
|
{ id: '42014', name: 'Computer Cases' },
|
||||||
|
{ id: '11176', name: 'Networking Equipment' },
|
||||||
|
{ id: '80053', name: 'Monitors' },
|
||||||
|
{ id: '177', name: 'Laptops' },
|
||||||
|
{ id: '179', name: 'Desktop Computers' },
|
||||||
|
]},
|
||||||
|
{ label: 'Mobile', cats: [
|
||||||
|
{ id: '9355', name: 'Smartphones' },
|
||||||
|
{ id: '171485', name: 'Tablets & eReaders' },
|
||||||
|
]},
|
||||||
|
{ label: 'Gaming', cats: [
|
||||||
|
{ id: '139971', name: 'Game Consoles' },
|
||||||
|
{ id: '1249', name: 'Video Games' },
|
||||||
|
]},
|
||||||
|
{ label: 'Audio & Video', cats: [
|
||||||
|
{ id: '14969', name: 'Home Audio' },
|
||||||
|
{ id: '32852', name: 'TVs' },
|
||||||
|
]},
|
||||||
|
{ label: 'Cameras', cats: [
|
||||||
|
{ id: '625', name: 'Cameras & Photo' },
|
||||||
|
]},
|
||||||
|
{ label: 'Collectibles', cats: [
|
||||||
|
{ id: '183454', name: 'Trading Cards' },
|
||||||
|
{ id: '64482', name: 'Sports Memorabilia' },
|
||||||
|
{ id: '11116', name: 'Coins & Currency' },
|
||||||
|
{ id: '20081', name: 'Antiques' },
|
||||||
|
]},
|
||||||
|
]
|
||||||
|
|
||||||
const CONDITIONS = [
|
const CONDITIONS = [
|
||||||
{ value: 'new', label: 'New' },
|
{ value: 'new', label: 'New' },
|
||||||
|
|
@ -277,7 +510,18 @@ function passesFilter(listing: Listing): boolean {
|
||||||
|
|
||||||
// Keyword filtering — substring match on lowercased title
|
// Keyword filtering — substring match on lowercased title
|
||||||
const title = listing.title.toLowerCase()
|
const title = listing.title.toLowerCase()
|
||||||
if (parsedMustInclude.value.some(term => !title.includes(term))) return false
|
if (parsedMustInclude.value.length) {
|
||||||
|
const mode = filters.mustIncludeMode ?? 'all'
|
||||||
|
if (mode === 'any') {
|
||||||
|
if (!parsedMustInclude.value.some(term => title.includes(term))) return false
|
||||||
|
} else if (mode === 'groups') {
|
||||||
|
// CNF: must match at least one alternative from every group
|
||||||
|
if (!parsedMustIncludeGroups.value.every(group => group.some(alt => title.includes(alt)))) return false
|
||||||
|
} else {
|
||||||
|
// 'all': every term must appear
|
||||||
|
if (parsedMustInclude.value.some(term => !title.includes(term))) return false
|
||||||
|
}
|
||||||
|
}
|
||||||
if (parsedMustExclude.value.some(term => title.includes(term))) return false
|
if (parsedMustExclude.value.some(term => title.includes(term))) return false
|
||||||
|
|
||||||
if (filters.minTrustScore && trust && trust.composite_score < filters.minTrustScore) return false
|
if (filters.minTrustScore && trust && trust.composite_score < filters.minTrustScore) return false
|
||||||
|
|
@ -286,7 +530,7 @@ function passesFilter(listing: Listing): boolean {
|
||||||
if (filters.conditions?.length && !filters.conditions.includes(listing.condition)) return false
|
if (filters.conditions?.length && !filters.conditions.includes(listing.condition)) return false
|
||||||
|
|
||||||
if (seller) {
|
if (seller) {
|
||||||
if (filters.minAccountAgeDays && seller.account_age_days < filters.minAccountAgeDays) return false
|
if (filters.minAccountAgeDays && seller.account_age_days != null && seller.account_age_days < filters.minAccountAgeDays) return false
|
||||||
if (filters.minFeedbackCount && seller.feedback_count < filters.minFeedbackCount) return false
|
if (filters.minFeedbackCount && seller.feedback_count < filters.minFeedbackCount) return false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -296,6 +540,9 @@ function passesFilter(listing: Listing): boolean {
|
||||||
if (filters.hideNewAccounts && flags.includes('account_under_30_days')) return false
|
if (filters.hideNewAccounts && flags.includes('account_under_30_days')) return false
|
||||||
if (filters.hideSuspiciousPrice && flags.includes('suspicious_price')) return false
|
if (filters.hideSuspiciousPrice && flags.includes('suspicious_price')) return false
|
||||||
if (filters.hideDuplicatePhotos && flags.includes('duplicate_photo')) return false
|
if (filters.hideDuplicatePhotos && flags.includes('duplicate_photo')) return false
|
||||||
|
if (filters.hideScratchDent && flags.includes('scratch_dent_mentioned')) return false
|
||||||
|
if (filters.hideLongOnMarket && flags.includes('long_on_market')) return false
|
||||||
|
if (filters.hidePriceDrop && flags.includes('significant_price_drop')) return false
|
||||||
}
|
}
|
||||||
|
|
||||||
return true
|
return true
|
||||||
|
|
@ -336,6 +583,30 @@ async function onSearch() {
|
||||||
max-width: 760px;
|
max-width: 760px;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.search-category-select {
|
||||||
|
padding: var(--space-3) var(--space-3);
|
||||||
|
background: var(--color-surface-raised);
|
||||||
|
border: 1.5px solid var(--color-border);
|
||||||
|
border-radius: var(--radius-md);
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
font-family: var(--font-body);
|
||||||
|
font-size: 0.875rem;
|
||||||
|
cursor: pointer;
|
||||||
|
white-space: nowrap;
|
||||||
|
flex-shrink: 0;
|
||||||
|
max-width: 160px;
|
||||||
|
transition: border-color 150ms ease, color 150ms ease;
|
||||||
|
}
|
||||||
|
.search-category-select--active {
|
||||||
|
border-color: var(--app-primary);
|
||||||
|
color: var(--color-text);
|
||||||
|
font-weight: 500;
|
||||||
|
}
|
||||||
|
.search-category-select:focus {
|
||||||
|
outline: none;
|
||||||
|
border-color: var(--app-primary);
|
||||||
|
}
|
||||||
|
|
||||||
.search-input {
|
.search-input {
|
||||||
flex: 1;
|
flex: 1;
|
||||||
padding: var(--space-3) var(--space-4);
|
padding: var(--space-3) var(--space-4);
|
||||||
|
|
@ -376,6 +647,49 @@ async function onSearch() {
|
||||||
.search-btn:disabled { opacity: 0.55; cursor: not-allowed; }
|
.search-btn:disabled { opacity: 0.55; cursor: not-allowed; }
|
||||||
.search-btn-icon { width: 1.1rem; height: 1.1rem; }
|
.search-btn-icon { width: 1.1rem; height: 1.1rem; }
|
||||||
|
|
||||||
|
.cancel-btn {
|
||||||
|
padding: var(--space-3) var(--space-4);
|
||||||
|
background: transparent;
|
||||||
|
border: 1.5px solid var(--color-error);
|
||||||
|
border-radius: var(--radius-md);
|
||||||
|
color: var(--color-error);
|
||||||
|
font-family: var(--font-body);
|
||||||
|
font-size: 0.875rem;
|
||||||
|
font-weight: 600;
|
||||||
|
cursor: pointer;
|
||||||
|
white-space: nowrap;
|
||||||
|
flex-shrink: 0;
|
||||||
|
transition: background 150ms ease;
|
||||||
|
}
|
||||||
|
.cancel-btn:hover { background: rgba(248, 81, 73, 0.1); }
|
||||||
|
|
||||||
|
.save-bookmark-btn {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
padding: var(--space-3);
|
||||||
|
background: var(--color-surface-raised);
|
||||||
|
border: 1.5px solid var(--color-border);
|
||||||
|
border-radius: var(--radius-md);
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
cursor: pointer;
|
||||||
|
transition: border-color 150ms ease, color 150ms ease;
|
||||||
|
flex-shrink: 0;
|
||||||
|
}
|
||||||
|
.save-bookmark-btn:hover:not(:disabled) {
|
||||||
|
border-color: var(--app-primary);
|
||||||
|
color: var(--app-primary);
|
||||||
|
}
|
||||||
|
.save-bookmark-btn:disabled { opacity: 0.4; cursor: not-allowed; }
|
||||||
|
|
||||||
|
.save-inline-form {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: var(--space-2);
|
||||||
|
padding: var(--space-2) 0 0;
|
||||||
|
max-width: 760px;
|
||||||
|
}
|
||||||
|
|
||||||
/* Two-column layout */
|
/* Two-column layout */
|
||||||
.search-body {
|
.search-body {
|
||||||
display: flex;
|
display: flex;
|
||||||
|
|
@ -404,6 +718,38 @@ async function onSearch() {
|
||||||
margin-bottom: var(--space-2);
|
margin-bottom: var(--space-2);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Section headings that separate eBay Search params from local filters */
|
||||||
|
.filter-section-heading {
|
||||||
|
font-size: 0.6875rem;
|
||||||
|
font-weight: 700;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.08em;
|
||||||
|
padding: var(--space-1) var(--space-2);
|
||||||
|
border-radius: var(--radius-sm);
|
||||||
|
margin-top: var(--space-1);
|
||||||
|
}
|
||||||
|
.filter-section-heading--search {
|
||||||
|
color: var(--app-primary);
|
||||||
|
background: color-mix(in srgb, var(--app-primary) 10%, transparent);
|
||||||
|
}
|
||||||
|
.filter-section-heading--filter {
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
background: color-mix(in srgb, var(--color-text-muted) 8%, transparent);
|
||||||
|
}
|
||||||
|
|
||||||
|
.filter-section-hint {
|
||||||
|
font-size: 0.6875rem;
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
opacity: 0.75;
|
||||||
|
margin-top: calc(-1 * var(--space-2));
|
||||||
|
}
|
||||||
|
|
||||||
|
.filter-section-divider {
|
||||||
|
height: 1px;
|
||||||
|
background: var(--color-border);
|
||||||
|
margin: var(--space-2) 0;
|
||||||
|
}
|
||||||
|
|
||||||
.filter-group {
|
.filter-group {
|
||||||
border: none;
|
border: none;
|
||||||
padding: 0;
|
padding: 0;
|
||||||
|
|
@ -472,6 +818,25 @@ async function onSearch() {
|
||||||
font-size: 0.75rem;
|
font-size: 0.75rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.adapter-badge {
|
||||||
|
display: inline-block;
|
||||||
|
margin-left: var(--space-2);
|
||||||
|
padding: 1px var(--space-2);
|
||||||
|
border-radius: var(--radius-sm);
|
||||||
|
font-size: 0.625rem;
|
||||||
|
font-weight: 700;
|
||||||
|
letter-spacing: 0.04em;
|
||||||
|
text-transform: uppercase;
|
||||||
|
vertical-align: middle;
|
||||||
|
}
|
||||||
|
.adapter-badge--api { background: color-mix(in srgb, var(--app-primary) 15%, transparent); color: var(--app-primary); }
|
||||||
|
.adapter-badge--scraper { background: color-mix(in srgb, var(--color-warning) 15%, transparent); color: var(--color-warning); }
|
||||||
|
|
||||||
|
.filter-category-select {
|
||||||
|
cursor: pointer;
|
||||||
|
appearance: auto;
|
||||||
|
}
|
||||||
|
|
||||||
.filter-input--exclude {
|
.filter-input--exclude {
|
||||||
border-color: color-mix(in srgb, var(--color-error) 40%, var(--color-border));
|
border-color: color-mix(in srgb, var(--color-error) 40%, var(--color-border));
|
||||||
}
|
}
|
||||||
|
|
@ -481,6 +846,51 @@ async function onSearch() {
|
||||||
border-color: var(--color-error);
|
border-color: var(--color-error);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Mode toggle row — same pill style as pages buttons */
|
||||||
|
.filter-mode-row {
|
||||||
|
display: flex;
|
||||||
|
gap: var(--space-1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Collapsible condition picker */
|
||||||
|
.filter-collapsible {
|
||||||
|
border: none;
|
||||||
|
padding: 0;
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.filter-collapsible-summary {
|
||||||
|
font-size: 0.8125rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
cursor: pointer;
|
||||||
|
list-style: none;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: var(--space-2);
|
||||||
|
user-select: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.filter-collapsible-summary::after {
|
||||||
|
content: '›';
|
||||||
|
font-size: 1rem;
|
||||||
|
line-height: 1;
|
||||||
|
transition: transform 150ms ease;
|
||||||
|
}
|
||||||
|
|
||||||
|
.filter-collapsible[open] .filter-collapsible-summary::after {
|
||||||
|
transform: rotate(90deg);
|
||||||
|
}
|
||||||
|
|
||||||
|
.filter-collapsible-summary::-webkit-details-marker { display: none; }
|
||||||
|
|
||||||
|
.filter-collapsible-body {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: var(--space-2);
|
||||||
|
padding-top: var(--space-2);
|
||||||
|
}
|
||||||
|
|
||||||
.filter-pages {
|
.filter-pages {
|
||||||
display: flex;
|
display: flex;
|
||||||
gap: var(--space-1);
|
gap: var(--space-1);
|
||||||
|
|
@ -568,6 +978,76 @@ async function onSearch() {
|
||||||
|
|
||||||
.results-hidden { color: var(--color-warning); }
|
.results-hidden { color: var(--color-warning); }
|
||||||
|
|
||||||
|
.toolbar-actions {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: var(--space-2);
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.save-btn {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: var(--space-1);
|
||||||
|
padding: var(--space-2) var(--space-3);
|
||||||
|
background: transparent;
|
||||||
|
border: 1px solid var(--color-border);
|
||||||
|
border-radius: var(--radius-md);
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
font-family: var(--font-body);
|
||||||
|
font-size: 0.8125rem;
|
||||||
|
cursor: pointer;
|
||||||
|
transition: border-color 150ms ease, color 150ms ease;
|
||||||
|
}
|
||||||
|
.save-btn:hover { border-color: var(--app-primary); color: var(--app-primary); }
|
||||||
|
.save-btn-icon { width: 0.9rem; height: 0.9rem; }
|
||||||
|
|
||||||
|
.save-form {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: var(--space-1);
|
||||||
|
}
|
||||||
|
.save-name-input {
|
||||||
|
padding: var(--space-1) var(--space-2);
|
||||||
|
background: var(--color-surface-raised);
|
||||||
|
border: 1px solid var(--app-primary);
|
||||||
|
border-radius: var(--radius-sm);
|
||||||
|
color: var(--color-text);
|
||||||
|
font-family: var(--font-body);
|
||||||
|
font-size: 0.8125rem;
|
||||||
|
width: 160px;
|
||||||
|
}
|
||||||
|
.save-name-input:focus { outline: none; }
|
||||||
|
.save-confirm-btn {
|
||||||
|
padding: var(--space-1) var(--space-3);
|
||||||
|
background: var(--app-primary);
|
||||||
|
border: none;
|
||||||
|
border-radius: var(--radius-sm);
|
||||||
|
color: var(--color-text-inverse);
|
||||||
|
font-size: 0.8125rem;
|
||||||
|
font-weight: 600;
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
.save-cancel-btn {
|
||||||
|
padding: var(--space-1) var(--space-2);
|
||||||
|
background: transparent;
|
||||||
|
border: none;
|
||||||
|
color: var(--color-text-muted);
|
||||||
|
font-size: 0.875rem;
|
||||||
|
cursor: pointer;
|
||||||
|
line-height: 1;
|
||||||
|
}
|
||||||
|
.save-success {
|
||||||
|
font-size: 0.8125rem;
|
||||||
|
color: var(--color-success);
|
||||||
|
font-weight: 600;
|
||||||
|
}
|
||||||
|
.save-error {
|
||||||
|
font-size: 0.75rem;
|
||||||
|
color: var(--color-error);
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
.sort-select {
|
.sort-select {
|
||||||
padding: var(--space-2) var(--space-3);
|
padding: var(--space-2) var(--space-3);
|
||||||
background: var(--color-surface-raised);
|
background: var(--color-surface-raised);
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,10 @@ import UnoCSS from 'unocss/vite'
|
||||||
export default defineConfig({
|
export default defineConfig({
|
||||||
plugins: [vue(), UnoCSS()],
|
plugins: [vue(), UnoCSS()],
|
||||||
base: process.env.VITE_BASE_URL ?? '/',
|
base: process.env.VITE_BASE_URL ?? '/',
|
||||||
|
build: {
|
||||||
|
// 16-char content hash prevents filename collisions that break immutable caching
|
||||||
|
rollupOptions: { output: { hashCharacters: 'base64', entryFileNames: 'assets/[name]-[hash:16].js', chunkFileNames: 'assets/[name]-[hash:16].js', assetFileNames: 'assets/[name]-[hash:16].[ext]' } },
|
||||||
|
},
|
||||||
server: {
|
server: {
|
||||||
host: '0.0.0.0',
|
host: '0.0.0.0',
|
||||||
port: 5174,
|
port: 5174,
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue