Compare commits

..

No commits in common. "main" and "v0.5.6" have entirely different histories.
main ... v0.5.6

57 changed files with 750 additions and 4728 deletions

View file

@ -98,25 +98,16 @@ CF_APP_NAME=snipe
# OLLAMA_HOST=http://localhost:11434 # OLLAMA_HOST=http://localhost:11434
# OLLAMA_MODEL=llava:7b # OLLAMA_MODEL=llava:7b
# GPU Server — routes vision/LLM tasks to a cf-orch coordinator for VRAM management. # CF Orchestrator — routes vision/LLM tasks to a cf-orch coordinator for VRAM management.
# Self-hosted: point at a local cf-orch coordinator if you have one running. # Self-hosted: point at a local cf-orch coordinator if you have one running.
# Cloud (internal): managed coordinator at orch.circuitforge.tech. # Cloud (internal): managed coordinator at orch.circuitforge.tech.
# Leave unset to run vision tasks inline (no VRAM coordination). # Leave unset to run vision tasks inline (no VRAM coordination).
# GPU_SERVER_URL=http://10.1.10.71:7700 # CF_ORCH_URL=http://10.1.10.71:7700
#
# CF_ORCH_URL is accepted as a backward-compat alias for GPU_SERVER_URL.
# #
# cf-orch agent (compose --profile orch) — coordinator URL for the sidecar agent. # cf-orch agent (compose --profile orch) — coordinator URL for the sidecar agent.
# Defaults to GPU_SERVER_URL if unset. # Defaults to CF_ORCH_URL if unset.
# CF_ORCH_COORDINATOR_URL=http://10.1.10.71:7700 # CF_ORCH_COORDINATOR_URL=http://10.1.10.71:7700
# ── Shared Postgres (optional — strongly recommended for cloud/multi-user) ────
# When set, sellers, market_comps, reported_sellers, and scammer_blocklist are
# stored in Postgres instead of SQLite. Required to avoid database-locked errors
# under concurrent load (>10 simultaneous search users).
# Cloud instances: set to the cf-postgres DSN. Self-hosted: leave unset for SQLite.
# SNIPE_SHARED_DB_URL=postgresql://snipe:<password>@localhost:5432/snipe_shared
# ── Community DB (optional) ────────────────────────────────────────────────── # ── Community DB (optional) ──────────────────────────────────────────────────
# When set, seller trust signals (confirmed scammers added to blocklist) are # When set, seller trust signals (confirmed scammers added to blocklist) are
# published to the shared community PostgreSQL for cross-user signal aggregation. # published to the shared community PostgreSQL for cross-user signal aggregation.

View file

@ -5,7 +5,6 @@ WORKDIR /app
# System deps for Playwright/Chromium # System deps for Playwright/Chromium
RUN apt-get update && apt-get install -y --no-install-recommends \ RUN apt-get update && apt-get install -y --no-install-recommends \
xvfb \ xvfb \
libpq-dev \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
# Install circuitforge-core from sibling directory (compose sets context: ..) # Install circuitforge-core from sibling directory (compose sets context: ..)

314
README.md
View file

@ -1,87 +1,29 @@
<!-- Logo coming soon — replace docs/snipe-logo.svg when final icon ships --> # Snipe — Auction Sniping & Listing Intelligence
<div align="center">
<img src="docs/snipe-logo.svg" alt="Snipe logo" width="120" />
# Snipe > *Part of the Circuit Forge LLC "AI for the tasks you hate most" suite.*
**Auction intelligence and sniping for people who don't trust the platform.** **Status:** Active — eBay listing intelligence MVP complete (search, trust scoring, affiliate links, feedback FAB, vision task scheduling). Auction sniping engine and multi-platform support are next.
[![License: MIT / BSL 1.1](https://img.shields.io/badge/license-MIT%20%2F%20BSL%201.1-blue)](LICENSE) **[Documentation](https://docs.circuitforge.tech/snipe/)** · [circuitforge.tech](https://circuitforge.tech)
[![Status: Beta](https://img.shields.io/badge/status-beta-yellow)]()
[![Version](https://img.shields.io/badge/version-0.5.1-green)](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/releases)
[![Forgejo](https://img.shields.io/badge/primary%20repo-Forgejo-orange)](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe)
[![Docs](https://img.shields.io/badge/docs-docs.circuitforge.tech%2Fsnipe-green)](https://docs.circuitforge.tech/snipe)
*Part of the Circuit Forge LLC suite — "AI for the tasks the system made hard on purpose."* ## Quick install (self-hosted)
</div>
--- **Requirements:** Docker with Compose plugin, Git. No API keys needed to get started.
<table>
<tr>
<td><img src="docs/screenshots/hero.png" alt="Snipe search page with filter panel and feature overview"/></td>
<td><img src="docs/screenshots/results.png" alt="Search results — trust score badges, STEAL price flags, seller feedback, and market price comparison"/></td>
</tr>
</table>
---
## Why Snipe?
Auction platforms are designed to make you act fast and trust blindly. The closing countdown, the hidden price history, the new-account seller with one feedback — all of it is structured against the buyer.
Snipe inverts that. Before you place a bid, you get a trust score built from five independently sourced signals: seller account age, feedback volume, feedback ratio, price versus recent completed sales, and category history. A hard-coded red flag for new accounts or bad actors overrides the composite. Soft flags surface buried damage disclosures, duplicate photos, and listings that have been sitting unsold for weeks. When the listing is priced well below market, you see a STEAL badge — sourced from eBay Marketplace Insights, not from the seller's description.
The sniping engine — precise last-second bid submission with NTP (network time protocol) synchronization and soft-close handling — is next on the roadmap. The intelligence layer is live now.
---
## Features
### Listing intelligence (live)
- **Trust scoring** — five-signal composite score (0100) per listing: account age, feedback count, feedback ratio, price vs. market, category history
- **Red flag detection** — hard flags for new accounts and established bad actors; soft flags for damage keywords, evasive language, duplicate photos, long-on-market listings, and significant price drops
- **Price vs. market** — listing price compared against completed-sale medians via eBay Marketplace Insights API (Browse API fallback)
- **Keyword filtering** — must-include (AND / ANY / OR-groups), must-exclude, category, price range; OR-groups expand into multiple targeted queries so eBay relevance doesn't silently drop variants
- **Saved searches** — one-click re-run that restores all filter settings
- **Background enrichment** — seller account age scraped via Playwright + Xvfb (Kasada/Cloudflare-safe headed Chromium); on-demand re-score per listing without re-searching
- **LLM query builder** — describe what you want in plain language; an LLM builds the search terms (paid tier)
- **Vision photo assessment** — condition scoring from listing photos via moondream2 locally or Claude vision (paid/cloud); VRAM-aware scheduling via circuitforge-core task scheduler
- **Affiliate link builder** — eBay Partner Network wrapping with user BYOK support and per-retailer disclosure
### Platforms
| Platform | Search | Trust scoring | Completed-sale comps |
|----------|--------|---------------|----------------------|
| **eBay** | Browse API + Playwright fallback | All 5 signals | Marketplace Insights + Browse fallback |
| **Mercari** | Playwright scraper | 3/5 signals (partial) | Phase 3 |
| CT Bids, HiBid, AuctionZip, Invaluable, GovPlanet, Bidsquare, Proxibid | Planned | Planned | Planned |
### Auction sniping engine (roadmap)
- NTP-synchronized last-second bid submission
- Soft-close detection and strategy adjustment
- Proxy bid ladder with configurable max
- Human approval gate before any bid executes
- Post-win workflow: payment routing, shipping coordination, provenance documentation
---
## Quick Start
**Requirements:** Docker with Compose plugin, Git. No API keys required to get started.
```bash ```bash
# One-line install — clones to ~/snipe by default # One-line install — clones to ~/snipe by default
bash <(curl -fsSL https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/raw/branch/main/install.sh) bash <(curl -fsSL https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/raw/branch/main/install.sh)
# Or clone manually and run the script:
git clone https://git.opensourcesolarpunk.com/Circuit-Forge/snipe.git
bash snipe/install.sh
``` ```
Then open **http://localhost:8509**. Then open **http://localhost:8509**.
### Manual setup ### Manual setup (if you prefer)
Snipe's API image builds from a parent context that includes `circuitforge-core`. Both repos must sit as siblings: Snipe's API image is built from a parent context that includes `circuitforge-core`. Both repos must sit as siblings in the same directory:
``` ```
workspace/ workspace/
@ -94,88 +36,236 @@ mkdir snipe-workspace && cd snipe-workspace
git clone https://git.opensourcesolarpunk.com/Circuit-Forge/snipe.git git clone https://git.opensourcesolarpunk.com/Circuit-Forge/snipe.git
git clone https://git.opensourcesolarpunk.com/Circuit-Forge/circuitforge-core.git git clone https://git.opensourcesolarpunk.com/Circuit-Forge/circuitforge-core.git
cd snipe cd snipe
cp .env.example .env # add eBay API credentials if you have them (optional) cp .env.example .env # edit if you have eBay API credentials (optional)
./manage.sh start ./manage.sh start
``` ```
### Optional: eBay API credentials ### Optional: eBay API credentials
Snipe works without credentials using its Playwright scraper fallback. Adding credentials unlocks faster searches and inline seller account age without an extra scrape: Snipe works without any credentials using its Playwright scraper fallback. Adding eBay API credentials unlocks faster searches and inline seller account age (no extra scrape needed):
1. Register at [developer.ebay.com](https://developer.ebay.com/my/keys) 1. Register at [developer.ebay.com](https://developer.ebay.com/my/keys)
2. Copy your Production **App ID** and **Cert ID** into `.env` 2. Copy your Production **App ID** and **Cert ID** into `.env`
3. `./manage.sh restart` 3. Restart: `./manage.sh restart`
--- ---
## Tiers ## What it does
| Tier | What you get | Snipe has two layers that work together:
|------|-------------|
| **Free** | eBay + Mercari search, full trust scoring, keyword filtering, saved searches — local LLM only |
| **Paid** | LLM query builder, background saved-search monitoring with alerts, cloud LLM option |
| **Premium** | Vision photo condition assessment, fine-tuned trust models, multi-user |
| **Ultra** | Human-in-the-loop operator — handles CAPTCHAs, phone calls, anything automation can't |
License key format: `CFG-SNPE-XXXX-XXXX-XXXX` **Layer 1 — Listing intelligence (MVP, implemented)**
Before you bid, Snipe tells you whether a listing is worth your time. It fetches eBay listings, scores each seller's trustworthiness across five signals, flags suspicious pricing relative to completed sales, and surfaces red flags like new accounts, cosmetic damage buried in titles, and listings that have been sitting unsold for weeks.
**Layer 2 — Auction sniping (roadmap)**
Snipe manages the bid itself: monitors listings across platforms, schedules last-second bids, handles soft-close extensions, and guides you through the post-win logistics (payment routing, shipping coordination, provenance documentation for antiques).
The name is the origin of the word "sniping" — common snipes are notoriously elusive birds, secretive and camouflaged, that flush suddenly from cover. Shooting one required extreme patience, stillness, and a precise last-second shot. That's the auction strategy.
--- ---
## Screenshots
**Landing page — no account required**
![Snipe landing hero showing search bar and three feature tiles: Seller trust score, Price vs. market, Red flag detection](docs/screenshots/01-hero.png)
**Search results with trust scores**
![Search results for vintage film camera listings, each card showing a trust score badge, seller feedback, price, and market comparison](docs/screenshots/02-results.png)
**STEAL badge — price significantly below market**
![Listing cards with STEAL badge highlighting listings priced well below completed sales median](docs/screenshots/03-steal-badge.png)
> Red flag and Triple Red screenshots coming — captured opportunistically from real scammy listings.
---
## Implemented: eBay Listing Intelligence
### Search & filtering
- Full-text eBay search via Browse API (with Playwright scraper fallback when no API credentials configured)
- Price range, must-include keywords (AND / ANY / OR-groups mode), must-exclude terms, eBay category filter
- OR-group mode expands keyword combinations into multiple targeted queries and deduplicates results — eBay relevance won't silently drop variants
- Pages-to-fetch control: each Browse API page returns up to 200 listings
- Saved searches with one-click re-run that restores all filter settings
### Seller trust scoring
Five signals, each scored 020, composited to 0100:
| Signal | What it measures |
|--------|-----------------|
| `account_age` | Days since eBay account registration |
| `feedback_count` | Total feedback received |
| `feedback_ratio` | Positive feedback percentage |
| `price_vs_market` | Listing price vs. median of recent completed sales |
| `category_history` | Whether seller has history selling in this category |
Scores are marked **partial** when signals are unavailable (e.g. account age not yet enriched). Partial scores are displayed with a visual indicator rather than penalizing the seller for missing data.
### Red flags
Hard filters that override the composite score:
- `new_account` — account registered within 7 days
- `established_bad_actor` — feedback ratio < 80% with 20+ reviews
Soft flags surfaced as warnings:
- `account_under_30_days` — account under 30 days old
- `low_feedback_count` — fewer than 10 reviews
- `suspicious_price` — listing price below 50% of market median *(suppressed automatically when the search returns a heterogeneous price distribution — e.g. mixed laptop generations — to prevent false positives)*
- `duplicate_photo` — same image found on another listing (perceptual hash)
- `scratch_dent_mentioned` — title keywords indicating cosmetic damage, functional problems, or evasive language (see below)
- `long_on_market` — listing has been seen 5+ times over 14+ days without selling
- `significant_price_drop` — current price more than 20% below first-seen price
### Scratch & dent title detection
Scans listing titles for signals the item may have undisclosed damage or problems:
- **Explicit damage**: scratch, scuff, dent, crack, chip, blemish, worn
- **Condition catch-alls**: as is, for parts, parts only, spares or repair
- **Evasive redirects**: "see description", "read description", "see photos for" (seller hiding damage detail in listing body)
- **Functional problems**: "not working", "stopped working", "no power", "dead on arrival", "powers on but", "faulty", "broken screen/hinge/port"
- **DIY/repair listings**: "needs repair", "needs tlc", "project laptop", "for repair", "sold as is"
### Seller enrichment
- **Inline (API adapter)**: account age filled from Browse API `registrationDate` field
- **Background (scraper)**: `/itm/` listing pages scraped for seller "Joined" date via Playwright + Xvfb (Kasada-safe headed Chromium)
- **On-demand**: ↻ button on any listing card triggers `POST /api/enrich` — runs enrichment and re-scores without waiting for a second search
- **Category history**: derived from the seller's accumulated listing data (Browse API `categories` field); improves with every search, no extra API calls
### Affiliate link builder
Listing cards surface eBay affiliate-wrapped URLs. Uses `circuitforge_core.affiliates.wrap_url` — resolution order: user opted out → plain URL; user has BYOK affiliate ID → their ID; CF env var set (`EBAY_AFFILIATE_ID`) → CF's ID; otherwise plain URL. Users can configure their own eBay Partner Network ID or opt out entirely in Settings.
Disclosure tooltip appears on first encounter per-session and on each wrapped link (per-retailer copy from `get_disclosure_text`).
### Feedback FAB
In-app feedback button (bottom-right FAB) opens a modal: title, description, optional screenshot. Posts to the CF feedback endpoint. Status probed on load; FAB hidden if endpoint unreachable.
### Vision task scheduling
Photo condition assessment tasks queued through `circuitforge_core.tasks.TaskScheduler` — VRAM-aware slot management shared with any other LLM workloads on the same host. Runs moondream2 locally (free tier) or Claude vision (paid/cloud). Results stored per-listing and update the trust score card.
### Market price comparison
Completed sales fetched via eBay Marketplace Insights API (with Browse API fallback for app tiers that don't have Insights access). Median stored per query hash, used to score `price_vs_market` across all listings in a search.
### Adapters
| Adapter | When used | Signals available |
|---------|-----------|-------------------|
| Browse API (`api`) | eBay API credentials configured | All signals; account age inline |
| Playwright scraper (`scraper`) | No credentials / forced | All signals except account age (async BTF enrichment) |
| `auto` (default) | — | API if credentials present, scraper otherwise |
---
## Stack
| Layer | Tech | Port |
|-------|------|------|
| Frontend | Vue 3 + Pinia + UnoCSS + Vite (nginx) | 8509 |
| API | FastAPI (uvicorn) | 8510 |
| Scraper | Playwright + playwright-stealth + Xvfb | — |
| DB | SQLite (`data/snipe.db`) | — |
| Core | circuitforge-core (editable install) | — |
## Running ## Running
```bash ```bash
./manage.sh start # start all services ./manage.sh start # start all services
./manage.sh stop # stop ./manage.sh stop # stop
./manage.sh restart # restart
./manage.sh logs # tail logs ./manage.sh logs # tail logs
./manage.sh open # open in browser ./manage.sh open # open in browser
``` ```
--- Cloud stack (shared DB, multi-user):
```bash
## Stack docker compose -f compose.cloud.yml -p snipe-cloud up -d
docker compose -f compose.cloud.yml -p snipe-cloud build api # after Python changes
| Layer | Technology | Port | ```
|-------|-----------|------|
| Frontend | Vue 3 + Pinia + UnoCSS + Vite (served via nginx) | 8509 |
| API | FastAPI (uvicorn) | 8510 |
| Scraper | Playwright + playwright-stealth + Xvfb (Kasada/Cloudflare-safe headed Chromium) | — |
| Database | SQLite (`data/snipe.db`) | — |
| Core | circuitforge-core (editable install) | — |
The scraper stack uses headed Chromium via Xvfb (X virtual framebuffer) with playwright-stealth for all platform access. Headless and `requests`-based approaches are blocked by eBay and Mercari.
--- ---
## Documentation ## Roadmap
Full documentation at **[docs.circuitforge.tech/snipe](https://docs.circuitforge.tech/snipe)** — setup guide, trust scoring algorithm, platform adapter reference, API docs, and self-hosting notes. ### Near-term (eBay)
| Issue | Feature |
|-------|---------|
| [#1](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/1) | SSE/WebSocket live score push — enriched data appears without re-search |
| [#2](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/2) | eBay OAuth (Connect eBay Account) for full trust score access via Trading API |
| [#4](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/4) | Scammer database: community blocklist + batch eBay Trust & Safety reporting |
| [#5](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/5) | UPC/product lookup → LLM-crafted search terms (paid tier) |
| [#8](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/8) | "Triple Red" easter egg: CSS animation when all hard flags fire simultaneously |
| [#11](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/11) | Vision-based photo condition assessment — moondream2 (local) / Claude vision (cloud, paid) |
| [#12](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/12) | Background saved-search monitoring with configurable alerts |
### Cloud / infrastructure
| Issue | Feature |
|-------|---------|
| [#6](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/6) | Shared seller/scammer/comps DB across cloud users (public data, no re-scraping) |
| [#7](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/7) | Shared image hash DB — requires explicit opt-in consent (CF privacy-by-architecture) |
### Auction sniping engine
| Issue | Feature |
|-------|---------|
| [#9](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/9) | Bid scheduling + snipe execution (NTP-synchronized, soft-close handling, human approval gate) |
| [#13](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/13) | Post-win workflow: payment routing, shipping coordination, provenance documentation |
### Multi-platform expansion
| Issue | Feature |
|-------|---------|
| [#10](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe/issues/10) | CT Bids, HiBid, AuctionZip, Invaluable, GovPlanet, Bidsquare, Proxibid |
--- ---
## Forgejo-primary ## Primary platforms (full vision)
Snipe is developed and maintained on Forgejo at [git.opensourcesolarpunk.com/Circuit-Forge/snipe](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe). GitHub and Codeberg are read-only mirrors. File issues and submit pull requests on Forgejo. - **eBay** — general + collectibles *(search + trust scoring: implemented)*
- **CT Bids** — Connecticut state surplus and municipal auctions
- **GovPlanet / IronPlanet** — government surplus equipment
- **AuctionZip** — antique auction house aggregator (1,000+ houses)
- **Invaluable / LiveAuctioneers** — fine art and antiques
- **Bidsquare** — antiques and collectibles
- **HiBid** — estate auctions
- **Proxibid** — industrial and collector auctions
--- ## Why auctions are hard
## Contributing Online auctions are frustrating because:
- Winning requires being present at the exact closing moment — sometimes 2 AM
- Platforms vary wildly: some allow proxy bids, some don't; closing times extend on activity
- Scammers exploit auction urgency — new accounts, stolen photos, pressure to pay outside platform
- Price history is hidden — you don't know if an item is underpriced or a trap
- Sellers hide damage in descriptions rather than titles to avoid automated filters
- Shipping logistics for large / fragile antiques require coordination with the auction house
- Provenance documentation is inconsistent across auction houses
Bug reports and feature requests: open an issue on Forgejo. The discovery pipeline (scrapers, adapters, signal extraction) is MIT-licensed — pull requests welcome. AI trust-scoring features are BSL 1.1 — contributions are accepted but the license terms apply. ## Bidding strategy engine (planned)
--- - **Hard snipe**: submit bid N seconds before close (default: 8s)
- **Soft-close handling**: detect if platform extends on last-minute bids; adjust strategy
- **Proxy ladder**: set max and let the engine bid in increments, reserve snipe for final window
- **Reserve detection**: identify likely reserve price from bid history patterns
- **Comparable sales**: pull recent auction results for same/similar items across platforms
## License ## Post-win workflow (planned)
Snipe uses a dual license: 1. Payment method routing (platform-specific: CC, wire, check)
2. Shipping quote requests to approved carriers (freight / large items via uShip; parcel via FedEx/UPS)
3. Condition report request from auction house
4. Provenance packet generation (for antiques / fine art resale or insurance)
5. Add to inventory (for dealers / collectors tracking portfolio value)
| Component | License | ## Product code (license key)
|-----------|---------|
| Discovery pipeline — scrapers, platform adapters, search, keyword filtering | [MIT](LICENSE-MIT) |
| LLM trust-scoring, query builder, vision assessment, AI features | [BSL 1.1](LICENSE-BSL) — free for personal non-commercial self-hosting; commercial use requires a paid license; converts to MIT after 4 years |
Humans own design, architecture, code review, testing, and verification. LLMs are part of our development workflow. [Our positions on LLM use →](https://circuitforge.tech/positions) `CFG-SNPE-XXXX-XXXX-XXXX`
Privacy · Safety · Accessibility — co-equal, non-negotiable. ## Tech notes
[circuitforge.tech](https://circuitforge.tech) - Shared `circuitforge-core` scaffold (DB, LLM router, tier system, config)
- Platform adapters: currently eBay only; AuctionZip, Invaluable, HiBid, CT Bids planned (Playwright + API where available)
- Bid execution: Playwright automation with precise timing (NTP-synchronized)
- Soft-close detection: platform-specific rules engine
- Comparable sales: eBay completed listings via Marketplace Insights API + Browse API fallback
- Vision module: condition assessment from listing photos — moondream2 / Claude vision (paid tier stub in `app/trust/photo.py`)
- **Kasada bypass**: headed Chromium via Xvfb; all scraping uses this path — headless and `requests`-based approaches are blocked by eBay

View file

@ -24,7 +24,7 @@ from circuitforge_core.affiliates import wrap_url as _wrap_affiliate_url
from circuitforge_core.api import make_corrections_router as _make_corrections_router from circuitforge_core.api import make_corrections_router as _make_corrections_router
from circuitforge_core.api import make_feedback_router as _make_feedback_router from circuitforge_core.api import make_feedback_router as _make_feedback_router
from circuitforge_core.config import load_env from circuitforge_core.config import load_env
from fastapi import Depends, FastAPI, File, HTTPException, Query, Request, Response, UploadFile from fastapi import Depends, FastAPI, File, HTTPException, Request, Response, UploadFile
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import StreamingResponse from fastapi.responses import StreamingResponse
from pydantic import BaseModel from pydantic import BaseModel
@ -33,9 +33,8 @@ from api.cloud_session import CloudUser, compute_features, get_session
from api.ebay_webhook import router as ebay_webhook_router from api.ebay_webhook import router as ebay_webhook_router
from app.db.models import SavedSearch as SavedSearchModel from app.db.models import SavedSearch as SavedSearchModel
from app.db.models import ScammerEntry from app.db.models import ScammerEntry
from app.db.protocol import SharedTableProtocol
from app.db.store import Store from app.db.store import Store
from app.platforms import SUPPORTED_PLATFORMS, SearchFilters from app.platforms import SearchFilters
from app.platforms.ebay.adapter import EbayAdapter from app.platforms.ebay.adapter import EbayAdapter
from app.platforms.ebay.auth import EbayTokenManager from app.platforms.ebay.auth import EbayTokenManager
from app.platforms.ebay.query_builder import expand_queries, parse_groups from app.platforms.ebay.query_builder import expand_queries, parse_groups
@ -143,19 +142,6 @@ def _get_community_store() -> "SnipeCommunityStore | None":
return _community_store return _community_store
# ── Shared Postgres backend (optional — active when SNIPE_SHARED_DB_URL is set) ─
# Replaces the SQLite shared.db for sellers, market_comps, reported_sellers, and
# scammer_blocklist. ThreadedConnectionPool is thread-safe; one instance per process.
_pg_shared_store: "SharedTableProtocol | None" = None
def _make_shared_store(path: Path) -> SharedTableProtocol:
"""Return the active shared backend — Postgres if configured, SQLite otherwise."""
if _pg_shared_store is not None:
return _pg_shared_store
return Store(path)
# ── LLM Query Builder singletons (optional — requires LLM backend) ──────────── # ── LLM Query Builder singletons (optional — requires LLM backend) ────────────
_category_cache = None _category_cache = None
_query_translator = None _query_translator = None
@ -167,7 +153,7 @@ def _get_query_translator():
@asynccontextmanager @asynccontextmanager
async def _lifespan(app: FastAPI): async def _lifespan(app: FastAPI):
global _community_store, _pg_shared_store global _community_store
# Pre-warm the Chromium browser pool so the first scrape request does not # Pre-warm the Chromium browser pool so the first scrape request does not
# pay the full cold-start cost (5-10s Xvfb + browser launch). # pay the full cold-start cost (5-10s Xvfb + browser launch).
# Pool size is controlled via BROWSER_POOL_SIZE env var (default: 2). # Pool size is controlled via BROWSER_POOL_SIZE env var (default: 2).
@ -192,21 +178,6 @@ async def _lifespan(app: FastAPI):
get_scheduler(sched_db) get_scheduler(sched_db)
log.info("Snipe task scheduler started (db=%s)", sched_db) log.info("Snipe task scheduler started (db=%s)", sched_db)
# Shared Postgres backend — optional. Replaces SQLite for sellers, market_comps,
# reported_sellers, and scammer_blocklist under concurrent load.
snipe_shared_dsn = os.environ.get("SNIPE_SHARED_DB_URL", "")
if snipe_shared_dsn:
try:
from app.db.pg_shared import SnipeSharedDB, SnipeSharedStore as _SnipeSharedStore
_pg_db = SnipeSharedDB(snipe_shared_dsn)
_pg_db.run_migrations()
_pg_shared_store = _SnipeSharedStore(_pg_db)
log.info("Shared Postgres backend ready (sellers, market_comps, blocklist)")
except Exception:
log.exception(
"SNIPE_SHARED_DB_URL set but Postgres init failed — falling back to SQLite"
)
# Community DB — optional. Skipped gracefully if COMMUNITY_DB_URL is unset. # Community DB — optional. Skipped gracefully if COMMUNITY_DB_URL is unset.
community_db_url = os.environ.get("COMMUNITY_DB_URL", "") community_db_url = os.environ.get("COMMUNITY_DB_URL", "")
if community_db_url: if community_db_url:
@ -238,21 +209,13 @@ async def _lifespan(app: FastAPI):
_category_cache.refresh(token_manager=None) # bootstrap fallback _category_cache.refresh(token_manager=None) # bootstrap fallback
try: try:
cforch_url = os.getenv("GPU_SERVER_URL") or os.getenv("CF_ORCH_URL") or None
if cforch_url:
_query_translator = QueryTranslator(
category_cache=_category_cache,
cforch_url=cforch_url,
)
log.info("LLM query builder ready (cf-orch).")
else:
from app.llm.router import LLMRouter from app.llm.router import LLMRouter
_llm_router = LLMRouter() _llm_router = LLMRouter()
_query_translator = QueryTranslator( _query_translator = QueryTranslator(
category_cache=_category_cache, category_cache=_category_cache,
llm_router=_llm_router, llm_router=_llm_router,
) )
log.info("LLM query builder ready (local LLM).") log.info("LLM query builder ready.")
except Exception: except Exception:
log.info("No LLM backend configured — query builder disabled.") log.info("No LLM backend configured — query builder disabled.")
except Exception: except Exception:
@ -475,7 +438,7 @@ def session_info(response: Response, session: CloudUser = Depends(get_session)):
def _trigger_scraper_enrichment( def _trigger_scraper_enrichment(
listings: list, listings: list,
shared_store: SharedTableProtocol, shared_store: Store,
shared_db: Path, shared_db: Path,
user_db: Path | None = None, user_db: Path | None = None,
query: str = "", query: str = "",
@ -541,7 +504,7 @@ def _trigger_scraper_enrichment(
if not session_id or session_id not in _update_queues: if not session_id or session_id not in _update_queues:
return return
q = _update_queues[session_id] q = _update_queues[session_id]
thread_shared = shared_store.clone() thread_shared = Store(shared_db)
thread_user = Store(user_db or shared_db) thread_user = Store(user_db or shared_db)
scorer = TrustScorer(thread_shared) scorer = TrustScorer(thread_shared)
comp = thread_shared.get_market_comp("ebay", hashlib.md5(query.encode()).hexdigest()) comp = thread_shared.get_market_comp("ebay", hashlib.md5(query.encode()).hexdigest())
@ -567,7 +530,7 @@ def _trigger_scraper_enrichment(
def _run(): def _run():
try: try:
enricher = ScrapedEbayAdapter(shared_store.clone()) enricher = ScrapedEbayAdapter(Store(shared_db))
if needs_btf: if needs_btf:
enricher.enrich_sellers_btf(needs_btf, max_workers=2) enricher.enrich_sellers_btf(needs_btf, max_workers=2)
log.info("BTF enrichment complete for %d sellers", len(needs_btf)) log.info("BTF enrichment complete for %d sellers", len(needs_btf))
@ -701,10 +664,10 @@ def _try_trading_api_enrichment(
return enriched return enriched
def _make_adapter(shared_store: Store, force: str = "auto", platform: str = "ebay"): def _make_adapter(shared_store: Store, force: str = "auto"):
"""Return the appropriate adapter for the given platform. """Return the appropriate adapter.
force: "auto" | "api" | "scraper" (ignored for non-eBay platforms) force: "auto" | "api" | "scraper"
auto API if creds present, else scraper auto API if creds present, else scraper
api Browse API (raises if no creds) api Browse API (raises if no creds)
scraper Playwright scraper regardless of creds scraper Playwright scraper regardless of creds
@ -712,11 +675,6 @@ def _make_adapter(shared_store: Store, force: str = "auto", platform: str = "eba
Adapters receive shared_store because they only read/write sellers and Adapters receive shared_store because they only read/write sellers and
market_comps never listings. Listings are returned and saved by the caller. market_comps never listings. Listings are returned and saved by the caller.
""" """
if platform == "mercari":
from app.platforms.mercari import MercariAdapter
return MercariAdapter(shared_store)
# eBay
client_id, client_secret, env = _ebay_creds() client_id, client_secret, env = _ebay_creds()
has_creds = bool(client_id and client_secret) has_creds = bool(client_id and client_secret)
@ -733,10 +691,8 @@ def _make_adapter(shared_store: Store, force: str = "auto", platform: str = "eba
return ScrapedEbayAdapter(shared_store) return ScrapedEbayAdapter(shared_store)
def _adapter_name(force: str = "auto", platform: str = "ebay") -> str: def _adapter_name(force: str = "auto") -> str:
"""Return the name of the adapter that would be used — without creating it.""" """Return the name of the adapter that would be used — without creating it."""
if platform != "ebay":
return platform
client_id, client_secret, _ = _ebay_creds() client_id, client_secret, _ = _ebay_creds()
if force == "scraper": if force == "scraper":
return "scraper" return "scraper"
@ -757,15 +713,8 @@ def search(
category_id: str = "", # eBay category ID — forwarded to Browse API / scraper _sacat category_id: str = "", # eBay category ID — forwarded to Browse API / scraper _sacat
adapter: str = "auto", # "auto" | "api" | "scraper" — override adapter selection adapter: str = "auto", # "auto" | "api" | "scraper" — override adapter selection
refresh: bool = False, # when True, bypass cache read (still writes fresh result) refresh: bool = False, # when True, bypass cache read (still writes fresh result)
platform: str = Query("ebay", description="Marketplace platform to search"),
session: CloudUser = Depends(get_session), session: CloudUser = Depends(get_session),
): ):
if platform not in SUPPORTED_PLATFORMS:
raise HTTPException(
status_code=422,
detail=f"Platform {platform!r} is not yet supported. Supported: {sorted(SUPPORTED_PLATFORMS)}",
)
# If the user pasted an eBay listing or checkout URL, extract the item ID # If the user pasted an eBay listing or checkout URL, extract the item ID
# and use it as the search query so the exact item surfaces in results. # and use it as the search query so the exact item surfaces in results.
ebay_item_id = _extract_ebay_item_id(q) ebay_item_id = _extract_ebay_item_id(q)
@ -774,7 +723,7 @@ def search(
q = ebay_item_id q = ebay_item_id
if not q.strip(): if not q.strip():
return {"listings": [], "trust_scores": {}, "sellers": {}, "market_price": None, "adapter_used": _adapter_name(adapter, platform=platform)} return {"listings": [], "trust_scores": {}, "sellers": {}, "market_price": None, "adapter_used": _adapter_name(adapter)}
# Cap pages to the tier's maximum — free cloud users get 1 page, local gets unlimited. # Cap pages to the tier's maximum — free cloud users get 1 page, local gets unlimited.
features = compute_features(session.tier) features = compute_features(session.tier)
@ -782,8 +731,9 @@ def search(
must_exclude_terms = _parse_terms(must_exclude) must_exclude_terms = _parse_terms(must_exclude)
# OR-group expansion is eBay-specific; other platforms use the base query directly. # In Groups mode, expand OR groups into multiple targeted eBay queries to
if platform == "ebay" and must_include_mode == "groups" and must_include.strip(): # guarantee comprehensive result coverage — eBay relevance won't silently drop variants.
if must_include_mode == "groups" and must_include.strip():
or_groups = parse_groups(must_include) or_groups = parse_groups(must_include)
ebay_queries = expand_queries(q, or_groups) ebay_queries = expand_queries(q, or_groups)
else: else:
@ -810,7 +760,7 @@ def search(
category_id=category_id.strip() or None, category_id=category_id.strip() or None,
) )
adapter_used = _adapter_name(adapter, platform=platform) adapter_used = _adapter_name(adapter)
shared_db = session.shared_db shared_db = session.shared_db
user_db = session.user_db user_db = session.user_db
@ -841,7 +791,7 @@ def search(
_update_queues[session_id] = _queue.SimpleQueue() _update_queues[session_id] = _queue.SimpleQueue()
try: try:
shared_store = _make_shared_store(shared_db) shared_store = Store(shared_db)
user_store = Store(user_db) user_store = Store(user_db)
# Re-hydrate Listing dataclass instances from the cached dicts so the # Re-hydrate Listing dataclass instances from the cached dicts so the
@ -870,11 +820,11 @@ def search(
} }
seller_map = { seller_map = {
listing.seller_platform_id: dataclasses.asdict( listing.seller_platform_id: dataclasses.asdict(
shared_store.get_seller(platform, listing.seller_platform_id) shared_store.get_seller("ebay", listing.seller_platform_id)
) )
for listing in listings for listing in listings
if listing.seller_platform_id if listing.seller_platform_id
and shared_store.get_seller(platform, listing.seller_platform_id) and shared_store.get_seller("ebay", listing.seller_platform_id)
} }
_is_unauthed = session.user_id == "anonymous" or session.user_id.startswith("guest:") _is_unauthed = session.user_id == "anonymous" or session.user_id.startswith("guest:")
@ -926,14 +876,13 @@ def search(
_evict_expired_cache() _evict_expired_cache()
log.info("cache: miss key=%s q=%r", cache_key, q) log.info("cache: miss key=%s q=%r", cache_key, q)
# Each thread creates its own store via clone() — sqlite3 check_same_thread=True; # Each thread creates its own Store — sqlite3 check_same_thread=True.
# SnipeSharedStore.clone() returns self (ThreadedConnectionPool is thread-safe).
def _run_search(ebay_query: str) -> list: def _run_search(ebay_query: str) -> list:
return _make_adapter(_make_shared_store(shared_db), adapter, platform=platform).search(ebay_query, base_filters) return _make_adapter(Store(shared_db), adapter).search(ebay_query, base_filters)
def _run_comps() -> None: def _run_comps() -> None:
try: try:
_make_adapter(_make_shared_store(shared_db), adapter, platform=platform).get_completed_sales(comp_query, pages) _make_adapter(Store(shared_db), adapter).get_completed_sales(comp_query, pages)
except Exception: except Exception:
log.warning("comps: unhandled exception for %r", comp_query, exc_info=True) log.warning("comps: unhandled exception for %r", comp_query, exc_info=True)
@ -960,8 +909,8 @@ def search(
raise HTTPException(status_code=502, detail=f"eBay search failed: {e}") raise HTTPException(status_code=502, detail=f"eBay search failed: {e}")
log.info( log.info(
"search platform=%s auth=%s tier=%s adapter=%s pages=%d queries=%d listings=%d q=%r", "search auth=%s tier=%s adapter=%s pages=%d queries=%d listings=%d q=%r",
platform, _auth_label(session.user_id), session.tier, adapter_used, _auth_label(session.user_id), session.tier, adapter_used,
pages, len(ebay_queries), len(listings), q, pages, len(ebay_queries), len(listings), q,
) )
@ -974,30 +923,33 @@ def search(
_update_queues[session_id] = _queue.SimpleQueue() _update_queues[session_id] = _queue.SimpleQueue()
try: try:
# Main-thread stores — shared_store may be Postgres (sellers, market_comps); # Main-thread stores — fresh connections, same thread.
# user_store is always per-user SQLite (listings, trust_scores, saved_searches). # shared_store: sellers, market_comps (all users share this data)
shared_store = _make_shared_store(shared_db) # user_store: listings, saved_searches (per-user in cloud mode, same file in local mode)
shared_store = Store(shared_db)
user_store = Store(user_db) user_store = Store(user_db)
user_store.save_listings(listings) user_store.save_listings(listings)
# Derive category_history from accumulated listing data — eBay only # Derive category_history from accumulated listing data — free for API adapter
# (category_name comes from Browse API response; other platforms return None). # (category_name comes from Browse API response), no-op for scraper listings (category_name=None).
# Reads listings from user_store, writes seller categories to shared_store.
seller_ids = list({l.seller_platform_id for l in listings if l.seller_platform_id}) seller_ids = list({l.seller_platform_id for l in listings if l.seller_platform_id})
if platform == "ebay":
n_cat = shared_store.refresh_seller_categories("ebay", seller_ids, listing_store=user_store) n_cat = shared_store.refresh_seller_categories("ebay", seller_ids, listing_store=user_store)
if n_cat: if n_cat:
log.info("Category history derived for %d sellers from listing data", n_cat) log.info("Category history derived for %d sellers from listing data", n_cat)
# Re-fetch to hydrate staging fields (times_seen, first_seen_at, id, price_at_first_seen) # Re-fetch to hydrate staging fields (times_seen, first_seen_at, id, price_at_first_seen)
# that are only available from the DB after the upsert. # that are only available from the DB after the upsert.
staged = user_store.get_listings_staged(platform, [l.platform_listing_id for l in listings]) staged = user_store.get_listings_staged("ebay", [l.platform_listing_id for l in listings])
listings = [staged.get(l.platform_listing_id, l) for l in listings] listings = [staged.get(l.platform_listing_id, l) for l in listings]
# Trading API enrichment and BTF scraping are eBay-specific. # Trading API enrichment: if the user has connected their eBay account, use
_main_adapter = _make_adapter(shared_store, adapter, platform=platform) # Trading API GetUser to instantly fill account_age_days for sellers missing it.
trading_api_enriched: set[str] = set() # This is synchronous (~200ms per seller) but only runs for sellers that need
if platform == "ebay": # enrichment — typically a small subset. Sellers resolved here are excluded from
# the slower BTF Playwright background pass.
_main_adapter = _make_adapter(shared_store, adapter)
sellers_needing_age = [ sellers_needing_age = [
l.seller_platform_id for l in listings l.seller_platform_id for l in listings
if l.seller_platform_id if l.seller_platform_id
@ -1011,7 +963,9 @@ def search(
_main_adapter, sellers_needing_age, user_db _main_adapter, sellers_needing_age, user_db
) )
# BTF enrichment: scrape /itm/ pages for sellers still missing account_age_days. # BTF enrichment: scrape /itm/ pages for sellers still missing account_age_days
# after the Trading API pass. Runs in the background so it doesn't delay the
# response. Live score updates are pushed to the pre-registered SSE queue.
_trigger_scraper_enrichment( _trigger_scraper_enrichment(
listings, shared_store, shared_db, listings, shared_store, shared_db,
user_db=user_db, query=comp_query, session_id=session_id, user_db=user_db, query=comp_query, session_id=session_id,
@ -1030,7 +984,7 @@ def search(
_enqueue_vision_tasks(listings, trust_scores_list, session) _enqueue_vision_tasks(listings, trust_scores_list, session)
query_hash = hashlib.md5(comp_query.encode()).hexdigest() query_hash = hashlib.md5(comp_query.encode()).hexdigest()
comp = shared_store.get_market_comp(platform, query_hash) comp = shared_store.get_market_comp("ebay", query_hash)
market_price = comp.median_price if comp else None market_price = comp.median_price if comp else None
# Store raw listings (as dicts) + market_price in cache. # Store raw listings (as dicts) + market_price in cache.
@ -1049,11 +1003,11 @@ def search(
} }
seller_map = { seller_map = {
listing.seller_platform_id: dataclasses.asdict( listing.seller_platform_id: dataclasses.asdict(
shared_store.get_seller(platform, listing.seller_platform_id) shared_store.get_seller("ebay", listing.seller_platform_id)
) )
for listing in listings for listing in listings
if listing.seller_platform_id if listing.seller_platform_id
and shared_store.get_seller(platform, listing.seller_platform_id) and shared_store.get_seller("ebay", listing.seller_platform_id)
} }
# Build a preference reader for affiliate URL wrapping. # Build a preference reader for affiliate URL wrapping.
@ -1119,7 +1073,6 @@ def search_async(
category_id: str = "", category_id: str = "",
adapter: str = "auto", adapter: str = "auto",
refresh: bool = False, # when True, bypass cache read (still writes fresh result) refresh: bool = False, # when True, bypass cache read (still writes fresh result)
platform: str = Query("ebay", description="Marketplace platform to search"),
session: CloudUser = Depends(get_session), session: CloudUser = Depends(get_session),
): ):
"""Async variant of GET /api/search. """Async variant of GET /api/search.
@ -1135,12 +1088,6 @@ def search_async(
"seller": {...}, "market_price": ...} (enrichment updates) "seller": {...}, "market_price": ...} (enrichment updates)
None (sentinel stream finished) None (sentinel stream finished)
""" """
if platform not in SUPPORTED_PLATFORMS:
raise HTTPException(
status_code=422,
detail=f"Platform {platform!r} is not yet supported. Supported: {sorted(SUPPORTED_PLATFORMS)}",
)
# Validate / normalise params — same logic as synchronous endpoint. # Validate / normalise params — same logic as synchronous endpoint.
ebay_item_id = _extract_ebay_item_id(q) ebay_item_id = _extract_ebay_item_id(q)
if ebay_item_id: if ebay_item_id:
@ -1157,7 +1104,7 @@ def search_async(
"trust_scores": {}, "trust_scores": {},
"sellers": {}, "sellers": {},
"market_price": None, "market_price": None,
"adapter_used": _adapter_name(adapter, platform=platform), "adapter_used": _adapter_name(adapter),
"affiliate_active": bool(os.environ.get("EBAY_AFFILIATE_CAMPAIGN_ID", "").strip()), "affiliate_active": bool(os.environ.get("EBAY_AFFILIATE_CAMPAIGN_ID", "").strip()),
}) })
_update_queues[empty_id].put(None) _update_queues[empty_id].put(None)
@ -1186,8 +1133,7 @@ def search_async(
q_norm = q # captured from outer scope q_norm = q # captured from outer scope
must_exclude_terms = _parse_terms(must_exclude) must_exclude_terms = _parse_terms(must_exclude)
# OR-group expansion is eBay-specific; other platforms use the base query directly. if must_include_mode == "groups" and must_include.strip():
if platform == "ebay" and must_include_mode == "groups" and must_include.strip():
or_groups = parse_groups(must_include) or_groups = parse_groups(must_include)
ebay_queries = expand_queries(q_norm, or_groups) ebay_queries = expand_queries(q_norm, or_groups)
else: else:
@ -1209,7 +1155,7 @@ def search_async(
category_id=category_id.strip() or None, category_id=category_id.strip() or None,
) )
adapter_used = _adapter_name(adapter, platform=platform) adapter_used = _adapter_name(adapter)
q_ref = _update_queues.get(session_id) q_ref = _update_queues.get(session_id)
if q_ref is None: if q_ref is None:
return # client disconnected before we even started return # client disconnected before we even started
@ -1236,7 +1182,7 @@ def search_async(
cached_listings_raw = payload["listings"] cached_listings_raw = payload["listings"]
cached_market_price = payload["market_price"] cached_market_price = payload["market_price"]
try: try:
shared_store = _make_shared_store(_shared_db) shared_store = Store(_shared_db)
user_store = Store(_user_db) user_store = Store(_user_db)
listings = [_Listing(**d) for d in cached_listings_raw] listings = [_Listing(**d) for d in cached_listings_raw]
user_store.save_listings(listings) user_store.save_listings(listings)
@ -1316,11 +1262,11 @@ def search_async(
try: try:
def _run_search(ebay_query: str) -> list: def _run_search(ebay_query: str) -> list:
return _make_adapter(_make_shared_store(_shared_db), adapter, platform=platform).search(ebay_query, base_filters) return _make_adapter(Store(_shared_db), adapter).search(ebay_query, base_filters)
def _run_comps() -> None: def _run_comps() -> None:
try: try:
_make_adapter(_make_shared_store(_shared_db), adapter, platform=platform).get_completed_sales(comp_query, pages) _make_adapter(Store(_shared_db), adapter).get_completed_sales(comp_query, pages)
except Exception: except Exception:
log.warning("async comps: unhandled exception for %r", comp_query, exc_info=True) log.warning("async comps: unhandled exception for %r", comp_query, exc_info=True)
@ -1339,27 +1285,24 @@ def search_async(
comps_future.result() comps_future.result()
log.info( log.info(
"async_search platform=%s auth=%s tier=%s adapter=%s pages=%d listings=%d q=%r", "async_search auth=%s tier=%s adapter=%s pages=%d listings=%d q=%r",
platform, _auth_label(_user_id), _tier, adapter_used, pages, len(listings), q_norm, _auth_label(_user_id), _tier, adapter_used, pages, len(listings), q_norm,
) )
shared_store = _make_shared_store(_shared_db) shared_store = Store(_shared_db)
user_store = Store(_user_db) user_store = Store(_user_db)
user_store.save_listings(listings) user_store.save_listings(listings)
seller_ids = list({l.seller_platform_id for l in listings if l.seller_platform_id}) seller_ids = list({l.seller_platform_id for l in listings if l.seller_platform_id})
if platform == "ebay":
n_cat = shared_store.refresh_seller_categories("ebay", seller_ids, listing_store=user_store) n_cat = shared_store.refresh_seller_categories("ebay", seller_ids, listing_store=user_store)
if n_cat: if n_cat:
log.info("async_search: category history derived for %d sellers", n_cat) log.info("async_search: category history derived for %d sellers", n_cat)
staged = user_store.get_listings_staged(platform, [l.platform_listing_id for l in listings]) staged = user_store.get_listings_staged("ebay", [l.platform_listing_id for l in listings])
listings = [staged.get(l.platform_listing_id, l) for l in listings] listings = [staged.get(l.platform_listing_id, l) for l in listings]
_main_adapter = _make_adapter(shared_store, adapter, platform=platform) _main_adapter = _make_adapter(shared_store, adapter)
sellers_needing_age: list[str] = []
if platform == "ebay":
sellers_needing_age = [ sellers_needing_age = [
l.seller_platform_id for l in listings l.seller_platform_id for l in listings
if l.seller_platform_id if l.seller_platform_id
@ -1369,7 +1312,7 @@ def search_async(
seen_set: set[str] = set() seen_set: set[str] = set()
sellers_needing_age = [s for s in sellers_needing_age if not (s in seen_set or seen_set.add(s))] # type: ignore[func-returns-value] sellers_needing_age = [s for s in sellers_needing_age if not (s in seen_set or seen_set.add(s))] # type: ignore[func-returns-value]
# Use a temporary CloudUser-like object for Trading API enrichment (eBay only) # Use a temporary CloudUser-like object for Trading API enrichment
from api.cloud_session import CloudUser as _CloudUser from api.cloud_session import CloudUser as _CloudUser
_session_stub = _CloudUser( _session_stub = _CloudUser(
user_id=_user_id, user_id=_user_id,
@ -1377,8 +1320,6 @@ def search_async(
shared_db=_shared_db, shared_db=_shared_db,
user_db=_user_db, user_db=_user_db,
) )
trading_api_enriched: set[str] = set()
if platform == "ebay":
trading_api_enriched = _try_trading_api_enrichment( trading_api_enriched = _try_trading_api_enrichment(
_main_adapter, sellers_needing_age, _user_db _main_adapter, sellers_needing_age, _user_db
) )
@ -1393,7 +1334,7 @@ def search_async(
_enqueue_vision_tasks(listings, trust_scores_list, _session_stub) _enqueue_vision_tasks(listings, trust_scores_list, _session_stub)
query_hash = _hashlib_local.md5(comp_query.encode()).hexdigest() query_hash = _hashlib_local.md5(comp_query.encode()).hexdigest()
comp = shared_store.get_market_comp(platform, query_hash) comp = shared_store.get_market_comp("ebay", query_hash)
market_price = comp.median_price if comp else None market_price = comp.median_price if comp else None
# Store raw listings + market_price in cache (trust scores excluded). # Store raw listings + market_price in cache (trust scores excluded).
@ -1409,11 +1350,11 @@ def search_async(
} }
seller_map = { seller_map = {
listing.seller_platform_id: dataclasses.asdict( listing.seller_platform_id: dataclasses.asdict(
shared_store.get_seller(platform, listing.seller_platform_id) shared_store.get_seller("ebay", listing.seller_platform_id)
) )
for listing in listings for listing in listings
if listing.seller_platform_id if listing.seller_platform_id
and shared_store.get_seller(platform, listing.seller_platform_id) and shared_store.get_seller("ebay", listing.seller_platform_id)
} }
_is_unauthed = _user_id == "anonymous" or _user_id.startswith("guest:") _is_unauthed = _user_id == "anonymous" or _user_id.startswith("guest:")
@ -1444,17 +1385,12 @@ def search_async(
"session_id": session_id, "session_id": session_id,
}) })
# BTF background enrichment is eBay-specific. # Kick off background enrichment — it pushes "update" events and the sentinel.
if platform == "ebay":
_trigger_scraper_enrichment( _trigger_scraper_enrichment(
listings, shared_store, _shared_db, listings, shared_store, _shared_db,
user_db=_user_db, query=comp_query, session_id=session_id, user_db=_user_db, query=comp_query, session_id=session_id,
skip_seller_ids=trading_api_enriched, skip_seller_ids=trading_api_enriched,
) )
else:
# For non-eBay platforms, push the sentinel directly since there's no
# background enrichment pass.
_push(None)
except _sqlite3.OperationalError as e: except _sqlite3.OperationalError as e:
log.warning("async_search DB contention: %s", e) log.warning("async_search DB contention: %s", e)
@ -1502,7 +1438,7 @@ def enrich_seller(
""" """
import threading import threading
shared_store = _make_shared_store(session.shared_db) shared_store = Store(session.shared_db)
user_store = Store(session.user_db) user_store = Store(session.user_db)
shared_db = session.shared_db shared_db = session.shared_db
@ -1531,7 +1467,7 @@ def enrich_seller(
def _btf(): def _btf():
try: try:
ScrapedEbayAdapter(shared_store.clone()).enrich_sellers_btf( ScrapedEbayAdapter(Store(shared_db)).enrich_sellers_btf(
{seller: listing_id}, max_workers=1 {seller: listing_id}, max_workers=1
) )
except Exception as e: except Exception as e:
@ -1539,7 +1475,7 @@ def enrich_seller(
def _ssn(): def _ssn():
try: try:
ScrapedEbayAdapter(shared_store.clone()).enrich_sellers_categories( ScrapedEbayAdapter(Store(shared_db)).enrich_sellers_categories(
[seller], max_workers=1 [seller], max_workers=1
) )
except Exception as e: except Exception as e:
@ -1810,7 +1746,7 @@ class BlocklistAdd(BaseModel):
@app.get("/api/blocklist") @app.get("/api/blocklist")
def list_blocklist(session: CloudUser = Depends(get_session)): def list_blocklist(session: CloudUser = Depends(get_session)):
store = _make_shared_store(session.shared_db) store = Store(session.shared_db)
return {"entries": [dataclasses.asdict(e) for e in store.list_blocklist()]} return {"entries": [dataclasses.asdict(e) for e in store.list_blocklist()]}
@ -1821,7 +1757,7 @@ def add_to_blocklist(body: BlocklistAdd, session: CloudUser = Depends(get_sessio
status_code=403, status_code=403,
detail="Sign in to report sellers to the community blocklist.", detail="Sign in to report sellers to the community blocklist.",
) )
store = _make_shared_store(session.shared_db) store = Store(session.shared_db)
entry = store.add_to_blocklist(ScammerEntry( entry = store.add_to_blocklist(ScammerEntry(
platform=body.platform, platform=body.platform,
platform_seller_id=body.platform_seller_id, platform_seller_id=body.platform_seller_id,
@ -1855,13 +1791,13 @@ def add_to_blocklist(body: BlocklistAdd, session: CloudUser = Depends(get_sessio
@app.delete("/api/blocklist/{platform_seller_id}", status_code=204) @app.delete("/api/blocklist/{platform_seller_id}", status_code=204)
def remove_from_blocklist(platform_seller_id: str, session: CloudUser = Depends(get_session)): def remove_from_blocklist(platform_seller_id: str, session: CloudUser = Depends(get_session)):
_make_shared_store(session.shared_db).remove_from_blocklist("ebay", platform_seller_id) Store(session.shared_db).remove_from_blocklist("ebay", platform_seller_id)
@app.get("/api/blocklist/export") @app.get("/api/blocklist/export")
def export_blocklist(session: CloudUser = Depends(get_session)): def export_blocklist(session: CloudUser = Depends(get_session)):
"""Download the blocklist as a CSV file.""" """Download the blocklist as a CSV file."""
entries = _make_shared_store(session.shared_db).list_blocklist() entries = Store(session.shared_db).list_blocklist()
buf = io.StringIO() buf = io.StringIO()
writer = csv.writer(buf) writer = csv.writer(buf)
writer.writerow(["platform", "platform_seller_id", "username", "reason", "source", "created_at"]) writer.writerow(["platform", "platform_seller_id", "username", "reason", "source", "created_at"])
@ -1893,7 +1829,7 @@ async def import_blocklist(
except UnicodeDecodeError: except UnicodeDecodeError:
raise HTTPException(status_code=400, detail="File must be UTF-8 encoded") raise HTTPException(status_code=400, detail="File must be UTF-8 encoded")
store = _make_shared_store(session.shared_db) store = Store(session.shared_db)
imported = 0 imported = 0
errors: list[str] = [] errors: list[str] = []
reader = csv.DictReader(io.StringIO(text)) reader = csv.DictReader(io.StringIO(text))
@ -2042,7 +1978,7 @@ async def build_search_query(
if translator is None: if translator is None:
raise HTTPException( raise HTTPException(
status_code=503, status_code=503,
detail="No LLM backend configured. Set CF_ORCH_URL (cloud) or OLLAMA_HOST / ANTHROPIC_API_KEY / OPENAI_API_KEY (local).", detail="No LLM backend configured. Set OLLAMA_HOST, ANTHROPIC_API_KEY, or OPENAI_API_KEY.",
) )
from app.llm.query_translator import QueryTranslatorError from app.llm.query_translator import QueryTranslatorError

View file

@ -1,20 +0,0 @@
-- Migration 013: eBay user OAuth tokens
--
-- Stores per-user eBay Authorization Code tokens so the app can call
-- Trading API GetUser for instant account_age_days + category feedback
-- without Playwright scraping.
--
-- Stored in the per-user DB (user.db), never the shared DB.
-- access_token is short-lived (2h); refresh_token is valid 18 months.
-- The API layer refreshes access_token automatically before expiry.
CREATE TABLE IF NOT EXISTS ebay_user_tokens (
id INTEGER PRIMARY KEY,
-- Single row per user DB — upsert on reconnect
access_token TEXT NOT NULL,
refresh_token TEXT NOT NULL,
expires_at REAL NOT NULL, -- epoch seconds; access token expiry
scopes TEXT NOT NULL DEFAULT '',
connected_at TEXT NOT NULL DEFAULT (datetime('now')),
last_refreshed TEXT
);

View file

@ -1,24 +0,0 @@
-- Migration 014: background monitor settings on saved_searches + watch_alerts table
ALTER TABLE saved_searches ADD COLUMN monitor_enabled INTEGER NOT NULL DEFAULT 0;
ALTER TABLE saved_searches ADD COLUMN poll_interval_min INTEGER NOT NULL DEFAULT 60;
ALTER TABLE saved_searches ADD COLUMN min_trust_score INTEGER NOT NULL DEFAULT 60;
ALTER TABLE saved_searches ADD COLUMN last_checked_at TEXT;
CREATE TABLE IF NOT EXISTS watch_alerts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
saved_search_id INTEGER NOT NULL REFERENCES saved_searches(id) ON DELETE CASCADE,
platform_listing_id TEXT NOT NULL,
title TEXT NOT NULL,
price REAL NOT NULL,
currency TEXT NOT NULL DEFAULT 'USD',
trust_score INTEGER NOT NULL,
url TEXT,
first_alerted_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
dismissed_at TEXT,
UNIQUE(saved_search_id, platform_listing_id)
);
CREATE INDEX IF NOT EXISTS idx_watch_alerts_undismissed
ON watch_alerts(saved_search_id)
WHERE dismissed_at IS NULL;

View file

@ -1,20 +0,0 @@
-- Migration 015: cross-user monitor registry for the background polling loop
--
-- In cloud mode this table lives in shared.db — the polling loop queries it
-- to find all due monitors without scanning per-user DB files.
-- In local mode it lives in the single local DB (same result, one user).
--
-- user_db_path references the per-user snipe user.db so the poller knows
-- which DB to open for the full SavedSearch config and to write alerts.
CREATE TABLE IF NOT EXISTS active_monitors (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_db_path TEXT NOT NULL,
saved_search_id INTEGER NOT NULL,
poll_interval_min INTEGER NOT NULL DEFAULT 60,
last_checked_at TEXT,
UNIQUE(user_db_path, saved_search_id)
);
CREATE INDEX IF NOT EXISTS idx_active_monitors_due
ON active_monitors(last_checked_at);

View file

@ -81,26 +81,6 @@ class SavedSearch:
id: Optional[int] = None id: Optional[int] = None
created_at: Optional[str] = None created_at: Optional[str] = None
last_run_at: Optional[str] = None last_run_at: Optional[str] = None
# Monitor settings (migration 014)
monitor_enabled: bool = False
poll_interval_min: int = 60
min_trust_score: int = 60
last_checked_at: Optional[str] = None
@dataclass
class WatchAlert:
"""A new listing surfaced by the background monitor for a saved search."""
saved_search_id: int
platform_listing_id: str
title: str
price: float
trust_score: int
currency: str = "USD"
url: Optional[str] = None
id: Optional[int] = None
first_alerted_at: Optional[str] = None
dismissed_at: Optional[str] = None
@dataclass @dataclass

View file

@ -1,49 +0,0 @@
-- Snipe shared tables: sellers, market_comps, reported_sellers
-- Replaces the equivalent tables in shared.db (SQLite).
-- Per-user tables (listings, trust_scores, saved_searches) remain in SQLite.
CREATE TABLE IF NOT EXISTS sellers (
id BIGSERIAL PRIMARY KEY,
platform TEXT NOT NULL,
platform_seller_id TEXT NOT NULL,
username TEXT NOT NULL,
account_age_days INTEGER,
feedback_count INTEGER NOT NULL DEFAULT 0,
feedback_ratio DOUBLE PRECISION NOT NULL DEFAULT 0,
category_history_json TEXT NOT NULL DEFAULT '{}',
fetched_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (platform, platform_seller_id)
);
CREATE TABLE IF NOT EXISTS market_comps (
id BIGSERIAL PRIMARY KEY,
platform TEXT NOT NULL,
query_hash TEXT NOT NULL,
median_price DOUBLE PRECISION NOT NULL,
sample_count INTEGER NOT NULL,
fetched_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
expires_at TIMESTAMPTZ NOT NULL,
UNIQUE (platform, query_hash)
);
CREATE TABLE IF NOT EXISTS reported_sellers (
id BIGSERIAL PRIMARY KEY,
platform TEXT NOT NULL,
platform_seller_id TEXT NOT NULL,
username TEXT,
reported_by TEXT NOT NULL DEFAULT 'user',
reported_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (platform, platform_seller_id)
);
CREATE TABLE IF NOT EXISTS scammer_blocklist (
id BIGSERIAL PRIMARY KEY,
platform TEXT NOT NULL,
platform_seller_id TEXT NOT NULL,
username TEXT NOT NULL,
reason TEXT,
source TEXT NOT NULL DEFAULT 'manual',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (platform, platform_seller_id)
);

View file

@ -1,380 +0,0 @@
from __future__ import annotations
import logging
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional
import psycopg2
from psycopg2.pool import ThreadedConnectionPool
from app.db.models import MarketComp, ScammerEntry, Seller
log = logging.getLogger(__name__)
_MIN_CONN = 2
_MAX_CONN = 20
class SnipeSharedDB:
"""Thread-safe Postgres connection pool for Snipe shared tables."""
def __init__(self, dsn: str) -> None:
self._pool = ThreadedConnectionPool(_MIN_CONN, _MAX_CONN, dsn=dsn)
def getconn(self):
return self._pool.getconn()
def putconn(self, conn) -> None:
self._pool.putconn(conn)
def close(self) -> None:
self._pool.closeall()
def run_migrations(self) -> None:
"""Apply pg_migrations/*.sql in filename order. Idempotent."""
migrations_dir = Path(__file__).parent / "pg_migrations"
files = sorted(migrations_dir.glob("*.sql"), key=lambda p: p.name)
conn = self.getconn()
try:
with conn.cursor() as cur:
cur.execute("""
CREATE TABLE IF NOT EXISTS _snipe_shared_migrations (
filename TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
)
""")
conn.commit()
for f in files:
cur.execute(
"SELECT 1 FROM _snipe_shared_migrations WHERE filename = %s",
(f.name,),
)
if cur.fetchone():
continue
log.info("Applying migration: %s", f.name)
cur.execute(f.read_text())
cur.execute(
"INSERT INTO _snipe_shared_migrations (filename) VALUES (%s)",
(f.name,),
)
conn.commit()
except Exception:
conn.rollback()
raise
finally:
self.putconn(conn)
class SnipeSharedStore:
"""Postgres-backed store for sellers, market_comps, and reported_sellers.
Satisfies SharedTableProtocol. clone() returns self ThreadedConnectionPool
is already thread-safe, so no new instance is needed per thread.
"""
def __init__(self, db: SnipeSharedDB) -> None:
self._db = db
def clone(self) -> "SnipeSharedStore":
return self
# Sellers
def save_seller(self, seller: Seller) -> None:
self.save_sellers([seller])
def save_sellers(self, sellers: list[Seller]) -> None:
if not sellers:
return
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.executemany(
"""
INSERT INTO sellers
(platform, platform_seller_id, username, account_age_days,
feedback_count, feedback_ratio, category_history_json)
VALUES (%s, %s, %s, %s, %s, %s, %s)
ON CONFLICT (platform, platform_seller_id) DO UPDATE SET
username = EXCLUDED.username,
feedback_count = EXCLUDED.feedback_count,
feedback_ratio = EXCLUDED.feedback_ratio,
account_age_days = COALESCE(
EXCLUDED.account_age_days,
sellers.account_age_days
),
category_history_json = COALESCE(
NULLIF(NULLIF(EXCLUDED.category_history_json, '{}'), ''),
NULLIF(NULLIF(sellers.category_history_json, '{}'), ''),
'{}'
),
fetched_at = NOW()
""",
[
(s.platform, s.platform_seller_id, s.username, s.account_age_days,
s.feedback_count, s.feedback_ratio, s.category_history_json or "{}")
for s in sellers
],
)
conn.commit()
except Exception:
conn.rollback()
raise
finally:
self._db.putconn(conn)
def get_seller(self, platform: str, platform_seller_id: str) -> Optional[Seller]:
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.execute(
"""
SELECT platform, platform_seller_id, username, account_age_days,
feedback_count, feedback_ratio, category_history_json,
id, fetched_at
FROM sellers
WHERE platform = %s AND platform_seller_id = %s
""",
(platform, platform_seller_id),
)
row = cur.fetchone()
if not row:
return None
return Seller(*row[:7], id=row[7], fetched_at=str(row[8]))
finally:
self._db.putconn(conn)
def delete_seller_data(self, platform: str, platform_seller_id: str) -> None:
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.execute(
"DELETE FROM sellers WHERE platform = %s AND platform_seller_id = %s",
(platform, platform_seller_id),
)
conn.commit()
except Exception:
conn.rollback()
raise
finally:
self._db.putconn(conn)
# MarketComps
def save_market_comp(self, comp: MarketComp) -> None:
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.execute(
"""
INSERT INTO market_comps
(platform, query_hash, median_price, sample_count, expires_at)
VALUES (%s, %s, %s, %s, %s::TIMESTAMPTZ)
ON CONFLICT (platform, query_hash) DO UPDATE SET
median_price = EXCLUDED.median_price,
sample_count = EXCLUDED.sample_count,
expires_at = EXCLUDED.expires_at,
fetched_at = NOW()
""",
(comp.platform, comp.query_hash, comp.median_price,
comp.sample_count, comp.expires_at),
)
conn.commit()
except Exception:
conn.rollback()
raise
finally:
self._db.putconn(conn)
def get_market_comp(self, platform: str, query_hash: str) -> Optional[MarketComp]:
now = datetime.now(timezone.utc).isoformat()
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.execute(
"""
SELECT platform, query_hash, median_price, sample_count,
expires_at, id, fetched_at
FROM market_comps
WHERE platform = %s AND query_hash = %s AND expires_at > %s::TIMESTAMPTZ
""",
(platform, query_hash, now),
)
row = cur.fetchone()
if not row:
return None
return MarketComp(*row[:5], id=row[5], fetched_at=str(row[6]))
finally:
self._db.putconn(conn)
# Reported Sellers
def mark_reported(
self,
platform: str,
platform_seller_id: str,
username: Optional[str] = None,
reported_by: str = "user",
) -> None:
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.execute(
"""
INSERT INTO reported_sellers
(platform, platform_seller_id, username, reported_by)
VALUES (%s, %s, %s, %s)
ON CONFLICT (platform, platform_seller_id) DO NOTHING
""",
(platform, platform_seller_id, username, reported_by),
)
conn.commit()
except Exception:
conn.rollback()
raise
finally:
self._db.putconn(conn)
def list_reported(self, platform: str = "ebay") -> list[str]:
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.execute(
"SELECT platform_seller_id FROM reported_sellers WHERE platform = %s",
(platform,),
)
return [row[0] for row in cur.fetchall()]
finally:
self._db.putconn(conn)
# Seller Category Refresh
def refresh_seller_categories(
self,
platform: str,
seller_ids: list[str],
listing_store=None, # always a SQLite Store in practice
) -> int:
"""Derive category_history_json from listing data and update sellers in Postgres.
listing_store must be provided (it's always the per-user SQLite Store).
Returns count of sellers updated.
"""
from app.platforms.ebay.scraper import _classify_category_label # lazy to avoid circular
import json
if not seller_ids or listing_store is None:
return 0
updated = 0
for sid in seller_ids:
seller = self.get_seller(platform, sid)
if not seller or seller.category_history_json not in ("{}", "", None):
continue
# listing_store is always a SQLite Store; access _conn directly for the query.
rows = listing_store._conn.execute(
"SELECT category_name, COUNT(*) FROM listings "
"WHERE platform=? AND seller_platform_id=? AND category_name IS NOT NULL "
"GROUP BY category_name",
(platform, sid),
).fetchall()
if not rows:
continue
counts: dict[str, int] = {}
for cat_name, cnt in rows:
key = _classify_category_label(cat_name)
if key:
counts[key] = counts.get(key, 0) + cnt
if counts:
from dataclasses import replace
self.save_sellers([replace(seller, category_history_json=json.dumps(counts))])
updated += 1
return updated
# Scammer Blocklist
def is_blocklisted(self, platform: str, platform_seller_id: str) -> bool:
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.execute(
"SELECT 1 FROM scammer_blocklist "
"WHERE platform = %s AND platform_seller_id = %s LIMIT 1",
(platform, platform_seller_id),
)
return cur.fetchone() is not None
finally:
self._db.putconn(conn)
def add_to_blocklist(self, entry: ScammerEntry) -> ScammerEntry:
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.execute(
"""
INSERT INTO scammer_blocklist
(platform, platform_seller_id, username, reason, source)
VALUES (%s, %s, %s, %s, %s)
ON CONFLICT (platform, platform_seller_id) DO UPDATE SET
username = EXCLUDED.username,
reason = COALESCE(EXCLUDED.reason, scammer_blocklist.reason),
source = EXCLUDED.source
""",
(entry.platform, entry.platform_seller_id, entry.username,
entry.reason, entry.source),
)
conn.commit()
cur.execute(
"SELECT id, created_at FROM scammer_blocklist "
"WHERE platform = %s AND platform_seller_id = %s",
(entry.platform, entry.platform_seller_id),
)
row = cur.fetchone()
from dataclasses import replace
return replace(entry, id=row[0], created_at=str(row[1]))
except Exception:
conn.rollback()
raise
finally:
self._db.putconn(conn)
def remove_from_blocklist(self, platform: str, platform_seller_id: str) -> None:
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.execute(
"DELETE FROM scammer_blocklist "
"WHERE platform = %s AND platform_seller_id = %s",
(platform, platform_seller_id),
)
conn.commit()
except Exception:
conn.rollback()
raise
finally:
self._db.putconn(conn)
def list_blocklist(self, platform: str = "ebay") -> list[ScammerEntry]:
conn = self._db.getconn()
try:
with conn.cursor() as cur:
cur.execute(
"""
SELECT platform, platform_seller_id, username, reason, source, id, created_at
FROM scammer_blocklist
WHERE platform = %s
ORDER BY created_at DESC
""",
(platform,),
)
return [
ScammerEntry(
platform=r[0], platform_seller_id=r[1], username=r[2],
reason=r[3], source=r[4], id=r[5], created_at=str(r[6]),
)
for r in cur.fetchall()
]
finally:
self._db.putconn(conn)

View file

@ -1,86 +0,0 @@
"""Protocol (duck-type interface) for shared table backends (SQLite and Postgres)."""
from __future__ import annotations
from typing import Any, Optional, Protocol, runtime_checkable
from app.db.models import MarketComp, ScammerEntry, Seller
@runtime_checkable
class SharedTableProtocol(Protocol):
"""Protocol that both Store (SQLite) and SnipeSharedStore (Postgres) must satisfy.
This enables code that reads/writes shared tables (sellers, market_comps,
reported_sellers, scammer_blocklist) to remain agnostic to the underlying backend.
"""
def save_seller(self, seller: Seller) -> None:
"""Persist a single seller record."""
...
def save_sellers(self, sellers: list[Seller]) -> None:
"""Persist multiple seller records (batch upsert)."""
...
def get_seller(self, platform: str, platform_seller_id: str) -> Optional[Seller]:
"""Fetch a single seller by platform and platform_seller_id."""
...
def save_market_comp(self, comp: MarketComp) -> None:
"""Persist a market comparison record."""
...
def get_market_comp(self, platform: str, query_hash: str) -> Optional[MarketComp]:
"""Fetch a market comparison by platform and query_hash."""
...
def mark_reported(
self,
platform: str,
platform_seller_id: str,
username: Optional[str] = None,
reported_by: str = "user",
) -> None:
"""Record that a seller has been reported to the platform."""
...
def list_reported(self, platform: str = "ebay") -> list[str]:
"""Return all platform_seller_ids that have been reported."""
...
def delete_seller_data(self, platform: str, platform_seller_id: str) -> None:
"""Permanently erase a seller and all related data (GDPR/eBay compliance)."""
...
def refresh_seller_categories(
self,
platform: str,
seller_ids: list[str],
listing_store: Optional[Any] = None,
) -> int:
"""Derive category_history_json for sellers that lack it from stored listings.
listing_store: Store holding listings (may differ from self in split-DB mode).
Returns count of sellers updated.
"""
...
def is_blocklisted(self, platform: str, platform_seller_id: str) -> bool:
"""Return True if a seller is on the community scammer blocklist."""
...
def add_to_blocklist(self, entry: ScammerEntry) -> ScammerEntry:
"""Upsert a seller into the blocklist. Returns the saved entry with id and created_at."""
...
def remove_from_blocklist(self, platform: str, platform_seller_id: str) -> None:
"""Remove a seller from the blocklist."""
...
def list_blocklist(self, platform: str = "ebay") -> list[ScammerEntry]:
"""Return all blocklisted sellers for a platform, newest first."""
...
def clone(self) -> SharedTableProtocol:
"""Create a new independent instance pointing to the same backend."""
...

View file

@ -8,7 +8,7 @@ from typing import Optional
from circuitforge_core.db import get_connection, run_migrations from circuitforge_core.db import get_connection, run_migrations
from .models import Listing, MarketComp, SavedSearch, ScammerEntry, Seller, TrustScore, WatchAlert from .models import Listing, MarketComp, SavedSearch, ScammerEntry, Seller, TrustScore
MIGRATIONS_DIR = Path(__file__).parent / "migrations" MIGRATIONS_DIR = Path(__file__).parent / "migrations"
@ -21,10 +21,6 @@ class Store:
# WAL mode: allows concurrent readers + one writer without blocking # WAL mode: allows concurrent readers + one writer without blocking
self._conn.execute("PRAGMA journal_mode=WAL") self._conn.execute("PRAGMA journal_mode=WAL")
def clone(self) -> Store:
"""Create a new independent instance pointing to the same database."""
return Store(self._db_path)
# --- Seller --- # --- Seller ---
def delete_seller_data(self, platform: str, platform_seller_id: str) -> None: def delete_seller_data(self, platform: str, platform_seller_id: str) -> None:
@ -314,66 +310,15 @@ class Store:
def list_saved_searches(self) -> list[SavedSearch]: def list_saved_searches(self) -> list[SavedSearch]:
rows = self._conn.execute( rows = self._conn.execute(
"SELECT name, query, platform, filters_json, id, created_at, last_run_at, " "SELECT name, query, platform, filters_json, id, created_at, last_run_at "
"monitor_enabled, poll_interval_min, min_trust_score, last_checked_at "
"FROM saved_searches ORDER BY created_at DESC" "FROM saved_searches ORDER BY created_at DESC"
).fetchall() ).fetchall()
return [ return [
SavedSearch( SavedSearch(name=r[0], query=r[1], platform=r[2], filters_json=r[3],
name=r[0], query=r[1], platform=r[2], filters_json=r[3], id=r[4], created_at=r[5], last_run_at=r[6])
id=r[4], created_at=r[5], last_run_at=r[6],
monitor_enabled=bool(r[7]), poll_interval_min=r[8],
min_trust_score=r[9], last_checked_at=r[10],
)
for r in rows for r in rows
] ]
def update_monitor_settings(
self,
saved_id: int,
*,
monitor_enabled: bool,
poll_interval_min: int,
min_trust_score: int,
) -> None:
self._conn.execute(
"UPDATE saved_searches "
"SET monitor_enabled=?, poll_interval_min=?, min_trust_score=? "
"WHERE id=?",
(int(monitor_enabled), poll_interval_min, min_trust_score, saved_id),
)
self._conn.commit()
def list_monitored_searches(self) -> list[SavedSearch]:
"""Return all saved searches with monitoring enabled (used by background poller)."""
rows = self._conn.execute(
"SELECT name, query, platform, filters_json, id, created_at, last_run_at, "
"monitor_enabled, poll_interval_min, min_trust_score, last_checked_at "
"FROM saved_searches WHERE monitor_enabled=1"
).fetchall()
return [
SavedSearch(
name=r[0], query=r[1], platform=r[2], filters_json=r[3],
id=r[4], created_at=r[5], last_run_at=r[6],
monitor_enabled=True, poll_interval_min=r[8],
min_trust_score=r[9], last_checked_at=r[10],
)
for r in rows
]
def mark_search_checked(self, saved_id: int) -> None:
self._conn.execute(
"UPDATE saved_searches SET last_checked_at=? WHERE id=?",
(datetime.now(timezone.utc).isoformat(), saved_id),
)
self._conn.commit()
def count_active_monitors(self) -> int:
row = self._conn.execute(
"SELECT COUNT(*) FROM saved_searches WHERE monitor_enabled=1"
).fetchone()
return row[0] if row else 0
def delete_saved_search(self, saved_id: int) -> None: def delete_saved_search(self, saved_id: int) -> None:
self._conn.execute("DELETE FROM saved_searches WHERE id=?", (saved_id,)) self._conn.execute("DELETE FROM saved_searches WHERE id=?", (saved_id,))
self._conn.commit() self._conn.commit()
@ -385,112 +330,6 @@ class Store:
) )
self._conn.commit() self._conn.commit()
# --- WatchAlerts ---
def upsert_alert(self, alert: WatchAlert) -> tuple[int, bool]:
"""Insert alert if not already present. Returns (id, is_new)."""
existing = self._conn.execute(
"SELECT id FROM watch_alerts WHERE saved_search_id=? AND platform_listing_id=?",
(alert.saved_search_id, alert.platform_listing_id),
).fetchone()
if existing:
return existing[0], False
cur = self._conn.execute(
"INSERT INTO watch_alerts "
"(saved_search_id, platform_listing_id, title, price, currency, trust_score, url) "
"VALUES (?,?,?,?,?,?,?)",
(alert.saved_search_id, alert.platform_listing_id, alert.title,
alert.price, alert.currency, alert.trust_score, alert.url),
)
self._conn.commit()
return cur.lastrowid, True
def list_alerts(self, *, include_dismissed: bool = False) -> list[WatchAlert]:
where = "" if include_dismissed else "WHERE dismissed_at IS NULL"
rows = self._conn.execute(
f"SELECT id, saved_search_id, platform_listing_id, title, price, currency, "
f"trust_score, url, first_alerted_at, dismissed_at "
f"FROM watch_alerts {where} ORDER BY first_alerted_at DESC"
).fetchall()
return [
WatchAlert(
id=r[0], saved_search_id=r[1], platform_listing_id=r[2],
title=r[3], price=r[4], currency=r[5], trust_score=r[6],
url=r[7], first_alerted_at=r[8], dismissed_at=r[9],
)
for r in rows
]
def count_undismissed_alerts(self) -> int:
row = self._conn.execute(
"SELECT COUNT(*) FROM watch_alerts WHERE dismissed_at IS NULL"
).fetchone()
return row[0] if row else 0
def dismiss_alert(self, alert_id: int) -> None:
self._conn.execute(
"UPDATE watch_alerts SET dismissed_at=? WHERE id=?",
(datetime.now(timezone.utc).isoformat(), alert_id),
)
self._conn.commit()
def dismiss_all_alerts(self) -> int:
"""Dismiss all undismissed alerts. Returns count dismissed."""
cur = self._conn.execute(
"UPDATE watch_alerts SET dismissed_at=? WHERE dismissed_at IS NULL",
(datetime.now(timezone.utc).isoformat(),),
)
self._conn.commit()
return cur.rowcount
# --- ActiveMonitors (sched_db / shared_db) ---
def upsert_active_monitor(
self,
user_db_path: str,
saved_search_id: int,
poll_interval_min: int,
) -> None:
"""Register or update a monitor in the cross-user registry (sched_db)."""
self._conn.execute(
"INSERT INTO active_monitors (user_db_path, saved_search_id, poll_interval_min) "
"VALUES (?,?,?) "
"ON CONFLICT(user_db_path, saved_search_id) DO UPDATE SET "
" poll_interval_min=excluded.poll_interval_min",
(user_db_path, saved_search_id, poll_interval_min),
)
self._conn.commit()
def remove_active_monitor(self, user_db_path: str, saved_search_id: int) -> None:
self._conn.execute(
"DELETE FROM active_monitors WHERE user_db_path=? AND saved_search_id=?",
(user_db_path, saved_search_id),
)
self._conn.commit()
def list_due_active_monitors(self) -> list[tuple[str, int, int]]:
"""Return (user_db_path, saved_search_id, poll_interval_min) for monitors that are due.
Due = never checked OR last_checked_at is old enough given poll_interval_min.
Uses SQLite's strftime('%s') for epoch arithmetic without Python datetime overhead.
"""
rows = self._conn.execute(
"SELECT user_db_path, saved_search_id, poll_interval_min "
"FROM active_monitors "
"WHERE last_checked_at IS NULL "
" OR (strftime('%s','now') - strftime('%s', last_checked_at)) "
" >= poll_interval_min * 60"
).fetchall()
return [(r[0], r[1], r[2]) for r in rows]
def mark_active_monitor_checked(self, user_db_path: str, saved_search_id: int) -> None:
self._conn.execute(
"UPDATE active_monitors SET last_checked_at=? "
"WHERE user_db_path=? AND saved_search_id=?",
(datetime.now(timezone.utc).isoformat(), user_db_path, saved_search_id),
)
self._conn.commit()
# --- ScammerBlocklist --- # --- ScammerBlocklist ---
def add_to_blocklist(self, entry: ScammerEntry) -> ScammerEntry: def add_to_blocklist(self, entry: ScammerEntry) -> ScammerEntry:

View file

@ -2,15 +2,9 @@
# BSL 1.1 License # BSL 1.1 License
"""LLM query builder — translates natural language to eBay SearchFilters. """LLM query builder — translates natural language to eBay SearchFilters.
Supports two backends, selected at construction time: The QueryTranslator calls LLMRouter.complete() (synchronous) with a domain-aware
system prompt. The prompt includes category hints injected from EbayCategoryCache.
cforch_url cf-orch task endpoint (cloud/premium). The coordinator resolves The LLM returns a single JSON object matching SearchParamsResponse.
product+task to a model and returns an allocation. The caller
POSTs to the allocated service URL, then DELETEs the allocation.
llm_router circuitforge_core.LLMRouter (local installs: ollama/vllm/api keys).
Exactly one of cforch_url or llm_router must be supplied.
""" """
from __future__ import annotations from __future__ import annotations
@ -19,8 +13,6 @@ import logging
from dataclasses import dataclass from dataclasses import dataclass
from typing import TYPE_CHECKING, Optional from typing import TYPE_CHECKING, Optional
import httpx
if TYPE_CHECKING: if TYPE_CHECKING:
from app.platforms.ebay.categories import EbayCategoryCache from app.platforms.ebay.categories import EbayCategoryCache
@ -136,23 +128,11 @@ class QueryTranslator:
Args: Args:
category_cache: An EbayCategoryCache instance (may have empty cache). category_cache: An EbayCategoryCache instance (may have empty cache).
cforch_url: cf-orch coordinator base URL (cloud/premium path). llm_router: An LLMRouter instance from circuitforge_core.
llm_router: A circuitforge_core LLMRouter instance (local path).
Exactly one of cforch_url or llm_router must be provided.
""" """
def __init__( def __init__(self, category_cache: "EbayCategoryCache", llm_router: object) -> None:
self,
category_cache: "EbayCategoryCache",
*,
cforch_url: str | None = None,
llm_router: object | None = None,
) -> None:
if cforch_url is None and llm_router is None:
raise ValueError("Either cforch_url or llm_router must be provided")
self._cache = category_cache self._cache = category_cache
self._cforch_url = cforch_url
self._llm_router = llm_router self._llm_router = llm_router
def translate(self, natural_language: str) -> SearchParamsResponse: def translate(self, natural_language: str) -> SearchParamsResponse:
@ -174,58 +154,14 @@ class QueryTranslator:
system_prompt = _SYSTEM_PROMPT_TEMPLATE.format(category_hints=category_hints) system_prompt = _SYSTEM_PROMPT_TEMPLATE.format(category_hints=category_hints)
try: try:
if self._cforch_url: raw = self._llm_router.complete(
raw = self._call_orch(system_prompt, natural_language) natural_language,
else: system=system_prompt,
raw = self._call_local(system_prompt, natural_language) max_tokens=512,
except QueryTranslatorError: )
raise
except Exception as exc: except Exception as exc:
raise QueryTranslatorError( raise QueryTranslatorError(
f"LLM backend error: {exc}", raw="" f"LLM backend error: {exc}", raw=""
) from exc ) from exc
return _parse_response(raw) return _parse_response(raw)
def _call_orch(self, system_prompt: str, user_message: str) -> str:
"""Allocate via cf-orch task endpoint, call the model, release the slot."""
alloc_resp = httpx.post(
f"{self._cforch_url}/api/inference/task",
json={"product": "snipe", "task": "query_translation"},
timeout=10.0,
)
alloc_resp.raise_for_status()
alloc = alloc_resp.json()
service_url = alloc["url"]
allocation_id = alloc["allocation_id"]
try:
resp = httpx.post(
f"{service_url}/v1/chat/completions",
json={
"model": "__auto__",
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_message},
],
"max_tokens": 512,
},
timeout=60.0,
)
resp.raise_for_status()
return resp.json()["choices"][0]["message"]["content"]
finally:
try:
httpx.delete(
f"{self._cforch_url}/api/services/cf-text/allocations/{allocation_id}",
timeout=5.0,
)
except Exception:
log.warning("Failed to release cf-orch allocation %s", allocation_id)
def _call_local(self, system_prompt: str, user_message: str) -> str:
"""Call the locally-configured LLMRouter (ollama/vllm/api keys)."""
return self._llm_router.complete( # type: ignore[union-attr]
user_message,
system=system_prompt,
max_tokens=512,
)

View file

@ -6,7 +6,7 @@ Snipe LLMRouter shim — tri-level config path priority.
Config lookup order: Config lookup order:
1. <repo>/config/llm.yaml per-install local override 1. <repo>/config/llm.yaml per-install local override
2. ~/.config/circuitforge/llm.yaml user-level config (circuitforge-core default) 2. ~/.config/circuitforge/llm.yaml user-level config (circuitforge-core default)
3. env-var auto-config (ANTHROPIC_API_KEY, OPENAI_API_KEY, OLLAMA_HOST, GPU_SERVER_URL) 3. env-var auto-config (ANTHROPIC_API_KEY, OPENAI_API_KEY, OLLAMA_HOST, CF_ORCH_URL)
""" """
from pathlib import Path from pathlib import Path

View file

@ -7,10 +7,6 @@ from typing import Optional
from app.db.models import Listing, Seller from app.db.models import Listing, Seller
# Single source of truth for platform validation.
# Phase 2 will extend this set as new adapters are implemented.
SUPPORTED_PLATFORMS: frozenset[str] = frozenset({"ebay", "mercari"})
@dataclass @dataclass
class SearchFilters: class SearchFilters:
@ -22,8 +18,6 @@ class SearchFilters:
must_include: list[str] = field(default_factory=list) # client-side title filter must_include: list[str] = field(default_factory=list) # client-side title filter
must_exclude: list[str] = field(default_factory=list) # forwarded to eBay -term AND client-side must_exclude: list[str] = field(default_factory=list) # forwarded to eBay -term AND client-side
category_id: Optional[str] = None # eBay category ID (e.g. "27386" = GPUs) category_id: Optional[str] = None # eBay category ID (e.g. "27386" = GPUs)
must_include_mode: str = "all" # "all" | "any" | "groups"
adapter: str = "auto" # "auto" | "api" | "scraper"
class PlatformAdapter(ABC): class PlatformAdapter(ABC):

View file

@ -1,9 +1,8 @@
"""eBay Browse + Trading API adapter.""" """eBay Browse API adapter."""
from __future__ import annotations from __future__ import annotations
import hashlib import hashlib
import logging import logging
import xml.etree.ElementTree as ET
from dataclasses import replace from dataclasses import replace
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from typing import Optional from typing import Optional
@ -22,7 +21,7 @@ _SHOPPING_API_INTER_REQUEST_DELAY = 0.5 # seconds between successive calls
_SELLER_ENRICH_TTL_HOURS = 24 # skip re-enrichment within this window _SELLER_ENRICH_TTL_HOURS = 24 # skip re-enrichment within this window
from app.db.models import Listing, MarketComp, Seller from app.db.models import Listing, MarketComp, Seller
from app.db.protocol import SharedTableProtocol from app.db.store import Store
from app.platforms import PlatformAdapter, SearchFilters from app.platforms import PlatformAdapter, SearchFilters
from app.platforms.ebay.auth import EbayTokenManager from app.platforms.ebay.auth import EbayTokenManager
from app.platforms.ebay.normaliser import normalise_listing, normalise_seller from app.platforms.ebay.normaliser import normalise_listing, normalise_seller
@ -67,7 +66,7 @@ BROWSE_BASE = {
class EbayAdapter(PlatformAdapter): class EbayAdapter(PlatformAdapter):
def __init__(self, token_manager: EbayTokenManager, shared_store: SharedTableProtocol, env: str = "production"): def __init__(self, token_manager: EbayTokenManager, shared_store: Store, env: str = "production"):
self._tokens = token_manager self._tokens = token_manager
self._store = shared_store self._store = shared_store
self._env = env self._env = env
@ -211,70 +210,6 @@ class EbayAdapter(PlatformAdapter):
except Exception as e: except Exception as e:
log.debug("Shopping API enrich failed for %s: %s", username, e) log.debug("Shopping API enrich failed for %s: %s", username, e)
# ── Trading API GetUser (requires user OAuth token) ───────────────────────
_TRADING_API_URL = "https://api.ebay.com/ws/api.dll"
_TRADING_API_COMPATIBILITY = "1283"
def enrich_seller_trading_api(self, username: str, user_access_token: str) -> bool:
"""Enrich a seller's account_age_days using Trading API GetUser.
Uses the connected user's OAuth access token (Authorization Code flow),
which bypasses Shopping API rate limits and works even when the Shopping
API GetUserProfile call is throttled.
Unlike BTF scraping, this is a clean API call (~200ms, no Playwright).
Called from the search endpoint when the requesting user has connected
their eBay account.
Returns True if enrichment succeeded, False on any failure.
"""
xml_body = (
'<?xml version="1.0" encoding="utf-8"?>'
'<GetUserRequest xmlns="urn:ebay:apis:eBLBaseComponents">'
f'<UserID>{username}</UserID>'
'</GetUserRequest>'
)
try:
resp = requests.post(
self._TRADING_API_URL,
headers={
"X-EBAY-API-CALL-NAME": "GetUser",
"X-EBAY-API-SITEID": "0",
"X-EBAY-API-COMPATIBILITY-LEVEL": self._TRADING_API_COMPATIBILITY,
"X-EBAY-API-IAF-TOKEN": f"Bearer {user_access_token}",
"Content-Type": "text/xml",
},
data=xml_body.encode("utf-8"),
timeout=10,
)
resp.raise_for_status()
root = ET.fromstring(resp.text)
ns = {"e": "urn:ebay:apis:eBLBaseComponents"}
ack = root.findtext("e:Ack", namespaces=ns)
if ack not in ("Success", "Warning"):
errors = [e.findtext("e:LongMessage", namespaces=ns, default="")
for e in root.findall("e:Errors", namespaces=ns)]
log.debug("Trading API GetUser failed for %s: %s", username, errors)
return False
reg_date = root.findtext("e:User/e:RegistrationDate", namespaces=ns)
if not reg_date:
return False
dt = datetime.fromisoformat(reg_date.replace("Z", "+00:00"))
age_days = (datetime.now(timezone.utc) - dt).days
seller = self._store.get_seller("ebay", username)
if seller:
self._store.save_seller(replace(seller, account_age_days=age_days))
log.debug("Trading API GetUser: %s registered %d days ago", username, age_days)
return True
except Exception as exc:
log.debug("Trading API GetUser failed for %s: %s", username, exc)
return False
def get_seller(self, seller_platform_id: str) -> Optional[Seller]: def get_seller(self, seller_platform_id: str) -> Optional[Seller]:
cached = self._store.get_seller("ebay", seller_platform_id) cached = self._store.get_seller("ebay", seller_platform_id)
if cached: if cached:

View file

@ -1,58 +1,57 @@
"""Thread-local Playwright browser manager for the eBay scraper. """Pre-warmed Chromium browser pool for the eBay scraper.
Each uvicorn worker thread that calls fetch_html() gets its own Playwright Eliminates cold-start latency (5-10s per call) by keeping a small pool of
instance, browser, and context created lazily on first use. This avoids long-lived Playwright browser instances with fresh contexts ready to serve.
the "cannot switch to a different thread" error that arises when Playwright
sync API instances are shared across threads (they bind their greenlet event
loop to the creating thread).
Key design: Key design:
- Thread-local: _thread_local.slot holds the _PooledBrowser for the current - Pool slots: ``(xvfb_proc, pw_instance, browser, context, display_num, last_used_ts)``
thread. No slot is ever handed to another thread. One headed Chromium browser per slot keeps the Kasada fingerprint clean.
- Lazy creation: slots are created on first fetch_html() call per thread, not - Thread safety: ``queue.Queue`` with blocking get (timeout=3s before fresh fallback).
at startup. start() is a lightweight lifecycle marker only. - Replenishment: after each use, the dirty context is closed and a new context is
- Registry: _slot_registry (keyed by thread-id) lets stop() close every active opened on the *same* browser, then returned to the queue. Browser launch overhead
slot across all threads without walking thread-local storage. is only paid at startup and during idle-cleanup replenishment.
- Replenishment: after each use the dirty context is closed and a fresh one - Idle cleanup: daemon thread closes slots idle for >5 minutes to avoid memory leaks
opened on the same browser. Browser launch overhead is paid at most once when the service is quiet.
per worker thread lifetime. - Graceful degradation: if Playwright / Xvfb is unavailable (host-side test env),
- Graceful degradation: if Playwright / Xvfb is unavailable, fetch_html falls ``fetch_html`` falls back to launching a fresh browser per call same behavior
back to _fetch_fresh (identical behavior to before this module existed). as before this module existed.
Pool size is read from BROWSER_POOL_SIZE env var (default: 2) but is now a Pool size is controlled via ``BROWSER_POOL_SIZE`` env var (default: 2).
soft limit used only for documentation; actual concurrency is bounded by
uvicorn's thread count.
""" """
from __future__ import annotations from __future__ import annotations
import itertools import itertools
import logging import logging
import os import os
import queue
import subprocess import subprocess
import threading import threading
import time import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Optional from typing import Optional
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
_pool_display_counter = itertools.cycle(range(200, 400)) # Reuse the same display counter namespace as scraper.py to avoid collisions.
# Pool uses :100-:199; scraper.py fallback uses :200-:299.
_pool_display_counter = itertools.cycle(range(100, 200))
_IDLE_TIMEOUT_SECS = 300 # 5 minutes
_CLEANUP_INTERVAL_SECS = 60
_QUEUE_TIMEOUT_SECS = 3.0
_CHROMIUM_ARGS = ["--no-sandbox", "--disable-dev-shm-usage"] _CHROMIUM_ARGS = ["--no-sandbox", "--disable-dev-shm-usage"]
_XVFB_ARGS = ["-screen", "0", "1280x800x24", "-ac"]
_USER_AGENT = ( _USER_AGENT = (
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 " "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" "(KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36"
) )
_VIEWPORT = {"width": 1280, "height": 800} _VIEWPORT = {"width": 1280, "height": 800}
# Thread-local storage: each thread gets its own _PooledBrowser slot.
_thread_local = threading.local()
@dataclass @dataclass
class _PooledBrowser: class _PooledBrowser:
"""One browser slot, bound to a single thread.""" """One slot in the browser pool."""
xvfb: subprocess.Popen xvfb: subprocess.Popen
pw: object # playwright instance (sync_playwright().__enter__()) pw: object # playwright instance (sync_playwright().__enter__())
browser: object # playwright Browser browser: object # playwright Browser
@ -61,13 +60,13 @@ class _PooledBrowser:
last_used_ts: float = field(default_factory=time.time) last_used_ts: float = field(default_factory=time.time)
def _launch_slot() -> _PooledBrowser: def _launch_slot() -> "_PooledBrowser":
"""Launch a new Xvfb display + headed Chromium browser + fresh context. """Launch a new Xvfb display + headed Chromium browser + fresh context.
Must be called from the thread that will use the slot. Raises on failure callers must catch and handle gracefully.
""" """
from playwright.sync_api import sync_playwright from playwright.sync_api import sync_playwright
from playwright_stealth import Stealth # noqa: F401 from playwright_stealth import Stealth # noqa: F401 — imported here to confirm availability
display_num = next(_pool_display_counter) display_num = next(_pool_display_counter)
display = f":{display_num}" display = f":{display_num}"
@ -75,10 +74,11 @@ def _launch_slot() -> _PooledBrowser:
env["DISPLAY"] = display env["DISPLAY"] = display
xvfb = subprocess.Popen( xvfb = subprocess.Popen(
["Xvfb", display] + _XVFB_ARGS, ["Xvfb", display, "-screen", "0", "1280x800x24"],
stdout=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
) )
# Small grace period for Xvfb to bind the display socket.
time.sleep(0.3) time.sleep(0.3)
pw = sync_playwright().start() pw = sync_playwright().start()
@ -109,7 +109,7 @@ def _launch_slot() -> _PooledBrowser:
def _close_slot(slot: _PooledBrowser) -> None: def _close_slot(slot: _PooledBrowser) -> None:
"""Cleanly close a slot: context → browser → Playwright → Xvfb.""" """Cleanly close a pool slot: context → browser → Playwright → Xvfb."""
try: try:
slot.ctx.close() slot.ctx.close()
except Exception: except Exception:
@ -130,7 +130,11 @@ def _close_slot(slot: _PooledBrowser) -> None:
def _replenish_slot(slot: _PooledBrowser) -> _PooledBrowser: def _replenish_slot(slot: _PooledBrowser) -> _PooledBrowser:
"""Close the used context and open a fresh one on the same browser.""" """Close the used context and open a fresh one on the same browser.
Returns a new _PooledBrowser sharing the same xvfb/pw/browser but with a
clean context avoids paying browser launch overhead on every fetch.
"""
try: try:
slot.ctx.close() slot.ctx.close()
except Exception: except Exception:
@ -151,27 +155,26 @@ def _replenish_slot(slot: _PooledBrowser) -> _PooledBrowser:
class BrowserPool: class BrowserPool:
"""Thread-local Playwright browser manager. """Thread-safe pool of pre-warmed Playwright browser contexts."""
Each thread that calls fetch_html() owns its own browser instance.
No slots are shared between threads.
"""
def __init__(self, size: int = 2) -> None: def __init__(self, size: int = 2) -> None:
self._size = size self._size = size
self._q: queue.Queue[_PooledBrowser] = queue.Queue()
self._lock = threading.Lock() self._lock = threading.Lock()
self._started = False self._started = False
self._stopped = False self._stopped = False
self._playwright_available: Optional[bool] = None self._playwright_available: Optional[bool] = None # cached after first check
# Registry of all active slots keyed by thread id — used only by stop().
self._slot_registry: dict[int, _PooledBrowser] = {}
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Lifecycle # Lifecycle
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def start(self) -> None: def start(self) -> None:
"""Mark the pool as started. Slots are created lazily per thread.""" """Pre-warm N browser slots in background threads.
Non-blocking: returns immediately; slots appear in the queue as they
finish launching. Safe to call multiple times (no-op after first).
"""
with self._lock: with self._lock:
if self._started: if self._started:
return return
@ -184,108 +187,91 @@ class BrowserPool:
) )
return return
log.info("BrowserPool: started (thread-local mode, size hint=%d)", self._size) def _warm_one(_: int) -> None:
try:
slot = _launch_slot()
self._q.put(slot)
log.debug("BrowserPool: slot :%d ready", slot.display_num)
except Exception as exc:
log.warning("BrowserPool: pre-warm failed: %s", exc)
with ThreadPoolExecutor(max_workers=self._size) as ex:
futures = [ex.submit(_warm_one, i) for i in range(self._size)]
# Don't wait — executor exits after submitting, threads continue.
# Actually ThreadPoolExecutor.__exit__ waits for completion, which
# is fine: pre-warming completes in background relative to FastAPI
# startup because this whole method is called from a thread.
for f in as_completed(futures):
pass # propagate exceptions via logging, not raises
_idle_cleaner = threading.Thread(
target=self._idle_cleanup_loop, daemon=True, name="browser-pool-idle-cleaner"
)
_idle_cleaner.start()
log.info("BrowserPool: started with %d slots", self._q.qsize())
def stop(self) -> None: def stop(self) -> None:
"""Close all active slots across all threads.""" """Drain and close all pool slots. Called at FastAPI shutdown."""
with self._lock: with self._lock:
self._stopped = True self._stopped = True
registry_snapshot = dict(self._slot_registry)
closed = 0 closed = 0
for slot in registry_snapshot.values(): while True:
try:
slot = self._q.get_nowait()
_close_slot(slot) _close_slot(slot)
closed += 1 closed += 1
self._slot_registry.clear() except queue.Empty:
break
log.info("BrowserPool: stopped, closed %d slot(s)", closed) log.info("BrowserPool: stopped, closed %d slot(s)", closed)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Core fetch # Core fetch
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def fetch_html( def fetch_html(self, url: str, delay: float = 1.0) -> str:
self,
url: str,
delay: float = 1.0,
wait_for_selector: Optional[str] = None,
wait_for_timeout_ms: int = 2000,
) -> str:
"""Navigate to *url* and return the rendered HTML. """Navigate to *url* and return the rendered HTML.
Uses the calling thread's browser slot (creates one if needed). Borrows a browser context from the pool (blocks up to 3s), uses it to
Falls back to a fresh browser if Playwright is unavailable or the fetch the page, then replenishes the slot with a fresh context.
slot fails.
Falls back to a fully fresh browser if the pool is empty after the
timeout or if Playwright is unavailable.
""" """
time.sleep(delay) time.sleep(delay)
slot = self._get_or_create_thread_slot() slot: Optional[_PooledBrowser] = None
try:
slot = self._q.get(timeout=_QUEUE_TIMEOUT_SECS)
except queue.Empty:
log.debug("BrowserPool: pool empty after %.1fs — using fresh browser", _QUEUE_TIMEOUT_SECS)
if slot is not None: if slot is not None:
try: try:
html = self._fetch_with_slot( html = self._fetch_with_slot(slot, url)
slot, url, # Replenish: close dirty context, open fresh one, return to queue.
wait_for_selector=wait_for_selector,
wait_for_timeout_ms=wait_for_timeout_ms,
)
try: try:
fresh_slot = _replenish_slot(slot) fresh_slot = _replenish_slot(slot)
self._register_slot(fresh_slot) self._q.put(fresh_slot)
except Exception as exc: except Exception as exc:
log.warning("BrowserPool: replenish failed, slot discarded: %s", exc) log.warning("BrowserPool: replenish failed, slot discarded: %s", exc)
_close_slot(slot) _close_slot(slot)
self._unregister_slot()
return html return html
except Exception as exc: except Exception as exc:
log.warning("BrowserPool: pooled fetch failed (%s) — closing slot", exc) log.warning("BrowserPool: pooled fetch failed (%s) — closing slot", exc)
_close_slot(slot) _close_slot(slot)
self._unregister_slot() # Fall through to fresh browser below.
return self._fetch_fresh( # Fallback: fresh browser (same code as old scraper._fetch_url).
url, return self._fetch_fresh(url)
wait_for_selector=wait_for_selector,
wait_for_timeout_ms=wait_for_timeout_ms,
)
# ------------------------------------------------------------------
# Thread-local slot management
# ------------------------------------------------------------------
def _get_or_create_thread_slot(self) -> Optional[_PooledBrowser]:
"""Return the calling thread's slot, creating it if absent."""
if not self._check_playwright():
return None
slot: Optional[_PooledBrowser] = getattr(_thread_local, "slot", None)
if slot is not None:
return slot
try:
slot = _launch_slot()
self._register_slot(slot)
log.debug("BrowserPool: launched slot :%d for thread %d",
slot.display_num, threading.get_ident())
return slot
except Exception as exc:
log.warning("BrowserPool: slot launch failed: %s", exc)
return None
def _register_slot(self, slot: _PooledBrowser) -> None:
"""Bind slot to the calling thread (both thread-local and registry)."""
_thread_local.slot = slot
with self._lock:
self._slot_registry[threading.get_ident()] = slot
def _unregister_slot(self) -> None:
"""Remove the calling thread's slot from thread-local and registry."""
_thread_local.slot = None
with self._lock:
self._slot_registry.pop(threading.get_ident(), None)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Internal helpers # Internal helpers
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def _check_playwright(self) -> bool: def _check_playwright(self) -> bool:
"""Return True if Playwright and Xvfb are importable/runnable."""
if self._playwright_available is not None: if self._playwright_available is not None:
return self._playwright_available return self._playwright_available
try: try:
@ -296,26 +282,15 @@ class BrowserPool:
self._playwright_available = False self._playwright_available = False
return self._playwright_available return self._playwright_available
def _fetch_with_slot( def _fetch_with_slot(self, slot: _PooledBrowser, url: str) -> str:
self, """Open a new page on *slot.ctx*, navigate to *url*, return HTML."""
slot: _PooledBrowser,
url: str,
wait_for_selector: Optional[str] = None,
wait_for_timeout_ms: int = 2000,
) -> str:
from playwright_stealth import Stealth from playwright_stealth import Stealth
page = slot.ctx.new_page() page = slot.ctx.new_page()
try: try:
Stealth().apply_stealth_sync(page) Stealth().apply_stealth_sync(page)
page.goto(url, wait_until="domcontentloaded", timeout=30_000) page.goto(url, wait_until="domcontentloaded", timeout=30_000)
if wait_for_selector: page.wait_for_timeout(2000)
try:
page.wait_for_selector(wait_for_selector, timeout=15_000)
except Exception:
pass
else:
page.wait_for_timeout(wait_for_timeout_ms)
return page.content() return page.content()
finally: finally:
try: try:
@ -323,12 +298,8 @@ class BrowserPool:
except Exception: except Exception:
pass pass
def _fetch_fresh( def _fetch_fresh(self, url: str) -> str:
self, """Launch a fully fresh browser, fetch *url*, close everything."""
url: str,
wait_for_selector: Optional[str] = None,
wait_for_timeout_ms: int = 2000,
) -> str:
import subprocess as _subprocess import subprocess as _subprocess
try: try:
@ -336,7 +307,7 @@ class BrowserPool:
from playwright_stealth import Stealth from playwright_stealth import Stealth
except ImportError as exc: except ImportError as exc:
raise RuntimeError( raise RuntimeError(
"Playwright not installed — cannot fetch pages. " "Playwright not installed — cannot fetch eBay pages. "
"Install playwright and playwright-stealth in the Docker image." "Install playwright and playwright-stealth in the Docker image."
) from exc ) from exc
@ -346,11 +317,10 @@ class BrowserPool:
env["DISPLAY"] = display env["DISPLAY"] = display
xvfb = _subprocess.Popen( xvfb = _subprocess.Popen(
["Xvfb", display] + _XVFB_ARGS, ["Xvfb", display, "-screen", "0", "1280x800x24"],
stdout=_subprocess.DEVNULL, stdout=_subprocess.DEVNULL,
stderr=_subprocess.DEVNULL, stderr=_subprocess.DEVNULL,
) )
time.sleep(0.3)
try: try:
with sync_playwright() as pw: with sync_playwright() as pw:
browser = pw.chromium.launch( browser = pw.chromium.launch(
@ -365,13 +335,7 @@ class BrowserPool:
page = ctx.new_page() page = ctx.new_page()
Stealth().apply_stealth_sync(page) Stealth().apply_stealth_sync(page)
page.goto(url, wait_until="domcontentloaded", timeout=30_000) page.goto(url, wait_until="domcontentloaded", timeout=30_000)
if wait_for_selector: page.wait_for_timeout(2000)
try:
page.wait_for_selector(wait_for_selector, timeout=15_000)
except Exception:
pass
else:
page.wait_for_timeout(wait_for_timeout_ms)
html = page.content() html = page.content()
browser.close() browser.close()
finally: finally:
@ -380,6 +344,32 @@ class BrowserPool:
return html return html
def _idle_cleanup_loop(self) -> None:
"""Daemon thread: drain slots idle for >5 minutes every 60 seconds."""
while not self._stopped:
time.sleep(_CLEANUP_INTERVAL_SECS)
if self._stopped:
break
now = time.time()
idle_cutoff = now - _IDLE_TIMEOUT_SECS
# Drain the entire queue, keep non-idle slots, close idle ones.
kept: list[_PooledBrowser] = []
closed = 0
while True:
try:
slot = self._q.get_nowait()
except queue.Empty:
break
if slot.last_used_ts < idle_cutoff:
_close_slot(slot)
closed += 1
else:
kept.append(slot)
for slot in kept:
self._q.put(slot)
if closed:
log.info("BrowserPool: idle cleanup closed %d slot(s)", closed)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Module-level singleton # Module-level singleton
@ -390,7 +380,11 @@ _pool_lock = threading.Lock()
def get_pool() -> BrowserPool: def get_pool() -> BrowserPool:
"""Return the module-level BrowserPool singleton (creates it if needed).""" """Return the module-level BrowserPool singleton (creates it if needed).
Pool size is read from ``BROWSER_POOL_SIZE`` env var (default: 2).
Call ``get_pool().start()`` at FastAPI startup to pre-warm slots.
"""
global _pool global _pool
if _pool is None: if _pool is None:
with _pool_lock: with _pool_lock:

View file

@ -25,7 +25,7 @@ log = logging.getLogger(__name__)
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
from app.db.models import Listing, MarketComp, Seller from app.db.models import Listing, MarketComp, Seller
from app.db.protocol import SharedTableProtocol from app.db.store import Store
from app.platforms import PlatformAdapter, SearchFilters from app.platforms import PlatformAdapter, SearchFilters
EBAY_SEARCH_URL = "https://www.ebay.com/sch/i.html" EBAY_SEARCH_URL = "https://www.ebay.com/sch/i.html"
@ -286,7 +286,7 @@ class ScrapedEbayAdapter(PlatformAdapter):
category_history) cause TrustScorer to set score_is_partial=True. category_history) cause TrustScorer to set score_is_partial=True.
""" """
def __init__(self, shared_store: SharedTableProtocol, delay: float = 1.0): def __init__(self, shared_store: Store, delay: float = 1.0):
self._store = shared_store self._store = shared_store
self._delay = delay self._delay = delay
@ -374,6 +374,8 @@ class ScrapedEbayAdapter(PlatformAdapter):
Does not raise failures per-seller are silently skipped so the main Does not raise failures per-seller are silently skipped so the main
search response is never blocked. search response is never blocked.
""" """
db_path = self._store._db_path # capture for thread-local Store creation
def _enrich_one(item: tuple[str, str]) -> None: def _enrich_one(item: tuple[str, str]) -> None:
seller_id, listing_id = item seller_id, listing_id = item
try: try:
@ -386,7 +388,7 @@ class ScrapedEbayAdapter(PlatformAdapter):
) )
if age_days is None and fb_count is None: if age_days is None and fb_count is None:
return # nothing new to write return # nothing new to write
thread_store = self._store.clone() thread_store = Store(db_path)
seller = thread_store.get_seller("ebay", seller_id) seller = thread_store.get_seller("ebay", seller_id)
if not seller: if not seller:
log.warning("BTF enrich: seller %s not found in DB", seller_id) log.warning("BTF enrich: seller %s not found in DB", seller_id)

View file

@ -1,4 +0,0 @@
"""Mercari platform adapter."""
from app.platforms.mercari.adapter import MercariAdapter
__all__ = ["MercariAdapter"]

View file

@ -1,173 +0,0 @@
"""MercariAdapter — scraper-based Mercari platform adapter.
Trust signal coverage vs eBay:
feedback_count (NumSales from listing page)
feedback_ratio (ReviewStarsWrapper data-stars / 5)
account_age_days (requires seller profile page future work)
category_history (not exposed in HTML future work)
price_vs_market (computed by trust scorer from comps, same as eBay)
Because account_age and category_history are always None, TrustScore.score_is_partial
will be True for all Mercari results. The aggregator handles this correctly
by scoring only from available signals.
seller_platform_id on Listing objects holds the product_id (e.g. "m86032668393")
rather than the seller username, because search results don't expose seller identity.
get_seller() resolves the product_id seller by fetching the listing page.
The DB lookup key is (platform="mercari", platform_seller_id=product_id).
"""
from __future__ import annotations
import json
import logging
import time
from typing import Optional
from app.db.models import Listing, MarketComp, Seller
from app.db.store import Store
from app.platforms import PlatformAdapter, SearchFilters
from app.platforms.mercari.scraper import (
build_search_url,
parse_listing_html,
parse_search_html,
)
log = logging.getLogger(__name__)
_SELLER_CACHE_TTL_HOURS = 6
_BETWEEN_LISTING_FETCH_SECS = 1.5
class MercariAdapter(PlatformAdapter):
def __init__(self, store: Store) -> None:
self._store = store
def search(self, query: str, filters: SearchFilters) -> list[Listing]:
from app.platforms.ebay.browser_pool import get_pool
url = build_search_url(query, filters.max_price, filters.min_price)
log.info("mercari: fetching search URL: %s", url)
html = get_pool().fetch_html(
url,
delay=1.0,
wait_for_timeout_ms=8000,
)
raw_listings = parse_search_html(html)
listings: list[Listing] = []
seen: set[str] = set()
for raw in raw_listings:
pid = raw["product_id"]
if pid in seen:
continue
seen.add(pid)
listings.append(_normalise_listing(raw, query))
log.info("mercari: parsed %d listings for %r", len(listings), query)
# Client-side keyword filter (mirrors eBay scraper behaviour).
if filters.must_include:
listings = _apply_keyword_filter(listings, filters.must_include, filters.must_include_mode)
if filters.must_exclude:
listings = _apply_exclude_filter(listings, filters.must_exclude)
return listings
def get_seller(self, seller_platform_id: str) -> Optional[Seller]:
"""Fetch seller data from the listing page identified by seller_platform_id.
For Mercari, seller_platform_id is the product_id (e.g. "m86032668393")
because seller usernames aren't available from search results HTML.
"""
cached = self._store.get_seller("mercari", seller_platform_id)
if cached:
return cached
from app.platforms.ebay.browser_pool import get_pool
url = f"https://www.mercari.com/us/item/{seller_platform_id}/"
try:
time.sleep(_BETWEEN_LISTING_FETCH_SECS)
html = get_pool().fetch_html(
url,
delay=0.5,
wait_for_timeout_ms=6000,
)
raw = parse_listing_html(html, seller_platform_id)
seller = _normalise_seller(raw)
self._store.save_seller(seller)
return seller
except Exception as exc:
log.warning("mercari: get_seller failed for %s: %s", seller_platform_id, exc)
return None
def get_completed_sales(self, query: str, pages: int = 1) -> list[Listing]:
"""Mercari sold-listing comps — stubbed for Phase 3.
Mercari exposes sold listings via ?status=ITEM_STATUS_TRADING but the
data is sparse. Phase 3 will implement comp extraction here; for now
the trust scorer falls back to price_vs_market=None (partial score).
"""
return []
# ---------------------------------------------------------------------------
# Normalisation helpers
# ---------------------------------------------------------------------------
def _normalise_listing(raw: dict, query: str) -> Listing:
return Listing(
platform="mercari",
platform_listing_id=raw["product_id"],
title=raw["title"],
price=raw["price"],
currency="USD",
condition="", # not available from search results; get_seller() populates this
seller_platform_id=raw["product_id"], # see module docstring
url=raw["url"],
photo_urls=[raw["photo_url"]] if raw.get("photo_url") else [],
listing_age_days=0,
buying_format="fixed_price",
category_name=None,
)
def _normalise_seller(raw: dict) -> Seller:
stars = raw.get("stars", 0.0)
feedback_ratio = min(stars / 5.0, 1.0) if stars > 0 else 0.0
return Seller(
platform="mercari",
platform_seller_id=raw["product_id"],
username=raw.get("username", ""),
account_age_days=None, # not available without seller profile page
feedback_count=raw.get("num_sales", 0),
feedback_ratio=feedback_ratio,
category_history_json=json.dumps({}),
)
def _apply_keyword_filter(listings: list[Listing], must_include: list[str], mode: str) -> list[Listing]:
if not must_include:
return listings
def _matches(listing: Listing) -> bool:
title = listing.title.lower()
if mode == "any":
return any(kw.lower() in title for kw in must_include)
# "all" (default) and "groups" both require all terms present
return all(kw.lower() in title for kw in must_include)
return [l for l in listings if _matches(l)]
def _apply_exclude_filter(listings: list[Listing], must_exclude: list[str]) -> list[Listing]:
if not must_exclude:
return listings
def _clean(listing: Listing) -> bool:
title = listing.title.lower()
return not any(term.lower() in title for term in must_exclude)
return [l for l in listings if _clean(l)]

View file

@ -1,165 +0,0 @@
"""Mercari search + listing page scraper.
Uses the shared eBay browser pool (headed Chromium + Xvfb + playwright-stealth)
which already bypasses Cloudflare Turnstile. Import the pool singleton from
ebay.browser_pool so both platforms share the same warm Chromium instances.
Seller data is NOT available from search results HTML only from individual
listing pages. The adapter lazily fetches listing pages in get_seller().
"""
from __future__ import annotations
import logging
import re
from typing import Optional
from urllib.parse import urlencode
from bs4 import BeautifulSoup, NavigableString
log = logging.getLogger(__name__)
_BASE = "https://www.mercari.com"
_SEARCH_PATH = "/search/"
_ITEM_PATH = "/us/item/"
_PRICE_RE = re.compile(r"[\d,]+\.?\d*")
_POSTED_RE = re.compile(r"(\d{2})/(\d{2})/(\d{2,4})") # MM/DD/YY or MM/DD/YYYY
def build_search_url(query: str, max_price: Optional[float] = None, min_price: Optional[float] = None) -> str:
# No explicit sortBy — Mercari's default (relevance) is the most useful order.
# "sortBy=SORT_SCORE" was a deprecated value that returns an empty results page.
params: dict = {"keyword": query}
# Mercari accepts priceMin/priceMax as whole dollar strings (not cents)
if min_price is not None and min_price > 0:
params["priceMin"] = str(int(min_price))
if max_price is not None and max_price > 0:
params["priceMax"] = str(int(max_price))
return f"{_BASE}{_SEARCH_PATH}?{urlencode(params)}"
def parse_search_html(html: str) -> list[dict]:
"""Parse Mercari search results HTML into a list of raw listing dicts."""
soup = BeautifulSoup(html, "html.parser")
results: list[dict] = []
for item in soup.find_all(attrs={"data-testid": "ItemContainer"}):
pid = item.get("data-productid", "")
if not pid:
continue
parent = item.parent
href = parent.get("href") if parent and parent.name == "a" else None
url = f"{_BASE}{href}" if href else f"{_BASE}{_ITEM_PATH}{pid}/"
name_el = item.find(attrs={"data-testid": "ItemName"})
title = name_el.get_text(strip=True) if name_el else ""
price = _extract_current_price(item)
img_el = item.find("img")
photo_url = img_el.get("src", "") if img_el else ""
results.append({
"product_id": pid,
"url": url,
"title": title,
"price": price,
"photo_url": photo_url,
"brand": item.get("data-brand", ""),
"is_on_sale": item.get("data-is-on-sale") == "true",
})
return results
def _extract_current_price(item: BeautifulSoup) -> float:
"""Return the current (non-strikethrough) price from an ItemContainer."""
price_el = item.find(attrs={"data-testid": "ProductThumbItemPrice"})
if not price_el:
return 0.0
# Direct text nodes are the current price; the nested span is the original.
price_text = "".join(
str(c) for c in price_el.children if isinstance(c, NavigableString)
).strip()
m = _PRICE_RE.search(price_text)
if m:
try:
return float(m.group().replace(",", ""))
except ValueError:
pass
return 0.0
def parse_listing_html(html: str, product_id: str) -> dict:
"""Parse a Mercari listing page into a raw seller dict."""
soup = BeautifulSoup(html, "html.parser")
def _text(testid: str) -> str:
el = soup.find(attrs={"data-testid": testid})
return el.get_text(strip=True) if el else ""
username_raw = _text("ItemDetailsSellerUserName")
username = username_raw.lstrip("@")
num_sales = _safe_int(_text("NumSales"))
rating_count = _safe_int(_text("SellerRatingCount"))
stars = 0.0
rw = soup.find(attrs={"data-testid": "ReviewStarsWrapper"})
if rw:
try:
stars = float(rw.get("data-stars", 0))
except (ValueError, TypeError):
pass
condition = _text("ItemDetailsCondition").lower()
posted_text = _text("ItemDetailsPosted")
listing_age_days = _parse_listing_age(posted_text)
price_text = _text("ItemPrice")
price = 0.0
m = _PRICE_RE.search(price_text.replace(",", ""))
if m:
try:
price = float(m.group())
except ValueError:
pass
return {
"product_id": product_id,
"username": username,
"num_sales": num_sales, # completed sales → maps to feedback_count
"rating_count": rating_count, # number of reviews (additional signal)
"stars": stars, # 0.05.0 → divide by 5 = feedback_ratio
"condition": condition,
"listing_age_days": listing_age_days,
"price": price,
}
def _safe_int(text: str) -> int:
m = _PRICE_RE.search(text.replace(",", ""))
if m:
try:
return int(float(m.group()))
except ValueError:
pass
return 0
def _parse_listing_age(posted_text: str) -> int:
"""Convert a posted date like '04/10/26' to days since posted."""
from datetime import datetime, timezone
m = _POSTED_RE.search(posted_text)
if not m:
return 0
try:
month, day, year = int(m.group(1)), int(m.group(2)), int(m.group(3))
if year < 100:
year += 2000
posted = datetime(year, month, day, tzinfo=timezone.utc)
return (datetime.now(timezone.utc) - posted).days
except (ValueError, OverflowError):
return 0

View file

@ -1,145 +0,0 @@
# app/tasks/monitor.py
"""Background saved-search monitor — polls eBay and writes WatchAlerts for new listings.
Design notes:
- Runs synchronously inside an asyncio.to_thread() call from the polling loop.
- Uses the same eBay adapter + trust scoring pipeline as the live search endpoint.
- Dedup via watch_alerts (saved_search_id, platform_listing_id) UNIQUE constraint.
- Never takes any transactional action alert only.
"""
from __future__ import annotations
import json
import logging
from pathlib import Path
from app.db.models import SavedSearch, WatchAlert
from app.db.store import Store
log = logging.getLogger(__name__)
_AUCTION_ALERT_WINDOW_HOURS = 24 # alert on auctions ending within this window
def should_alert(
*,
trust_score: int,
score_is_partial: bool,
price: float,
buying_format: str,
min_trust_score: int,
ends_at: "str | None" = None,
) -> bool:
"""Return True if a listing qualifies for a watch alert.
BIN (fixed_price / best_offer): alert immediately these sell on a first-come
basis, so speed matters. Require a higher trust bar on partial scores to reduce
false positives while BTF scraping is still in flight.
Auction: only alert when the auction is within _AUCTION_ALERT_WINDOW_HOURS of
ending. Alerting on a 7-day auction 6 days early is noise the user can't act
usefully until the end window anyway. Bid scheduling (paid+) and sniping algo
(premium) are separate features built on top of this alert layer.
"""
from datetime import datetime, timezone
# Partial scores: apply a +10 buffer so we don't surface unreliable signals.
effective_min = min_trust_score + 10 if score_is_partial else min_trust_score
if trust_score < effective_min:
return False
if buying_format in ("fixed_price", "best_offer"):
# BIN: alert immediately — inventory can disappear any time.
return True
if buying_format == "auction":
if not ends_at:
# No end time recorded — alert anyway rather than silently skip.
return True
try:
end = datetime.fromisoformat(ends_at.replace("Z", "+00:00"))
hours_remaining = (end - datetime.now(timezone.utc)).total_seconds() / 3600
return 0 < hours_remaining <= _AUCTION_ALERT_WINDOW_HOURS
except (ValueError, TypeError):
log.debug("should_alert: could not parse ends_at=%r, alerting anyway", ends_at)
return True
# Unknown format — alert and let the user decide.
return True
def run_monitor_search(
search: SavedSearch,
*,
user_db: Path,
shared_db: Path,
) -> int:
"""Execute one background monitor run for a saved search.
Fetches current listings, scores them, writes new high-trust finds
to watch_alerts. Returns the count of new alerts written.
Called from the async polling loop via asyncio.to_thread().
"""
from app.platforms.ebay.adapter import EbayAdapter
from app.trust import TrustScorer
log.info("Monitor: checking saved search %d (%r)", search.id, search.name)
filters = json.loads(search.filters_json or "{}")
query = filters.pop("query_raw", search.query)
try:
adapter = EbayAdapter()
raw_listings = adapter.search(query, **filters)
except Exception as exc:
log.warning("Monitor: eBay search failed for search %d: %s", search.id, exc)
return 0
shared_store = Store(shared_db)
user_store = Store(user_db)
scorer = TrustScorer(shared_store)
try:
trust_scores = scorer.score_batch(raw_listings, query)
except Exception as exc:
log.warning("Monitor: trust scoring failed for search %d: %s", search.id, exc)
return 0
new_alert_count = 0
for listing, trust in zip(raw_listings, trust_scores):
qualifies = should_alert(
trust_score=trust.composite_score,
score_is_partial=trust.score_is_partial,
price=listing.price,
buying_format=listing.buying_format,
min_trust_score=search.min_trust_score,
ends_at=listing.ends_at,
)
if not qualifies:
continue
alert = WatchAlert(
saved_search_id=search.id,
platform_listing_id=listing.platform_listing_id,
title=listing.title,
price=listing.price,
currency=listing.currency,
trust_score=trust.composite_score,
url=listing.url,
)
_, is_new = user_store.upsert_alert(alert)
if is_new:
new_alert_count += 1
log.info(
"Monitor: new alert — search %d, listing %s, score=%d",
search.id, listing.platform_listing_id, trust.composite_score,
)
user_store.mark_search_checked(search.id)
log.info(
"Monitor: search %d done — %d new alerts from %d listings",
search.id, new_alert_count, len(raw_listings),
)
return new_alert_count

View file

@ -7,30 +7,28 @@ Current task types:
trust_photo_analysis download primary photo, run vision LLM, write trust_photo_analysis download primary photo, run vision LLM, write
result to trust_scores.photo_analysis_json (Paid tier). result to trust_scores.photo_analysis_json (Paid tier).
Image assessment routing: Prompt note: The vision prompt is a functional first pass. Tune against real
Cloud (GPU_SERVER_URL set): allocates via cf-orch task endpoint eBay listings before GA specifically stock-photo vs genuine-product distinction
product=snipe, task=image_assessment. and the damage vocabulary.
Local (no GPU_SERVER_URL) or TaskNotFound fallback: uses LLMRouter
with a vision-capable local backend (moondream2, llava, etc.).
""" """
from __future__ import annotations from __future__ import annotations
import base64 import base64
import json import json
import logging import logging
import os
from pathlib import Path from pathlib import Path
import httpx
import requests import requests
from circuitforge_core.db import get_connection from circuitforge_core.db import get_connection
from circuitforge_core.llm import LLMRouter
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
LLM_TASK_TYPES: frozenset[str] = frozenset({"trust_photo_analysis"}) LLM_TASK_TYPES: frozenset[str] = frozenset({"trust_photo_analysis"})
VRAM_BUDGETS: dict[str, float] = { VRAM_BUDGETS: dict[str, float] = {
"trust_photo_analysis": 6000, # Q5_K_M Qwen2-VL via cf-orch; LLMRouter fallback uses 2.0 GB # moondream2 / vision-capable LLM — single image, short response
"trust_photo_analysis": 2.0,
} }
_VISION_SYSTEM_PROMPT = ( _VISION_SYSTEM_PROMPT = (
@ -53,7 +51,8 @@ def insert_task(
) -> tuple[int, bool]: ) -> tuple[int, bool]:
"""Insert a background task if no identical task is already in-flight. """Insert a background task if no identical task is already in-flight.
Returns (task_id, is_new). Uses get_connection() so WAL mode and timeout=30 apply same as all other
Snipe DB access. Returns (task_id, is_new).
""" """
conn = get_connection(db_path) conn = get_connection(db_path)
conn.row_factory = __import__("sqlite3").Row conn.row_factory = __import__("sqlite3").Row
@ -121,26 +120,32 @@ def _run_trust_photo_analysis(
p = json.loads(params or "{}") p = json.loads(params or "{}")
photo_url = p.get("photo_url", "") photo_url = p.get("photo_url", "")
listing_title = p.get("listing_title", "") listing_title = p.get("listing_title", "")
# user_db: per-user DB in cloud mode; same as db_path in local mode.
result_db = Path(p.get("user_db", str(db_path))) result_db = Path(p.get("user_db", str(db_path)))
if not photo_url: if not photo_url:
raise ValueError("trust_photo_analysis: 'photo_url' is required in params") raise ValueError("trust_photo_analysis: 'photo_url' is required in params")
# Download and base64-encode the photo
resp = requests.get(photo_url, timeout=10) resp = requests.get(photo_url, timeout=10)
resp.raise_for_status() resp.raise_for_status()
image_b64 = base64.b64encode(resp.content).decode() image_b64 = base64.b64encode(resp.content).decode()
image_data_url = f"data:image/jpeg;base64,{image_b64}"
user_prompt = "Assess this listing image." # Build user prompt with optional title context
user_prompt = "Evaluate this eBay listing photo."
if listing_title: if listing_title:
user_prompt = f"Assess this eBay listing image: {listing_title}" user_prompt = f"Evaluate this eBay listing photo for: {listing_title}"
cforch_url = os.getenv("GPU_SERVER_URL") or os.getenv("CF_ORCH_URL") # Call LLMRouter with vision capability
if cforch_url: router = LLMRouter()
raw = _assess_via_orch(cforch_url, image_data_url, user_prompt) raw = router.complete(
else: user_prompt,
raw = _assess_via_local_llm(image_b64, user_prompt) system=_VISION_SYSTEM_PROMPT,
images=[image_b64],
max_tokens=128,
)
# Parse — be lenient: strip markdown fences if present
try: try:
cleaned = raw.strip().removeprefix("```json").removeprefix("```").removesuffix("```").strip() cleaned = raw.strip().removeprefix("```json").removeprefix("```").removesuffix("```").strip()
analysis = json.loads(cleaned) analysis = json.loads(cleaned)
@ -163,54 +168,3 @@ def _run_trust_photo_analysis(
analysis.get("visible_damage"), analysis.get("visible_damage"),
analysis.get("confidence"), analysis.get("confidence"),
) )
def _assess_via_orch(cforch_url: str, image_data_url: str, user_prompt: str) -> str:
"""Run photo assessment via cf-orch task endpoint (cloud path)."""
from circuitforge_orch.client import CFOrchClient, TaskNotFound
client = CFOrchClient(cforch_url)
try:
with client.task_allocate("snipe", "image_assessment") as alloc:
resp = httpx.post(
f"{alloc.url}/v1/chat/completions",
json={
"model": alloc.model or "__auto__",
"messages": [
{
"role": "system",
"content": _VISION_SYSTEM_PROMPT,
},
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": image_data_url}},
{"type": "text", "text": user_prompt},
],
},
],
"max_tokens": 128,
},
timeout=60.0,
)
resp.raise_for_status()
return resp.json()["choices"][0]["message"]["content"]
except TaskNotFound:
log.warning(
"snipe.image_assessment not registered in cf-orch — falling back to local LLM"
)
image_b64 = image_data_url.split(",", 1)[1]
return _assess_via_local_llm(image_b64, user_prompt)
def _assess_via_local_llm(image_b64: str, user_prompt: str) -> str:
"""Run photo assessment via local LLMRouter (local/self-hosted path)."""
from app.llm.router import LLMRouter
router = LLMRouter()
return router.complete(
user_prompt,
system=_VISION_SYSTEM_PROMPT,
images=[image_b64],
max_tokens=128,
)

View file

@ -2,7 +2,7 @@ import hashlib
import math import math
from app.db.models import Listing, TrustScore from app.db.models import Listing, TrustScore
from app.db.protocol import SharedTableProtocol from app.db.store import Store
from .aggregator import Aggregator from .aggregator import Aggregator
from .metadata import MetadataScorer from .metadata import MetadataScorer
@ -12,7 +12,7 @@ from .photo import PhotoScorer
class TrustScorer: class TrustScorer:
"""Orchestrates metadata + photo scoring for a batch of listings.""" """Orchestrates metadata + photo scoring for a batch of listings."""
def __init__(self, shared_store: SharedTableProtocol): def __init__(self, shared_store: Store):
self._store = shared_store self._store = shared_store
self._meta = MetadataScorer() self._meta = MetadataScorer()
self._photo = PhotoScorer() self._photo = PhotoScorer()

View file

@ -126,12 +126,7 @@ class Aggregator:
# Hard filters # Hard filters
if seller and seller.account_age_days is not None and seller.account_age_days < HARD_FILTER_AGE_DAYS: if seller and seller.account_age_days is not None and seller.account_age_days < HARD_FILTER_AGE_DAYS:
red_flags.append("new_account") red_flags.append("new_account")
if seller and seller.feedback_ratio == 0.0 and seller.feedback_count > 0: if seller and seller.feedback_ratio < HARD_FILTER_BAD_RATIO_THRESHOLD:
# 12-month ratio missing from page — returning seller or buyer-only account.
# Score will be partial (metadata._feedback_ratio returns None). Soft flag
# only: do NOT fire established_bad_actor on what is likely missing data.
red_flags.append("no_recent_seller_data")
elif seller and seller.feedback_ratio < HARD_FILTER_BAD_RATIO_THRESHOLD:
if HARD_FILTER_BAD_RATIO_MIN_COUNT < seller.feedback_count <= HARD_FILTER_BAD_RATIO_MAX_COUNT: if HARD_FILTER_BAD_RATIO_MIN_COUNT < seller.feedback_count <= HARD_FILTER_BAD_RATIO_MAX_COUNT:
# Moderate-volume account with consistently bad ratio → hard flag. # Moderate-volume account with consistently bad ratio → hard flag.
red_flags.append("established_bad_actor") red_flags.append("established_bad_actor")

View file

@ -44,13 +44,7 @@ class MetadataScorer:
if count < 200: return 15 if count < 200: return 15
return 20 return 20
def _feedback_ratio(self, ratio: float, count: int) -> Optional[int]: def _feedback_ratio(self, ratio: float, count: int) -> int:
# ratio=0.0 with count>0 means the 12-month percentage wasn't on the page —
# eBay omits the ratio for returning/buyer-only sellers with no recent sales.
# Treat as missing rather than "literally 0% positive" (which eBay doesn't allow
# on active accounts — those get suspended long before reaching 0%).
if ratio == 0.0 and count > 0:
return None
if ratio < 0.80 and count > 20: return 0 if ratio < 0.80 and count > 20: return 0
if ratio < 0.90: return 5 if ratio < 0.90: return 5
if ratio < 0.95: return 10 if ratio < 0.95: return 10

View file

@ -20,12 +20,9 @@ services:
CLOUD_MODE: "true" CLOUD_MODE: "true"
CLOUD_DATA_ROOT: /devl/snipe-cloud-data CLOUD_DATA_ROOT: /devl/snipe-cloud-data
# DIRECTUS_JWT_SECRET, HEIMDALL_URL, HEIMDALL_ADMIN_TOKEN — set in .env (never commit) # DIRECTUS_JWT_SECRET, HEIMDALL_URL, HEIMDALL_ADMIN_TOKEN — set in .env (never commit)
# GPU_SERVER_URL routes LLM query builder through cf-orch for VRAM-aware scheduling. # CF_ORCH_URL routes LLM query builder through cf-orch for VRAM-aware scheduling.
# Override in .env to use a different coordinator URL. # Override in .env to use a different coordinator URL.
GPU_SERVER_URL: "http://host.docker.internal:7700" CF_ORCH_URL: "http://host.docker.internal:7700"
# SNIPE_SHARED_DB_URL — Postgres DSN for shared tables (sellers, market_comps, blocklist).
# Required for production multi-user deployments. Set in .env (never commit).
# SNIPE_SHARED_DB_URL: "postgresql://snipe:<password>@postgres:5432/snipe_shared"
CF_APP_NAME: snipe CF_APP_NAME: snipe
extra_hosts: extra_hosts:
- "host.docker.internal:host-gateway" - "host.docker.internal:host-gateway"

View file

@ -18,8 +18,8 @@ services:
environment: environment:
- RELOAD=true - RELOAD=true
# Point the LLM/vision task scheduler at the local cf-orch coordinator. # Point the LLM/vision task scheduler at the local cf-orch coordinator.
# Only has effect when GPU_SERVER_URL is set (uncomment in .env, or set inline). # Only has effect when CF_ORCH_URL is set (uncomment in .env, or set inline).
# - GPU_SERVER_URL=http://10.1.10.71:7700 # - CF_ORCH_URL=http://10.1.10.71:7700
# cf-orch agent — routes trust_photo_analysis vision tasks to the GPU coordinator. # cf-orch agent — routes trust_photo_analysis vision tasks to the GPU coordinator.
# Only starts when you pass --profile orch: # Only starts when you pass --profile orch:

View file

@ -6,7 +6,7 @@
# (claude_code, copilot) are intentionally excluded here. # (claude_code, copilot) are intentionally excluded here.
# #
# CF Orchestrator routes both ollama and vllm allocations for VRAM-aware # CF Orchestrator routes both ollama and vllm allocations for VRAM-aware
# scheduling. GPU_SERVER_URL must be set in .env for allocations to resolve; # scheduling. CF_ORCH_URL must be set in .env for allocations to resolve;
# if cf-orch is unreachable the backend falls back to its static base_url. # if cf-orch is unreachable the backend falls back to its static base_url.
# #
# Model choice for query builder: llama3.1:8b # Model choice for query builder: llama3.1:8b

View file

@ -16,10 +16,6 @@ server {
# Forward the session header injected by Caddy from the cf_session cookie. # Forward the session header injected by Caddy from the cf_session cookie.
# Caddy adds: header_up X-CF-Session {http.request.cookie.cf_session} # Caddy adds: header_up X-CF-Session {http.request.cookie.cf_session}
proxy_set_header X-CF-Session $http_x_cf_session; proxy_set_header X-CF-Session $http_x_cf_session;
# eBay search + comps can take 60-90s (Marketplace Insights 404 → Browse fallback).
# Default 60s proxy_read_timeout drops slow searches with a NetworkError on the client.
proxy_read_timeout 120s;
proxy_send_timeout 120s;
} }
# index.html — never cache; ensures clients always get the latest entry point # index.html — never cache; ensures clients always get the latest entry point

Binary file not shown.

Before

Width:  |  Height:  |  Size: 122 KiB

After

Width:  |  Height:  |  Size: 118 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 164 KiB

View file

@ -8,7 +8,7 @@ version = "0.3.0"
description = "Auction listing monitor and trust scorer" description = "Auction listing monitor and trust scorer"
requires-python = ">=3.11" requires-python = ">=3.11"
dependencies = [ dependencies = [
"circuitforge-core[community]>=0.8.0", "circuitforge-core>=0.8.0",
"streamlit>=1.32", "streamlit>=1.32",
"requests>=2.31", "requests>=2.31",
"imagehash>=4.3", "imagehash>=4.3",
@ -23,20 +23,14 @@ dependencies = [
"playwright-stealth>=1.0", "playwright-stealth>=1.0",
"cryptography>=42.0", "cryptography>=42.0",
"PyJWT>=2.8", "PyJWT>=2.8",
"httpx>=0.27",
] ]
[project.optional-dependencies] [project.optional-dependencies]
orchestration = [
# Paid+ tier only — not published to PyPI. Install from source or Forgejo Packages.
# pip install -e ../circuitforge-orch (dev)
# pip install snipe[orchestration] (self-hosted Paid+)
"circuitforge-orch>=0.1.0",
]
dev = [ dev = [
"pytest>=8.0", "pytest>=8.0",
"pytest-cov>=5.0", "pytest-cov>=5.0",
"ruff>=0.4", "ruff>=0.4",
"httpx>=0.27", # FastAPI test client
] ]
[tool.setuptools.packages.find] [tool.setuptools.packages.find]

View file

@ -1,64 +0,0 @@
"""Reproduce the exact FastAPI code path: pool warmup → slot close → _fetch_fresh.
Run inside the container:
docker exec -it snipe-api-1 python /app/snipe/scripts/debug_fetch_fresh.py
"""
import sys, time, threading
sys.path.insert(0, '/app/snipe')
from bs4 import BeautifulSoup
from app.platforms.ebay.browser_pool import BrowserPool, _close_slot
URL = "https://www.mercari.com/search/?keyword=rtx+4090&sortBy=SORT_SCORE&priceMax=800"
print("=== Test 1: _fetch_fresh with no pool (baseline) ===", flush=True)
pool0 = BrowserPool(size=0)
t0 = time.time()
html = pool0._fetch_fresh(URL, wait_for_timeout_ms=8000)
items = BeautifulSoup(html, "html.parser").find_all(attrs={"data-testid": "ItemContainer"})
print(f"Items: {len(items)}, HTML: {len(html)}b, elapsed: {time.time()-t0:.1f}s", flush=True)
print("\n=== Test 2: pool warmup (size=2), grab slot, close it, then _fetch_fresh ===", flush=True)
pool2 = BrowserPool(size=2)
# Warmup in background (blocks until done)
warm_done = threading.Event()
def do_warmup():
pool2.start()
warm_done.set()
t = threading.Thread(target=do_warmup, daemon=True)
t.start()
warm_done.wait(timeout=30)
print(f"Pool size after warmup: {pool2._q.qsize()}", flush=True)
# Grab a slot and close it (simulating the thread-error path)
import queue
try:
slot = pool2._q.get(timeout=3.0)
print(f"Got slot on display :{slot.display_num}", flush=True)
_close_slot(slot)
print("Slot closed", flush=True)
except queue.Empty:
print("Pool empty — no slot to simulate", flush=True)
# Now call _fetch_fresh in this thread (same as FastAPI handler thread)
print("Calling _fetch_fresh from warmup-thread context...", flush=True)
t0 = time.time()
html2 = pool2._fetch_fresh(URL, wait_for_timeout_ms=8000)
items2 = BeautifulSoup(html2, "html.parser").find_all(attrs={"data-testid": "ItemContainer"})
print(f"Items: {len(items2)}, HTML: {len(html2)}b, elapsed: {time.time()-t0:.1f}s", flush=True)
# Save HTML for inspection if empty
if len(items2) == 0:
with open("/tmp/debug_mercari.html", "w") as f:
f.write(html2)
print("Saved HTML to /tmp/debug_mercari.html", flush=True)
title = BeautifulSoup(html2, "html.parser").find("title")
print("Page title:", title.get_text() if title else "(none)", flush=True)
if "Just a moment" in html2 or "turnstile" in html2.lower():
print("BLOCKED: Cloudflare challenge", flush=True)
else:
body = BeautifulSoup(html2, "html.parser").find("body")
if body:
print("Body snippet:", body.get_text(separator=" ", strip=True)[:300], flush=True)

View file

@ -1,113 +0,0 @@
"""One-shot Mercari probe using the same headed Chromium + Xvfb + stealth stack
as the eBay scraper. Run inside the snipe-api container:
docker exec -it snipe-api-1 python /app/scripts/probe_mercari.py
"""
from __future__ import annotations
import itertools
import os
import subprocess
import sys
import time
_display_counter = itertools.count(200)
_CHROMIUM_ARGS = ["--no-sandbox", "--disable-dev-shm-usage"]
_USER_AGENT = (
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36"
)
SEARCH_URL = "https://www.mercari.com/search/?keyword=rtx+4090"
# Give Cloudflare challenge time to resolve (if it does)
WAIT_MS = 8_000
def probe(url: str) -> str:
from playwright.sync_api import sync_playwright
from playwright_stealth import Stealth
display_num = next(_display_counter)
display = f":{display_num}"
env = os.environ.copy()
env["DISPLAY"] = display
xvfb = subprocess.Popen(
["Xvfb", display, "-screen", "0", "1280x800x24"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
time.sleep(0.5)
try:
with sync_playwright() as pw:
browser = pw.chromium.launch(
headless=False,
env=env,
args=_CHROMIUM_ARGS,
)
ctx = browser.new_context(
user_agent=_USER_AGENT,
viewport={"width": 1280, "height": 800},
)
page = ctx.new_page()
Stealth().apply_stealth_sync(page)
print(f"[probe] Navigating to {url}", flush=True)
response = page.goto(url, wait_until="domcontentloaded", timeout=40_000)
print(f"[probe] HTTP status: {response.status if response else 'unknown'}", flush=True)
print(f"[probe] Waiting {WAIT_MS}ms for JS / Turnstile …", flush=True)
page.wait_for_timeout(WAIT_MS)
html = page.content()
title = page.title()
print(f"[probe] Page title: {title!r}", flush=True)
browser.close()
finally:
xvfb.terminate()
xvfb.wait()
return html
def analyse(html: str) -> None:
from bs4 import BeautifulSoup
soup = BeautifulSoup(html, "html.parser")
# Cloudflare challenge indicators
if "Just a moment" in html or "cf-challenge" in html or "turnstile" in html.lower():
print("[result] BLOCKED — Cloudflare Turnstile still active")
return
print("[result] Cloudflare challenge NOT detected — page appears to have loaded")
# Try to find listing cards
# Mercari US uses data-testid or item cards in the DOM
candidates = [
soup.select("[data-testid='ItemCell']"),
soup.select("[data-testid='item-cell']"),
soup.select("li[data-testid]"),
soup.select(".merList .merListItem"),
soup.select("[class*='ItemCell']"),
soup.select("[class*='item-cell']"),
]
for sel_result in candidates:
if sel_result:
print(f"[result] Found {len(sel_result)} listing card(s) via selector")
card = sel_result[0]
print(f"[result] First card snippet:\n{card.prettify()[:800]}")
return
# Fallback: show body text summary
body = soup.find("body")
text = body.get_text(separator=" ", strip=True)[:500] if body else html[:500]
print(f"[result] No listing cards found. Body text preview:\n{text}")
# Save full HTML for manual inspection
out = "/tmp/mercari_probe.html"
with open(out, "w") as fh:
fh.write(html)
print(f"[result] Full HTML saved to {out}")
if __name__ == "__main__":
html = probe(SEARCH_URL)
analyse(html)

View file

@ -1,17 +0,0 @@
import os
import pytest
def pytest_configure(config):
config.addinivalue_line(
"markers",
"postgres: mark test as requiring a live Postgres instance (SNIPE_SHARED_DB_URL must be set)",
)
@pytest.fixture
def postgres_dsn():
dsn = os.environ.get("SNIPE_SHARED_DB_URL")
if not dsn:
pytest.skip("SNIPE_SHARED_DB_URL not set — skipping Postgres tests")
return dsn

View file

@ -1,157 +0,0 @@
"""Tests for SnipeSharedStore — requires live Postgres via SNIPE_SHARED_DB_URL."""
import pytest
from app.db.models import MarketComp, Seller
from app.db.pg_shared import SnipeSharedDB, SnipeSharedStore
from app.db.protocol import SharedTableProtocol
@pytest.mark.postgres
def test_snipe_shared_store_satisfies_protocol(postgres_dsn):
assert issubclass(SnipeSharedStore, SharedTableProtocol)
@pytest.mark.postgres
def test_save_and_get_seller(postgres_dsn):
db = SnipeSharedDB(postgres_dsn)
db.run_migrations()
store = SnipeSharedStore(db)
seller = Seller(
platform="ebay",
platform_seller_id="test-seller-001",
username="testseller",
account_age_days=365,
feedback_count=100,
feedback_ratio=0.99,
category_history_json='{"electronics": 5}',
)
store.save_seller(seller)
result = store.get_seller("ebay", "test-seller-001")
assert result is not None
assert result.username == "testseller"
assert result.feedback_count == 100
store.delete_seller_data("ebay", "test-seller-001")
db.close()
@pytest.mark.postgres
def test_save_sellers_coalesce_preserves_age(postgres_dsn):
db = SnipeSharedDB(postgres_dsn)
db.run_migrations()
store = SnipeSharedStore(db)
seller_with_age = Seller(
platform="ebay", platform_seller_id="coalesce-test",
username="u", account_age_days=730,
feedback_count=50, feedback_ratio=0.95, category_history_json="{}",
)
store.save_seller(seller_with_age)
seller_without_age = Seller(
platform="ebay", platform_seller_id="coalesce-test",
username="u", account_age_days=None,
feedback_count=60, feedback_ratio=0.96, category_history_json="{}",
)
store.save_sellers([seller_without_age])
result = store.get_seller("ebay", "coalesce-test")
assert result.account_age_days == 730
assert result.feedback_count == 60
store.delete_seller_data("ebay", "coalesce-test")
db.close()
@pytest.mark.postgres
def test_market_comp_cache(postgres_dsn):
from datetime import datetime, timedelta, timezone
db = SnipeSharedDB(postgres_dsn)
db.run_migrations()
store = SnipeSharedStore(db)
expires = (datetime.now(timezone.utc) + timedelta(hours=1)).isoformat()
comp = MarketComp(
platform="ebay", query_hash="abc123",
median_price=49.99, sample_count=10, expires_at=expires,
)
store.save_market_comp(comp)
result = store.get_market_comp("ebay", "abc123")
assert result is not None
assert result.median_price == 49.99
db.close()
@pytest.mark.postgres
def test_reported_sellers(postgres_dsn):
db = SnipeSharedDB(postgres_dsn)
db.run_migrations()
store = SnipeSharedStore(db)
store.mark_reported("ebay", "bad-seller-99", username="badguy")
reported = store.list_reported("ebay")
assert "bad-seller-99" in reported
store.mark_reported("ebay", "bad-seller-99") # idempotent
db.close()
@pytest.mark.postgres
def test_clone_returns_self(postgres_dsn):
db = SnipeSharedDB(postgres_dsn)
store = SnipeSharedStore(db)
assert store.clone() is store
db.close()
@pytest.mark.postgres
def test_blocklist_add_get_remove(postgres_dsn):
from app.db.models import ScammerEntry
db = SnipeSharedDB(postgres_dsn)
db.run_migrations()
store = SnipeSharedStore(db)
assert not store.is_blocklisted("ebay", "bad-999")
entry = store.add_to_blocklist(ScammerEntry(
platform="ebay", platform_seller_id="bad-999",
username="scammer1", reason="sold fakes", source="manual",
))
assert entry.id is not None
assert store.is_blocklisted("ebay", "bad-999")
entries = store.list_blocklist("ebay")
assert any(e.platform_seller_id == "bad-999" for e in entries)
store.remove_from_blocklist("ebay", "bad-999")
assert not store.is_blocklisted("ebay", "bad-999")
db.close()
@pytest.mark.postgres
def test_blocklist_upsert_is_idempotent(postgres_dsn):
from app.db.models import ScammerEntry
db = SnipeSharedDB(postgres_dsn)
db.run_migrations()
store = SnipeSharedStore(db)
store.add_to_blocklist(ScammerEntry(
platform="ebay", platform_seller_id="dup-test",
username="seller", reason="reason1", source="manual",
))
# Second add — should not raise, should update username but preserve reason via COALESCE
store.add_to_blocklist(ScammerEntry(
platform="ebay", platform_seller_id="dup-test",
username="seller_updated", reason=None, source="community",
))
entries = [e for e in store.list_blocklist("ebay") if e.platform_seller_id == "dup-test"]
assert len(entries) == 1
assert entries[0].username == "seller_updated"
assert entries[0].reason == "reason1" # COALESCE preserved original reason
store.remove_from_blocklist("ebay", "dup-test")
db.close()

View file

@ -1,39 +0,0 @@
"""Verify Store satisfies SharedTableProtocol at import time."""
from app.db.protocol import SharedTableProtocol
from app.db.store import Store
def test_store_satisfies_protocol():
assert issubclass(Store, SharedTableProtocol)
def test_store_clone_returns_new_instance(tmp_path):
db = tmp_path / "test.db"
s = Store(db)
clone = s.clone()
assert isinstance(clone, Store)
assert clone is not s
assert clone._db_path == db
def test_ebay_adapter_accepts_protocol():
from app.platforms.ebay.adapter import EbayAdapter
import tempfile
import pathlib
from unittest.mock import MagicMock
with tempfile.TemporaryDirectory() as tmp:
s = Store(pathlib.Path(tmp) / "t.db")
adapter = EbayAdapter(token_manager=MagicMock(), shared_store=s)
assert adapter._store is s
def test_scraped_adapter_no_db_path_ref():
from app.platforms.ebay.scraper import ScrapedEbayAdapter
import tempfile
import pathlib
with tempfile.TemporaryDirectory() as tmp:
s = Store(pathlib.Path(tmp) / "t.db")
adapter = ScrapedEbayAdapter(shared_store=s)
assert not hasattr(adapter, '_db_path_ref')

View file

@ -1,15 +1,16 @@
"""Tests for app.platforms.ebay.browser_pool (thread-local design). """Tests for app.platforms.ebay.browser_pool.
All tests run without real Chromium / Xvfb / Playwright. All tests run without real Chromium / Xvfb / Playwright.
Playwright, Xvfb subprocess calls, and Stealth are mocked throughout. Playwright, Xvfb subprocess calls, and Stealth are mocked throughout.
""" """
from __future__ import annotations from __future__ import annotations
import queue
import subprocess import subprocess
import threading import threading
import time import time
from typing import Any from typing import Any
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch, call
import pytest import pytest
@ -18,35 +19,40 @@ import pytest
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def _reset_pool_singleton(): def _reset_pool_singleton():
"""Force the module-level _pool singleton back to None."""
import app.platforms.ebay.browser_pool as _mod import app.platforms.ebay.browser_pool as _mod
_mod._pool = None _mod._pool = None
def _reset_thread_local(): # ---------------------------------------------------------------------------
import app.platforms.ebay.browser_pool as _mod # Fixtures
_mod._thread_local.slot = None # ---------------------------------------------------------------------------
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def reset_pool(): def reset_singleton():
"""Reset the singleton before and after every test."""
_reset_pool_singleton() _reset_pool_singleton()
_reset_thread_local()
yield yield
_reset_pool_singleton() _reset_pool_singleton()
_reset_thread_local()
def _make_fake_slot(): def _make_fake_slot():
"""Build a mock _PooledBrowser with all necessary attributes."""
from app.platforms.ebay.browser_pool import _PooledBrowser from app.platforms.ebay.browser_pool import _PooledBrowser
xvfb = MagicMock(spec=subprocess.Popen) xvfb = MagicMock(spec=subprocess.Popen)
pw = MagicMock() pw = MagicMock()
browser = MagicMock() browser = MagicMock()
ctx = MagicMock() ctx = MagicMock()
return _PooledBrowser( slot = _PooledBrowser(
xvfb=xvfb, pw=pw, browser=browser, ctx=ctx, xvfb=xvfb,
display_num=100, last_used_ts=time.time(), pw=pw,
browser=browser,
ctx=ctx,
display_num=100,
last_used_ts=time.time(),
) )
return slot
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@ -56,7 +62,9 @@ def _make_fake_slot():
class TestGetPoolSingleton: class TestGetPoolSingleton:
def test_returns_same_instance(self): def test_returns_same_instance(self):
from app.platforms.ebay.browser_pool import get_pool, BrowserPool from app.platforms.ebay.browser_pool import get_pool, BrowserPool
assert get_pool() is get_pool() p1 = get_pool()
p2 = get_pool()
assert p1 is p2
def test_returns_browser_pool_instance(self): def test_returns_browser_pool_instance(self):
from app.platforms.ebay.browser_pool import get_pool, BrowserPool from app.platforms.ebay.browser_pool import get_pool, BrowserPool
@ -64,12 +72,14 @@ class TestGetPoolSingleton:
def test_default_size_is_two(self): def test_default_size_is_two(self):
from app.platforms.ebay.browser_pool import get_pool from app.platforms.ebay.browser_pool import get_pool
assert get_pool()._size == 2 pool = get_pool()
assert pool._size == 2
def test_custom_size_from_env(self, monkeypatch): def test_custom_size_from_env(self, monkeypatch):
monkeypatch.setenv("BROWSER_POOL_SIZE", "5") monkeypatch.setenv("BROWSER_POOL_SIZE", "5")
from app.platforms.ebay.browser_pool import get_pool from app.platforms.ebay.browser_pool import get_pool
assert get_pool()._size == 5 pool = get_pool()
assert pool._size == 5
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@ -78,15 +88,17 @@ class TestGetPoolSingleton:
class TestLifecycle: class TestLifecycle:
def test_start_is_noop_when_playwright_unavailable(self): def test_start_is_noop_when_playwright_unavailable(self):
"""Pool should handle missing Playwright gracefully — no error raised."""
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=2) pool = BrowserPool(size=2)
with patch.object(pool, "_check_playwright", return_value=False): with patch.object(pool, "_check_playwright", return_value=False):
pool.start() pool.start() # must not raise
assert pool._started is True # Pool queue is empty — no slots launched.
assert pool._slot_registry == {} assert pool._q.empty()
def test_start_only_runs_once(self): def test_start_only_runs_once(self):
"""Calling start() twice must not double-warm."""
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=1) pool = BrowserPool(size=1)
@ -95,68 +107,68 @@ class TestLifecycle:
pool.start() pool.start()
assert pool._started is True assert pool._started is True
def test_stop_closes_all_registry_slots(self): def test_stop_drains_queue(self):
"""stop() should close every slot in the queue."""
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=2) pool = BrowserPool(size=2)
slot1 = _make_fake_slot() slot1 = _make_fake_slot()
slot2 = _make_fake_slot() slot2 = _make_fake_slot()
pool._slot_registry[1001] = slot1 pool._q.put(slot1)
pool._slot_registry[1002] = slot2 pool._q.put(slot2)
with patch("app.platforms.ebay.browser_pool._close_slot") as mock_close: with patch("app.platforms.ebay.browser_pool._close_slot") as mock_close:
pool.stop() pool.stop()
assert mock_close.call_count == 2 assert mock_close.call_count == 2
assert pool._slot_registry == {} assert pool._q.empty()
assert pool._stopped is True assert pool._stopped is True
def test_stop_on_empty_registry_is_safe(self): def test_stop_on_empty_pool_is_safe(self):
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
BrowserPool(size=2).stop() pool = BrowserPool(size=2)
pool.stop() # must not raise
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# fetch_html — thread-local slot hit path # fetch_html — pool hit path
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
class TestFetchHtmlSlotHit: class TestFetchHtmlPoolHit:
def test_uses_existing_slot_and_replenishes(self): def test_uses_pooled_slot_and_replenishes(self):
"""fetch_html should borrow a slot, call _fetch_with_slot, replenish."""
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
import app.platforms.ebay.browser_pool as _mod
pool = BrowserPool(size=1) pool = BrowserPool(size=1)
slot = _make_fake_slot() slot = _make_fake_slot()
_mod._thread_local.slot = slot pool._q.put(slot)
fresh_slot = _make_fake_slot() fresh_slot = _make_fake_slot()
with ( with (
patch.object(pool, "_fetch_with_slot", return_value="<html>ok</html>") as mock_fetch, patch.object(pool, "_fetch_with_slot", return_value="<html>ok</html>") as mock_fetch,
patch("app.platforms.ebay.browser_pool._replenish_slot", return_value=fresh_slot), patch("app.platforms.ebay.browser_pool._replenish_slot", return_value=fresh_slot) as mock_replenish,
patch.object(pool, "_register_slot") as mock_register,
patch("time.sleep"), patch("time.sleep"),
): ):
html = pool.fetch_html("https://www.ebay.com/sch/i.html?_nkw=test", delay=0) html = pool.fetch_html("https://www.ebay.com/sch/i.html?_nkw=test", delay=0)
assert html == "<html>ok</html>" assert html == "<html>ok</html>"
mock_fetch.assert_called_once_with( mock_fetch.assert_called_once_with(slot, "https://www.ebay.com/sch/i.html?_nkw=test")
slot, "https://www.ebay.com/sch/i.html?_nkw=test", mock_replenish.assert_called_once_with(slot)
wait_for_selector=None, wait_for_timeout_ms=2000, # Fresh slot returned to queue
) assert pool._q.get_nowait() is fresh_slot
mock_register.assert_called_once_with(fresh_slot)
def test_delay_is_respected(self): def test_delay_is_respected(self):
"""fetch_html must call time.sleep(delay)."""
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
import app.platforms.ebay.browser_pool as _mod
pool = BrowserPool(size=1) pool = BrowserPool(size=1)
_mod._thread_local.slot = _make_fake_slot() slot = _make_fake_slot()
pool._q.put(slot)
with ( with (
patch.object(pool, "_fetch_with_slot", return_value="<html/>"), patch.object(pool, "_fetch_with_slot", return_value="<html/>"),
patch("app.platforms.ebay.browser_pool._replenish_slot", return_value=_make_fake_slot()), patch("app.platforms.ebay.browser_pool._replenish_slot", return_value=_make_fake_slot()),
patch.object(pool, "_register_slot"),
patch("app.platforms.ebay.browser_pool.time") as mock_time, patch("app.platforms.ebay.browser_pool.time") as mock_time,
): ):
pool.fetch_html("https://example.com", delay=1.5) pool.fetch_html("https://example.com", delay=1.5)
@ -165,41 +177,40 @@ class TestFetchHtmlSlotHit:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# fetch_html — no slot / fallback path # fetch_html — pool empty / fallback path
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
class TestFetchHtmlFallback: class TestFetchHtmlFallback:
def test_falls_back_when_no_slot_and_playwright_unavailable(self): def test_falls_back_to_fresh_browser_when_pool_empty(self):
"""When pool is empty after timeout, _fetch_fresh should be called."""
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=1) pool = BrowserPool(size=1)
# No thread-local slot; playwright unavailable → _get_or_create returns None. # Queue is empty — no slots available.
with ( with (
patch.object(pool, "_get_or_create_thread_slot", return_value=None),
patch.object(pool, "_fetch_fresh", return_value="<html>fresh</html>") as mock_fresh, patch.object(pool, "_fetch_fresh", return_value="<html>fresh</html>") as mock_fresh,
patch("time.sleep"), patch("time.sleep"),
# Make Queue.get raise Empty after a short wait.
patch.object(pool._q, "get", side_effect=queue.Empty),
): ):
html = pool.fetch_html("https://www.ebay.com/sch/i.html?_nkw=widget", delay=0) html = pool.fetch_html("https://www.ebay.com/sch/i.html?_nkw=widget", delay=0)
assert html == "<html>fresh</html>" assert html == "<html>fresh</html>"
mock_fresh.assert_called_once_with( mock_fresh.assert_called_once_with("https://www.ebay.com/sch/i.html?_nkw=widget")
"https://www.ebay.com/sch/i.html?_nkw=widget",
wait_for_selector=None, wait_for_timeout_ms=2000,
)
def test_falls_back_when_pooled_fetch_raises(self): def test_falls_back_when_pooled_fetch_raises(self):
"""If _fetch_with_slot raises, the slot is closed and _fetch_fresh is used."""
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
import app.platforms.ebay.browser_pool as _mod
pool = BrowserPool(size=1) pool = BrowserPool(size=1)
slot = _make_fake_slot() slot = _make_fake_slot()
_mod._thread_local.slot = slot pool._q.put(slot)
with ( with (
patch.object(pool, "_fetch_with_slot", side_effect=RuntimeError("Chromium crashed")), patch.object(pool, "_fetch_with_slot", side_effect=RuntimeError("Chromium crashed")),
patch.object(pool, "_fetch_fresh", return_value="<html>recovered</html>") as mock_fresh, patch.object(pool, "_fetch_fresh", return_value="<html>recovered</html>") as mock_fresh,
patch("app.platforms.ebay.browser_pool._close_slot") as mock_close, patch("app.platforms.ebay.browser_pool._close_slot") as mock_close,
patch.object(pool, "_unregister_slot"),
patch("time.sleep"), patch("time.sleep"),
): ):
html = pool.fetch_html("https://www.ebay.com/", delay=0) html = pool.fetch_html("https://www.ebay.com/", delay=0)
@ -209,107 +220,19 @@ class TestFetchHtmlFallback:
mock_fresh.assert_called_once() mock_fresh.assert_called_once()
# ---------------------------------------------------------------------------
# Thread-local slot management
# ---------------------------------------------------------------------------
class TestThreadLocalSlotManagement:
def test_get_or_create_returns_existing_slot(self):
import app.platforms.ebay.browser_pool as _mod
from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=1)
pool._playwright_available = True
existing = _make_fake_slot()
_mod._thread_local.slot = existing
result = pool._get_or_create_thread_slot()
assert result is existing
def test_get_or_create_launches_new_slot_when_absent(self):
import app.platforms.ebay.browser_pool as _mod
from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=1)
pool._playwright_available = True
_mod._thread_local.slot = None
new_slot = _make_fake_slot()
with (
patch("app.platforms.ebay.browser_pool._launch_slot", return_value=new_slot),
patch.object(pool, "_register_slot") as mock_register,
):
result = pool._get_or_create_thread_slot()
assert result is new_slot
mock_register.assert_called_once_with(new_slot)
def test_get_or_create_returns_none_when_playwright_unavailable(self):
from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=1)
pool._playwright_available = False
assert pool._get_or_create_thread_slot() is None
def test_register_slot_sets_thread_local_and_registry(self):
import app.platforms.ebay.browser_pool as _mod
from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=1)
slot = _make_fake_slot()
pool._register_slot(slot)
assert _mod._thread_local.slot is slot
assert threading.get_ident() in pool._slot_registry
def test_unregister_slot_clears_thread_local_and_registry(self):
import app.platforms.ebay.browser_pool as _mod
from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=1)
slot = _make_fake_slot()
pool._register_slot(slot)
pool._unregister_slot()
assert getattr(_mod._thread_local, "slot", None) is None
assert threading.get_ident() not in pool._slot_registry
def test_different_threads_get_independent_slots(self):
from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=2)
pool._playwright_available = True
slots_seen: list = []
errors: list = []
def worker():
new_slot = _make_fake_slot()
with patch("app.platforms.ebay.browser_pool._launch_slot", return_value=new_slot):
s = pool._get_or_create_thread_slot()
slots_seen.append(s)
t1 = threading.Thread(target=worker)
t2 = threading.Thread(target=worker)
t1.start(); t2.start()
t1.join(); t2.join()
assert len(slots_seen) == 2
# Each thread got its own slot object (they may differ or coincidentally share
# the same mock; what matters is both threads succeeded without interference).
assert all(s is not None for s in slots_seen)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# ImportError graceful fallback # ImportError graceful fallback
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
class TestImportErrorHandling: class TestImportErrorHandling:
def test_check_playwright_returns_false_on_import_error(self): def test_check_playwright_returns_false_on_import_error(self):
"""_check_playwright should cache False when playwright is not installed."""
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=2) pool = BrowserPool(size=2)
with patch.dict("sys.modules", {"playwright": None, "playwright_stealth": None}): with patch.dict("sys.modules", {"playwright": None, "playwright_stealth": None}):
# Force re-check by clearing the cached value.
pool._playwright_available = None pool._playwright_available = None
result = pool._check_playwright() result = pool._check_playwright()
@ -317,11 +240,12 @@ class TestImportErrorHandling:
assert pool._playwright_available is False assert pool._playwright_available is False
def test_start_logs_warning_when_playwright_missing(self, caplog): def test_start_logs_warning_when_playwright_missing(self, caplog):
"""start() should log a warning and not crash when Playwright is absent."""
import logging import logging
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=1) pool = BrowserPool(size=1)
pool._playwright_available = False pool._playwright_available = False # simulate missing
with patch.object(pool, "_check_playwright", return_value=False): with patch.object(pool, "_check_playwright", return_value=False):
with caplog.at_level(logging.WARNING, logger="app.platforms.ebay.browser_pool"): with caplog.at_level(logging.WARNING, logger="app.platforms.ebay.browser_pool"):
@ -330,14 +254,87 @@ class TestImportErrorHandling:
assert any("not available" in r.message for r in caplog.records) assert any("not available" in r.message for r in caplog.records)
def test_fetch_fresh_raises_runtime_error_when_playwright_missing(self): def test_fetch_fresh_raises_runtime_error_when_playwright_missing(self):
"""_fetch_fresh must raise RuntimeError (not ImportError) when PW absent."""
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
pool = BrowserPool(size=1) pool = BrowserPool(size=1)
with patch.dict("sys.modules", {"playwright": None, "playwright.sync_api": None}): with patch.dict("sys.modules", {"playwright": None, "playwright.sync_api": None}):
with pytest.raises(RuntimeError, match="Playwright not installed"): with pytest.raises(RuntimeError, match="Playwright not installed"):
pool._fetch_fresh("https://www.ebay.com/") pool._fetch_fresh("https://www.ebay.com/")
# ---------------------------------------------------------------------------
# Idle cleanup
# ---------------------------------------------------------------------------
class TestIdleCleanup:
def test_idle_cleanup_closes_stale_slots(self):
"""_idle_cleanup_loop should close slots whose last_used_ts is too old."""
from app.platforms.ebay.browser_pool import BrowserPool, _IDLE_TIMEOUT_SECS
pool = BrowserPool(size=2)
stale_slot = _make_fake_slot()
stale_slot.last_used_ts = time.time() - (_IDLE_TIMEOUT_SECS + 60)
fresh_slot = _make_fake_slot()
fresh_slot.last_used_ts = time.time()
pool._q.put(stale_slot)
pool._q.put(fresh_slot)
closed_slots = []
def fake_close(s):
closed_slots.append(s)
with patch("app.platforms.ebay.browser_pool._close_slot", side_effect=fake_close):
# Run one cleanup tick directly (not the full loop).
now = time.time()
idle_cutoff = now - _IDLE_TIMEOUT_SECS
kept = []
while True:
try:
s = pool._q.get_nowait()
except queue.Empty:
break
if s.last_used_ts < idle_cutoff:
fake_close(s)
else:
kept.append(s)
for s in kept:
pool._q.put(s)
assert stale_slot in closed_slots
assert fresh_slot not in closed_slots
assert pool._q.qsize() == 1
def test_idle_cleanup_loop_stops_when_pool_stopped(self):
"""Cleanup daemon should exit when _stopped is True."""
from app.platforms.ebay.browser_pool import BrowserPool, _CLEANUP_INTERVAL_SECS
pool = BrowserPool(size=1)
pool._stopped = True
# The loop should return after one iteration of the while check.
# Use a very short sleep mock so the test doesn't actually wait 60s.
sleep_calls = []
def fake_sleep(secs):
sleep_calls.append(secs)
with patch("app.platforms.ebay.browser_pool.time") as mock_time:
mock_time.time.return_value = time.time()
mock_time.sleep.side_effect = fake_sleep
# Run in a thread with a short timeout to confirm it exits.
t = threading.Thread(target=pool._idle_cleanup_loop)
t.start()
t.join(timeout=2.0)
assert not t.is_alive(), "idle cleanup loop did not exit when _stopped=True"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# _replenish_slot helper # _replenish_slot helper
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@ -352,8 +349,12 @@ class TestReplenishSlot:
browser.new_context.return_value = new_ctx browser.new_context.return_value = new_ctx
slot = _PooledBrowser( slot = _PooledBrowser(
xvfb=MagicMock(), pw=MagicMock(), browser=browser, xvfb=MagicMock(),
ctx=old_ctx, display_num=101, last_used_ts=time.time() - 10, pw=MagicMock(),
browser=browser,
ctx=old_ctx,
display_num=101,
last_used_ts=time.time() - 10,
) )
result = _replenish_slot(slot) result = _replenish_slot(slot)
@ -363,6 +364,7 @@ class TestReplenishSlot:
assert result.ctx is new_ctx assert result.ctx is new_ctx
assert result.browser is browser assert result.browser is browser
assert result.xvfb is slot.xvfb assert result.xvfb is slot.xvfb
# last_used_ts is refreshed
assert result.last_used_ts > slot.last_used_ts assert result.last_used_ts > slot.last_used_ts
@ -383,6 +385,7 @@ class TestCloseSlot:
xvfb=xvfb, pw=pw, browser=browser, ctx=ctx, xvfb=xvfb, pw=pw, browser=browser, ctx=ctx,
display_num=102, last_used_ts=time.time(), display_num=102, last_used_ts=time.time(),
) )
_close_slot(slot) _close_slot(slot)
ctx.close.assert_called_once() ctx.close.assert_called_once()
@ -392,6 +395,7 @@ class TestCloseSlot:
xvfb.wait.assert_called_once() xvfb.wait.assert_called_once()
def test_close_slot_ignores_exceptions(self): def test_close_slot_ignores_exceptions(self):
"""_close_slot must not raise even if components throw."""
from app.platforms.ebay.browser_pool import _close_slot, _PooledBrowser from app.platforms.ebay.browser_pool import _close_slot, _PooledBrowser
xvfb = MagicMock(spec=subprocess.Popen) xvfb = MagicMock(spec=subprocess.Popen)
@ -408,6 +412,7 @@ class TestCloseSlot:
xvfb=xvfb, pw=pw, browser=browser, ctx=ctx, xvfb=xvfb, pw=pw, browser=browser, ctx=ctx,
display_num=103, last_used_ts=time.time(), display_num=103, last_used_ts=time.time(),
) )
_close_slot(slot) # must not raise _close_slot(slot) # must not raise
@ -417,6 +422,7 @@ class TestCloseSlot:
class TestScraperUsesPool: class TestScraperUsesPool:
def test_fetch_url_delegates_to_pool(self): def test_fetch_url_delegates_to_pool(self):
"""ScrapedEbayAdapter._fetch_url must use the pool, not launch its own browser."""
from app.platforms.ebay.browser_pool import BrowserPool from app.platforms.ebay.browser_pool import BrowserPool
from app.platforms.ebay.scraper import ScrapedEbayAdapter from app.platforms.ebay.scraper import ScrapedEbayAdapter
from app.db.store import Store from app.db.store import Store
@ -428,6 +434,7 @@ class TestScraperUsesPool:
fake_pool.fetch_html.return_value = "<html>pooled</html>" fake_pool.fetch_html.return_value = "<html>pooled</html>"
with patch("app.platforms.ebay.browser_pool.get_pool", return_value=fake_pool): with patch("app.platforms.ebay.browser_pool.get_pool", return_value=fake_pool):
# Clear the cache so fetch_url actually hits the pool.
import app.platforms.ebay.scraper as scraper_mod import app.platforms.ebay.scraper as scraper_mod
scraper_mod._html_cache.clear() scraper_mod._html_cache.clear()
html = adapter._fetch_url("https://www.ebay.com/sch/i.html?_nkw=test") html = adapter._fetch_url("https://www.ebay.com/sch/i.html?_nkw=test")
@ -438,6 +445,7 @@ class TestScraperUsesPool:
) )
def test_fetch_url_uses_cache_before_pool(self): def test_fetch_url_uses_cache_before_pool(self):
"""_fetch_url should return cached HTML without hitting the pool."""
from app.platforms.ebay.scraper import ScrapedEbayAdapter, _html_cache, _HTML_CACHE_TTL from app.platforms.ebay.scraper import ScrapedEbayAdapter, _html_cache, _HTML_CACHE_TTL
from app.db.store import Store from app.db.store import Store
@ -453,4 +461,6 @@ class TestScraperUsesPool:
assert html == "<html>cached</html>" assert html == "<html>cached</html>"
fake_pool.fetch_html.assert_not_called() fake_pool.fetch_html.assert_not_called()
# Cleanup
_html_cache.pop(url, None) _html_cache.pop(url, None)

View file

@ -1,4 +1,4 @@
"""Unit tests for QueryTranslator — LLMRouter and cf-orch backends mocked at boundary.""" """Unit tests for QueryTranslator — LLMRouter mocked at boundary."""
from __future__ import annotations from __future__ import annotations
import json import json
@ -73,7 +73,7 @@ def test_parse_response_missing_required_field():
_parse_response(raw) _parse_response(raw)
# ── Fixtures ────────────────────────────────────────────────────────────────── # ── QueryTranslator (integration with mocked LLMRouter) ──────────────────────
from app.platforms.ebay.categories import EbayCategoryCache from app.platforms.ebay.categories import EbayCategoryCache
from circuitforge_core.db import get_connection, run_migrations from circuitforge_core.db import get_connection, run_migrations
@ -88,7 +88,16 @@ def db_with_categories(tmp_path):
return conn return conn
_VALID_LLM_RESPONSE = json.dumps({ def _make_translator(db_conn, llm_response: str) -> QueryTranslator:
from app.platforms.ebay.categories import EbayCategoryCache
cache = EbayCategoryCache(db_conn)
mock_router = MagicMock()
mock_router.complete.return_value = llm_response
return QueryTranslator(category_cache=cache, llm_router=mock_router)
def test_translate_returns_search_params(db_with_categories):
llm_out = json.dumps({
"base_query": "RTX 3080", "base_query": "RTX 3080",
"must_include_mode": "groups", "must_include_mode": "groups",
"must_include": "rtx|geforce, 3080", "must_include": "rtx|geforce, 3080",
@ -99,20 +108,7 @@ _VALID_LLM_RESPONSE = json.dumps({
"category_id": "27386", "category_id": "27386",
"explanation": "Searching for used RTX 3080 GPUs under $300.", "explanation": "Searching for used RTX 3080 GPUs under $300.",
}) })
t = _make_translator(db_with_categories, llm_out)
# ── Local LLMRouter backend ───────────────────────────────────────────────────
def _make_local_translator(db_conn, llm_response: str) -> QueryTranslator:
from app.platforms.ebay.categories import EbayCategoryCache
cache = EbayCategoryCache(db_conn)
mock_router = MagicMock()
mock_router.complete.return_value = llm_response
return QueryTranslator(category_cache=cache, llm_router=mock_router)
def test_translate_returns_search_params(db_with_categories):
t = _make_local_translator(db_with_categories, _VALID_LLM_RESPONSE)
result = t.translate("used RTX 3080 under $300 no mining") result = t.translate("used RTX 3080 under $300 no mining")
assert result.base_query == "RTX 3080" assert result.base_query == "RTX 3080"
assert result.max_price == 300.0 assert result.max_price == 300.0
@ -120,7 +116,18 @@ def test_translate_returns_search_params(db_with_categories):
def test_translate_injects_category_hints(db_with_categories): def test_translate_injects_category_hints(db_with_categories):
"""The system prompt sent to the LLM must contain category_id hints.""" """The system prompt sent to the LLM must contain category_id hints."""
t = _make_local_translator(db_with_categories, _VALID_LLM_RESPONSE) llm_out = json.dumps({
"base_query": "GPU",
"must_include_mode": "all",
"must_include": "",
"must_exclude": "",
"max_price": None,
"min_price": None,
"condition": [],
"category_id": None,
"explanation": "Searching for GPUs.",
})
t = _make_translator(db_with_categories, llm_out)
t.translate("GPU") t.translate("GPU")
call_args = t._llm_router.complete.call_args call_args = t._llm_router.complete.call_args
system_prompt = call_args.kwargs.get("system") or call_args.args[1] system_prompt = call_args.kwargs.get("system") or call_args.args[1]
@ -134,7 +141,7 @@ def test_translate_empty_category_cache_still_works(tmp_path):
conn = get_connection(tmp_path / "empty.db") conn = get_connection(tmp_path / "empty.db")
run_migrations(conn, Path("app/db/migrations")) run_migrations(conn, Path("app/db/migrations"))
# Do NOT seed bootstrap — empty cache # Do NOT seed bootstrap — empty cache
t = _make_local_translator(conn, json.dumps({ llm_out = json.dumps({
"base_query": "vinyl", "base_query": "vinyl",
"must_include_mode": "all", "must_include_mode": "all",
"must_include": "", "must_include": "",
@ -144,7 +151,8 @@ def test_translate_empty_category_cache_still_works(tmp_path):
"condition": [], "condition": [],
"category_id": None, "category_id": None,
"explanation": "Searching for vinyl records.", "explanation": "Searching for vinyl records.",
})) })
t = _make_translator(conn, llm_out)
result = t.translate("vinyl records") result = t.translate("vinyl records")
assert result.base_query == "vinyl" assert result.base_query == "vinyl"
call_args = t._llm_router.complete.call_args call_args = t._llm_router.complete.call_args
@ -160,101 +168,3 @@ def test_translate_llm_error_raises_query_translator_error(db_with_categories):
t = QueryTranslator(category_cache=cache, llm_router=mock_router) t = QueryTranslator(category_cache=cache, llm_router=mock_router)
with pytest.raises(QueryTranslatorError, match="LLM backend"): with pytest.raises(QueryTranslatorError, match="LLM backend"):
t.translate("used GPU") t.translate("used GPU")
# ── cf-orch backend ───────────────────────────────────────────────────────────
def _make_orch_translator(db_conn) -> QueryTranslator:
from app.platforms.ebay.categories import EbayCategoryCache
cache = EbayCategoryCache(db_conn)
return QueryTranslator(category_cache=cache, cforch_url="http://orch.local:8700")
def _mock_alloc_response() -> MagicMock:
resp = MagicMock()
resp.json.return_value = {
"url": "http://cf-text.local:11434",
"allocation_id": "alloc-abc123",
"node_id": "heimdall",
}
resp.raise_for_status.return_value = None
return resp
def _mock_chat_response(content: str) -> MagicMock:
resp = MagicMock()
resp.json.return_value = {
"choices": [{"message": {"content": content}}]
}
resp.raise_for_status.return_value = None
return resp
def _mock_delete_response() -> MagicMock:
resp = MagicMock()
resp.raise_for_status.return_value = None
return resp
def test_orch_translate_returns_search_params(db_with_categories):
t = _make_orch_translator(db_with_categories)
with patch("httpx.post") as mock_post, patch("httpx.delete") as mock_delete:
mock_post.side_effect = [
_mock_alloc_response(),
_mock_chat_response(_VALID_LLM_RESPONSE),
]
mock_delete.return_value = _mock_delete_response()
result = t.translate("used RTX 3080 under $300")
assert result.base_query == "RTX 3080"
assert result.max_price == 300.0
def test_orch_allocates_with_correct_task_tag(db_with_categories):
t = _make_orch_translator(db_with_categories)
with patch("httpx.post") as mock_post, patch("httpx.delete"):
mock_post.side_effect = [
_mock_alloc_response(),
_mock_chat_response(_VALID_LLM_RESPONSE),
]
t.translate("GPU")
alloc_call = mock_post.call_args_list[0]
assert alloc_call.args[0] == "http://orch.local:8700/api/inference/task"
body = alloc_call.kwargs.get("json") or alloc_call.args[1]
assert body == {"product": "snipe", "task": "query_translation"}
def test_orch_releases_allocation_after_success(db_with_categories):
t = _make_orch_translator(db_with_categories)
with patch("httpx.post") as mock_post, patch("httpx.delete") as mock_delete:
mock_post.side_effect = [
_mock_alloc_response(),
_mock_chat_response(_VALID_LLM_RESPONSE),
]
mock_delete.return_value = _mock_delete_response()
t.translate("GPU")
mock_delete.assert_called_once()
delete_url = mock_delete.call_args.args[0]
assert "alloc-abc123" in delete_url
def test_orch_releases_allocation_on_inference_failure(db_with_categories):
"""Allocation must be released even when the inference call fails."""
t = _make_orch_translator(db_with_categories)
with patch("httpx.post") as mock_post, patch("httpx.delete") as mock_delete:
mock_post.side_effect = [
_mock_alloc_response(),
Exception("inference timeout"),
]
mock_delete.return_value = _mock_delete_response()
with pytest.raises(QueryTranslatorError, match="LLM backend"):
t.translate("GPU")
mock_delete.assert_called_once()
def test_init_requires_at_least_one_backend(tmp_path):
from circuitforge_core.db import get_connection, run_migrations
conn = get_connection(tmp_path / "test.db")
run_migrations(conn, Path("app/db/migrations"))
cache = EbayCategoryCache(conn)
with pytest.raises(ValueError, match="cforch_url or llm_router"):
QueryTranslator(category_cache=cache)

View file

@ -1,372 +0,0 @@
"""Tests for the background monitor: should_alert logic, store alert methods, and run_monitor_search."""
from __future__ import annotations
import sqlite3
from datetime import datetime, timedelta, timezone
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from app.tasks.monitor import _AUCTION_ALERT_WINDOW_HOURS, should_alert
# ---------------------------------------------------------------------------
# should_alert — pure function, no I/O
# ---------------------------------------------------------------------------
class TestShouldAlert:
def test_bin_above_threshold_alerts(self):
assert should_alert(
trust_score=70, score_is_partial=False,
price=100.0, buying_format="fixed_price",
min_trust_score=60,
) is True
def test_bin_below_threshold_no_alert(self):
assert should_alert(
trust_score=55, score_is_partial=False,
price=100.0, buying_format="fixed_price",
min_trust_score=60,
) is False
def test_partial_score_applies_buffer(self):
# Score 65 with min 60 passes normally but fails with the +10 partial buffer.
assert should_alert(
trust_score=65, score_is_partial=True,
price=100.0, buying_format="fixed_price",
min_trust_score=60,
) is False
def test_partial_score_above_buffered_threshold_alerts(self):
assert should_alert(
trust_score=75, score_is_partial=True,
price=100.0, buying_format="fixed_price",
min_trust_score=60,
) is True
def test_best_offer_treated_like_bin(self):
assert should_alert(
trust_score=80, score_is_partial=False,
price=200.0, buying_format="best_offer",
min_trust_score=60,
) is True
def test_auction_within_window_alerts(self):
soon = (datetime.now(timezone.utc) + timedelta(hours=12)).isoformat()
assert should_alert(
trust_score=70, score_is_partial=False,
price=100.0, buying_format="auction",
min_trust_score=60, ends_at=soon,
) is True
def test_auction_outside_window_no_alert(self):
far = (datetime.now(timezone.utc) + timedelta(hours=48)).isoformat()
assert should_alert(
trust_score=70, score_is_partial=False,
price=100.0, buying_format="auction",
min_trust_score=60, ends_at=far,
) is False
def test_auction_no_ends_at_alerts_anyway(self):
assert should_alert(
trust_score=70, score_is_partial=False,
price=100.0, buying_format="auction",
min_trust_score=60, ends_at=None,
) is True
def test_auction_bad_ends_at_alerts_anyway(self):
assert should_alert(
trust_score=70, score_is_partial=False,
price=100.0, buying_format="auction",
min_trust_score=60, ends_at="not-a-date",
) is True
def test_auction_expired_no_alert(self):
past = (datetime.now(timezone.utc) - timedelta(hours=1)).isoformat()
assert should_alert(
trust_score=70, score_is_partial=False,
price=100.0, buying_format="auction",
min_trust_score=60, ends_at=past,
) is False
def test_unknown_format_alerts(self):
# Fail-open: unknown buying_format should not silently suppress.
assert should_alert(
trust_score=70, score_is_partial=False,
price=100.0, buying_format="mystery_format",
min_trust_score=60,
) is True
def test_score_exactly_at_threshold_passes(self):
assert should_alert(
trust_score=60, score_is_partial=False,
price=100.0, buying_format="fixed_price",
min_trust_score=60,
) is True
def test_auction_exactly_at_window_boundary_alerts(self):
boundary = (datetime.now(timezone.utc) + timedelta(hours=_AUCTION_ALERT_WINDOW_HOURS - 0.1)).isoformat()
assert should_alert(
trust_score=70, score_is_partial=False,
price=100.0, buying_format="auction",
min_trust_score=60, ends_at=boundary,
) is True
# ---------------------------------------------------------------------------
# Store alert methods — integration against real SQLite
# ---------------------------------------------------------------------------
def _create_monitor_db(path: Path) -> None:
conn = sqlite3.connect(path)
conn.executescript("""
CREATE TABLE IF NOT EXISTS saved_searches (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
query TEXT NOT NULL,
platform TEXT NOT NULL DEFAULT 'ebay',
filters_json TEXT,
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_run_at TEXT,
monitor_enabled INTEGER NOT NULL DEFAULT 0,
poll_interval_min INTEGER NOT NULL DEFAULT 60,
min_trust_score INTEGER NOT NULL DEFAULT 60,
last_checked_at TEXT
);
CREATE TABLE IF NOT EXISTS watch_alerts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
saved_search_id INTEGER NOT NULL REFERENCES saved_searches(id) ON DELETE CASCADE,
platform_listing_id TEXT NOT NULL,
title TEXT NOT NULL,
price REAL NOT NULL,
currency TEXT NOT NULL DEFAULT 'USD',
trust_score INTEGER NOT NULL,
url TEXT,
first_alerted_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
dismissed_at TEXT,
UNIQUE(saved_search_id, platform_listing_id)
);
INSERT INTO saved_searches (name, query, monitor_enabled) VALUES ('RTX 4090', 'rtx 4090', 1);
""")
conn.commit()
conn.close()
@pytest.fixture
def monitor_db(tmp_path: Path) -> Path:
db = tmp_path / "snipe.db"
_create_monitor_db(db)
return db
class TestStoreAlertMethods:
def test_upsert_alert_new(self, monitor_db: Path):
from app.db.models import WatchAlert
from app.db.store import Store
store = Store(monitor_db)
alert = WatchAlert(
saved_search_id=1, platform_listing_id="ebay-001",
title="RTX 4090", price=750.0, trust_score=72, currency="USD",
url="https://ebay.com/itm/001",
)
alert_id, is_new = store.upsert_alert(alert)
assert is_new is True
assert alert_id > 0
def test_upsert_alert_dedup(self, monitor_db: Path):
from app.db.models import WatchAlert
from app.db.store import Store
store = Store(monitor_db)
alert = WatchAlert(
saved_search_id=1, platform_listing_id="ebay-002",
title="RTX 4090 FE", price=800.0, trust_score=68,
)
id1, new1 = store.upsert_alert(alert)
id2, new2 = store.upsert_alert(alert)
assert id1 == id2
assert new1 is True
assert new2 is False
def test_list_alerts_returns_undismissed(self, monitor_db: Path):
from app.db.models import WatchAlert
from app.db.store import Store
store = Store(monitor_db)
alert = WatchAlert(
saved_search_id=1, platform_listing_id="ebay-003",
title="Test listing", price=500.0, trust_score=75,
)
store.upsert_alert(alert)
alerts = store.list_alerts(include_dismissed=False)
assert len(alerts) == 1
assert alerts[0].platform_listing_id == "ebay-003"
def test_count_undismissed_alerts(self, monitor_db: Path):
from app.db.models import WatchAlert
from app.db.store import Store
store = Store(monitor_db)
for i in range(3):
store.upsert_alert(WatchAlert(
saved_search_id=1, platform_listing_id=f"ebay-{i:03d}",
title=f"Listing {i}", price=float(100 + i), trust_score=70,
))
assert store.count_undismissed_alerts() == 3
def test_dismiss_alert(self, monitor_db: Path):
from app.db.models import WatchAlert
from app.db.store import Store
store = Store(monitor_db)
alert = WatchAlert(
saved_search_id=1, platform_listing_id="ebay-dismiss",
title="To dismiss", price=400.0, trust_score=65,
)
alert_id, _ = store.upsert_alert(alert)
store.dismiss_alert(alert_id)
alerts = store.list_alerts(include_dismissed=False)
assert all(a.id != alert_id for a in alerts)
def test_dismiss_all_alerts(self, monitor_db: Path):
from app.db.models import WatchAlert
from app.db.store import Store
store = Store(monitor_db)
for i in range(3):
store.upsert_alert(WatchAlert(
saved_search_id=1, platform_listing_id=f"all-{i}",
title=f"All {i}", price=float(100 * i), trust_score=70,
))
count = store.dismiss_all_alerts()
assert count == 3
assert store.count_undismissed_alerts() == 0
def test_mark_search_checked_updates_timestamp(self, monitor_db: Path):
from app.db.store import Store
store = Store(monitor_db)
store.mark_search_checked(1)
searches = store.list_monitored_searches()
assert searches[0].last_checked_at is not None
# ---------------------------------------------------------------------------
# run_monitor_search — mocked adapter + trust aggregator
# ---------------------------------------------------------------------------
class TestRunMonitorSearch:
def test_new_qualifying_listing_creates_alert(self, monitor_db: Path):
from app.db.models import Listing, SavedSearch, TrustScore
from app.db.store import Store
from app.tasks.monitor import run_monitor_search
search = SavedSearch(
id=1, name="RTX 4090", query="rtx 4090",
platform="ebay", monitor_enabled=True,
min_trust_score=60,
)
mock_listing = Listing(
platform="ebay", platform_listing_id="ebay-new",
title="ASUS RTX 4090", price=750.0, currency="USD",
condition="used", url="https://ebay.com/itm/new",
buying_format="fixed_price", seller_platform_id="seller123",
)
mock_trust = TrustScore(
listing_id=0, composite_score=72, score_is_partial=False,
account_age_score=0, feedback_count_score=0, feedback_ratio_score=0,
price_vs_market_score=0, category_history_score=0,
)
with patch("app.platforms.ebay.adapter.EbayAdapter") as MockAdapter, \
patch("app.trust.TrustScorer") as MockAgg:
MockAdapter.return_value.search.return_value = [mock_listing]
MockAgg.return_value.score_batch.return_value = [mock_trust]
count = run_monitor_search(search, user_db=monitor_db, shared_db=monitor_db)
assert count == 1
alerts = Store(monitor_db).list_alerts()
assert len(alerts) == 1
assert alerts[0].platform_listing_id == "ebay-new"
def test_below_threshold_listing_not_alerted(self, monitor_db: Path):
from app.db.models import Listing, SavedSearch, TrustScore
from app.tasks.monitor import run_monitor_search
search = SavedSearch(
id=1, name="RTX 4090", query="rtx 4090",
platform="ebay", monitor_enabled=True,
min_trust_score=70,
)
mock_listing = Listing(
platform="ebay", platform_listing_id="ebay-low",
title="Sketchy RTX 4090", price=500.0, currency="USD",
condition="used", url="https://ebay.com/itm/low",
buying_format="fixed_price", seller_platform_id="s1",
)
mock_trust = TrustScore(
listing_id=0, composite_score=55, score_is_partial=False,
account_age_score=0, feedback_count_score=0, feedback_ratio_score=0,
price_vs_market_score=0, category_history_score=0,
)
with patch("app.platforms.ebay.adapter.EbayAdapter") as MockAdapter, \
patch("app.trust.TrustScorer") as MockAgg:
MockAdapter.return_value.search.return_value = [mock_listing]
MockAgg.return_value.score_batch.return_value = [mock_trust]
count = run_monitor_search(search, user_db=monitor_db, shared_db=monitor_db)
assert count == 0
def test_duplicate_listing_not_double_alerted(self, monitor_db: Path):
from app.db.models import Listing, SavedSearch, TrustScore
from app.tasks.monitor import run_monitor_search
search = SavedSearch(
id=1, name="RTX 4090", query="rtx 4090",
platform="ebay", monitor_enabled=True, min_trust_score=60,
)
mock_listing = Listing(
platform="ebay", platform_listing_id="ebay-dupe",
title="RTX 4090", price=700.0, currency="USD",
condition="used", url="https://ebay.com/itm/dupe",
buying_format="fixed_price", seller_platform_id="s1",
)
mock_trust = TrustScore(
listing_id=0, composite_score=75, score_is_partial=False,
account_age_score=0, feedback_count_score=0, feedback_ratio_score=0,
price_vs_market_score=0, category_history_score=0,
)
with patch("app.platforms.ebay.adapter.EbayAdapter") as MockAdapter, \
patch("app.trust.TrustScorer") as MockAgg:
MockAdapter.return_value.search.return_value = [mock_listing]
MockAgg.return_value.score_batch.return_value = [mock_trust]
count1 = run_monitor_search(search, user_db=monitor_db, shared_db=monitor_db)
count2 = run_monitor_search(search, user_db=monitor_db, shared_db=monitor_db)
assert count1 == 1
assert count2 == 0 # deduped by UNIQUE constraint
def test_adapter_failure_returns_zero(self, monitor_db: Path):
from app.db.models import SavedSearch
from app.tasks.monitor import run_monitor_search
search = SavedSearch(
id=1, name="RTX 4090", query="rtx 4090",
platform="ebay", monitor_enabled=True, min_trust_score=60,
)
with patch("app.platforms.ebay.adapter.EbayAdapter") as MockAdapter:
MockAdapter.return_value.search.side_effect = RuntimeError("eBay down")
count = run_monitor_search(search, user_db=monitor_db, shared_db=monitor_db)
assert count == 0

View file

@ -4,7 +4,7 @@ from __future__ import annotations
import json import json
import sqlite3 import sqlite3
from pathlib import Path from pathlib import Path
from unittest.mock import MagicMock, patch, call from unittest.mock import patch
import pytest import pytest
@ -47,19 +47,6 @@ def tmp_db(tmp_path: Path) -> Path:
return db return db
_VISION_JSON = json.dumps({
"is_stock_photo": False,
"visible_damage": False,
"authenticity_signal": "genuine_product_photo",
"confidence": "high",
})
_PARAMS = json.dumps({
"photo_url": "https://example.com/photo.jpg",
"listing_title": "Used iPhone 13",
})
def test_llm_task_types_defined(): def test_llm_task_types_defined():
assert "trust_photo_analysis" in LLM_TASK_TYPES assert "trust_photo_analysis" in LLM_TASK_TYPES
@ -88,17 +75,29 @@ def test_insert_task_dedup(tmp_db: Path):
assert new2 is False assert new2 is False
# ── Local LLMRouter path ────────────────────────────────────────────────────── def test_run_task_photo_analysis_success(tmp_db: Path):
"""Vision analysis result is written to trust_scores.photo_analysis_json."""
params = json.dumps({
"listing_id": 1,
"photo_url": "https://example.com/photo.jpg",
"listing_title": "Used iPhone 13",
})
task_id, _ = insert_task(tmp_db, "trust_photo_analysis", job_id=1, params=params)
def test_run_task_photo_analysis_local_success(tmp_db: Path): vision_result = {
"""Local path: vision result is written to trust_scores.photo_analysis_json.""" "is_stock_photo": False,
task_id, _ = insert_task(tmp_db, "trust_photo_analysis", job_id=1, params=_PARAMS) "visible_damage": False,
"authenticity_signal": "genuine_product_photo",
"confidence": "high",
}
with patch("app.tasks.runner.requests") as mock_req, \ with patch("app.tasks.runner.requests") as mock_req, \
patch("app.tasks.runner._assess_via_local_llm", return_value=_VISION_JSON): patch("app.tasks.runner.LLMRouter") as MockRouter:
mock_req.get.return_value.content = b"fake_image_bytes" mock_req.get.return_value.content = b"fake_image_bytes"
mock_req.get.return_value.raise_for_status = lambda: None mock_req.get.return_value.raise_for_status = lambda: None
run_task(tmp_db, task_id, "trust_photo_analysis", 1, _PARAMS) instance = MockRouter.return_value
instance.complete.return_value = json.dumps(vision_result)
run_task(tmp_db, task_id, "trust_photo_analysis", 1, params)
conn = sqlite3.connect(tmp_db) conn = sqlite3.connect(tmp_db)
score_row = conn.execute( score_row = conn.execute(
@ -111,16 +110,20 @@ def test_run_task_photo_analysis_local_success(tmp_db: Path):
assert task_row[0] == "completed" assert task_row[0] == "completed"
parsed = json.loads(score_row[0]) parsed = json.loads(score_row[0])
assert parsed["is_stock_photo"] is False assert parsed["is_stock_photo"] is False
assert parsed["confidence"] == "high"
def test_run_task_photo_fetch_failure_marks_failed(tmp_db: Path): def test_run_task_photo_fetch_failure_marks_failed(tmp_db: Path):
"""If photo download fails, task is marked failed without crashing.""" """If photo download fails, task is marked failed without crashing."""
task_id, _ = insert_task(tmp_db, "trust_photo_analysis", job_id=1, params=_PARAMS) params = json.dumps({
"listing_id": 1,
"photo_url": "https://example.com/bad.jpg",
"listing_title": "Laptop",
})
task_id, _ = insert_task(tmp_db, "trust_photo_analysis", job_id=1, params=params)
with patch("app.tasks.runner.requests") as mock_req: with patch("app.tasks.runner.requests") as mock_req:
mock_req.get.side_effect = ConnectionError("fetch failed") mock_req.get.side_effect = ConnectionError("fetch failed")
run_task(tmp_db, task_id, "trust_photo_analysis", 1, _PARAMS) run_task(tmp_db, task_id, "trust_photo_analysis", 1, params)
conn = sqlite3.connect(tmp_db) conn = sqlite3.connect(tmp_db)
row = conn.execute( row = conn.execute(
@ -153,169 +156,3 @@ def test_run_task_unknown_type_marks_failed(tmp_db: Path):
).fetchone() ).fetchone()
conn.close() conn.close()
assert row[0] == "failed" assert row[0] == "failed"
# ── cf-orch path ──────────────────────────────────────────────────────────────
def _make_orch_client_mock(vision_json: str) -> MagicMock:
"""Build a CFOrchClient mock whose task_allocate context manager returns an Allocation."""
alloc = MagicMock()
alloc.url = "http://cf-vlm.local:8000"
alloc.model = "bartowski--qwen2-vl-7b-instruct-gguf"
cm = MagicMock()
cm.__enter__ = MagicMock(return_value=alloc)
cm.__exit__ = MagicMock(return_value=False)
client = MagicMock()
client.task_allocate.return_value = cm
return client
def test_run_task_photo_analysis_orch_success(tmp_db: Path):
"""Cloud path: CFOrchClient.task_allocate is used when GPU_SERVER_URL is set."""
task_id, _ = insert_task(tmp_db, "trust_photo_analysis", job_id=1, params=_PARAMS)
chat_resp = MagicMock()
chat_resp.json.return_value = {"choices": [{"message": {"content": _VISION_JSON}}]}
chat_resp.raise_for_status = MagicMock()
with patch("app.tasks.runner.requests") as mock_req, \
patch.dict("os.environ", {"GPU_SERVER_URL": "http://cf-orch.local:8700"}), \
patch("app.tasks.runner.httpx") as mock_httpx, \
patch("circuitforge_orch.client.CFOrchClient") as MockClient:
mock_req.get.return_value.content = b"fake_image_bytes"
mock_req.get.return_value.raise_for_status = lambda: None
mock_httpx.post.return_value = chat_resp
client_instance = _make_orch_client_mock(_VISION_JSON)
MockClient.return_value = client_instance
run_task(tmp_db, task_id, "trust_photo_analysis", 1, _PARAMS)
conn = sqlite3.connect(tmp_db)
score_row = conn.execute(
"SELECT photo_analysis_json FROM trust_scores WHERE listing_id=1"
).fetchone()
task_row = conn.execute(
"SELECT status FROM background_tasks WHERE id=?", (task_id,)
).fetchone()
conn.close()
assert task_row[0] == "completed"
parsed = json.loads(score_row[0])
assert parsed["authenticity_signal"] == "genuine_product_photo"
def test_run_task_photo_analysis_orch_uses_image_assessment_task(tmp_db: Path):
"""task_allocate must be called with product='snipe', task='image_assessment'."""
task_id, _ = insert_task(tmp_db, "trust_photo_analysis", job_id=1, params=_PARAMS)
chat_resp = MagicMock()
chat_resp.json.return_value = {"choices": [{"message": {"content": _VISION_JSON}}]}
chat_resp.raise_for_status = MagicMock()
with patch("app.tasks.runner.requests") as mock_req, \
patch.dict("os.environ", {"GPU_SERVER_URL": "http://cf-orch.local:8700"}), \
patch("app.tasks.runner.httpx") as mock_httpx, \
patch("circuitforge_orch.client.CFOrchClient") as MockClient:
mock_req.get.return_value.content = b"fake_image_bytes"
mock_req.get.return_value.raise_for_status = lambda: None
mock_httpx.post.return_value = chat_resp
client_instance = _make_orch_client_mock(_VISION_JSON)
MockClient.return_value = client_instance
run_task(tmp_db, task_id, "trust_photo_analysis", 1, _PARAMS)
client_instance.task_allocate.assert_called_once_with("snipe", "image_assessment")
def test_run_task_photo_analysis_orch_sends_image_url_content(tmp_db: Path):
"""Vision payload must include image_url content block with data URI."""
task_id, _ = insert_task(tmp_db, "trust_photo_analysis", job_id=1, params=_PARAMS)
captured_body: dict = {}
def capture_post(url, **kwargs):
nonlocal captured_body
if "/v1/chat/completions" in url:
captured_body = kwargs.get("json", {})
resp = MagicMock()
resp.json.return_value = {"choices": [{"message": {"content": _VISION_JSON}}]}
resp.raise_for_status = MagicMock()
return resp
with patch("app.tasks.runner.requests") as mock_req, \
patch.dict("os.environ", {"GPU_SERVER_URL": "http://cf-orch.local:8700"}), \
patch("app.tasks.runner.httpx") as mock_httpx, \
patch("circuitforge_orch.client.CFOrchClient") as MockClient:
mock_req.get.return_value.content = b"fake_image_bytes"
mock_req.get.return_value.raise_for_status = lambda: None
mock_httpx.post.side_effect = capture_post
client_instance = _make_orch_client_mock(_VISION_JSON)
MockClient.return_value = client_instance
run_task(tmp_db, task_id, "trust_photo_analysis", 1, _PARAMS)
user_content = captured_body["messages"][1]["content"]
image_blocks = [b for b in user_content if b.get("type") == "image_url"]
assert image_blocks, "No image_url content block found in vision payload"
url = image_blocks[0]["image_url"]["url"]
assert url.startswith("data:image/jpeg;base64,"), f"Unexpected image URL format: {url[:40]}"
def test_run_task_photo_analysis_orch_task_not_found_falls_back(tmp_db: Path):
"""TaskNotFound from cf-orch → graceful fallback to local LLMRouter."""
from circuitforge_orch.client import TaskNotFound
task_id, _ = insert_task(tmp_db, "trust_photo_analysis", job_id=1, params=_PARAMS)
cm = MagicMock()
cm.__enter__ = MagicMock(side_effect=TaskNotFound("snipe", "image_assessment"))
cm.__exit__ = MagicMock(return_value=False)
client_instance = MagicMock()
client_instance.task_allocate.return_value = cm
with patch("app.tasks.runner.requests") as mock_req, \
patch.dict("os.environ", {"GPU_SERVER_URL": "http://cf-orch.local:8700"}), \
patch("circuitforge_orch.client.CFOrchClient", return_value=client_instance), \
patch("app.tasks.runner._assess_via_local_llm", return_value=_VISION_JSON) as mock_local:
mock_req.get.return_value.content = b"fake_image_bytes"
mock_req.get.return_value.raise_for_status = lambda: None
run_task(tmp_db, task_id, "trust_photo_analysis", 1, _PARAMS)
mock_local.assert_called_once()
conn = sqlite3.connect(tmp_db)
task_row = conn.execute(
"SELECT status FROM background_tasks WHERE id=?", (task_id,)
).fetchone()
conn.close()
assert task_row[0] == "completed"
def test_run_task_photo_analysis_non_json_response_writes_raw(tmp_db: Path):
"""Non-JSON LLM response is stored with parse_error flag rather than crashing."""
task_id, _ = insert_task(tmp_db, "trust_photo_analysis", job_id=1, params=_PARAMS)
with patch("app.tasks.runner.requests") as mock_req, \
patch("app.tasks.runner._assess_via_local_llm", return_value="not valid json at all"):
mock_req.get.return_value.content = b"fake_image_bytes"
mock_req.get.return_value.raise_for_status = lambda: None
run_task(tmp_db, task_id, "trust_photo_analysis", 1, _PARAMS)
conn = sqlite3.connect(tmp_db)
score_row = conn.execute(
"SELECT photo_analysis_json FROM trust_scores WHERE listing_id=1"
).fetchone()
conn.close()
parsed = json.loads(score_row[0])
assert parsed.get("parse_error") is True
assert "raw_response" in parsed

View file

@ -296,37 +296,3 @@ def test_non_retailer_does_not_suppress_duplicate_photo():
) )
result = agg.aggregate(_ALL_20.copy(), photo_hash_duplicate=True, seller=seller) result = agg.aggregate(_ALL_20.copy(), photo_hash_duplicate=True, seller=seller)
assert "duplicate_photo" in result.red_flags_json assert "duplicate_photo" in result.red_flags_json
# ── #52: buyer-only / returning seller (ratio=0.0, count>0) ──────────────────
def test_zero_ratio_with_count_gives_no_recent_seller_data_flag():
"""Seller with 117 lifetime feedbacks (buyer-only) has ratio=0.0 parsed from page.
Must get no_recent_seller_data soft flag, NOT established_bad_actor."""
agg = Aggregator()
scores = {k: 10 for k in ["account_age", "feedback_count",
"feedback_ratio", "price_vs_market", "category_history"]}
buyer_only = Seller(
platform="ebay", platform_seller_id="u", username="jjcpryz",
account_age_days=1200, feedback_count=117, feedback_ratio=0.0,
category_history_json="{}",
)
result = agg.aggregate(scores, photo_hash_duplicate=False, seller=buyer_only)
assert "no_recent_seller_data" in result.red_flags_json
assert "established_bad_actor" not in result.red_flags_json
def test_established_bad_actor_still_fires_for_genuinely_bad_ratio():
"""ratio=0.75 (not zero) with moderate count → established_bad_actor still fires."""
agg = Aggregator()
scores = {k: 10 for k in ["account_age", "feedback_count",
"feedback_ratio", "price_vs_market", "category_history"]}
bad = Seller(
platform="ebay", platform_seller_id="u", username="u",
account_age_days=500, feedback_count=100, feedback_ratio=0.75,
category_history_json="{}",
)
result = agg.aggregate(scores, photo_hash_duplicate=False, seller=bad)
assert "established_bad_actor" in result.red_flags_json
assert "no_recent_seller_data" not in result.red_flags_json

View file

@ -43,26 +43,3 @@ def test_no_market_data_returns_none():
scores = scorer.score(_seller(), market_median=None, listing_price=950.0) scores = scorer.score(_seller(), market_median=None, listing_price=950.0)
# None signals "data unavailable" — aggregator will set score_is_partial=True # None signals "data unavailable" — aggregator will set score_is_partial=True
assert scores["price_vs_market"] is None assert scores["price_vs_market"] is None
def test_zero_ratio_with_nonzero_count_returns_none():
"""ratio=0.0 with count>0 means eBay didn't show a 12-month percentage.
Must return None (missing data) not 0 (catastrophically bad)."""
scorer = MetadataScorer()
scores = scorer.score(
_seller(feedback_ratio=0.0, feedback_count=117),
market_median=None, listing_price=500.0,
)
assert scores["feedback_ratio"] is None
def test_zero_ratio_with_zero_count_scores_low():
"""feedback_ratio=0.0 with count=0 is a real 'no data at all' case, not missing."""
scorer = MetadataScorer()
scores = scorer.score(
_seller(feedback_ratio=0.0, feedback_count=0),
market_median=None, listing_price=500.0,
)
# count=0 means zero_feedback; ratio=0 with count=0 is the standard no-history path
# (not the "missing 12-month window" path)
assert scores["feedback_ratio"] == 5 # ratio < 0.90 → 5

View file

@ -21,7 +21,6 @@ import { useMotion } from './composables/useMotion'
import { useSnipeMode } from './composables/useSnipeMode' import { useSnipeMode } from './composables/useSnipeMode'
import { useTheme } from './composables/useTheme' import { useTheme } from './composables/useTheme'
import { useKonamiCode } from './composables/useKonamiCode' import { useKonamiCode } from './composables/useKonamiCode'
import { useCandycoreMode } from './composables/useCandycoreMode'
import { useSessionStore } from './stores/session' import { useSessionStore } from './stores/session'
import { useBlocklistStore } from './stores/blocklist' import { useBlocklistStore } from './stores/blocklist'
import { usePreferencesStore } from './stores/preferences' import { usePreferencesStore } from './stores/preferences'
@ -32,8 +31,6 @@ import FeedbackButton from './components/FeedbackButton.vue'
const motion = useMotion() const motion = useMotion()
const { activate, restore } = useSnipeMode() const { activate, restore } = useSnipeMode()
const { restore: restoreTheme } = useTheme() const { restore: restoreTheme } = useTheme()
const { restore: restoreCandy, useWordTrigger } = useCandycoreMode()
useWordTrigger()
const session = useSessionStore() const session = useSessionStore()
const blocklistStore = useBlocklistStore() const blocklistStore = useBlocklistStore()
const preferencesStore = usePreferencesStore() const preferencesStore = usePreferencesStore()
@ -45,7 +42,6 @@ useKonamiCode(activate)
onMounted(async () => { onMounted(async () => {
restore() // re-apply snipe mode from localStorage on hard reload restore() // re-apply snipe mode from localStorage on hard reload
restoreTheme() // re-apply explicit theme override on hard reload restoreTheme() // re-apply explicit theme override on hard reload
restoreCandy() // re-apply candycore mode from localStorage on hard reload
await session.bootstrap() // fetch tier + feature flags from API await session.bootstrap() // fetch tier + feature flags from API
blocklistStore.fetchBlocklist() // pre-load so card block buttons reflect state immediately blocklistStore.fetchBlocklist() // pre-load so card block buttons reflect state immediately
preferencesStore.load() // load user preferences after session resolves preferencesStore.load() // load user preferences after session resolves
@ -61,12 +57,6 @@ onMounted(async () => {
padding: 0; padding: 0;
} }
/* Global keyboard focus indicator — safety net so no stylesheet can silently remove focus rings */
:focus-visible {
outline: 2px solid var(--app-primary);
outline-offset: 2px;
}
html { html {
font-family: var(--font-body, sans-serif); font-family: var(--font-body, sans-serif);
color: var(--color-text, #e6edf3); color: var(--color-text, #e6edf3);

View file

@ -87,7 +87,7 @@
Snipe Mode data attribute overrides this via higher specificity. Snipe Mode data attribute overrides this via higher specificity.
*/ */
/* Explicit dark override — beats OS preference when user forces dark in Settings */ /* Explicit dark override — beats OS preference when user forces dark in Settings */
[data-theme="dark"]:not([data-snipe-mode="active"]):not([data-candycore="active"]) { [data-theme="dark"]:not([data-snipe-mode="active"]) {
--color-surface: #0d1117; --color-surface: #0d1117;
--color-surface-2: #161b22; --color-surface-2: #161b22;
--color-surface-raised: #1c2129; --color-surface-raised: #1c2129;
@ -113,7 +113,7 @@
} }
@media (prefers-color-scheme: light) { @media (prefers-color-scheme: light) {
:root:not([data-theme="dark"]):not([data-snipe-mode="active"]):not([data-candycore="active"]) { :root:not([data-theme="dark"]):not([data-snipe-mode="active"]) {
/* Surfaces — warm cream, like a tactical field notebook */ /* Surfaces — warm cream, like a tactical field notebook */
--color-surface: #f8f5ee; --color-surface: #f8f5ee;
--color-surface-2: #f0ece3; --color-surface-2: #f0ece3;
@ -153,7 +153,7 @@
} }
/* Explicit light override — beats OS preference when user forces light in Settings */ /* Explicit light override — beats OS preference when user forces light in Settings */
[data-theme="light"]:not([data-snipe-mode="active"]):not([data-candycore="active"]) { [data-theme="light"]:not([data-snipe-mode="active"]) {
--color-surface: #f8f5ee; --color-surface: #f8f5ee;
--color-surface-2: #f0ece3; --color-surface-2: #f0ece3;
--color-surface-raised: #e8e3d8; --color-surface-raised: #e8e3d8;
@ -178,56 +178,6 @@
--shadow-lg: 0 10px 30px rgba(60,45,20,0.2), 0 4px 8px rgba(60,45,20,0.1); --shadow-lg: 0 10px 30px rgba(60,45,20,0.2), 0 4px 8px rgba(60,45,20,0.1);
} }
/* Candycore easter egg theme
Activated by typing "neon" outside a form field (tribute to artist Neon).
Palette sourced from snipe_v0_Neon_IPad_Paint.jpeg:
purple-black sky + lavender primary + cyan glow + yellow crown + pink text.
Stored as 'cf-candycore' in localStorage.
Applied: document.documentElement.dataset.candycore = 'active'
NOTE: Snipe Mode is declared last and overrides this when both are active.
*/
[data-candycore="active"] {
--app-primary: #c77dff;
--app-primary-hover: #a855f7;
--app-primary-light: rgba(199, 125, 255, 0.15);
/* Purple-black night sky */
--color-surface: #08051a;
--color-surface-2: #100d28;
--color-surface-raised: #1a1248;
/* Purple glow borders */
--color-border: rgba(199, 125, 255, 0.20);
--color-border-light: rgba(199, 125, 255, 0.10);
/* Candy-floss text — pink-white, muted bubblegum */
--color-text: #ffd6f5;
--color-text-muted: #f09099;
--color-text-inverse: #08051a;
/* Trust signals — straight from the painting */
--trust-high: #00c8e0; /* cyan (outline glow) = good */
--trust-mid: #ffe520; /* yellow (crown stripe) = caution */
--trust-low: #ff6eb4; /* hot pink = danger */
/* Semantic */
--color-success: #00c8e0;
--color-error: #ff6eb4;
--color-warning: #ffe520;
--color-info: #c77dff;
--color-accent: #00c8e0; /* cyan accent */
/* Purple glow shadows */
--shadow-sm: 0 1px 3px rgba(199, 125, 255, 0.12);
--shadow-md: 0 4px 12px rgba(199, 125, 255, 0.20);
--shadow-lg: 0 10px 30px rgba(199, 125, 255, 0.28);
/* Glow helpers (used in scoped styles if needed) */
--candy-glow-xs: rgba(199, 125, 255, 0.08);
--candy-glow-sm: rgba(199, 125, 255, 0.18);
--candy-glow-md: rgba(199, 125, 255, 0.45);
}
/* ── Snipe Mode easter egg theme ─────────────────── */ /* ── Snipe Mode easter egg theme ─────────────────── */
/* Activated by Konami code; stored as 'cf-snipe-mode' in localStorage */ /* Activated by Konami code; stored as 'cf-snipe-mode' in localStorage */
/* Applied: document.documentElement.dataset.snipeMode = 'active' */ /* Applied: document.documentElement.dataset.snipeMode = 'active' */

View file

@ -1,398 +0,0 @@
<template>
<div class="alert-bell-wrap" ref="wrapRef">
<!-- Bell trigger button -->
<button
ref="bellRef"
class="alert-bell"
:class="{ 'alert-bell--active': panelOpen }"
:aria-label="unreadCount > 0 ? `${unreadCount} new watch alert${unreadCount === 1 ? '' : 's'}` : 'Watch alerts'"
:aria-expanded="panelOpen"
aria-haspopup="true"
@click="togglePanel"
>
<BellIcon class="alert-bell__icon" aria-hidden="true" />
<span
v-if="unreadCount > 0"
class="alert-badge"
aria-hidden="true"
>{{ unreadCount > 99 ? '99+' : unreadCount }}</span>
</button>
<!-- Polite live region announces count changes without moving focus -->
<div aria-live="polite" aria-atomic="true" class="sr-only">
{{ liveAnnouncement }}
</div>
<!-- Alert panel -->
<Transition name="panel">
<div
v-if="panelOpen"
class="alert-panel"
role="dialog"
aria-label="Watch alerts"
aria-modal="false"
>
<div class="alert-panel__header">
<span class="alert-panel__title">Watch Alerts</span>
<button
v-if="store.alerts.length > 0"
class="alert-panel__clear"
@click="onDismissAll"
>
Clear all
</button>
<button
class="alert-panel__close"
aria-label="Close alerts panel"
@click="closePanel"
>
</button>
</div>
<div v-if="store.loading" class="alert-panel__state">
Loading
</div>
<div v-else-if="store.alerts.length === 0" class="alert-panel__state">
<span aria-hidden="true">🔔</span>
<p>No new alerts. Enable monitoring on a saved search to get notified.</p>
</div>
<ul v-else class="alert-list" role="list">
<li
v-for="alert in store.alerts"
:key="alert.id"
class="alert-card"
>
<div class="alert-card__body">
<p class="alert-card__title">{{ alert.title }}</p>
<div class="alert-card__meta">
<span class="alert-card__price">${{ alert.price.toFixed(2) }}</span>
<span class="alert-card__score" :class="scoreClass(alert.trust_score)">
Trust {{ alert.trust_score }}
</span>
</div>
</div>
<div class="alert-card__actions">
<a
v-if="alert.url"
:href="alert.url"
target="_blank"
rel="noopener noreferrer"
class="alert-card__view"
:aria-label="`View listing: ${alert.title}`"
>
View on eBay
</a>
<button
class="alert-card__dismiss"
:aria-label="`Dismiss alert: ${alert.title}`"
@click="onDismiss(alert.id)"
@keydown.delete.prevent="onDismiss(alert.id)"
>
</button>
</div>
</li>
</ul>
</div>
</Transition>
</div>
</template>
<script setup lang="ts">
import { computed, onBeforeUnmount, onMounted, ref, watch } from 'vue'
import { BellIcon } from '@heroicons/vue/24/outline'
import { useAlertsStore } from '../stores/alerts'
const store = useAlertsStore()
const panelOpen = ref(false)
const bellRef = ref<HTMLButtonElement | null>(null)
const wrapRef = ref<HTMLDivElement | null>(null)
const liveAnnouncement = ref('')
const unreadCount = computed(() => store.unreadCount)
// Announce count changes to screen readers via the polite live region.
watch(unreadCount, (count, prev) => {
if (count > prev) {
liveAnnouncement.value = `${count} new watch alert${count === 1 ? '' : 's'}`
// Reset after announcement so repeat counts still fire.
setTimeout(() => { liveAnnouncement.value = '' }, 1500)
}
})
function togglePanel() {
panelOpen.value = !panelOpen.value
if (panelOpen.value) store.fetchAlerts()
}
function closePanel() {
panelOpen.value = false
bellRef.value?.focus()
}
async function onDismiss(id: number) {
await store.dismiss(id)
if (store.alerts.length === 0) {
// Return focus to bell when last alert is dismissed.
panelOpen.value = false
bellRef.value?.focus()
}
}
async function onDismissAll() {
await store.dismissAll()
panelOpen.value = false
bellRef.value?.focus()
}
function scoreClass(score: number) {
if (score >= 75) return 'score--high'
if (score >= 50) return 'score--medium'
return 'score--low'
}
// Close on outside click.
function handleOutsideClick(e: MouseEvent) {
if (wrapRef.value && !wrapRef.value.contains(e.target as Node)) {
panelOpen.value = false
}
}
onMounted(() => {
store.fetchAlerts()
// Poll for new alerts every 2 minutes while the app is open.
const interval = setInterval(() => store.fetchAlerts(), 120_000)
document.addEventListener('click', handleOutsideClick)
onBeforeUnmount(() => {
clearInterval(interval)
document.removeEventListener('click', handleOutsideClick)
})
})
</script>
<style scoped>
.sr-only {
position: absolute;
width: 1px;
height: 1px;
padding: 0;
margin: -1px;
overflow: hidden;
clip: rect(0, 0, 0, 0);
white-space: nowrap;
border: 0;
}
.alert-bell-wrap {
position: relative;
}
.alert-bell {
position: relative;
display: flex;
align-items: center;
justify-content: center;
width: 40px;
height: 40px;
background: transparent;
border: 1px solid var(--color-border);
border-radius: var(--radius-md);
color: var(--color-text-muted);
cursor: pointer;
transition: border-color 150ms ease, color 150ms ease, background 150ms ease;
}
.alert-bell:hover,
.alert-bell--active {
border-color: var(--app-primary);
color: var(--app-primary);
background: var(--app-primary-light);
}
.alert-bell__icon {
width: 1.25rem;
height: 1.25rem;
}
.alert-badge {
position: absolute;
top: -6px;
right: -6px;
min-width: 18px;
height: 18px;
padding: 0 4px;
background: var(--color-error, #ef4444);
color: #fff;
font-size: 0.625rem;
font-weight: 700;
font-family: var(--font-mono);
border-radius: 9px;
display: flex;
align-items: center;
justify-content: center;
line-height: 1;
}
/* Panel */
.alert-panel {
position: absolute;
top: calc(100% + 8px);
right: 0;
width: min(360px, 92vw);
background: var(--color-surface-2);
border: 1px solid var(--color-border);
border-radius: var(--radius-lg);
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.4);
z-index: 200;
overflow: hidden;
}
.alert-panel__header {
display: flex;
align-items: center;
gap: var(--space-2);
padding: var(--space-3) var(--space-4);
border-bottom: 1px solid var(--color-border);
}
.alert-panel__title {
flex: 1;
font-weight: 600;
font-size: 0.875rem;
color: var(--color-text);
}
.alert-panel__clear {
font-size: 0.75rem;
color: var(--color-text-muted);
background: none;
border: none;
cursor: pointer;
padding: var(--space-1) var(--space-2);
border-radius: var(--radius-sm);
transition: color 150ms ease;
}
.alert-panel__clear:hover { color: var(--color-error); }
.alert-panel__close {
background: none;
border: none;
color: var(--color-text-muted);
font-size: 0.75rem;
cursor: pointer;
padding: var(--space-1);
border-radius: var(--radius-sm);
line-height: 1;
transition: color 150ms ease;
min-width: 24px;
min-height: 24px;
}
.alert-panel__close:hover { color: var(--color-error); }
.alert-panel__state {
padding: var(--space-8) var(--space-4);
text-align: center;
color: var(--color-text-muted);
font-size: 0.875rem;
display: flex;
flex-direction: column;
align-items: center;
gap: var(--space-2);
}
/* Alert list */
.alert-list {
list-style: none;
max-height: 360px;
overflow-y: auto;
}
.alert-card {
display: flex;
align-items: center;
gap: var(--space-3);
padding: var(--space-3) var(--space-4);
border-bottom: 1px solid var(--color-border-light);
transition: background 150ms ease;
}
.alert-card:hover { background: var(--color-surface); }
.alert-card:last-child { border-bottom: none; }
.alert-card__body { flex: 1; min-width: 0; }
.alert-card__title {
font-size: 0.8125rem;
color: var(--color-text);
margin: 0 0 var(--space-1);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.alert-card__meta {
display: flex;
gap: var(--space-2);
align-items: center;
}
.alert-card__price {
font-family: var(--font-mono);
font-size: 0.75rem;
color: var(--app-primary);
}
.alert-card__score {
font-size: 0.6875rem;
font-weight: 600;
padding: 1px var(--space-2);
border-radius: var(--radius-sm);
}
.score--high { background: rgba(34,197,94,0.15); color: #22c55e; }
.score--medium { background: rgba(234,179,8,0.15); color: #eab308; }
.score--low { background: rgba(239,68,68,0.15); color: #ef4444; }
.alert-card__actions {
display: flex;
align-items: center;
gap: var(--space-2);
flex-shrink: 0;
}
.alert-card__view {
font-size: 0.75rem;
color: var(--app-primary);
text-decoration: none;
white-space: nowrap;
padding: var(--space-1) var(--space-2);
border-radius: var(--radius-sm);
transition: background 150ms ease;
}
.alert-card__view:hover { background: var(--app-primary-light); }
.alert-card__dismiss {
background: none;
border: 1px solid var(--color-border);
border-radius: var(--radius-sm);
color: var(--color-text-muted);
font-size: 0.625rem;
cursor: pointer;
min-width: 24px;
min-height: 24px;
transition: border-color 150ms ease, color 150ms ease;
}
.alert-card__dismiss:hover { border-color: var(--color-error); color: var(--color-error); }
/* Transition */
.panel-enter-active,
.panel-leave-active { transition: opacity 120ms ease, transform 120ms ease; }
.panel-enter-from,
.panel-leave-to { opacity: 0; transform: translateY(-6px); }
@media (prefers-reduced-motion: reduce) {
.panel-enter-active,
.panel-leave-active { transition: none; }
}
</style>

View file

@ -1,7 +1,7 @@
<template> <template>
<!-- Desktop: persistent sidebar (1024px) --> <!-- Desktop: persistent sidebar (1024px) -->
<!-- Mobile: bottom tab bar (<1024px) --> <!-- Mobile: bottom tab bar (<1024px) -->
<nav class="app-sidebar" role="navigation" aria-label="Sidebar"> <nav class="app-sidebar" role="navigation" aria-label="Main navigation">
<!-- Brand --> <!-- Brand -->
<div class="sidebar__brand"> <div class="sidebar__brand">
<RouterLink to="/" class="sidebar__logo"> <RouterLink to="/" class="sidebar__logo">
@ -32,20 +32,17 @@
</button> </button>
</div> </div>
<!-- Settings + alert bell at bottom --> <!-- Settings at bottom -->
<div class="sidebar__footer"> <div class="sidebar__footer">
<div class="sidebar__footer-row">
<RouterLink to="/settings" class="sidebar__link sidebar__link--footer" active-class="sidebar__link--active"> <RouterLink to="/settings" class="sidebar__link sidebar__link--footer" active-class="sidebar__link--active">
<Cog6ToothIcon class="sidebar__icon" aria-hidden="true" /> <Cog6ToothIcon class="sidebar__icon" aria-hidden="true" />
<span class="sidebar__label">Settings</span> <span class="sidebar__label">Settings</span>
</RouterLink> </RouterLink>
<AlertBell v-if="session.isLoggedIn || session.tier === 'local'" class="sidebar__bell" />
</div>
</div> </div>
</nav> </nav>
<!-- Mobile bottom tab bar --> <!-- Mobile bottom tab bar -->
<nav class="app-tabbar" role="navigation" aria-label="Tab bar"> <nav class="app-tabbar" role="navigation" aria-label="Main navigation">
<ul class="tabbar__links" role="list"> <ul class="tabbar__links" role="list">
<li v-for="link in mobileLinks" :key="link.to"> <li v-for="link in mobileLinks" :key="link.to">
<RouterLink <RouterLink
@ -72,11 +69,8 @@ import {
ShieldExclamationIcon, ShieldExclamationIcon,
} from '@heroicons/vue/24/outline' } from '@heroicons/vue/24/outline'
import { useSnipeMode } from '../composables/useSnipeMode' import { useSnipeMode } from '../composables/useSnipeMode'
import { useSessionStore } from '../stores/session'
import AlertBell from './AlertBell.vue'
const { active: isSnipeMode, deactivate } = useSnipeMode() const { active: isSnipeMode, deactivate } = useSnipeMode()
const session = useSessionStore()
const navLinks = computed(() => [ const navLinks = computed(() => [
{ to: '/', icon: MagnifyingGlassIcon, label: 'Search' }, { to: '/', icon: MagnifyingGlassIcon, label: 'Search' },
@ -87,7 +81,7 @@ const navLinks = computed(() => [
const mobileLinks = [ const mobileLinks = [
{ to: '/', icon: MagnifyingGlassIcon, label: 'Search' }, { to: '/', icon: MagnifyingGlassIcon, label: 'Search' },
{ to: '/saved', icon: BookmarkIcon, label: 'Saved' }, { to: '/saved', icon: BookmarkIcon, label: 'Saved' },
{ to: '/blocklist', icon: ShieldExclamationIcon, label: 'Blocklist' }, { to: '/blocklist', icon: ShieldExclamationIcon, label: 'Block' },
{ to: '/settings', icon: Cog6ToothIcon, label: 'Settings' }, { to: '/settings', icon: Cog6ToothIcon, label: 'Settings' },
] ]
</script> </script>
@ -208,20 +202,6 @@ const mobileLinks = [
border-top: 1px solid var(--color-border-light); border-top: 1px solid var(--color-border-light);
} }
.sidebar__footer-row {
display: flex;
align-items: center;
gap: var(--space-2);
}
.sidebar__footer-row .sidebar__link {
flex: 1;
}
.sidebar__bell {
flex-shrink: 0;
}
/* ── Mobile tab bar (<1024px) ───────────────────────── */ /* ── Mobile tab bar (<1024px) ───────────────────────── */
.app-tabbar { .app-tabbar {
display: none; display: none;

View file

@ -1,181 +0,0 @@
<template>
<div class="search-progress" role="status" aria-label="Search in progress" aria-live="polite">
<!-- Indeterminate progress bar -->
<div class="progress-track" aria-hidden="true">
<div class="progress-bar"></div>
</div>
<!-- Status line -->
<p class="progress-label">
Searching <strong>{{ platformLabel }}</strong> for <strong>{{ query }}</strong>
</p>
<!-- Skeleton listing cards -->
<div class="skeleton-list" aria-hidden="true">
<div v-for="n in 4" :key="n" class="skeleton-card">
<div class="skeleton-thumb"></div>
<div class="skeleton-body">
<div class="skeleton-line skeleton-line--title"></div>
<div class="skeleton-line skeleton-line--meta"></div>
<div class="skeleton-footer">
<div class="skeleton-chip"></div>
<div class="skeleton-chip skeleton-chip--price"></div>
</div>
</div>
</div>
</div>
</div>
</template>
<script setup lang="ts">
import { computed } from 'vue'
const props = defineProps<{ query: string; platform?: string }>()
const PLATFORM_LABELS: Record<string, string> = {
ebay: 'eBay',
mercari: 'Mercari',
poshmark: 'Poshmark',
}
const platformLabel = computed(() =>
PLATFORM_LABELS[props.platform ?? 'ebay'] ?? props.platform ?? 'eBay'
)
</script>
<style scoped>
.search-progress {
padding: var(--space-6);
display: flex;
flex-direction: column;
gap: var(--space-5);
}
/* ── Indeterminate progress bar ───────────────── */
.progress-track {
height: 3px;
background: var(--color-surface-raised);
border-radius: var(--radius-full);
overflow: hidden;
}
.progress-bar {
height: 100%;
width: 40%;
background: var(--app-primary);
border-radius: var(--radius-full);
animation: progress-slide 1.6s ease-in-out infinite;
}
@keyframes progress-slide {
0% { transform: translateX(-100%); }
100% { transform: translateX(300%); }
}
/* ── Status label ─────────────────────────────── */
.progress-label {
font-size: 0.875rem;
color: var(--color-text-muted);
margin: 0;
}
.progress-label strong {
color: var(--color-text);
font-weight: 600;
}
/* ── Skeleton cards ───────────────────────────── */
.skeleton-list {
display: flex;
flex-direction: column;
gap: var(--space-3);
}
.skeleton-card {
display: flex;
gap: var(--space-4);
padding: var(--space-4);
background: var(--color-surface-2);
border: 1px solid var(--color-border);
border-radius: var(--radius-lg);
}
.skeleton-thumb {
width: 100px;
height: 80px;
flex-shrink: 0;
background: var(--color-surface-raised);
border-radius: var(--radius-md);
animation: shimmer 1.8s ease-in-out infinite;
}
.skeleton-body {
flex: 1;
min-width: 0;
display: flex;
flex-direction: column;
gap: var(--space-3);
justify-content: center;
}
.skeleton-line {
height: 12px;
background: var(--color-surface-raised);
border-radius: var(--radius-sm);
animation: shimmer 1.8s ease-in-out infinite;
}
.skeleton-line--title {
width: 70%;
height: 14px;
}
.skeleton-line--meta {
width: 45%;
}
.skeleton-footer {
display: flex;
gap: var(--space-2);
margin-top: var(--space-1);
}
.skeleton-chip {
height: 22px;
width: 64px;
background: var(--color-surface-raised);
border-radius: var(--radius-full);
animation: shimmer 1.8s ease-in-out infinite;
}
.skeleton-chip--price {
width: 80px;
}
/* Stagger shimmer so cards don't all pulse in sync */
.skeleton-card:nth-child(2) .skeleton-line,
.skeleton-card:nth-child(2) .skeleton-thumb,
.skeleton-card:nth-child(2) .skeleton-chip { animation-delay: 0.15s; }
.skeleton-card:nth-child(3) .skeleton-line,
.skeleton-card:nth-child(3) .skeleton-thumb,
.skeleton-card:nth-child(3) .skeleton-chip { animation-delay: 0.3s; }
.skeleton-card:nth-child(4) .skeleton-line,
.skeleton-card:nth-child(4) .skeleton-thumb,
.skeleton-card:nth-child(4) .skeleton-chip { animation-delay: 0.45s; }
@keyframes shimmer {
0%, 100% { opacity: 1; }
50% { opacity: 0.4; }
}
@media (max-width: 480px) {
.skeleton-thumb {
width: 72px;
height: 60px;
}
.skeleton-line--title { width: 85%; }
}
</style>

View file

@ -1,98 +0,0 @@
import { ref, onMounted, onUnmounted } from 'vue'
import { useSnipeMode } from './useSnipeMode'
const LS_KEY = 'cf-candycore'
const DATA_ATTR = 'candycore'
// Module-level ref — shared across all callers
const active = ref(false)
/**
* Candycore easter egg theme activated by typing "neon" outside a form field.
* Tribute to artist Neon, whose iPad painting (snipe_v0_Neon_IPad_Paint.jpeg)
* defined the candy palette: lavender primary, cyan glow, yellow crown, bubblegum pink.
*
* Mutually exclusive with Snipe Mode (each deactivates the other).
* Stores state in localStorage under 'cf-candycore'.
*/
export function useCandycoreMode() {
const snipe = useSnipeMode(false /* no sound on deactivate */)
function _playCandySound() {
try {
const ctx = new AudioContext()
// Ascending arpeggio: C5 → E5 → G5 → C6
const notes = [523.25, 659.25, 783.99, 1046.50]
const step = 0.08
notes.forEach((freq, i) => {
const t = ctx.currentTime + i * step
const osc = ctx.createOscillator()
const gain = ctx.createGain()
osc.type = 'sine'
osc.frequency.setValueAtTime(freq, t)
gain.gain.setValueAtTime(0, t)
gain.gain.linearRampToValueAtTime(0.22, t + 0.01)
gain.gain.exponentialRampToValueAtTime(0.001, t + step * 1.4)
osc.connect(gain)
gain.connect(ctx.destination)
osc.start(t)
osc.stop(t + step * 1.5)
})
setTimeout(() => ctx.close(), (notes.length * step + 0.3) * 1000)
} catch {
// Web Audio API unavailable
}
}
function activate() {
// Deactivate Snipe Mode if it's running — can't have both
if (snipe.active.value) snipe.deactivate()
active.value = true
document.documentElement.dataset[DATA_ATTR] = 'active'
localStorage.setItem(LS_KEY, 'active')
_playCandySound()
}
function deactivate() {
active.value = false
delete document.documentElement.dataset[DATA_ATTR]
localStorage.removeItem(LS_KEY)
}
function restore() {
if (localStorage.getItem(LS_KEY) === 'active') {
active.value = true
document.documentElement.dataset[DATA_ATTR] = 'active'
}
}
/**
* Registers a document keydown listener that fires activate() when the user
* types "neon" outside of any form field. Call from component setup().
* The listener is automatically removed when the calling component unmounts.
*/
function useWordTrigger() {
const TARGET = 'neon'
let buffer = ''
function handleKey(e: KeyboardEvent) {
const tag = (e.target as HTMLElement | null)?.tagName ?? ''
if (tag === 'INPUT' || tag === 'TEXTAREA' || tag === 'SELECT') return
if (e.key.length !== 1) return // skip modifier/arrow keys
buffer = (buffer + e.key.toLowerCase()).slice(-TARGET.length)
if (buffer === TARGET) {
buffer = ''
if (active.value) deactivate()
else activate()
}
}
onMounted(() => document.addEventListener('keydown', handleKey))
onUnmounted(() => document.removeEventListener('keydown', handleKey))
}
return { active, activate, deactivate, restore, useWordTrigger }
}

View file

@ -58,9 +58,6 @@ export function useSnipeMode(audioEnabled = true) {
} }
function activate() { function activate() {
// Clear candycore if it's on — can't have both
delete document.documentElement.dataset.candycore
localStorage.removeItem('cf-candycore')
active.value = true active.value = true
document.documentElement.dataset[DATA_ATTR] = 'active' document.documentElement.dataset[DATA_ATTR] = 'active'
localStorage.setItem(LS_KEY, 'active') localStorage.setItem(LS_KEY, 'active')

View file

@ -1,63 +0,0 @@
import { defineStore } from 'pinia'
import { ref } from 'vue'
export interface WatchAlert {
id: number
saved_search_id: number
platform_listing_id: string
title: string
price: number
currency: string
trust_score: number
url: string | null
first_alerted_at: string
dismissed_at: string | null
}
const BASE = import.meta.env.VITE_API_BASE ?? ''
export const useAlertsStore = defineStore('alerts', () => {
const alerts = ref<WatchAlert[]>([])
const unreadCount = ref(0)
const loading = ref(false)
const error = ref<string | null>(null)
async function fetchAlerts(includeDismissed = false) {
loading.value = true
error.value = null
try {
const res = await fetch(
`${BASE}/api/alerts${includeDismissed ? '?include_dismissed=true' : ''}`,
{ credentials: 'include' },
)
if (!res.ok) throw new Error(`${res.status}`)
const data = await res.json()
alerts.value = data.alerts
unreadCount.value = data.unread_count
} catch (e) {
error.value = e instanceof Error ? e.message : 'Failed to load alerts'
} finally {
loading.value = false
}
}
async function dismiss(alertId: number) {
await fetch(`${BASE}/api/alerts/${alertId}/dismiss`, {
method: 'POST',
credentials: 'include',
})
alerts.value = alerts.value.filter((a) => a.id !== alertId)
unreadCount.value = Math.max(0, unreadCount.value - 1)
}
async function dismissAll() {
await fetch(`${BASE}/api/alerts/dismiss-all`, {
method: 'POST',
credentials: 'include',
})
alerts.value = []
unreadCount.value = 0
}
return { alerts, unreadCount, loading, error, fetchAlerts, dismiss, dismissAll }
})

View file

@ -59,11 +59,6 @@ export interface SavedSearch {
filters_json: string // JSON blob of SearchFilters subset filters_json: string // JSON blob of SearchFilters subset
created_at: string | null created_at: string | null
last_run_at: string | null last_run_at: string | null
// Monitor settings (migration 014)
monitor_enabled: boolean
poll_interval_min: number
min_trust_score: number
last_checked_at: string | null
} }
export interface SearchParamsResult { export interface SearchParamsResult {
@ -98,7 +93,6 @@ export interface SearchFilters {
mustExclude?: string // comma-separated; forwarded to eBay -term AND client-side mustExclude?: string // comma-separated; forwarded to eBay -term AND client-side
categoryId?: string // eBay category ID (e.g. "27386" = Graphics/Video Cards) categoryId?: string // eBay category ID (e.g. "27386" = Graphics/Video Cards)
adapter?: 'auto' | 'api' | 'scraper' // override adapter selection adapter?: 'auto' | 'api' | 'scraper' // override adapter selection
platform?: string // target platform; defaults to 'ebay' when omitted
} }
// ── Session cache ───────────────────────────────────────────────────────────── // ── Session cache ─────────────────────────────────────────────────────────────
@ -179,7 +173,6 @@ export const useSearchStore = defineStore('search', () => {
if (filters.mustExclude?.trim()) params.set('must_exclude', filters.mustExclude.trim()) if (filters.mustExclude?.trim()) params.set('must_exclude', filters.mustExclude.trim())
if (filters.categoryId?.trim()) params.set('category_id', filters.categoryId.trim()) if (filters.categoryId?.trim()) params.set('category_id', filters.categoryId.trim())
if (filters.adapter && filters.adapter !== 'auto') params.set('adapter', filters.adapter) if (filters.adapter && filters.adapter !== 'auto') params.set('adapter', filters.adapter)
if (filters.platform && filters.platform !== 'ebay') params.set('platform', filters.platform)
// Use the async endpoint: returns 202 immediately with a session_id, then // Use the async endpoint: returns 202 immediately with a session_id, then
// streams listings + trust scores via SSE as the scrape completes. // streams listings + trust scores via SSE as the scrape completes.

View file

@ -31,63 +31,8 @@
<span v-if="item.last_run_at">Last run {{ formatDate(item.last_run_at) }}</span> <span v-if="item.last_run_at">Last run {{ formatDate(item.last_run_at) }}</span>
<span v-else>Never run</span> <span v-else>Never run</span>
· Saved {{ formatDate(item.created_at) }} · Saved {{ formatDate(item.created_at) }}
<span v-if="item.last_checked_at" class="saved-card-checked">
· Monitored {{ formatDate(item.last_checked_at) }}
</span>
</p> </p>
</div> </div>
<div class="saved-card-right">
<!-- Monitor toggle only shown to paid+ users -->
<div v-if="session.isPaid || session.tier === 'local'" class="monitor-section">
<label class="monitor-toggle-label">
<input
type="checkbox"
class="monitor-toggle-input"
:checked="item.monitor_enabled"
:aria-label="`Monitor ${item.name}`"
@change="onToggleMonitor(item, ($event.target as HTMLInputElement).checked)"
/>
<span class="monitor-toggle-track" aria-hidden="true" />
<span class="monitor-toggle-text">Monitor</span>
</label>
<!-- Inline settings only when enabled -->
<Transition name="slide">
<div v-if="item.monitor_enabled" class="monitor-settings">
<label class="monitor-setting-label">
Check every
<input
type="number"
class="monitor-setting-input"
:value="item.poll_interval_min"
min="15"
max="1440"
step="15"
:aria-label="`Poll interval for ${item.name} in minutes`"
@change="onIntervalChange(item, ($event.target as HTMLInputElement).valueAsNumber)"
/>
min
<span class="monitor-hint">Min 15. 60 = hourly.</span>
</label>
<label class="monitor-setting-label">
Trust
<input
type="number"
class="monitor-setting-input"
:value="item.min_trust_score"
min="0"
max="100"
step="5"
:aria-label="`Minimum trust score for ${item.name}`"
@change="onThresholdChange(item, ($event.target as HTMLInputElement).valueAsNumber)"
/>
<span class="monitor-hint">0100. 60 = medium confidence.</span>
</label>
</div>
</Transition>
</div>
<div class="saved-card-actions"> <div class="saved-card-actions">
<button class="saved-run-btn" type="button" @click="onRun(item)"> <button class="saved-run-btn" type="button" @click="onRun(item)">
Run Run
@ -96,47 +41,31 @@
class="saved-delete-btn" class="saved-delete-btn"
type="button" type="button"
:aria-label="`Delete saved search: ${item.name}`" :aria-label="`Delete saved search: ${item.name}`"
@click="onDelete(item)" @click="onDelete(item.id)"
> >
</button> </button>
</div> </div>
</div>
</li> </li>
</ul> </ul>
<!-- Undo toast for delete -->
<Transition name="toast">
<div v-if="pendingDelete" class="undo-toast" role="status" aria-live="polite">
<span>Deleted "{{ pendingDelete.name }}"</span>
<button class="undo-btn" @click="onUndoDelete">Undo</button>
</div>
</Transition>
</div> </div>
</template> </template>
<script setup lang="ts"> <script setup lang="ts">
import { onMounted, ref } from 'vue' import { onMounted } from 'vue'
import { useRouter, RouterLink } from 'vue-router' import { useRouter, RouterLink } from 'vue-router'
import { useSavedSearchesStore } from '../stores/savedSearches' import { useSavedSearchesStore } from '../stores/savedSearches'
import { useSessionStore } from '../stores/session'
import type { SavedSearch } from '../stores/savedSearches' import type { SavedSearch } from '../stores/savedSearches'
const store = useSavedSearchesStore() const store = useSavedSearchesStore()
const session = useSessionStore()
const router = useRouter() const router = useRouter()
const BASE = import.meta.env.VITE_API_BASE ?? ''
// Soft-delete state holds for 3 seconds before committing
const pendingDelete = ref<SavedSearch | null>(null)
let deleteTimer: ReturnType<typeof setTimeout> | null = null
onMounted(() => store.fetchAll()) onMounted(() => store.fetchAll())
function formatDate(iso: string | null): string { function formatDate(iso: string | null): string {
if (!iso) return '—' if (!iso) return '—'
return new Date(iso).toLocaleDateString(undefined, { month: 'short', day: 'numeric', year: 'numeric' }) const d = new Date(iso)
return d.toLocaleDateString(undefined, { month: 'short', day: 'numeric', year: 'numeric' })
} }
async function onRun(item: SavedSearch) { async function onRun(item: SavedSearch) {
@ -146,65 +75,8 @@ async function onRun(item: SavedSearch) {
router.push({ path: '/', query }) router.push({ path: '/', query })
} }
function onDelete(item: SavedSearch) { async function onDelete(id: number) {
// Soft-delete: show undo toast, commit after 3s. await store.remove(id)
if (deleteTimer) clearTimeout(deleteTimer)
pendingDelete.value = item
deleteTimer = setTimeout(async () => {
if (pendingDelete.value?.id === item.id) {
await store.remove(item.id)
pendingDelete.value = null
}
}, 3000)
}
function onUndoDelete() {
if (deleteTimer) clearTimeout(deleteTimer)
pendingDelete.value = null
}
async function onToggleMonitor(item: SavedSearch, enabled: boolean) {
await fetch(`${BASE}/api/saved-searches/${item.id}/monitor`, {
method: 'PATCH',
credentials: 'include',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
monitor_enabled: enabled,
poll_interval_min: item.poll_interval_min,
min_trust_score: item.min_trust_score,
}),
})
await store.fetchAll()
}
async function onIntervalChange(item: SavedSearch, minutes: number) {
if (isNaN(minutes) || minutes < 15) return
await fetch(`${BASE}/api/saved-searches/${item.id}/monitor`, {
method: 'PATCH',
credentials: 'include',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
monitor_enabled: item.monitor_enabled,
poll_interval_min: minutes,
min_trust_score: item.min_trust_score,
}),
})
await store.fetchAll()
}
async function onThresholdChange(item: SavedSearch, score: number) {
if (isNaN(score)) return
await fetch(`${BASE}/api/saved-searches/${item.id}/monitor`, {
method: 'PATCH',
credentials: 'include',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
monitor_enabled: item.monitor_enabled,
poll_interval_min: item.poll_interval_min,
min_trust_score: score,
}),
})
await store.fetchAll()
} }
</script> </script>
@ -255,12 +127,12 @@ async function onThresholdChange(item: SavedSearch, score: number) {
display: flex; display: flex;
flex-direction: column; flex-direction: column;
gap: var(--space-3); gap: var(--space-3);
max-width: 800px; max-width: 720px;
} }
.saved-card { .saved-card {
display: flex; display: flex;
align-items: flex-start; align-items: center;
gap: var(--space-4); gap: var(--space-4);
padding: var(--space-4) var(--space-5); padding: var(--space-4) var(--space-5);
background: var(--color-surface-2); background: var(--color-surface-2);
@ -302,131 +174,13 @@ async function onThresholdChange(item: SavedSearch, score: number) {
margin: 0; margin: 0;
} }
.saved-card-checked {
color: var(--app-primary);
}
/* Right column: monitor section + action buttons */
.saved-card-right {
display: flex;
flex-direction: column;
align-items: flex-end;
gap: var(--space-3);
flex-shrink: 0;
}
.saved-card-actions { .saved-card-actions {
display: flex; display: flex;
align-items: center; align-items: center;
gap: var(--space-2); gap: var(--space-2);
}
/* Monitor toggle */
.monitor-section {
display: flex;
flex-direction: column;
align-items: flex-end;
gap: var(--space-2);
}
.monitor-toggle-label {
display: flex;
align-items: center;
gap: var(--space-2);
cursor: pointer;
user-select: none;
}
/* Visually hide the native checkbox but keep it accessible */
.monitor-toggle-input {
position: absolute;
width: 1px;
height: 1px;
opacity: 0;
pointer-events: none;
}
.monitor-toggle-track {
display: inline-block;
width: 32px;
height: 18px;
border-radius: 9px;
background: var(--color-border);
position: relative;
transition: background 150ms ease;
flex-shrink: 0; flex-shrink: 0;
} }
.monitor-toggle-track::after {
content: '';
position: absolute;
top: 2px;
left: 2px;
width: 14px;
height: 14px;
border-radius: 50%;
background: #fff;
transition: transform 150ms ease;
}
.monitor-toggle-input:checked + .monitor-toggle-track {
background: var(--app-primary);
}
.monitor-toggle-input:checked + .monitor-toggle-track::after {
transform: translateX(14px);
}
/* Focus ring on the label when the hidden checkbox is focused */
.monitor-toggle-label:has(.monitor-toggle-input:focus-visible) .monitor-toggle-track {
outline: 2px solid var(--app-primary);
outline-offset: 2px;
}
.monitor-toggle-text {
font-size: 0.8125rem;
color: var(--color-text-muted);
white-space: nowrap;
}
/* Inline monitor settings */
.monitor-settings {
display: flex;
flex-direction: column;
gap: var(--space-2);
padding: var(--space-3);
background: var(--color-surface);
border: 1px solid var(--color-border-light);
border-radius: var(--radius-md);
font-size: 0.8125rem;
color: var(--color-text-muted);
}
.monitor-setting-label {
display: flex;
align-items: center;
gap: var(--space-2);
flex-wrap: wrap;
}
.monitor-setting-input {
width: 60px;
padding: var(--space-1) var(--space-2);
background: var(--color-surface-2);
border: 1px solid var(--color-border);
border-radius: var(--radius-sm);
color: var(--color-text);
font-family: var(--font-mono);
font-size: 0.8125rem;
text-align: center;
}
.monitor-hint {
font-size: 0.6875rem;
color: var(--color-text-muted);
opacity: 0.75;
}
.saved-run-btn { .saved-run-btn {
padding: var(--space-2) var(--space-4); padding: var(--space-2) var(--space-4);
background: var(--app-primary); background: var(--app-primary);
@ -452,65 +206,13 @@ async function onThresholdChange(item: SavedSearch, score: number) {
cursor: pointer; cursor: pointer;
transition: border-color 150ms ease, color 150ms ease; transition: border-color 150ms ease, color 150ms ease;
min-width: 28px; min-width: 28px;
min-height: 28px;
} }
.saved-delete-btn:hover { border-color: var(--color-error); color: var(--color-error); } .saved-delete-btn:hover { border-color: var(--color-error); color: var(--color-error); }
/* Undo toast */
.undo-toast {
position: fixed;
bottom: calc(var(--space-6) + env(safe-area-inset-bottom));
left: 50%;
transform: translateX(-50%);
display: flex;
align-items: center;
gap: var(--space-3);
padding: var(--space-3) var(--space-5);
background: var(--color-surface-2);
border: 1px solid var(--color-border);
border-radius: var(--radius-lg);
box-shadow: 0 4px 16px rgba(0,0,0,0.4);
font-size: 0.875rem;
color: var(--color-text);
z-index: 300;
white-space: nowrap;
}
.undo-btn {
padding: var(--space-1) var(--space-3);
background: var(--app-primary);
border: none;
border-radius: var(--radius-sm);
color: var(--color-text-inverse);
font-family: var(--font-body);
font-size: 0.8125rem;
font-weight: 600;
cursor: pointer;
}
/* Transitions */
.slide-enter-active,
.slide-leave-active { transition: opacity 150ms ease, max-height 200ms ease; max-height: 200px; overflow: hidden; }
.slide-enter-from,
.slide-leave-to { opacity: 0; max-height: 0; }
.toast-enter-active,
.toast-leave-active { transition: opacity 200ms ease, transform 200ms ease; }
.toast-enter-from,
.toast-leave-to { opacity: 0; transform: translateX(-50%) translateY(8px); }
@media (prefers-reduced-motion: reduce) {
.slide-enter-active, .slide-leave-active,
.toast-enter-active, .toast-leave-active { transition: none; }
}
@media (max-width: 767px) { @media (max-width: 767px) {
.saved-header { padding: var(--space-4); } .saved-header { padding: var(--space-4); }
.saved-list { padding: var(--space-4); } .saved-list { padding: var(--space-4); }
.saved-card { flex-direction: column; align-items: flex-start; gap: var(--space-3); } .saved-card { flex-direction: column; align-items: flex-start; gap: var(--space-3); }
.saved-card-right { width: 100%; align-items: flex-start; }
.saved-card-actions { width: 100%; justify-content: flex-end; } .saved-card-actions { width: 100%; justify-content: flex-end; }
.monitor-section { width: 100%; align-items: flex-start; }
.monitor-settings { width: 100%; }
} }
</style> </style>

View file

@ -2,29 +2,8 @@
<div class="search-view"> <div class="search-view">
<!-- Search bar --> <!-- Search bar -->
<header class="search-header"> <header class="search-header">
<div class="platform-tabs" role="tablist" aria-label="Search platform">
<button
v-for="p in PLATFORMS"
:key="p.value"
type="button"
role="tab"
class="platform-tab"
:class="{
'platform-tab--active': filters.platform === p.value,
'platform-tab--soon': !p.available,
}"
:aria-selected="filters.platform === p.value"
:disabled="!p.available"
:title="p.available ? p.label : `${p.label} — coming soon`"
@click="p.available && (filters.platform = p.value)"
>
{{ p.label }}
<span v-if="!p.available" class="platform-tab__soon">soon</span>
</button>
</div>
<form class="search-form" @submit.prevent="onSearch" role="search"> <form class="search-form" @submit.prevent="onSearch" role="search">
<div class="search-form-row1"> <div class="search-form-row1">
<template v-if="filters.platform === 'ebay' || !filters.platform">
<label for="cat-select" class="sr-only">Category</label> <label for="cat-select" class="sr-only">Category</label>
<select <select
id="cat-select" id="cat-select"
@ -41,7 +20,6 @@
</option> </option>
</optgroup> </optgroup>
</select> </select>
</template>
<label for="search-input" class="sr-only">Search listings</label> <label for="search-input" class="sr-only">Search listings</label>
<input <input
id="search-input" id="search-input"
@ -138,7 +116,6 @@
<!-- eBay Search Parameters --> <!-- eBay Search Parameters -->
<!-- These are sent to eBay. Changes require a new search to take effect. --> <!-- These are sent to eBay. Changes require a new search to take effect. -->
<template v-if="filters.platform === 'ebay' || !filters.platform">
<h2 class="filter-section-heading filter-section-heading--search"> <h2 class="filter-section-heading filter-section-heading--search">
eBay Search eBay Search
</h2> </h2>
@ -239,7 +216,6 @@
<p class="filter-pages-hint">Excludes forwarded to eBay on re-search</p> <p class="filter-pages-hint">Excludes forwarded to eBay on re-search</p>
</div> </div>
</fieldset> </fieldset>
</template>
<!-- Post-search Filters --> <!-- Post-search Filters -->
<!-- Applied locally to current results no re-search needed. --> <!-- Applied locally to current results no re-search needed. -->
@ -379,9 +355,6 @@
</div> </div>
</div> </div>
<!-- Loading (scraping in progress, no results yet) -->
<SearchProgress v-else-if="store.loading && !store.results.length" :query="store.query" :platform="filters.platform ?? 'ebay'" />
<!-- No results --> <!-- No results -->
<div v-else-if="!store.results.length && !store.loading && store.query" class="results-empty"> <div v-else-if="!store.results.length && !store.loading && store.query" class="results-empty">
<p>No listings found for <strong>{{ store.query }}</strong>.</p> <p>No listings found for <strong>{{ store.query }}</strong>.</p>
@ -402,13 +375,8 @@
</span> </span>
</p> </p>
<div class="toolbar-actions"> <div class="toolbar-actions">
<!-- Re-search indicator loading while stale results are still visible -->
<span v-if="store.loading && store.results.length" class="enriching-badge enriching-badge--searching" aria-live="polite" title="Fetching new results…">
<span class="enriching-dot" aria-hidden="true"></span>
Re-searching
</span>
<!-- Live enrichment indicator visible while SSE stream is open --> <!-- Live enrichment indicator visible while SSE stream is open -->
<span v-else-if="store.enriching" class="enriching-badge" aria-live="polite" title="Scores updating as seller data arrives"> <span v-if="store.enriching" class="enriching-badge" aria-live="polite" title="Scores updating as seller data arrives">
<span class="enriching-dot" aria-hidden="true"></span> <span class="enriching-dot" aria-hidden="true"></span>
Updating scores Updating scores
</span> </span>
@ -488,7 +456,6 @@ import { useBlocklistStore } from '../stores/blocklist'
import { useReportedStore } from '../stores/reported' import { useReportedStore } from '../stores/reported'
import ListingCard from '../components/ListingCard.vue' import ListingCard from '../components/ListingCard.vue'
import LLMQueryPanel from '../components/LLMQueryPanel.vue' import LLMQueryPanel from '../components/LLMQueryPanel.vue'
import SearchProgress from '../components/SearchProgress.vue'
const route = useRoute() const route = useRoute()
const store = useSearchStore() const store = useSearchStore()
@ -660,7 +627,6 @@ const DEFAULT_FILTERS: SearchFilters = {
mustExclude: '', mustExclude: '',
categoryId: '', categoryId: '',
adapter: 'auto' as 'auto' | 'api' | 'scraper', adapter: 'auto' as 'auto' | 'api' | 'scraper',
platform: 'ebay',
} }
const filters = reactive<SearchFilters>({ ...DEFAULT_FILTERS }) const filters = reactive<SearchFilters>({ ...DEFAULT_FILTERS })
@ -696,12 +662,6 @@ const parsedMustIncludeGroups = computed(() =>
.filter(g => g.length > 0) .filter(g => g.length > 0)
) )
const PLATFORMS: { value: string; label: string; available: boolean }[] = [
{ value: 'ebay', label: 'eBay', available: true },
{ value: 'mercari', label: 'Mercari', available: true },
{ value: 'poshmark', label: 'Poshmark', available: false },
]
const INCLUDE_MODES: { value: MustIncludeMode; label: string }[] = [ const INCLUDE_MODES: { value: MustIncludeMode; label: string }[] = [
{ value: 'all', label: 'All' }, { value: 'all', label: 'All' },
{ value: 'any', label: 'Any' }, { value: 'any', label: 'Any' },
@ -1480,16 +1440,6 @@ async function onSearch() {
white-space: nowrap; white-space: nowrap;
} }
.enriching-badge--searching {
background: color-mix(in srgb, var(--color-info) 10%, transparent);
border-color: color-mix(in srgb, var(--color-info) 30%, transparent);
color: var(--color-info);
}
.enriching-badge--searching .enriching-dot {
background: var(--color-info);
}
.enriching-dot { .enriching-dot {
width: 6px; width: 6px;
height: 6px; height: 6px;
@ -1826,53 +1776,4 @@ async function onSearch() {
to { opacity: 1; transform: translateY(0); } to { opacity: 1; transform: translateY(0); }
} }
/* ── Platform tab strip ──────────────────────────────────────────────── */
.platform-tabs {
display: flex;
gap: var(--space-1);
margin-bottom: var(--space-3);
}
.platform-tab {
display: inline-flex;
align-items: center;
gap: var(--space-1);
padding: var(--space-1) var(--space-3);
background: transparent;
border: 1.5px solid var(--color-border);
border-radius: var(--radius-full);
color: var(--color-text-muted);
font-family: var(--font-body);
font-size: 0.8125rem;
font-weight: 500;
cursor: pointer;
transition: border-color 150ms ease, color 150ms ease, background 150ms ease;
white-space: nowrap;
}
.platform-tab:hover:not(:disabled):not(.platform-tab--active) {
border-color: var(--app-primary);
color: var(--app-primary);
}
.platform-tab--active {
background: var(--app-primary);
border-color: var(--app-primary);
color: var(--color-text-inverse);
font-weight: 600;
}
.platform-tab--soon {
opacity: 0.45;
cursor: not-allowed;
}
.platform-tab__soon {
font-size: 0.625rem;
font-weight: 700;
text-transform: uppercase;
letter-spacing: 0.06em;
opacity: 0.8;
}
</style> </style>

View file

@ -93,74 +93,6 @@
</div> </div>
</section> </section>
<!-- eBay Account Connection paid+ only -->
<section v-if="ebay.oauth_available && session.isLoggedIn" class="settings-section">
<h2 class="settings-section-title">eBay Account</h2>
<!-- Connected state -->
<div v-if="ebay.connected" class="ebay-connected">
<div class="ebay-status-row">
<span class="ebay-status-dot ebay-status-dot--on" aria-hidden="true" />
<span class="settings-toggle-label">Connected</span>
</div>
<p class="settings-toggle-desc">
Snipe uses your eBay account to fetch seller registration dates instantly
via the Trading API, without Playwright scraping. This means faster, more
accurate trust scores on every search.
<span v-if="ebay.access_token_expired" class="ebay-warn">
Your access token has expired reconnect to restore instant enrichment.
</span>
</p>
<div class="ebay-action-row">
<button
v-if="ebay.access_token_expired"
class="ebay-btn ebay-btn--primary"
:disabled="ebay.connecting"
@click="startConnect"
>
Reconnect eBay account
</button>
<button
class="ebay-btn ebay-btn--danger"
:disabled="ebay.disconnecting"
@click="disconnect"
>
{{ ebay.disconnecting ? 'Disconnecting…' : 'Disconnect' }}
</button>
</div>
</div>
<!-- Not connected paid tier -->
<div v-else-if="session.isPaid || session.isPremium" class="ebay-disconnected">
<p class="settings-toggle-desc">
Connect your eBay account to enable instant seller registration date lookup
via the Trading API. Without it, Snipe falls back to slower Playwright
scraping (or Shopping API rate-limited calls) to determine account age.
</p>
<button
class="ebay-btn ebay-btn--primary"
:disabled="ebay.connecting"
@click="startConnect"
>
{{ ebay.connecting ? 'Redirecting to eBay…' : 'Connect eBay account' }}
</button>
</div>
<!-- Not connected free tier upsell -->
<div v-else class="ebay-disconnected">
<p class="settings-toggle-desc">
Connect your eBay account for instant seller trust scoring without scraping.
Available on Paid tier and above.
</p>
<a class="ebay-btn ebay-btn--upsell" href="/pricing" rel="noopener">
Upgrade to Paid
</a>
</div>
<p v-if="ebay.error" class="settings-error" role="alert">{{ ebay.error }}</p>
<p v-if="ebay.success" class="settings-success" role="status">{{ ebay.success }}</p>
</section>
<!-- Affiliate Links only shown to signed-in cloud users --> <!-- Affiliate Links only shown to signed-in cloud users -->
<section v-if="session.isLoggedIn" class="settings-section"> <section v-if="session.isLoggedIn" class="settings-section">
<h2 class="settings-section-title">Affiliate Links</h2> <h2 class="settings-section-title">Affiliate Links</h2>
@ -242,16 +174,13 @@
</template> </template>
<script setup lang="ts"> <script setup lang="ts">
import { ref, computed, watch, reactive, onMounted } from 'vue' import { ref, computed, watch } from 'vue'
import { useRoute, useRouter } from 'vue-router'
import { useTrustSignalPref } from '../composables/useTrustSignalPref' import { useTrustSignalPref } from '../composables/useTrustSignalPref'
import { useTheme } from '../composables/useTheme' import { useTheme } from '../composables/useTheme'
import { useSessionStore } from '../stores/session' import { useSessionStore } from '../stores/session'
import { usePreferencesStore } from '../stores/preferences' import { usePreferencesStore } from '../stores/preferences'
import { useLLMQueryBuilder } from '../composables/useLLMQueryBuilder' import { useLLMQueryBuilder } from '../composables/useLLMQueryBuilder'
const route = useRoute()
const router = useRouter()
const { enabled: trustSignalEnabled, setEnabled } = useTrustSignalPref() const { enabled: trustSignalEnabled, setEnabled } = useTrustSignalPref()
const theme = useTheme() const theme = useTheme()
const themeOptions: { value: 'system' | 'dark' | 'light'; label: string }[] = [ const themeOptions: { value: 'system' | 'dark' | 'light'; label: string }[] = [
@ -283,90 +212,6 @@ watch(() => prefs.affiliateByokId, (val) => { byokInput.value = val })
function saveByokId() { function saveByokId() {
prefs.setAffiliateByokId(byokInput.value) prefs.setAffiliateByokId(byokInput.value)
} }
// eBay Account Connection
const ebay = reactive({
oauth_available: false,
connected: false,
access_token_expired: false,
scopes: [] as string[],
connecting: false,
disconnecting: false,
error: '',
success: '',
})
async function fetchEbayStatus() {
try {
const res = await fetch('/api/ebay/status')
if (!res.ok) return
const data = await res.json()
ebay.oauth_available = data.oauth_available ?? false
ebay.connected = data.connected ?? false
ebay.access_token_expired = data.access_token_expired ?? false
ebay.scopes = data.scopes ?? []
} catch {
// silently ignore section stays hidden if fetch fails
}
}
async function startConnect() {
ebay.connecting = true
ebay.error = ''
try {
const res = await fetch('/api/ebay/connect')
if (!res.ok) {
const body = await res.json().catch(() => ({}))
ebay.error = body.detail ?? 'eBay connection unavailable.'
return
}
const { auth_url } = await res.json()
window.location.href = auth_url
} catch {
ebay.error = 'Could not reach the server. Try again.'
ebay.connecting = false
}
}
async function disconnect() {
ebay.disconnecting = true
ebay.error = ''
ebay.success = ''
try {
const res = await fetch('/api/ebay/disconnect', { method: 'DELETE' })
if (res.ok || res.status === 204) {
ebay.connected = false
ebay.access_token_expired = false
ebay.scopes = []
ebay.success = 'eBay account disconnected.'
} else {
ebay.error = 'Disconnect failed. Try again.'
}
} catch {
ebay.error = 'Could not reach the server. Try again.'
} finally {
ebay.disconnecting = false
}
}
onMounted(async () => {
await fetchEbayStatus()
// Handle OAuth callback redirect params: ?ebay_connected=1 or ?ebay_error=access_denied
const connected = route.query.ebay_connected
const oauthError = route.query.ebay_error
if (connected) {
ebay.success = 'eBay account connected! Trust scores will now use the Trading API.'
await fetchEbayStatus()
router.replace({ query: { ...route.query, ebay_connected: undefined } })
} else if (oauthError) {
ebay.error = oauthError === 'access_denied'
? 'eBay authorization was cancelled.'
: `eBay OAuth error: ${oauthError}`
router.replace({ query: { ...route.query, ebay_error: undefined } })
}
})
</script> </script>
<style scoped> <style scoped>
@ -528,7 +373,7 @@ onMounted(async () => {
outline-offset: 2px; outline-offset: 2px;
} }
/* ---- Error / success feedback ---- */ /* ---- Error feedback ---- */
.settings-error { .settings-error {
font-size: 0.8125rem; font-size: 0.8125rem;
color: var(--color-danger, #f85149); color: var(--color-danger, #f85149);
@ -553,100 +398,6 @@ onMounted(async () => {
border-color: var(--app-primary); border-color: var(--app-primary);
} }
.settings-success {
font-size: 0.8125rem;
color: var(--color-success, #3fb950);
margin: 0;
}
/* ---- eBay Account section ---- */
.ebay-status-row {
display: flex;
align-items: center;
gap: var(--space-2);
}
.ebay-status-dot {
width: 10px;
height: 10px;
border-radius: 50%;
flex-shrink: 0;
background: var(--color-border);
}
.ebay-status-dot--on {
background: var(--color-success, #3fb950);
}
.ebay-connected,
.ebay-disconnected {
display: flex;
flex-direction: column;
gap: var(--space-3);
}
.ebay-warn {
display: block;
margin-top: var(--space-1);
color: var(--color-warning, #d29922);
}
.ebay-action-row {
display: flex;
gap: var(--space-2);
flex-wrap: wrap;
}
.ebay-btn {
padding: var(--space-2) var(--space-4);
border: none;
border-radius: var(--radius-md);
font-size: 0.875rem;
font-weight: 600;
cursor: pointer;
font-family: inherit;
white-space: nowrap;
transition: opacity 0.15s ease;
text-decoration: none;
display: inline-flex;
align-items: center;
}
.ebay-btn:disabled {
opacity: 0.55;
cursor: not-allowed;
}
.ebay-btn--primary {
background: var(--app-primary);
color: var(--color-text-inverse, #fff);
}
.ebay-btn--primary:hover:not(:disabled) { opacity: 0.85; }
.ebay-btn--danger {
background: transparent;
color: var(--color-danger, #f85149);
border: 1px solid var(--color-danger, #f85149);
}
.ebay-btn--danger:hover:not(:disabled) {
background: color-mix(in srgb, var(--color-danger, #f85149) 12%, transparent);
}
.ebay-btn--upsell {
background: var(--color-surface-raised);
color: var(--color-text);
border: 1px solid var(--color-border);
}
.ebay-btn--upsell:hover { opacity: 0.85; }
.ebay-btn:focus-visible {
outline: 2px solid var(--app-primary);
outline-offset: 2px;
}
.theme-btn-group { .theme-btn-group {
display: flex; display: flex;
gap: 0; gap: 0;