Trust score: buyer-only feedback history inflates seller feedback_count; 12-month ratio vs lifetime count mismatch #52

New issue

Closed

opened 2026-04-27 12:50:23 -07:00 by pyr0ball · 1 comment

pyr0ball commented

2026-04-27 12:50:23 -07:00

Owner

Problem

Two related edge cases in trust scoring that cause false positives for legitimate sellers:

1. Buyer feedback inflating `feedback_count`

eBay feedback is unified — buyer and seller transactions share one score. A user with 117 lifetime feedbacks all received as a buyer (sellers praising fast payment) will score 15/20 on feedback_count and clear the low_feedback_count soft flag, even though they have zero seller experience. The count appears to signal an established seller but reflects buyer history only.

2. 12-month ratio vs lifetime count cross-horizon mismatch

The scraper parses from search results:

feedback_count — eBay lifetime score (parenthetical)
feedback_ratio — eBay 12-month percentage

These come from different time windows. The hard filter in aggregator.py:

if seller.feedback_ratio < 0.80 and seller.feedback_count > 20:
    red_flags.append("established_bad_actor")

...fires when a seller has high lifetime count but a low 12-month ratio (e.g. returned after a year away, got a couple of negatives from their first few recent sales). This incorrectly maps "established account with recent problems" to established_bad_actor and zeroes the composite score.

Observed example

Seller jjcpryz: 117 lifetime feedbacks (all as buyer, all >1 year ago), 0 feedback in last 12 months, 89 active listings. First-time or returning seller — zero seller feedback history.

Proposed fix

Short term (aggregator.py)

Add a buyer_feedback_only soft flag: fires when feedback_count > 0 but 12-month window shows 0 transactions (returning/new seller with buyer history)
Change established_bad_actor hard filter to require a minimum recent transaction count, not just lifetime count
Add declining_ratio soft flag for high lifetime count + low ratio (replaces hard flag for established accounts)

Medium term (scraper + seller profile)

Scrape the seller profile page to get the 12-month feedback breakdown separately (positive/neutral/negative counts)
Store recent_feedback_count alongside lifetime feedback_count in the Seller model
Score ratio only against recent_feedback_count when available

Files

app/trust/aggregator.py — hard filter logic
app/trust/metadata.py — _feedback_count and _feedback_ratio scorers
app/platforms/ebay/scraper.py — feedback parsing from search results
app/db/models.py — Seller model (may need recent_feedback_count field)

## Problem Two related edge cases in trust scoring that cause false positives for legitimate sellers: ### 1. Buyer feedback inflating `feedback_count` eBay feedback is unified — buyer and seller transactions share one score. A user with 117 lifetime feedbacks all received **as a buyer** (sellers praising fast payment) will score 15/20 on `feedback_count` and clear the `low_feedback_count` soft flag, even though they have zero seller experience. The count appears to signal an established seller but reflects buyer history only. ### 2. 12-month ratio vs lifetime count cross-horizon mismatch The scraper parses from search results: - `feedback_count` — eBay lifetime score (parenthetical) - `feedback_ratio` — eBay 12-month percentage These come from different time windows. The hard filter in `aggregator.py`: ```python if seller.feedback_ratio < 0.80 and seller.feedback_count > 20: red_flags.append("established_bad_actor") ``` ...fires when a seller has high lifetime count but a low 12-month ratio (e.g. returned after a year away, got a couple of negatives from their first few recent sales). This incorrectly maps "established account with recent problems" to `established_bad_actor` and zeroes the composite score. ## Observed example Seller `jjcpryz`: 117 lifetime feedbacks (all as buyer, all >1 year ago), 0 feedback in last 12 months, 89 active listings. First-time or returning seller — zero seller feedback history. ## Proposed fix ### Short term (aggregator.py) - Add a `buyer_feedback_only` soft flag: fires when `feedback_count > 0` but 12-month window shows 0 transactions (returning/new seller with buyer history) - Change `established_bad_actor` hard filter to require a minimum recent transaction count, not just lifetime count - Add `declining_ratio` soft flag for high lifetime count + low ratio (replaces hard flag for established accounts) ### Medium term (scraper + seller profile) - Scrape the seller profile page to get the 12-month feedback breakdown separately (positive/neutral/negative counts) - Store `recent_feedback_count` alongside lifetime `feedback_count` in the Seller model - Score ratio only against `recent_feedback_count` when available ## Files - `app/trust/aggregator.py` — hard filter logic - `app/trust/metadata.py` — `_feedback_count` and `_feedback_ratio` scorers - `app/platforms/ebay/scraper.py` — feedback parsing from search results - `app/db/models.py` — Seller model (may need `recent_feedback_count` field)

pyr0ball referenced this issue from a commit

2026-05-04 08:26:28 -07:00

docs: update roadmap — mark shipped issues, add platform expansion section

pyr0ball referenced this issue from a commit

2026-05-06 08:59:37 -07:00

fix(trust): treat feedback_ratio=0.0 as missing data for buyer-only/returning sellers (#52)

pyr0ball commented

2026-05-06 23:01:02 -07:00

Author

Owner

Shipped in the Apr 19–May 4 sprint. Hard bad-actor filter now scoped to 20–500 lifetime count; declining_ratio soft flag added for 500+ accounts. False positive on high-volume established sellers resolved.

Shipped in the Apr 19–May 4 sprint. Hard bad-actor filter now scoped to 20–500 lifetime count; `declining_ratio` soft flag added for 500+ accounts. False positive on high-volume established sellers resolved.

pyr0ball closed this issue

2026-05-06 23:01:03 -07:00