Shared reranker service for RAG pipelines across the orchard #54

New issue

Open

opened 2026-04-21 11:37:50 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-04-21 11:37:50 -07:00

Owner

Context: Peregrine, Kiwi, Snipe, and future ND/adaptive products (Harrier, Ibis, Osprey) all need a reranker as the second stage of retrieval pipelines. Rather than each product implementing its own, build once in cf-core (MIT layer) as a shared service.

Scope:

cf_core.reranker module: rerank(query: str, candidates: list[str], top_n: int) -> list[tuple[str, float]]
Adapter pattern: local (RerankerAdapter via transformers) and remote (cf-orch allocated reranker instance)
Local-first: default to Qwen3-Reranker-0.6B or 1.5B on Free tier; 8B on Paid
Async-friendly: batch scoring, not one-at-a-time
BYOK unlock path for cloud reranker endpoint

Out of scope: Training or fine-tuning rerankers (Avocet handles that pipeline).

Acceptance criteria: Peregrine can call cf_core.reranker.rerank(resume_text, [listing1, listing2, ...]) and get back scored, sorted candidates.

Related:

peregrine#108 (job listing relevance)
kiwi#117 (recipe ranking)
snipe#51 (search result filtering)
Avocet has RerankerAdapter + text-ranking support; Qwen3-Reranker-8B queued for evaluation

**Context:** Peregrine, Kiwi, Snipe, and future ND/adaptive products (Harrier, Ibis, Osprey) all need a reranker as the second stage of retrieval pipelines. Rather than each product implementing its own, build once in cf-core (MIT layer) as a shared service. **Scope:** - [ ] `cf_core.reranker` module: `rerank(query: str, candidates: list[str], top_n: int) -> list[tuple[str, float]]` - [ ] Adapter pattern: local (RerankerAdapter via transformers) and remote (cf-orch allocated reranker instance) - [ ] Local-first: default to Qwen3-Reranker-0.6B or 1.5B on Free tier; 8B on Paid - [ ] Async-friendly: batch scoring, not one-at-a-time - [ ] BYOK unlock path for cloud reranker endpoint **Out of scope:** Training or fine-tuning rerankers (Avocet handles that pipeline). **Acceptance criteria:** Peregrine can call `cf_core.reranker.rerank(resume_text, [listing1, listing2, ...])` and get back scored, sorted candidates. **Related:** - peregrine#108 (job listing relevance) - kiwi#117 (recipe ranking) - snipe#51 (search result filtering) - Avocet has `RerankerAdapter` + `text-ranking` support; Qwen3-Reranker-8B queued for evaluation