Bi-encoder embeddings collapse a whole query into one vector, losing
multi-part reasoning structure — queries like "the procedure for setting
an IP on an AVC-X" or "what is the action economy for a fighter casting
a spell while prone" lose nuance. Agent-ModernColBERT is a late-interaction
retriever: per-token embeddings, scored via MaxSim at query time, built
specifically for agentic/multi-hop queries.
Implements Option A from the issue (in-process, via `pylate`) rather than
Option B (managed cf-orch service) — cf-orch already has `agent-moderncolbert`
registered in model_registry.yaml with a `pagepiper/retrieve` assignment
in assignments.yaml pointing at it and referencing this issue directly,
someone had already pre-wired that side.
- app/services/colbert_index.py: new ColBERTIndex class, mirrors
BM25Index's dirty-flag/rebuild-from-SQLite pattern exactly — no
separate per-shelve indexing step needed, just mark_dirty() on the
same callback that already marks BM25 dirty.
- app/services/retriever.py: hybrid_search's semantic half now merges
BM25 with ColBERT MaxSim scores (min-max normalized per-batch, since
MaxSim is unbounded unlike the old sqlite-vec L2-distance path) instead
of Ollama-embed + sqlite-vec cosine. BM25 merge/rank/per-doc-cap/
adjacent-chunk-window logic is unchanged.
- app/main.py / app/deps.py: per-user ColBERTIndex registry, same
pattern as the existing per-user BM25Index registry.
- Existing BYOK tier gate preserved exactly (llm is None check) — this
is a retrieval-technology swap, not a tier/licensing change. The
ColBERT model runs locally via pylate with no Ollama dependency, but
gating still follows product tiering.
- 12 new tests. pylate is intentionally NOT installed in the dev/test
env — see the cf-sysadmin skill's "Known Gotchas" for why (installing
it directly into the shared `cf` conda env broke several other
services' torch/transformers pins on 2026-07-10). Tests inject fake
pylate modules via sys.modules instead.
Known follow-up (not addressed here): shelve scripts still compute and
store Ollama embeddings into `page_vecs` at shelve time — that table is
no longer read by search/chat now that retrieval uses the ColBERT index.
Removing the now-redundant embedding step is separate cleanup.
Closes: #8
pyr0ball
89a58ec9b0