pyr0ball 89a58ec9b0 feat: replace nomic-embed-text retriever with Agent-ModernColBERT

Bi-encoder embeddings collapse a whole query into one vector, losing
multi-part reasoning structure — queries like "the procedure for setting
an IP on an AVC-X" or "what is the action economy for a fighter casting
a spell while prone" lose nuance. Agent-ModernColBERT is a late-interaction
retriever: per-token embeddings, scored via MaxSim at query time, built
specifically for agentic/multi-hop queries.

Implements Option A from the issue (in-process, via `pylate`) rather than
Option B (managed cf-orch service) — cf-orch already has `agent-moderncolbert`
registered in model_registry.yaml with a `pagepiper/retrieve` assignment
in assignments.yaml pointing at it and referencing this issue directly,
someone had already pre-wired that side.

- app/services/colbert_index.py: new ColBERTIndex class, mirrors
  BM25Index's dirty-flag/rebuild-from-SQLite pattern exactly — no
  separate per-shelve indexing step needed, just mark_dirty() on the
  same callback that already marks BM25 dirty.
- app/services/retriever.py: hybrid_search's semantic half now merges
  BM25 with ColBERT MaxSim scores (min-max normalized per-batch, since
  MaxSim is unbounded unlike the old sqlite-vec L2-distance path) instead
  of Ollama-embed + sqlite-vec cosine. BM25 merge/rank/per-doc-cap/
  adjacent-chunk-window logic is unchanged.
- app/main.py / app/deps.py: per-user ColBERTIndex registry, same
  pattern as the existing per-user BM25Index registry.
- Existing BYOK tier gate preserved exactly (llm is None check) — this
  is a retrieval-technology swap, not a tier/licensing change. The
  ColBERT model runs locally via pylate with no Ollama dependency, but
  gating still follows product tiering.
- 12 new tests. pylate is intentionally NOT installed in the dev/test
  env — see the cf-sysadmin skill's "Known Gotchas" for why (installing
  it directly into the shared `cf` conda env broke several other
  services' torch/transformers pins on 2026-07-10). Tests inject fake
  pylate modules via sys.modules instead.

Known follow-up (not addressed here): shelve scripts still compute and
store Ollama embeddings into `page_vecs` at shelve time — that table is
no longer read by search/chat now that retrieval uses the ColBERT index.
Removing the now-redundant embedding step is separate cleanup.

Closes: #8

2026-07-10 19:02:12 -07:00

2.5 KiB

Raw Blame History

Environment Variables

Copy .env.example to .env and configure as needed.

Core

Variable	Default	Description
`PAGEPIPER_DATA_DIR`	`data`	Directory for SQLite databases and uploads
`PAGEPIPER_WATCH_DIR`	(unset)	Directory scanned for PDFs/EPUBs on demand
`SECRET_KEY`	(required)	Random secret for internal signing

Ollama / BYOK

Variable	Default	Description
`PAGEPIPER_OLLAMA_URL`	(unset)	Ollama base URL, e.g. `http://localhost:11434`. Enables hybrid search and chat (BYOK gate — see below).
`PAGEPIPER_CHAT_MODEL`	`mistral:7b`	Ollama chat/completion model
`PAGEPIPER_EMBED_MODEL`	`nomic-embed-text`	Ollama embedding model — used for shelve-time embeddings only (`page_vecs`), not for search retrieval (see ColBERT below)
`PAGEPIPER_EMBED_DIMS`	`1024`	Embedding dimensions (must match `PAGEPIPER_EMBED_MODEL`)

Semantic search (ColBERT)

Variable	Default	Description
`PAGEPIPER_COLBERT_MODEL`	`lightonai/Agent-ModernColBERT`	HuggingFace model used for hybrid search's semantic half — a late-interaction retriever, runs locally via `pylate`, no Ollama call required. Gated behind the same BYOK check as the rest of hybrid search (`PAGEPIPER_OLLAMA_URL` or `CF_ORCH_URL`/`GPU_SERVER_URL` must be set).

Note: page_vecs (the sqlite-vec table populated at shelve time using PAGEPIPER_EMBED_MODEL) is no longer read by search or chat — retrieval was switched to the ColBERT index above (pagepiper#8). Shelving still computes and stores those embeddings for now; removing that redundant work is tracked as a follow-up.

cf-orch (managed deployments)

Variable	Default	Description
`CF_ORCH_URL`	(unset)	cf-orch coordinator URL for GPU allocation
`CF_LICENSE_KEY`	(unset)	License key for cf-orch authentication
`CF_APP_NAME`	`pagepiper`	Application identifier sent to cf-orch

License (cloud tier)

Variable	Default	Description
`PAGEPIPER_HEIMDALL_URL`	(unset)	Heimdall license server URL
`PAGEPIPER_HEIMDALL_TOKEN`	(unset)	Admin token for license validation

Feature flags

Variable	Default	Description
`PAGEPIPER_CHAT_FEEDBACK`	`false`	Enable thumbs up/down feedback UI on chat answers
`CLOUD_MODE`	`false`	Enable cloud-specific middleware (rate limiting, license checks)

2.5 KiB Raw Blame History