pyr0ball
89a58ec9b0
Bi-encoder embeddings collapse a whole query into one vector, losing multi-part reasoning structure — queries like "the procedure for setting an IP on an AVC-X" or "what is the action economy for a fighter casting a spell while prone" lose nuance. Agent-ModernColBERT is a late-interaction retriever: per-token embeddings, scored via MaxSim at query time, built specifically for agentic/multi-hop queries. Implements Option A from the issue (in-process, via `pylate`) rather than Option B (managed cf-orch service) — cf-orch already has `agent-moderncolbert` registered in model_registry.yaml with a `pagepiper/retrieve` assignment in assignments.yaml pointing at it and referencing this issue directly, someone had already pre-wired that side. - app/services/colbert_index.py: new ColBERTIndex class, mirrors BM25Index's dirty-flag/rebuild-from-SQLite pattern exactly — no separate per-shelve indexing step needed, just mark_dirty() on the same callback that already marks BM25 dirty. - app/services/retriever.py: hybrid_search's semantic half now merges BM25 with ColBERT MaxSim scores (min-max normalized per-batch, since MaxSim is unbounded unlike the old sqlite-vec L2-distance path) instead of Ollama-embed + sqlite-vec cosine. BM25 merge/rank/per-doc-cap/ adjacent-chunk-window logic is unchanged. - app/main.py / app/deps.py: per-user ColBERTIndex registry, same pattern as the existing per-user BM25Index registry. - Existing BYOK tier gate preserved exactly (llm is None check) — this is a retrieval-technology swap, not a tier/licensing change. The ColBERT model runs locally via pylate with no Ollama dependency, but gating still follows product tiering. - 12 new tests. pylate is intentionally NOT installed in the dev/test env — see the cf-sysadmin skill's "Known Gotchas" for why (installing it directly into the shared `cf` conda env broke several other services' torch/transformers pins on 2026-07-10). Tests inject fake pylate modules via sys.modules instead. Known follow-up (not addressed here): shelve scripts still compute and store Ollama embeddings into `page_vecs` at shelve time — that table is no longer read by search/chat now that retrieval uses the ColBERT index. Removing the now-redundant embedding step is separate cleanup. Closes: #8
50 lines
2.5 KiB
Markdown
50 lines
2.5 KiB
Markdown
# Environment Variables
|
|
|
|
Copy `.env.example` to `.env` and configure as needed.
|
|
|
|
## Core
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `PAGEPIPER_DATA_DIR` | `data` | Directory for SQLite databases and uploads |
|
|
| `PAGEPIPER_WATCH_DIR` | _(unset)_ | Directory scanned for PDFs/EPUBs on demand |
|
|
| `SECRET_KEY` | _(required)_ | Random secret for internal signing |
|
|
|
|
## Ollama / BYOK
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `PAGEPIPER_OLLAMA_URL` | _(unset)_ | Ollama base URL, e.g. `http://localhost:11434`. Enables hybrid search and chat (BYOK gate — see below). |
|
|
| `PAGEPIPER_CHAT_MODEL` | `mistral:7b` | Ollama chat/completion model |
|
|
| `PAGEPIPER_EMBED_MODEL` | `nomic-embed-text` | Ollama embedding model — used for shelve-time embeddings only (`page_vecs`), not for search retrieval (see ColBERT below) |
|
|
| `PAGEPIPER_EMBED_DIMS` | `1024` | Embedding dimensions (must match `PAGEPIPER_EMBED_MODEL`) |
|
|
|
|
## Semantic search (ColBERT)
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `PAGEPIPER_COLBERT_MODEL` | `lightonai/Agent-ModernColBERT` | HuggingFace model used for hybrid search's semantic half — a late-interaction retriever, runs locally via `pylate`, no Ollama call required. Gated behind the same BYOK check as the rest of hybrid search (`PAGEPIPER_OLLAMA_URL` or `CF_ORCH_URL`/`GPU_SERVER_URL` must be set). |
|
|
|
|
**Note:** `page_vecs` (the sqlite-vec table populated at shelve time using `PAGEPIPER_EMBED_MODEL`) is no longer read by search or chat — retrieval was switched to the ColBERT index above (pagepiper#8). Shelving still computes and stores those embeddings for now; removing that redundant work is tracked as a follow-up.
|
|
|
|
## cf-orch (managed deployments)
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `CF_ORCH_URL` | _(unset)_ | cf-orch coordinator URL for GPU allocation |
|
|
| `CF_LICENSE_KEY` | _(unset)_ | License key for cf-orch authentication |
|
|
| `CF_APP_NAME` | `pagepiper` | Application identifier sent to cf-orch |
|
|
|
|
## License (cloud tier)
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `PAGEPIPER_HEIMDALL_URL` | _(unset)_ | Heimdall license server URL |
|
|
| `PAGEPIPER_HEIMDALL_TOKEN` | _(unset)_ | Admin token for license validation |
|
|
|
|
## Feature flags
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `PAGEPIPER_CHAT_FEEDBACK` | `false` | Enable thumbs up/down feedback UI on chat answers |
|
|
| `CLOUD_MODE` | `false` | Enable cloud-specific middleware (rate limiting, license checks) |
|