pyr0ball
f941ebdeeb
Extends Pagepiper's document shelving pipeline (renamed from "ingest" — see below) to cover the formats most likely to appear in a real-world engineering document corpus, prompted by scoping a STERIS licensing pitch that needs DOCX/ODT coverage. - Rename the ingest pipeline to "shelve" throughout (scripts/, app/api, tests, docs, frontend). "Glean" (Turnstone's term) was considered and rejected — that's a harvest metaphor for log/knowledge extraction, not a fit for documents entering a library. Documented as a general CF naming principle in the org-level CLAUDE.md. - Wire DOCX into the upload/scan UI, README, and docs — the extraction logic (heading-based chunking, table serialization) already existed but wasn't exposed to users or covered by tests. - Add ODT support via odfpy, mirroring DOCX's chunking strategy. - Add Apple Pages support via headless LibreOffice conversion to ODT. No maintained Python library parses the IWA format directly; libreoffice bundles libetonyek, the only real open-source Pages parser. Adds libreoffice-writer to the Docker image (~300-400MB) for this. - 24 new/updated tests across shelve_docx, shelve_odt, and shelve_pages; full suite (72 tests) passing. Known gaps not addressed here: no Windchill/DocPortal connector exists yet (metadata-only PowerShell recon only), Excel/.xlsx is unsupported, and circuitforge_core.tasks.dispatch_task does not currently exist in circuitforge-core — cf-orch dispatch is dead code, always falling through to local BackgroundTasks. See circuitforge-plans/pagepiper/superpowers/plans/2026-07-10-steris-licensing-pitch.md for the full writeup.
30 lines
1.2 KiB
Text
30 lines
1.2 KiB
Text
# Copy to .env and fill in your values. .env is gitignored.
|
|
|
|
# Path to your PDF library on the host machine
|
|
PAGEPIPER_BOOKS_DIR=/path/to/your/pdfs
|
|
|
|
# Data directory (SQLite + vector DB stored here)
|
|
PAGEPIPER_DATA_DIR=data
|
|
|
|
# LLM backend — either option (or both) unlocks semantic search and RAG chat.
|
|
#
|
|
# Option A: direct Ollama URL
|
|
# PAGEPIPER_OLLAMA_URL=http://localhost:11434
|
|
#
|
|
# Option B: cf-orch coordinator (resolves service URL via GPU allocation).
|
|
# Set CF_ORCH_URL alone — no PAGEPIPER_OLLAMA_URL needed.
|
|
# PAGEPIPER_OLLAMA_URL is used as a fallback if cf-orch is unreachable.
|
|
# CF_ORCH_URL=http://localhost:7700
|
|
# CF_APP_NAME=pagepiper
|
|
# PAGEPIPER_ORCH_SERVICE=ollama # or cf-text for managed transformer inference
|
|
#
|
|
PAGEPIPER_CHAT_MODEL=mistral:7b
|
|
PAGEPIPER_EMBED_MODEL=nomic-embed-text
|
|
|
|
# Forgejo API token — enables the in-app feedback button (files Forgejo issues)
|
|
# Create a token at https://git.opensourcesolarpunk.com/user/settings/applications
|
|
# FORGEJO_API_TOKEN=
|
|
|
|
# Enable thumbs up/down on chat answers (stores retrieval quality signals locally)
|
|
# Off by default — opt in when you want to collect correction data
|
|
# PAGEPIPER_CHAT_FEEDBACK=true
|