pyr0ball
f941ebdeeb
Extends Pagepiper's document shelving pipeline (renamed from "ingest" — see below) to cover the formats most likely to appear in a real-world engineering document corpus, prompted by scoping a STERIS licensing pitch that needs DOCX/ODT coverage. - Rename the ingest pipeline to "shelve" throughout (scripts/, app/api, tests, docs, frontend). "Glean" (Turnstone's term) was considered and rejected — that's a harvest metaphor for log/knowledge extraction, not a fit for documents entering a library. Documented as a general CF naming principle in the org-level CLAUDE.md. - Wire DOCX into the upload/scan UI, README, and docs — the extraction logic (heading-based chunking, table serialization) already existed but wasn't exposed to users or covered by tests. - Add ODT support via odfpy, mirroring DOCX's chunking strategy. - Add Apple Pages support via headless LibreOffice conversion to ODT. No maintained Python library parses the IWA format directly; libreoffice bundles libetonyek, the only real open-source Pages parser. Adds libreoffice-writer to the Docker image (~300-400MB) for this. - 24 new/updated tests across shelve_docx, shelve_odt, and shelve_pages; full suite (72 tests) passing. Known gaps not addressed here: no Windchill/DocPortal connector exists yet (metadata-only PowerShell recon only), Excel/.xlsx is unsupported, and circuitforge_core.tasks.dispatch_task does not currently exist in circuitforge-core — cf-orch dispatch is dead code, always falling through to local BackgroundTasks. See circuitforge-plans/pagepiper/superpowers/plans/2026-07-10-steris-licensing-pitch.md for the full writeup.
36 lines
1.3 KiB
Markdown
36 lines
1.3 KiB
Markdown
# Quick Start
|
|
|
|
This guide gets you from zero to searching your first document in under five minutes.
|
|
|
|
## 1. Start Pagepiper
|
|
|
|
```bash
|
|
./manage.sh start
|
|
```
|
|
|
|
Open `http://localhost:8521` in your browser.
|
|
|
|
## 2. Add a document
|
|
|
|
You have two options:
|
|
|
|
**Upload directly** — click **Upload PDF / EPUB / DOCX / ODT / Pages** in the library header and pick a file from your computer.
|
|
|
|
**Scan a directory** — set `PAGEPIPER_WATCH_DIR` in your `.env` to a folder of PDFs or EPUBs, then click **Scan for PDFs**. Pagepiper indexes every file it finds.
|
|
|
|
## 3. Wait for indexing
|
|
|
|
The document card shows progress while text is being extracted and embedded:
|
|
|
|
- **Extracting text...** (animated bar) — PDF/EPUB/DOCX/ODT/Pages is being parsed into page chunks
|
|
- **Embedding N / M pages (X%)** (filling bar) — vectors are being written to the vector store (only when Ollama is configured)
|
|
|
|
Once the badge shows **READY**, the document is searchable.
|
|
|
|
## 4. Search
|
|
|
|
Click **Search** in the navigation. Type any phrase and see ranked page excerpts with scores. Results are instant using BM25 full-text search — no Ollama required.
|
|
|
|
## 5. Chat (optional, requires Ollama)
|
|
|
|
See the [Ollama Setup](ollama-setup.md) guide to enable hybrid vector search and LLM-powered chat. Once configured, the **Chat** tab lets you ask natural-language questions and get answers with page citations.
|