Pagepiper

v0.1.0 | Self-hosted PDF and EPUB search for your personal library

Pagepiper lets you drop PDFs and EPUBs into a library, index them, and search across the full text. With Ollama configured, you also get hybrid vector search and an LLM (large language model) chat interface that cites specific page numbers when it answers.

Built for TTRPG (tabletop roleplaying game) players tired of ctrl-F'ing through Pathfinder core rulebooks. Works equally well for fan fiction EPUB collections, AO3 exports, and any personal document library.

Try it: pagepiper.circuitforge.tech

---

Features

Feature	Free tier	Paid (BYOK)
PDF and EPUB upload via browser drag-and-drop	Yes	Yes
Directory scan for existing files	Yes	Yes
BM25 full-text search (no LLM required)	Yes	Yes
Unlimited local ingestion	Yes	Yes
Hybrid BM25 + k-NN vector search	No	Yes (local Ollama)
LLM chat with page-level citations	No	Yes (local Ollama)
Thumbs up / down feedback on answers	No	Yes

BYOK (bring your own key) means you supply your own Ollama instance. No cloud API keys, no usage billing.

BM25 (Best Match 25) is a keyword ranking algorithm. It works without any LLM and runs entirely inside the Docker container. k-NN (k-nearest neighbor) vector search uses embeddings to find passages that are semantically similar to your question, even when the exact words don't match.

---

Tech Stack

• Backend: FastAPI + SQLite (BM25 via custom BM25Index, vectors via sqlite-vec)
• Frontend: Vue 3 SPA served by nginx
• Embedding model: nomic-embed-text via Ollama (1024-dim, optional)
• Chat LLM: mistral:7b via Ollama (optional, any Ollama model works)
• Deployment: Docker Compose

---

Quick Start (Self-Hosting)

Prerequisites

• Docker and Docker Compose
• PDFs or EPUBs you want to search
• Optional: Ollama for semantic search and RAG (retrieval-augmented generation) chat

1. Clone the repo

git clone https://git.opensourcesolarpunk.com/Circuit-Forge/pagepiper
cd pagepiper

2. Configure

cp .env.example .env

Open .env and set your paths:

# Directory to scan for PDFs/EPUBs (used by the "Scan" button in the UI)
PAGEPIPER_BOOKS_DIR=/path/to/your/pdfs

# Where Pagepiper stores its SQLite index and uploaded files
PAGEPIPER_DATA_DIR=data

To unlock hybrid search and LLM chat, uncomment and set the Ollama block:

PAGEPIPER_OLLAMA_URL=http://localhost:11434
PAGEPIPER_CHAT_MODEL=mistral:7b
PAGEPIPER_EMBED_MODEL=nomic-embed-text

3. Start

./manage.sh start

Open http://localhost:8521.

4. Add documents

Two ways to add files:

Upload via browser (easiest for small collections): Click Upload in the Library view and select a PDF or EPUB. The file saves to data/uploads/ and begins indexing automatically.

Scan a directory (best for large collections): Set PAGEPIPER_BOOKS_DIR in your .env to a folder of PDFs/EPUBs, then click Scan in the Library view. Pagepiper finds all files recursively and queues them for indexing.

5. Search and chat

Switch to the Chat tab and ask questions. On the free tier, BM25 keyword search returns matching passages. With Ollama configured, you get semantic search and an LLM-generated answer with page-number citations.

---

Ollama Setup (optional)

Install Ollama from ollama.com, then pull the models:

ollama pull mistral:7b
ollama pull nomic-embed-text

On a headless Linux server, make Ollama listen on all interfaces so the Docker container can reach it:

OLLAMA_HOST=0.0.0.0 ollama serve

On Docker Desktop (Linux or Mac), host.docker.internal resolves automatically. No extra network config needed.

---

Environment Variables

Variable	Default	Description
PAGEPIPER_BOOKS_DIR	./books	Host directory to scan for PDFs and EPUBs
PAGEPIPER_DATA_DIR	./data	SQLite index and uploaded files live here
PAGEPIPER_OLLAMA_URL	(unset)	Ollama base URL; leave blank for BM25-only mode
PAGEPIPER_EMBED_MODEL	nomic-embed-text	Ollama embedding model (1024-dim default)
PAGEPIPER_EMBED_DIMS	1024	Must match the embedding model's output dimensions
PAGEPIPER_CHAT_MODEL	mistral:7b	Ollama chat model; any Ollama model name works
PAGEPIPER_CHAT_FEEDBACK	(unset)	Set to true to enable thumbs up/down on chat answers

---

Management

./manage.sh start          # Build and start (dev)
./manage.sh stop           # Stop
./manage.sh restart        # Restart
./manage.sh status         # Show container status
./manage.sh logs [svc]     # Tail logs (default: all services; pass 'api' or 'web' to filter)
./manage.sh open           # Open the UI in your browser
./manage.sh build          # Rebuild images without cache

./manage.sh cloud:start    # Start the cloud managed instance (port 8533)
./manage.sh cloud:stop
./manage.sh cloud:restart
./manage.sh cloud:status
./manage.sh cloud:logs [svc]
./manage.sh cloud:build

---

Cloud Managed Instance

The cloud deployment runs at pagepiper.circuitforge.tech and at menagerie.circuitforge.tech/pagepiper. It uses compose.cloud.yml with LLM inference routed through the cf-orch coordinator.

To run your own cloud-style deployment:

cp .env.cloud.example .env
# Edit .env: set PAGEPIPER_OLLAMA_URL and data paths
./manage.sh cloud:start

Cloud instance listens on port 8533. The API is internal-only; nginx proxies /api/ to the backend.

---

Data and Backups

The data/ directory contains the SQLite index database and all uploaded files. Back it up to preserve your index. Pagepiper indexes documents at ingest time. If you modify or replace a source file, use the re-index button on the document card to rebuild its entry.

Large PDFs (hundreds of pages) can take a few minutes to index. The status badge on the document card updates as indexing progresses.

---

Licensing

Pagepiper uses a split license:

• MIT: BM25 full-text search, document library management, ingest pipeline, EPUB support
• BSL 1.1: Hybrid vector search (embedding + k-NN), RAG chat, LLM integration

BSL 1.1 is free for personal non-commercial self-hosting. SaaS re-hosting or commercial redistribution requires a license from CircuitForge. BSL 1.1 converts to MIT after four years.

License keys: circuitforge.tech

---

Contributing

Issues and PRs welcome at git.opensourcesolarpunk.com/Circuit-Forge/pagepiper.

The ingest pipeline and BM25 index are MIT-licensed. If you build a better PDF parser or add support for additional formats (CBZ, MOBI, etc.), the community benefits directly.