Extends the shelve pipeline to cover spreadsheets, closing the Excel gap
called out in the PR's original "known gaps" list — Windchill/DocPortal
corpora commonly include parts lists and spec sheets as spreadsheets, not
just prose documents.
- scripts/shelve_xlsx.py — openpyxl, chunked by sheet with row-window
splitting for large sheets (header row repeated in every window so
each chunk stays self-describing for retrieval).
- scripts/shelve_ods.py — same chunking strategy via odfpy (already a
dependency from ODT support), OpenDocumentSpreadsheet's Table/TableRow/
TableCell.
- scripts/shelve_numbers.py — converts via headless LibreOffice to XLSX
and delegates to shelve_xlsx, mirroring shelve_pages.py's pattern for
.pages. Adds libreoffice-calc to the Docker image alongside the
existing libreoffice-writer.
- Upload button text changed from an ever-growing format list to
"Upload Document or Spreadsheet" — the Supported Formats table in
README/docs is now the source of truth for the full list.
- 13 new tests (XLSX, ODS, Numbers); full suite (85 tests) passing.
Manually verified via Playwright against an isolated test instance:
XLSX and ODS both upload, shelve to "ready", and store correctly
row-serialized, header-repeated chunks (confirmed via sample-chunks).
BM25 search against a 2-chunk toy corpus returned no hits for terms
split 1-vs-1 across the two chunks — traced to Okapi BM25's IDF formula
giving an exact 0 for terms in exactly half a tiny corpus
(log((N-n+0.5)/(n+0.5)) = log(1.0) = 0, filtered by `score <= 0`), not a
defect in the new shelvers. The earlier DOCX/ODT/PDF Playwright pass
(5 chunks total) diluted this enough to return real results.
pyr0ball
d39cfbd87a