2 commits

Author	SHA1	Message	Date
pyr0ball	c18bfec8f5	fix(lint): add ruff config, fix all lint errors for GitHub CI ... - pyproject.toml: add [tool.ruff] config suppressing E402/W293 globally, F841/E741/E702 in tests, E741 in scripts - inventory.py: split semicolon import (E702) - recipes.py: fix logger -> log (F821 undefined name) - shopping.py: rename l -> lnk in list comprehension (E741) - format_conversion.py: noqa F841 on CUDA flag (used as future hook) - backfill_keywords.py: rename done -> _done (F841) - ingest_purplecarrot.py: drop == True comparison (E712) - Auto-fix: I001 import sorting, F401 unused imports across all files	2026-07-06 02:49:02 -07:00
pyr0ball	a9ab996bcc	feat(pipeline): purple carrot weekly menu scraper with CF bypass ... Add three new scripts for Purple Carrot recipe pipeline: - discover_current_menu.py: fetches this week's active menu slugs from /plant-based-recipes using requests (server-rendered HTML, no JS needed). Accumulates slugs across weekly runs for building a recipe corpus over time. - discover_slugs_categories.py: crawls recipe-category listing pages with ?page=N pagination to discover historical slug inventory. Note: category archive slugs (past menu items) 404 when scraped live; only use for identifying currently-featured recipes per category. - scrape_live.py: updated with --slugs-from flag (load slug inventory from any parquet, not just the default Wayback one) and fresh-context-per-slug pattern to bypass Cloudflare session-level bot detection (which fires on the 2nd+ request in a shared browser context). Discovery: the live site only renders full ingredient/instruction content for recipes currently on the active weekly menu. 23/23 current menu recipes scraped successfully (100% hit rate vs ~1% for archived slugs).	2026-05-21 16:16:32 -07:00