POST /recipes/scan/stream emits live status events while cf-docuvision
allocates and processes, replacing the static spinner with phase-aware labels:
allocating -> scanning -> structuring -> done|error
Uses asyncio.Queue bridge to route progress callbacks from the sync scanner
thread to the async SSE generator. Frontend updated to consume the stream via
fetch + ReadableStream (EventSource does not support POST multipart).
Closes kiwi#136 (companion to the docuvision routing fix).
Two-step pipeline: task_allocate("kiwi", "recipe_scan", service_hint="cf-docuvision")
acquires a docuvision allocation, calls /extract per image to get OCR text, then
LLMRouter structures the combined OCR output into recipe JSON via the text
extraction prompt.
Also fixes DocuvisionClient bugs:
- POST field was "image" (ignored by Pydantic) — should be "image_b64"
- Response read "text" key — docuvision returns "raw_text"
- Add hint parameter (use "text" for recipe cards, dense prose)
- Configurable timeout (default 120s; docuvision lazy-loads model on first request)
Replaces single-path cf-orch allocation with a three-tier strategy:
tier 1 task_allocate() (coordinator-driven), tier 2 direct CFOrchClient.allocate()
(TaskNotRegistered fallback), tier 3 local LLMRouter. Module-level imports for
CFOrchClient and LLMRouter make all three paths patchable in tests without
import caching issues.
Full ActivityPub implementation wired to cf-core.activitypub module:
Endpoints (root-level, not under /api/v1):
GET /.well-known/webfinger — WebFinger JRD (AP_ENABLED only)
GET /ap/actor — Instance actor document
POST /ap/actor/inbox — Incoming Follow/Undo (dedup + Accept dispatch)
GET /ap/outbox — OrderedCollection of community posts
GET /ap/posts/{slug} — Individual AP Note
GET /ap/followers — Follower count collection
GET /ap/following — Empty following collection
Mastodon OAuth (under /api/v1/social/mastodon/):
POST /connect — Dynamic app registration + OAuth flow start
GET /callback — Code exchange + token storage (Fernet-encrypted)
DELETE /disconnect — Token revocation
GET /status — Connection status
Config: AP_ENABLED, AP_HOST, AP_KEY_PATH, AP_TOKEN_ENCRYPTION_KEY
Migration 042: ap_followers, ap_deliveries, ap_received, mastodon_tokens tables
Key manager: auto-generates RSA-2048 keypair on first boot if AP_ENABLED
Delivery service: deliver_to_followers() with 3-retry exponential backoff + DB log
Post publish: background fan-out to AP followers + Mastodon when opted-in
All AP endpoints gracefully degrade (404) when AP_ENABLED=false.
Three-layer dedup check before community post submission:
- L1: title ILIKE search against existing posts in community DB
- L2: Jaccard ingredient overlap using local corpus (≥0.70 very_similar, ≥0.35 somewhat_similar)
- L3: similar_to_ref FK — user can explicitly mark post as variation of existing
New endpoint: POST /api/v1/community/check-similar (gracefully no-ops if community DB absent)
New service: app/services/community/dedup.py — jaccard(), similarity_tier(), build_similar_post_result()
Both publish modals (plan + outcome) now check similarity before submit; user can proceed as-is,
mark as variation, or cancel. similar_to_ref passed in final publish payload.
- New Ask tab in recipe browser tab bar (alongside Find/Browse/Saved)
- Text input + Search button; Enter to submit
- 4 example question chips shown in empty state
- Results as clickable recipe cards (opens RecipeDetailPanel)
- Pantry match_pct badge on each card when pantry items are available
- LLM-synthesized answer shown above results (paid tier)
- Session history: last 3 questions shown as re-runnable chips
- Keyboard navigable (tab key, Enter on card, Arrow keys on tab bar)
- ARIA: role=tabpanel, aria-labelledby, aria-live for error/answer regions
Also fixes pre-existing build issues now caught by vue-tsc:
- Move pantryItems/secondaryPantryItems declarations before auto-suggest
watcher that uses them (TS2448 block-scoped variable before declaration)
- Fix nullable regex capture group access in parsedStream computed (TS2532)
using optional chaining (titleMatch?.[1], ingMatch?.[1], etc.)
Closes#134
Free tier: keyword extraction + FTS ingredient search + title probe search.
Paid tier / BYOK: same search, then LLM synthesis of a conversational answer
(8s timeout so an unresponsive model degrades gracefully to recipe list only).
- AskRequest / AskRecipeHit / AskResponse schemas in recipe.py
- _extract_ask_keywords(): tokenize question, strip stopwords
- _ask_in_thread(): two-pronged search (ingredient FTS + title LIKE)
merges by ID, computes pantry match_pct when pantry_items provided
- Endpoint registered before /{recipe_id} to avoid integer coercion on /ask
- LLM synthesis gated to paid/premium/ultra only (not "local" dev tier)
Closes#134 (backend)
Parses the streamed LLM output (Title / Ingredients / Directions / Notes
plain-text format) on the fly as tokens arrive. Shows a shimmer skeleton
for each section while that section has not yet arrived, then swaps in
real content as the parse succeeds — title first, then ingredients, then
numbered steps, then notes on completion.
parsedStream computed: matches Title, Ingredients (comma-split), numbered
step lines, and Notes sections from the accumulating streamChunks string.
Skeleton shimmer is CSS-only (no JS); respects prefers-reduced-motion by
falling back to a static placeholder color. The stream-output <pre> block
is removed from the template entirely — raw tokens never reach the user.
Auto-suggest (L1/L2 only):
When the Find tab is activated with a non-empty pantry and no existing
results, suggestion fires immediately without user action. L3/L4 are
excluded to avoid unintended VRAM allocation and AI quota charges.
After the first auto-suggest completes, the Refine panel collapses so
the results are the first thing the user sees.
Live re-suggest (L1/L2 only):
A single filterKey computed wraps all filter state as JSON. Any filter
change while on the Find tab with existing results triggers a debounced
(1.2s) re-suggest, keeping the result list live without button clicks.
Refine collapsible:
Time budget, Dietary preferences, and Nutrition/Advanced filters are
wrapped in a v-show panel controlled by filtersOpen (persisted to
localStorage under kiwi:find_filters_open, default open). Level
selector, Hard Day Mode, and the Suggest button remain always visible.
Toggle button shows active filter count badge when any filter is set.
The time budget selector (hands-on and total time chips) was previously
gated behind the time_first_layout Settings preference. Removed the v-if
guard so both rows are always visible in the Find tab without requiring
a Settings change.
Added "No limit" clear buttons that appear next to the chip row when a
time limit is active, so users can reset a time filter in one tap without
needing to find the active chip and re-tap it.
The time_first_layout setting in Settings remains for users who want
control over the layout.
Renders domain › category › subcategory above the recipe grid whenever
a domain and category are active. Each ancestor crumb is a button that
navigates back up the hierarchy (selectDomain / selectCategory). The
leaf node is a plain span with aria-current="page". The nav has
aria-label="Browse location" for screen reader context.
Adds a summary bar that appears at the top of the Find Recipes panel
whenever any filter is active. Shows a count ("3 filters active") and
a Clear all button that resets all Find-tab filters in one tap:
constraints, allergies, excluded ingredients, shopping mode,
pantry-match-only, hard day mode, time budgets (active + total),
max missing, style, category, and all four nutrition limits.
Local input refs (constraintInput, allergyInput, etc.) are also cleared
so the text fields don't show stale uncommitted values after a clear.
Each setting now saves via a debounced (600ms) individual API call when
its value changes. A hydration guard (_hydrated flag + nextTick) prevents
watchers from firing during the initial load() fetch, ensuring the first
API round-trip does not generate spurious write calls.
Removed: five explicit Save buttons across Equipment, Sensory, Units,
Shopping Region, and Recipe Search Layout sections.
Added: "Changes save automatically." subtitle + fixed bottom-right toast
that appears for 2s after any successful save, with enter/leave
transitions that respect prefers-reduced-motion via the theme.
The full save() and saveSensory() actions are kept as internal fallbacks.
Screen readers had no way to determine which domain, category, subcategory,
or sort button was selected — the active CSS class is invisible to assistive
technology.
- aria-pressed on all toggle buttons (domain, category, subcategory, sort)
- aria-label="Previous page" / "Next page" on pagination buttons
- aria-live="polite" on results count span — announces filter result changes
- Equipment chip-remove: "Remove" → "Remove equipment: {item}"
Addresses WCAG 2.1 AA criteria 4.1.2 (Name, Role, Value) and 1.3.1
(Info and Relationships). Part of kiwi UX audit (2026-05-11).
Previously, get_or_create_product was only called when auto_add was true,
so scan responses with auto_add=false returned no product details. Now the
DB lookup always runs when product_info is available; inventory insertion
is still conditional on auto_add_to_inventory. Fixes preview-only barcode
scans returning empty product fields.
Two-phase streaming architecture:
Phase 1 (sync thread): IngredientClassifier builds element profiles +
gap list from SQLite — thread-safe, no async context needed
Phase 2 (async): LLMRecipeGenerator.stream_generate() yields tokens via
cf-orch warm vllm (existing /stream-token path) or AsyncOpenAI against
Ollama if the coordinator is unavailable
Backend (app/services/recipe/llm_recipe.py):
- stream_generate() async generator; _try_alloc_for_stream() sync helper
- _stream_openai_compat() static method handles __auto__ model resolution
- LLMRecipeGenerator(None) is safe for streaming (store not used)
Endpoint (app/api/endpoints/recipes.py):
- ?stream=true on POST /recipes/suggest returns StreamingResponse
- X-Accel-Buffering: no prevents nginx buffering without nginx.conf edits
Frontend (api.ts, recipes.ts, RecipesView.vue):
- suggestRecipeStream() uses fetch + ReadableStream (POST; EventSource
only supports GET)
- streamSuggest() action in recipes store builds request internally
- RecipesView.streamRecipe() silently falls back to native SSE when
cf-orch token fetch fails rather than surfacing an error
Exposes four read-only tools to Claude Code:
kiwi_query_corpus — parameterised SELECT against kiwi.db (200-row cap)
kiwi_count_fts — FTS5 MATCH hit count for keyword coverage audits
kiwi_sample_tags — tag frequency distribution by prefix
kiwi_browse_preview — first-page results from the live browse API
DB opened in SQLite URI read-only mode (mode=ro); any write statement is
rejected at the driver level. Configure via KIWI_DB_PATH and KIWI_API_URL
env vars (see module docstring for settings.json snippet).
Adds `update` (local stack) and `cloud-update` (menagerie) subcommands
to manage.sh. Both pull HEAD and rebuild/restart the Docker stack in one
step — required for post-merge deployment without manual compose commands.
Adds max_active_min request field and backend filter. Active time uses
parse_time_effort().active_min (passive waits excluded). Recipes with
no parsed active time signal are not excluded (avoid hiding unlabelled
results). Total and active limits are AND'd when both set.
UI: two pill rows — "Hands-on time" (15/30/45/1hr) and "Total time"
(30m/1hr/90m/2hr/3hr/4+hr). Replaces single row capped at 90 min.
Recipe cards were rendering full directions, all nutrition chips,
prep notes, swap candidates, and grocery links inline in the grid —
making each card tall enough to push the second row below the fold at
3-column widths. Cards now show title, match/complexity/time badges,
up to 4 pantry ingredient chips, missing count, and calorie hint.
Full detail remains in RecipeDetailPanel on "Make this".
ElementClassifier.classify_batch() was issuing N separate DB queries
(one per pantry item). Replaced with a single WHERE name IN (...)
query + heuristic fallback for misses — same result, one round-trip.
backfill_meal_tags.py merges meal: tags from title-only matching
into existing inferred_tags without re-deriving all other signals.
~10x faster than infer_recipe_tags.py --force for meal-tag-only
updates: 3.19M recipes in ~5-10min vs ~2.5h for full re-derivation.
Adds _MEAL_SIGNALS table to tag_inferrer with title-only matching for:
meal:Breakfast — pancakes, waffles, frittata, oatmeal, granola, etc.
meal:Dessert — cake, cookie, brownie, pudding, ice cream, tart, etc.
meal:Snack — dip, chips, popcorn, nachos, energy balls, etc.
meal:Beverage — smoothie, cocktail, juice, lemonade, etc.
meal:Lunch — sandwich, wrap, panini, grilled cheese, etc.
meal:Bread — bread, sourdough, focaccia, dinner roll, etc.
Uses word-boundary + optional-plural regex (\bWORD(?:s|es)?\b) so:
- "pancakes" matches the "pancake" pattern but "pancake" != "cake"
- "tartare" does not match "tart" (no word boundary after tart in tartare)
- "dipping" does not match "dip" (extra chars prevent boundary)
Title-only matching (not ingredient text) avoids false positives from
ingredient names like "cake flour" or "sandwich bread".
Estimated browse impact after backfill (--force on 3.19M recipes):
Breakfast: 43 → ~70k
Dessert: 372 → ~350k (real desserts, not flavor:Sweet)
Snack: 57 → ~60k
Beverage: 43 → ~36k
Lunch: 69 → ~26k
- Dinner: replace non-matching text keywords with main:X protein inferred tags (0 -> 815k results)
- All meal_type categories: add meal:X structured tag phrases
- Dietary: switch to dietary:X-only phrases; bare text keywords matched can_be:X
tags (nearly all recipes), inflating counts to 1.3M+ falsely
- Cuisine: add cuisine:X structured tag phrases to Italian, Mexican, Asian,
Indian, Mediterranean, American, BBQ, European, Latin American
- Side Dish: use main:Vegetables + main:Grains as proxy (no meal:Side Dish tag exists)
- Dessert: remove 'sweet' keyword (matched flavor:Sweet on all recipes)
- New dietary categories: Low-Sodium, Paleo
Closes#122. Partial progress on #123.
Follow-up: #125 (expand meal: tag inferrer coverage)
New feature: photograph a recipe card, cookbook page, or handwritten
note and have it extracted into a structured, editable recipe.
Backend:
- POST /recipes/scan: accept 1-4 photos, run VLM extraction, return
structured JSON for review (not auto-saved)
- POST /recipes/scan/save: persist a reviewed/edited recipe
- GET/DELETE /recipes/user: user-created recipe CRUD
- Vision backend priority: cf-orch -> local Qwen2.5-VL -> Anthropic BYOK
- 503 with clear config hint when no vision backend available
- Multi-photo support: facing pages (ingredients/directions) sent together
- Pantry cross-reference: marks which ingredients are already on hand
- migration 041: user_recipes table (title, servings, cook_time, steps,
ingredients JSON, source, pantry_match_pct)
- Tier gate: recipe_scan -> paid, BYOK-unlockable
Frontend:
- "Scan" button in the Recipes tab bar (camera icon)
- RecipeScanModal: upload step (drag-drop + file picker, up to 4 photos,
live previews), processing step (spinner), review/edit step (all
fields inline-editable before save), pantry match badge, warning banner
for low-confidence or incomplete scans
Tests: 35 new tests (23 unit + 12 API), 404 total passing
- RecipeBrowserPanel: fix onTagSearchInput using '_all' domain slug
(backend validates domain — was silently returning empty results)
- RecipeDetailPanel: fetch and display accepted community category tags
on recipe open; accepted tags shown with accent chip + checkmark,
pending tags shown in muted style
- browserAPI.listRecipeTags() was already in api.ts but not consumed —
now wired into RecipeDetailPanel onMounted as a background fetch
Style classifier (kiwi#27):
- app/services/recipe/style_classifier.py: LLM prompt with curated vocab,
cf-orch/LLMRouter fallback, JSON + regex tag extraction
- POST /recipes/saved/{recipe_id}/classify-style: Paid/BYOK tier gate,
fetches recipe from corpus, returns {suggested_tags:[...]}
- SaveRecipeModal.vue: "Suggest tags" button with loading state; merges
LLM suggestions into existing tags without overwriting user's choices
- 403/empty list silently ignored — button is a no-op when tier not met
Cooked leftovers shelf-life (kiwi#112):
- app/services/leftovers_predictor.py: deterministic FDA/USDA lookup table
with shortest-component-wins for proteins and dish-type override for
assembled dishes; special entries for ceviche (2d, acid != heat),
fermented/cured (kimchi 14d, confit/lardo 7d), soups, rice, pasta, etc.
- POST /recipes/{recipe_id}/leftovers: free tier, no gate
- RecipeDetailPanel.vue: shelf-life section appears after "I cooked this"
with fridge/freeze days, freeze-by advice, per-instance dismiss; calm
framing per no-panic UX policy
- LeftoversResponse Pydantic schema added to recipe.py
- Add TimeEffortProfile + StepAnalysis Pydantic schemas; serialised into
RecipeSuggestion so the frontend receives active/passive/total minutes,
effort label, and detected equipment per suggestion.
- parse_time_effort() now drives max_total_min filter (falls back to step-count
estimate when directions contain no explicit time mentions).
- _PRODUCT_TOKEN_STOPWORDS: strips marketing/packaging words from multi-word
product labels before adding individual ingredient tokens to pantry_set.
"Organic Extra Firm Tofu" → adds "tofu"; improves packaged-food pantry match.
- L1 candidate pool raised to 60 (was 20); min_match_ratio lowered to 0.35
(was 0.60) to keep enough results for plant-based / packaged-food pantries.
- household.py: tighten import to pull HEIMDALL_URL/ADMIN_TOKEN from
services.heimdall_orch (matches refactor in cloud_session.py).
- Migration 039: drop saved_recipes.recipe_id FK (SQLite table rebuild).
The FK referenced main.recipes but corpus lives in an ATTACH'd DB — caused
500 on every POST /recipes/saved in cloud mode.
- _to_summary: row.get("title") or "" to handle corpus JOIN returning NULL
title (e.g. placeholder recipe_id 99999).
- list_collections: return [] for Free tier instead of 403 — prevents
Promise.all in savedStore.load() from aborting the saved-recipes fetch.
- savedStore.load(): switched to Promise.allSettled so a collections failure
never blocks the saved list from populating.
- RecipesView: star indicator now reflects savedStore.isSaved() (server-side
saved state) rather than localStorage bookmarks; changed to <span> since
the star is now read-only visual feedback.
- Removed { immediate: true } from saved-tab watcher — premature bounce to
Build Your Own before onMounted load() completes.
Delegates JWT validation, Heimdall provision/tier-resolve, bypass-IP
handling, and guest session management to circuitforge_core. Kiwi keeps
its own CloudUser (db path, household fields, BYOK flag) and DB helpers.
detect_byok() is now imported from cf-core instead of a local copy.
household_id/is_household_owner/license_key flow through core_user.meta
(cf-core already forwards all Heimdall response extras into meta).
Removes ~217 lines of duplicated auth code.
Note: guest cookie name changes from kiwi_guest_id to cf_guest_id (cf-core
managed). Existing guest sessions get a new UUID on first visit — acceptable
for alpha.
Migrations 037 and 038 rebuild products and inventory_items tables
to add 'visual_capture' as a valid source value, which the label-confirm
endpoint sets when saving user-verified nutrition label data.
Adds 2 schema tests covering the new allowed value.
- Bump version to 0.6.0 (visual label capture release)
- Remove unused TimeEffortProfile import in RecipeDetailPanel.vue
- Prefix unused value params with _ in SettingsView.vue sensory fns
- Remove cf-orch agent sidecar from compose.override.yml (Sif now has
its own dedicated systemd cf-orch-agent service)
When a barcode scan finds no product in FDC/OFF, paid-tier users now see a
"Capture label" offer instead of a dead-end "add manually" prompt.
Backend:
- Migration 036: captured_products local cache table (keyed by barcode,
UPSERT on conflict so re-capture refreshes rather than errors)
- store.get_captured_product / save_captured_product (with JSON decode for
ingredient_names and allergens)
- app/services/label_capture.py: wraps cf-core VisionRouter (caption API);
graceful fallback to zero-confidence mock when stub/error; JSON fence
stripping; confidence clamped to [0,1]; KIWI_LABEL_CAPTURE_MOCK=1 for tests
- New schemas: LabelCaptureResponse, LabelConfirmRequest, LabelConfirmResponse
- POST /inventory/scan/label-capture — image to extraction (paid+ gate, 403)
- POST /inventory/scan/label-confirm — save confirmed product + optional
inventory add
- Both scan endpoints now: check captured_products cache before FDC/OFF;
set needs_visual_capture=True for gap products on paid tier; BarcodeScanResult
gains needs_visual_capture field
- visual_label_capture feature gate added to tiers.py (paid)
Tests: 42 new tests (service, store/migration, API endpoints) — 367 total passing
Frontend:
- InventoryList.vue: capturePhase state machine (offer => uploading => reviewing)
- Offer card appears after scan gap (calm UX: no urgency, Discard always visible)
- Review form: pre-populated from extraction; amber label highlights for
unread fields (confidence < 0.7); comma-separated ingredients/allergens
- api.ts: LabelCaptureResult + LabelConfirmRequest types; captureLabelPhoto()
and confirmLabelCapture() API methods
_user_constraints() loads dietary_constraints from user_settings once per
request. All 7 _enrich_item call sites now pass constraints so wine (and
any future alcohol-containing entries) are suppressed for halal/alcohol-free
users at the API response layer.
Adds 10 new secondary use entries and corrects all 8 existing ones.
New: apples/soft, leafy_greens/wilting, tomatoes/soft, cooked_pasta/day-old,
cooked_potatoes/day-old, yogurt/tangy, cream/sour, wine/open,
cooked_beans/day-old, cooked_meat/leftover.
Corrections: milk uses (specific recipes, not 'baking'/'sauces'); dairy uses
expanded; cheese label well-aged→rind-ready with named dishes (minestrone,
ribollita); rice uses (onigiri, arancini, congee); tortillas warning added;
bakery uses and synonyms expanded to named pastries; bananas synonyms
(spotty/brown/black/mushy); rice synonyms (old rice).
New fields on every SECONDARY_WINDOW entry:
- discard_signs: qualitative cues for when the item has gone past its
secondary window (shown in UI alongside uses)
- constraints_exclude: dietary labels that suppress the entry entirely
(wine suppressed for halal/alcohol-free)
ExpirationPredictor.filter_secondary_by_constraints() applies constraint
suppression; _enrich_item() now accepts user_constraints and passes
secondary_discard_signs through to the API response.
Integrates cf-core reranker into the L1/L2 recipe engine. Paid+ tier
gets a BGE cross-encoder pass over the top-20 FTS candidates, scoring
each recipe against the user's full context: pantry state, dietary
constraints, allergies, expiry urgency, style preference, and effort
preference. Free tier keeps the existing overlap sort unchanged.
- New app/services/recipe/reranker.py: build_query, build_candidate_string,
rerank_suggestions with tier gate (_RERANKER_TIERS) and graceful fallback
- rerank_score field added to RecipeSuggestion (None on free tier, float on paid+)
- recipe_engine.py: single call after candidate assembly, before final sort;
hard_day_mode tier grouping preserved as primary sort when reranker active
- Fix pre-existing circular import in app/services/__init__.py (eager import
of ReceiptService triggered store.py → services → receipt_service → store)
- 27 unit tests (mock backend, no model weights) + 2 engine-level tier tests;
325 tests passing, no regressions
