Compare commits
30 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 5a363f3b6c | |||
| a7d916f630 | |||
| c2ac55259d | |||
| 9f7fb45071 | |||
| 93d36346c1 | |||
| af66877b51 | |||
| 41c9830281 | |||
| fb3a4c697d | |||
| ccc6a15d94 | |||
| 0ddb3cbf07 | |||
| 7526092481 | |||
| 8e2d15bcd4 | |||
| a6d906bcbb | |||
| 0489f1111c | |||
| e6c69f25ae | |||
| 9492942623 | |||
| fe51914902 | |||
| ac45067ae7 | |||
| 408ab64c55 | |||
| bbb146b361 | |||
| 3be21ce452 | |||
| 73f694ed3a | |||
| 0f5ea86ab0 | |||
| cb3d186a58 | |||
| a62bff5f1e | |||
| 524cc62812 | |||
| a31e6099c6 | |||
| 5a4917d455 | |||
| 460530bb03 | |||
| b2b58913c7 |
37 changed files with 2792 additions and 94 deletions
1
.gitignore
vendored
1
.gitignore
vendored
|
|
@ -11,3 +11,4 @@ build/
|
||||||
|
|
||||||
# cf-orch private profiles (commit on personal/heimdall branch only)
|
# cf-orch private profiles (commit on personal/heimdall branch only)
|
||||||
circuitforge_core/resources/profiles/private/
|
circuitforge_core/resources/profiles/private/
|
||||||
|
.worktrees/
|
||||||
|
|
|
||||||
12
CHANGELOG.md
12
CHANGELOG.md
|
|
@ -6,6 +6,18 @@ Versions follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## [0.20.0] — 2026-05-05
|
||||||
|
|
||||||
|
### Fixed / Enhanced
|
||||||
|
|
||||||
|
**`circuitforge_core.llm.LLMRouter`** — Pagepiper-driven improvements (closes #59, #60)
|
||||||
|
|
||||||
|
- **#59 — dict init** (`LLMRouter(config_path: Path | dict)`): `__init__` now accepts an inline config dict in addition to a `Path`. Ingest scripts that construct Ollama URLs from product-specific env vars (e.g. `PAGEPIPER_OLLAMA_URL`) can pass the dict directly without writing a temp file. Passing a dict previously raised `AttributeError: 'dict' object has no attribute 'exists'`. Tests: `test_init_accepts_inline_dict`, `test_init_dict_is_used_directly`.
|
||||||
|
|
||||||
|
- **#60 — Ollama preflight** (`_check_ollama_model_pulled()`): Before the first `embed()` call on an Ollama backend, `GET /api/tags` is checked to verify the configured embedding model is pulled. If it is not, a `RuntimeError` with an actionable `ollama pull <model>` hint is raised immediately — replacing the opaque `All LLM backends exhausted for embed()` error. Results are cached per base URL for the router's lifetime (one HTTP call, not one per `embed()` invocation). Non-Ollama backends (vLLM, etc.) don't expose `/api/tags` — a non-200 response causes the check to be silently skipped. Tests: `test_embed_raises_actionable_error_when_model_not_pulled`, `test_embed_proceeds_when_model_is_pulled`, `test_embed_skips_preflight_when_tags_endpoint_unavailable`, `test_ollama_tags_cache_is_hit_only_once`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## [0.17.0] — 2026-04-27
|
## [0.17.0] — 2026-04-27
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|
|
||||||
193
README.md
193
README.md
|
|
@ -1,37 +1,186 @@
|
||||||
# circuitforge-core
|
<p align="center">
|
||||||
|
<img src="docs/cf-logo.png" alt="CircuitForge logo" width="120" />
|
||||||
|
</p>
|
||||||
|
|
||||||
Shared scaffold for CircuitForge products.
|
<h1 align="center">circuitforge-core</h1>
|
||||||
|
|
||||||
**Current version: 0.7.0**
|
<p align="center">Shared Python scaffold for privacy-first, self-hosted AI tools</p>
|
||||||
|
|
||||||
## Modules
|
<p align="center">
|
||||||
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green.svg" alt="MIT License" /></a>
|
||||||
|
<img src="https://img.shields.io/badge/version-0.20.0-blue.svg" alt="v0.20.0" />
|
||||||
|
<img src="https://img.shields.io/badge/python-3.11%2B-blue.svg" alt="Python 3.11+" />
|
||||||
|
<a href="https://git.opensourcesolarpunk.com/Circuit-Forge/circuitforge-core"><img src="https://img.shields.io/badge/repo-Forgejo-orange.svg" alt="Forgejo" /></a>
|
||||||
|
</p>
|
||||||
|
|
||||||
### Implemented
|
---
|
||||||
|
|
||||||
- `circuitforge_core.db` — SQLite connection factory and migration runner
|
## Why circuitforge-core?
|
||||||
- `circuitforge_core.llm` — LLM router with fallback chain (Ollama, vLLM, Anthropic, OpenAI-compatible)
|
|
||||||
- `circuitforge_core.tiers` — Tier system with BYOK and local vision unlocks
|
|
||||||
- `circuitforge_core.config` — Env validation and .env loader
|
|
||||||
- `circuitforge_core.hardware` — Hardware detection and LLM backend profile generation (VRAM tiers, GPU/CPU auto-select)
|
|
||||||
- `circuitforge_core.documents` — Document ingestion pipeline: PDF, DOCX, and image OCR → `StructuredDocument`
|
|
||||||
- `circuitforge_core.affiliates` — Affiliate URL wrapping with opt-out, BYOK user IDs, and CF env-var fallback (`wrap_url`)
|
|
||||||
- `circuitforge_core.preferences` — User preference store (local YAML file, pluggable backend); dot-path get/set API
|
|
||||||
- `circuitforge_core.tasks` — VRAM-aware LLM task scheduler; shared slot manager across services (`TaskScheduler`)
|
|
||||||
- `circuitforge_core.manage` — Cross-platform product process manager (Docker and native modes)
|
|
||||||
- `circuitforge_core.resources` — Resource coordinator and agent: VRAM allocation, eviction engine, GPU profile registry
|
|
||||||
|
|
||||||
### Stubs (in-tree, not yet implemented)
|
- **Local inference first.** The LLM router defaults to Ollama on localhost. Cloud APIs are a configurable fallback, not the default path. No telemetry, no round-trips you didn't ask for.
|
||||||
|
- **VRAM-aware scheduling.** The task scheduler and resource coordinator track GPU memory across concurrent services, allocate slots before loading models, and evict backends gracefully when VRAM is scarce.
|
||||||
|
- **Consistent tier system across products.** One `tiers` module handles Free / Paid / Premium / Ultra tiers, BYOK (bring your own key) unlocks, and local-vision capability gates — the same way in every product.
|
||||||
|
- **Uniform developer experience.** DB migrations, config validation, document ingestion, process management, and preference storage all share a single, tested implementation. Products extend, not reimplement.
|
||||||
|
|
||||||
- `circuitforge_core.vision` — Vision router base class (planned: moondream2 / Claude vision dispatch)
|
---
|
||||||
- `circuitforge_core.wizard` — First-run wizard base class (products subclass `BaseWizard`)
|
|
||||||
- `circuitforge_core.pipeline` — Staging queue base (`StagingDB`; products provide concrete schema)
|
|
||||||
|
|
||||||
## Install
|
## Install
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install -e .
|
# From PyPI
|
||||||
|
pip install circuitforge-core
|
||||||
|
|
||||||
|
# Editable install from source (recommended for product development)
|
||||||
|
pip install -e /path/to/circuitforge-core
|
||||||
|
|
||||||
|
# With optional extras
|
||||||
|
pip install circuitforge-core[pdf] # PDF/DOCX/OCR document ingestion
|
||||||
|
pip install circuitforge-core[vector] # SQLite-vec vector store
|
||||||
|
pip install circuitforge-core[text-transformers] # Local transformer inference (cf-text)
|
||||||
|
pip install circuitforge-core[stt-faster-whisper] # Speech-to-text via Faster Whisper
|
||||||
|
pip install circuitforge-core[tts-chatterbox] # Text-to-speech via Chatterbox
|
||||||
|
pip install circuitforge-core[reranker-qwen3] # Reranking via Qwen3
|
||||||
|
pip install circuitforge-core[community] # PostgreSQL-backed community store
|
||||||
|
pip install circuitforge-core[manage] # cf-manage CLI (Typer)
|
||||||
|
pip install circuitforge-core[dev] # All dev dependencies
|
||||||
```
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Modules
|
||||||
|
|
||||||
|
| Module | Status | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| `db` | Implemented | SQLite connection factory and migration runner |
|
||||||
|
| `llm` | Implemented | LLM router with priority fallback chain (Ollama, vLLM, Anthropic, OpenAI-compatible) |
|
||||||
|
| `tiers` | Implemented | Tier system with BYOK and local-vision unlocks (Free / Paid / Premium / Ultra) |
|
||||||
|
| `config` | Implemented | Env validation and `.env` loader with startup fail-fast |
|
||||||
|
| `hardware` | Implemented | GPU/CPU detection, VRAM profiling, backend profile generation |
|
||||||
|
| `documents` | Implemented | PDF, DOCX, and image OCR ingestion into `StructuredDocument` |
|
||||||
|
| `affiliates` | Implemented | Affiliate URL wrapping with per-user opt-out and env-var fallback |
|
||||||
|
| `preferences` | Implemented | User preference store — local YAML with pluggable backend; dot-path get/set |
|
||||||
|
| `tasks` | Implemented | VRAM-aware LLM task scheduler; shared slot manager across services |
|
||||||
|
| `manage` | Implemented | Cross-platform product process manager (Docker and native modes) |
|
||||||
|
| `resources` | Implemented | VRAM allocation, eviction engine, GPU profile registry |
|
||||||
|
| `text` | Implemented | Text processing utilities and local transformer inference service |
|
||||||
|
| `activitypub` | Implemented | ActivityPub actor, inbox, delivery, and Lemmy federation primitives |
|
||||||
|
| `audio` | Implemented | Audio buffer, format conversion, resampling, and VAD (voice activity detection) gate |
|
||||||
|
| `stt` | Implemented | Speech-to-text service (Faster Whisper backend) |
|
||||||
|
| `tts` | Implemented | Text-to-speech service (Chatterbox backend) |
|
||||||
|
| `musicgen` | Implemented | Music generation service (AudioCraft/MusicGen backend) |
|
||||||
|
| `reranker` | Implemented | Result reranking — BGE, Qwen3, cross-encoder, and Cohere adapters |
|
||||||
|
| `vector` | Implemented | SQLite-vec vector store with pluggable embedding backend |
|
||||||
|
| `api` | Implemented | Shared API helpers — corrections and feedback endpoints |
|
||||||
|
| `community` | Implemented | Community feed and social store (PostgreSQL-backed) |
|
||||||
|
| `platforms` | Implemented | Platform-specific integrations (eBay) |
|
||||||
|
| `cloud_session` | Implemented | Cloud session management primitives |
|
||||||
|
| `input` | Implemented | Input handling — MediaPipe gesture recognition |
|
||||||
|
| `job_quality` | Implemented | Job listing quality scoring and signal extraction |
|
||||||
|
| `vision` | Stub | Vision router (moondream2 / SigLIP dispatch — planned) |
|
||||||
|
| `wizard` | Stub | First-run wizard base class — products subclass `BaseWizard` |
|
||||||
|
| `pipeline` | Stub | Staging queue base — products provide concrete schema |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Usage: LLM Router
|
||||||
|
|
||||||
|
The LLM router reads a config file at `~/.config/circuitforge/llm.yaml`, tries each backend in fallback order, and skips unreachable or disabled entries transparently.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from circuitforge_core.llm import LLMRouter
|
||||||
|
|
||||||
|
# Auto-detects from env vars when llm.yaml is absent:
|
||||||
|
# ANTHROPIC_API_KEY, OPENAI_API_KEY / OPENAI_BASE_URL, OLLAMA_HOST
|
||||||
|
router = LLMRouter()
|
||||||
|
|
||||||
|
response = router.complete(
|
||||||
|
messages=[{"role": "user", "content": "Summarize this in one sentence."}],
|
||||||
|
system="You are a concise assistant.",
|
||||||
|
)
|
||||||
|
print(response)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example `llm.yaml`** (Ollama local, Anthropic cloud fallback):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
fallback_order:
|
||||||
|
- ollama
|
||||||
|
- anthropic
|
||||||
|
|
||||||
|
backends:
|
||||||
|
ollama:
|
||||||
|
type: openai_compat
|
||||||
|
enabled: true
|
||||||
|
base_url: http://localhost:11434/v1
|
||||||
|
model: llama3.2:3b
|
||||||
|
|
||||||
|
anthropic:
|
||||||
|
type: anthropic
|
||||||
|
enabled: true
|
||||||
|
model: claude-haiku-4-5-20251001
|
||||||
|
api_key_env: ANTHROPIC_API_KEY
|
||||||
|
supports_images: true
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Usage: Database + Migrations
|
||||||
|
|
||||||
|
```python
|
||||||
|
from circuitforge_core.db import get_connection, run_migrations
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Run product migrations on startup
|
||||||
|
run_migrations(db_path=Path("data/app.db"), migrations_dir=Path("db/migrations"))
|
||||||
|
|
||||||
|
# Get a connection anywhere in your app
|
||||||
|
with get_connection(Path("data/app.db")) as conn:
|
||||||
|
conn.execute("INSERT INTO items (name) VALUES (?)", ("example",))
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Used by
|
||||||
|
|
||||||
|
| Product | Description |
|
||||||
|
|---|---|
|
||||||
|
| [peregrine](https://git.opensourcesolarpunk.com/Circuit-Forge/peregrine) | Job search — discovery, cover letters, interview prep |
|
||||||
|
| [snipe](https://git.opensourcesolarpunk.com/Circuit-Forge/snipe) | Auction sniping — eBay trust scoring, bid timing |
|
||||||
|
| [kiwi](https://git.opensourcesolarpunk.com/Circuit-Forge/kiwi) | Pantry tracker with barcode/receipt OCR and recipe suggestions |
|
||||||
|
| [avocet](https://git.opensourcesolarpunk.com/Circuit-Forge/avocet) | Email classifier training and benchmark harness |
|
||||||
|
| [osprey](https://git.opensourcesolarpunk.com/Circuit-Forge/osprey) | Government hold-line automation |
|
||||||
|
| [linnet](https://git.opensourcesolarpunk.com/Circuit-Forge/linnet) | Real-time tone annotation and voice transcription |
|
||||||
|
| pagepiper | PDF/rulebook RAG (retrieval-augmented generation) search |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
circuitforge-core is MIT licensed. Contributions are welcome.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://git.opensourcesolarpunk.com/Circuit-Forge/circuitforge-core
|
||||||
|
cd circuitforge-core
|
||||||
|
pip install -e ".[dev]"
|
||||||
|
pytest
|
||||||
|
```
|
||||||
|
|
||||||
|
- New modules belong in `circuitforge_core/<module>/` as a package, not a flat file
|
||||||
|
- Keep modules focused — extract when a module exceeds 400 lines
|
||||||
|
- All public functions need type annotations
|
||||||
|
- Tests live in `tests/` — aim for 80% coverage on new code
|
||||||
|
- Use `ruff` for linting before submitting a PR
|
||||||
|
|
||||||
|
Open issues and PRs at: [git.opensourcesolarpunk.com/Circuit-Forge/circuitforge-core](https://git.opensourcesolarpunk.com/Circuit-Forge/circuitforge-core)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
BSL 1.1 — see LICENSE
|
MIT — see [LICENSE](LICENSE).
|
||||||
|
|
||||||
|
This is the fully open layer of the CircuitForge stack. Products built on top of circuitforge-core may carry different licenses (BSL 1.1 for AI features, proprietary for fine-tuned weights). The scaffold itself is and will remain MIT.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
Humans own design, architecture, code review, testing, and verification. LLMs are part of our development workflow. [Our positions on LLM use →](https://circuitforge.tech/positions)
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
__version__ = "0.16.1"
|
__version__ = "0.18.0"
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from circuitforge_core.community import CommunityDB, CommunityPost, SharedStore
|
from circuitforge_core.community import CommunityDB, CommunityPost, SharedStore
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,22 @@
|
||||||
|
-- 006_community_dedup.sql
|
||||||
|
-- Adds variation-linking and title search support for community recipe dedup.
|
||||||
|
-- Applies to: cf_community PostgreSQL database.
|
||||||
|
-- BSL boundary: MIT (data layer, no inference).
|
||||||
|
|
||||||
|
-- Nullable self-referential FK: user-declared "this is a variation of X"
|
||||||
|
ALTER TABLE community_posts
|
||||||
|
ADD COLUMN IF NOT EXISTS similar_to_ref TEXT REFERENCES community_posts(slug) ON DELETE SET NULL;
|
||||||
|
|
||||||
|
-- Index for variation lookup (find all variations of a parent post)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_community_posts_similar_ref
|
||||||
|
ON community_posts (similar_to_ref)
|
||||||
|
WHERE similar_to_ref IS NOT NULL;
|
||||||
|
|
||||||
|
-- Index to speed up title ILIKE prefix and substring searches
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_community_posts_title_lower
|
||||||
|
ON community_posts (lower(title));
|
||||||
|
|
||||||
|
-- Index on recipe_id for exact-recipe duplicate detection
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_community_posts_recipe_id
|
||||||
|
ON community_posts (recipe_id)
|
||||||
|
WHERE recipe_id IS NOT NULL;
|
||||||
|
|
@ -66,6 +66,9 @@ class CommunityPost:
|
||||||
protein_pct: float | None
|
protein_pct: float | None
|
||||||
moisture_pct: float | None
|
moisture_pct: float | None
|
||||||
|
|
||||||
|
# Variation link: slug of the parent post this is explicitly a variation of
|
||||||
|
similar_to_ref: str | None = None
|
||||||
|
|
||||||
def __post_init__(self) -> None:
|
def __post_init__(self) -> None:
|
||||||
# Coerce list fields to tuples (frozen dataclass: use object.__setattr__)
|
# Coerce list fields to tuples (frozen dataclass: use object.__setattr__)
|
||||||
for key in ("slots", "dietary_tags", "allergen_flags", "flavor_molecules"):
|
for key in ("slots", "dietary_tags", "allergen_flags", "flavor_molecules"):
|
||||||
|
|
|
||||||
|
|
@ -46,6 +46,7 @@ def _row_to_post(row: dict) -> CommunityPost:
|
||||||
fat_pct=row.get("fat_pct"),
|
fat_pct=row.get("fat_pct"),
|
||||||
protein_pct=row.get("protein_pct"),
|
protein_pct=row.get("protein_pct"),
|
||||||
moisture_pct=row.get("moisture_pct"),
|
moisture_pct=row.get("moisture_pct"),
|
||||||
|
similar_to_ref=row.get("similar_to_ref"),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -137,6 +138,61 @@ class SharedStore:
|
||||||
finally:
|
finally:
|
||||||
self._db.putconn(conn)
|
self._db.putconn(conn)
|
||||||
|
|
||||||
|
def search_similar_posts(
|
||||||
|
self,
|
||||||
|
title: str,
|
||||||
|
recipe_id: int | None = None,
|
||||||
|
post_type: str | None = None,
|
||||||
|
limit: int = 8,
|
||||||
|
) -> list[CommunityPost]:
|
||||||
|
"""Return posts similar to the given title or with the same recipe_id.
|
||||||
|
|
||||||
|
Used by the dedup check before a new post is submitted. Matches on:
|
||||||
|
- exact recipe_id (strongest signal)
|
||||||
|
- case-insensitive title substring match
|
||||||
|
|
||||||
|
Results are ordered: recipe_id matches first, then by published desc.
|
||||||
|
"""
|
||||||
|
conn = self._db.getconn()
|
||||||
|
try:
|
||||||
|
conditions: list[str] = []
|
||||||
|
params: list = []
|
||||||
|
|
||||||
|
title_condition = "lower(title) LIKE lower(%s)"
|
||||||
|
title_param = f"%{title.lower()[:80]}%"
|
||||||
|
|
||||||
|
if recipe_id is not None:
|
||||||
|
conditions.append(f"(recipe_id = %s OR {title_condition})")
|
||||||
|
params.extend([recipe_id, title_param])
|
||||||
|
else:
|
||||||
|
conditions.append(title_condition)
|
||||||
|
params.append(title_param)
|
||||||
|
|
||||||
|
if post_type:
|
||||||
|
conditions.append("post_type = %s")
|
||||||
|
params.append(post_type)
|
||||||
|
|
||||||
|
where = "WHERE " + " AND ".join(conditions)
|
||||||
|
params.append(limit)
|
||||||
|
|
||||||
|
order_clause = (
|
||||||
|
"ORDER BY (recipe_id = %s) DESC, published DESC"
|
||||||
|
if recipe_id is not None
|
||||||
|
else "ORDER BY published DESC"
|
||||||
|
)
|
||||||
|
if recipe_id is not None:
|
||||||
|
params.insert(-1, recipe_id)
|
||||||
|
|
||||||
|
with conn.cursor() as cur:
|
||||||
|
cur.execute(
|
||||||
|
f"SELECT * FROM community_posts {where} {order_clause} LIMIT %s",
|
||||||
|
params,
|
||||||
|
)
|
||||||
|
rows = cur.fetchall()
|
||||||
|
return [_row_to_post(_cursor_to_dict(cur, r)) for r in rows]
|
||||||
|
finally:
|
||||||
|
self._db.putconn(conn)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Writes
|
# Writes
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
@ -156,13 +212,14 @@ class SharedStore:
|
||||||
seasoning_score, richness_score, brightness_score,
|
seasoning_score, richness_score, brightness_score,
|
||||||
depth_score, aroma_score, structure_score, texture_profile,
|
depth_score, aroma_score, structure_score, texture_profile,
|
||||||
dietary_tags, allergen_flags, flavor_molecules,
|
dietary_tags, allergen_flags, flavor_molecules,
|
||||||
fat_pct, protein_pct, moisture_pct, source_product
|
fat_pct, protein_pct, moisture_pct, source_product,
|
||||||
|
similar_to_ref
|
||||||
) VALUES (
|
) VALUES (
|
||||||
%s, %s, %s, %s, %s, %s, %s,
|
%s, %s, %s, %s, %s, %s, %s,
|
||||||
%s::jsonb, %s, %s, %s, %s,
|
%s::jsonb, %s, %s, %s, %s,
|
||||||
%s, %s, %s, %s, %s, %s, %s,
|
%s, %s, %s, %s, %s, %s, %s,
|
||||||
%s::jsonb, %s::jsonb, %s::jsonb,
|
%s::jsonb, %s::jsonb, %s::jsonb,
|
||||||
%s, %s, %s, %s
|
%s, %s, %s, %s, %s
|
||||||
)
|
)
|
||||||
""",
|
""",
|
||||||
(
|
(
|
||||||
|
|
@ -178,6 +235,7 @@ class SharedStore:
|
||||||
json.dumps(list(post.flavor_molecules)),
|
json.dumps(list(post.flavor_molecules)),
|
||||||
post.fat_pct, post.protein_pct, post.moisture_pct,
|
post.fat_pct, post.protein_pct, post.moisture_pct,
|
||||||
self._source_product,
|
self._source_product,
|
||||||
|
post.similar_to_ref,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
conn.commit()
|
conn.commit()
|
||||||
|
|
|
||||||
133
circuitforge_core/documents/pdf.py
Normal file
133
circuitforge_core/documents/pdf.py
Normal file
|
|
@ -0,0 +1,133 @@
|
||||||
|
# circuitforge_core/documents/pdf.py
|
||||||
|
"""
|
||||||
|
circuitforge_core.documents.pdf — PDF text extraction and page-level chunking.
|
||||||
|
|
||||||
|
Primary path: pdfplumber (selectable text layers).
|
||||||
|
Fallback: pytesseract OCR (scanned / image-only pages).
|
||||||
|
|
||||||
|
Usage::
|
||||||
|
|
||||||
|
from circuitforge_core.documents.pdf import PDFExtractor
|
||||||
|
|
||||||
|
chunks = PDFExtractor().chunk_pages("/path/to/book.pdf")
|
||||||
|
for chunk in chunks:
|
||||||
|
print(f"[p.{chunk.page_number}] ({chunk.source}) {chunk.text[:80]}")
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import logging
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
try:
|
||||||
|
import pdfplumber
|
||||||
|
except ImportError: # pragma: no cover
|
||||||
|
pdfplumber = None # type: ignore[assignment]
|
||||||
|
|
||||||
|
try:
|
||||||
|
import pytesseract
|
||||||
|
except ImportError: # pragma: no cover
|
||||||
|
pytesseract = None # type: ignore[assignment]
|
||||||
|
|
||||||
|
try:
|
||||||
|
from PIL import Image
|
||||||
|
except ImportError: # pragma: no cover
|
||||||
|
Image = None # type: ignore[assignment]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PageChunk:
|
||||||
|
"""Text content extracted from a single PDF page."""
|
||||||
|
|
||||||
|
page_number: int # 1-indexed
|
||||||
|
text: str
|
||||||
|
source: str # "text_layer" | "ocr"
|
||||||
|
word_count: int
|
||||||
|
|
||||||
|
|
||||||
|
class PDFExtractor:
|
||||||
|
"""
|
||||||
|
Extract page-level text chunks from PDF files.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
ocr_min_words: Pages with fewer words from the text layer trigger OCR.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, ocr_min_words: int = 10) -> None:
|
||||||
|
self.ocr_min_words = ocr_min_words
|
||||||
|
|
||||||
|
def chunk_pages(self, pdf_path: str | Path) -> list[PageChunk]:
|
||||||
|
"""
|
||||||
|
Primary entry point. Returns one PageChunk per page.
|
||||||
|
|
||||||
|
Uses text-layer extraction per page; falls back to OCR when text is sparse.
|
||||||
|
Empty PDFs return an empty list.
|
||||||
|
"""
|
||||||
|
if pdfplumber is None:
|
||||||
|
raise ImportError(
|
||||||
|
"pdfplumber is required for PDF extraction. "
|
||||||
|
"Install it with: pip install pdfplumber"
|
||||||
|
)
|
||||||
|
|
||||||
|
path = Path(pdf_path)
|
||||||
|
chunks: list[PageChunk] = []
|
||||||
|
|
||||||
|
with pdfplumber.open(path) as pdf:
|
||||||
|
for i, page in enumerate(pdf.pages, start=1):
|
||||||
|
text = page.extract_text() or ""
|
||||||
|
words = text.split()
|
||||||
|
|
||||||
|
if len(words) >= self.ocr_min_words:
|
||||||
|
chunks.append(
|
||||||
|
PageChunk(
|
||||||
|
page_number=i,
|
||||||
|
text=text.strip(),
|
||||||
|
source="text_layer",
|
||||||
|
word_count=len(words),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
logger.debug(
|
||||||
|
"pdf: page %d sparse (%d words), falling back to OCR",
|
||||||
|
i,
|
||||||
|
len(words),
|
||||||
|
)
|
||||||
|
chunks.append(self._ocr_page(page, i))
|
||||||
|
|
||||||
|
return chunks
|
||||||
|
|
||||||
|
def _ocr_page(self, page: object, page_number: int) -> PageChunk:
|
||||||
|
"""Render page to image and extract text via tesseract."""
|
||||||
|
try:
|
||||||
|
rendered = page.to_image(resolution=200).original # type: ignore[attr-defined]
|
||||||
|
rendered = _ensure_pil_image(rendered)
|
||||||
|
text = pytesseract.image_to_string(rendered) # type: ignore[union-attr]
|
||||||
|
words = text.split()
|
||||||
|
return PageChunk(
|
||||||
|
page_number=page_number,
|
||||||
|
text=text.strip(),
|
||||||
|
source="ocr",
|
||||||
|
word_count=len(words),
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("pdf: OCR failed for page %d: %s", page_number, exc)
|
||||||
|
return PageChunk(
|
||||||
|
page_number=page_number, text="", source="ocr", word_count=0
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure_pil_image(rendered: object) -> object:
|
||||||
|
"""Return *rendered* as a PIL Image, converting from bytes if needed."""
|
||||||
|
if Image is None:
|
||||||
|
return rendered
|
||||||
|
try:
|
||||||
|
if not isinstance(rendered, Image.Image):
|
||||||
|
rendered = Image.open(io.BytesIO(rendered)) # type: ignore[arg-type]
|
||||||
|
except TypeError:
|
||||||
|
# Image may be patched (e.g. in tests); skip the conversion.
|
||||||
|
pass
|
||||||
|
return rendered
|
||||||
0
circuitforge_core/input/__init__.py
Normal file
0
circuitforge_core/input/__init__.py
Normal file
15
circuitforge_core/input/gestures/__init__.py
Normal file
15
circuitforge_core/input/gestures/__init__.py
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
"""
|
||||||
|
cf_input.gestures — camera capture, hand detection, landmark normalization.
|
||||||
|
|
||||||
|
Public API:
|
||||||
|
CameraCapture — OpenCV frame source
|
||||||
|
HandsDetector — MediaPipe Hands wrapper
|
||||||
|
HandLandmarks — immutable detected hand dataclass
|
||||||
|
normalize_hand() — scale/translation-invariant feature vector
|
||||||
|
"""
|
||||||
|
|
||||||
|
from circuitforge_core.input.gestures.camera import CameraCapture
|
||||||
|
from circuitforge_core.input.gestures.hands import HandLandmarks, HandsDetector
|
||||||
|
from circuitforge_core.input.gestures.normalizer import normalize_hand
|
||||||
|
|
||||||
|
__all__ = ["CameraCapture", "HandLandmarks", "HandsDetector", "normalize_hand"]
|
||||||
57
circuitforge_core/input/gestures/camera.py
Normal file
57
circuitforge_core/input/gestures/camera.py
Normal file
|
|
@ -0,0 +1,57 @@
|
||||||
|
"""
|
||||||
|
OpenCV camera capture — context manager wrapping VideoCapture.
|
||||||
|
|
||||||
|
Yields BGR frames. Callers convert to RGB before passing to HandsDetector:
|
||||||
|
frame_rgb = frame_bgr[:, :, ::-1]
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Iterator
|
||||||
|
|
||||||
|
import cv2
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
|
||||||
|
class CameraCapture:
|
||||||
|
"""
|
||||||
|
Thin wrapper around cv2.VideoCapture.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
with CameraCapture(device_index=0) as cam:
|
||||||
|
for frame_bgr in cam.frames():
|
||||||
|
process(frame_bgr)
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
device_index: int = 0,
|
||||||
|
width: int = 640,
|
||||||
|
height: int = 480,
|
||||||
|
fps: int = 30,
|
||||||
|
) -> None:
|
||||||
|
self._cap = cv2.VideoCapture(device_index)
|
||||||
|
self._cap.set(cv2.CAP_PROP_FRAME_WIDTH, width)
|
||||||
|
self._cap.set(cv2.CAP_PROP_FRAME_HEIGHT, height)
|
||||||
|
self._cap.set(cv2.CAP_PROP_FPS, fps)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_open(self) -> bool:
|
||||||
|
return self._cap.isOpened()
|
||||||
|
|
||||||
|
def frames(self) -> Iterator[np.ndarray]:
|
||||||
|
"""Yield BGR uint8 frames until camera fails or caller breaks."""
|
||||||
|
while self._cap.isOpened():
|
||||||
|
ok, frame = self._cap.read()
|
||||||
|
if not ok:
|
||||||
|
break
|
||||||
|
yield frame
|
||||||
|
|
||||||
|
def release(self) -> None:
|
||||||
|
self._cap.release()
|
||||||
|
|
||||||
|
def __enter__(self) -> CameraCapture:
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *_: object) -> None:
|
||||||
|
self.release()
|
||||||
91
circuitforge_core/input/gestures/hands.py
Normal file
91
circuitforge_core/input/gestures/hands.py
Normal file
|
|
@ -0,0 +1,91 @@
|
||||||
|
"""
|
||||||
|
MediaPipe Hands wrapper.
|
||||||
|
|
||||||
|
Produces immutable HandLandmarks dataclasses from RGB video frames.
|
||||||
|
The caller is responsible for BGR→RGB conversion before passing frames.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
import mediapipe as mp
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class HandLandmarks:
|
||||||
|
"""Immutable snapshot of one detected hand."""
|
||||||
|
|
||||||
|
points: np.ndarray # shape (21, 3) — x, y, z in [0,1] normalized image space
|
||||||
|
handedness: str # 'Left' | 'Right' (mirror of physical hand)
|
||||||
|
confidence: float # [0.0, 1.0]
|
||||||
|
|
||||||
|
|
||||||
|
class HandsDetector:
|
||||||
|
"""
|
||||||
|
Thin wrapper around mediapipe.solutions.hands.Hands.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
detector = HandsDetector()
|
||||||
|
for frame_bgr in camera.frames():
|
||||||
|
frame_rgb = frame_bgr[:, :, ::-1]
|
||||||
|
hands = detector.detect(frame_rgb)
|
||||||
|
for hand in hands:
|
||||||
|
vec = normalize_hand(hand.points)
|
||||||
|
...
|
||||||
|
detector.close()
|
||||||
|
|
||||||
|
Or use as a context manager:
|
||||||
|
with HandsDetector() as detector:
|
||||||
|
...
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
max_hands: int = 2,
|
||||||
|
min_detection_confidence: float = 0.7,
|
||||||
|
min_tracking_confidence: float = 0.5,
|
||||||
|
) -> None:
|
||||||
|
self._hands = mp.solutions.hands.Hands(
|
||||||
|
static_image_mode=False,
|
||||||
|
max_num_hands=max_hands,
|
||||||
|
min_detection_confidence=min_detection_confidence,
|
||||||
|
min_tracking_confidence=min_tracking_confidence,
|
||||||
|
)
|
||||||
|
|
||||||
|
def detect(self, rgb_frame: np.ndarray) -> list[HandLandmarks]:
|
||||||
|
"""
|
||||||
|
Run hand detection on one RGB frame.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
rgb_frame: (H, W, 3) uint8 RGB image.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of HandLandmarks, one per detected hand (up to max_hands).
|
||||||
|
Empty list if no hands detected.
|
||||||
|
"""
|
||||||
|
results = self._hands.process(rgb_frame)
|
||||||
|
if not results.multi_hand_landmarks:
|
||||||
|
return []
|
||||||
|
out: list[HandLandmarks] = []
|
||||||
|
for lm, hand in zip(results.multi_hand_landmarks, results.multi_handedness):
|
||||||
|
points = np.array([[p.x, p.y, p.z] for p in lm.landmark], dtype=np.float32)
|
||||||
|
points.flags.writeable = False # enforce immutability of stored array
|
||||||
|
out.append(
|
||||||
|
HandLandmarks(
|
||||||
|
points=points,
|
||||||
|
handedness=hand.classification[0].label,
|
||||||
|
confidence=float(hand.classification[0].score),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return out
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self._hands.close()
|
||||||
|
|
||||||
|
def __enter__(self) -> HandsDetector:
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *_: object) -> None:
|
||||||
|
self.close()
|
||||||
33
circuitforge_core/input/gestures/normalizer.py
Normal file
33
circuitforge_core/input/gestures/normalizer.py
Normal file
|
|
@ -0,0 +1,33 @@
|
||||||
|
"""
|
||||||
|
Landmark normalization for MediaPipe hand landmarks.
|
||||||
|
|
||||||
|
Converts raw (21, 3) landmark array into a 63-element translation- and
|
||||||
|
scale-invariant feature vector suitable for gesture classifiers.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_hand(points: np.ndarray) -> np.ndarray:
|
||||||
|
"""
|
||||||
|
Normalize 21 MediaPipe hand landmarks into a scale/translation-invariant
|
||||||
|
63-element feature vector.
|
||||||
|
|
||||||
|
Steps:
|
||||||
|
1. Translate so wrist (landmark 0) is at origin.
|
||||||
|
2. Scale so distance from wrist to middle-finger MCP (landmark 9) = 1.0.
|
||||||
|
If that distance is near-zero (degenerate hand), return zeros.
|
||||||
|
3. Flatten to shape (63,).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
points: (21, 3) float32 array — raw MediaPipe landmark coords.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
(63,) float32 feature vector.
|
||||||
|
"""
|
||||||
|
pts = points.astype(np.float32).copy()
|
||||||
|
pts -= pts[0] # translate: wrist → origin
|
||||||
|
scale = float(np.linalg.norm(pts[9])) # wrist-to-middle-MCP distance
|
||||||
|
if scale > 1e-6:
|
||||||
|
pts /= scale
|
||||||
|
return pts.flatten()
|
||||||
|
|
@ -43,6 +43,7 @@ When llm.yaml is absent, the router builds a minimal config from environment
|
||||||
variables: ANTHROPIC_API_KEY, OPENAI_API_KEY / OPENAI_BASE_URL, OLLAMA_HOST.
|
variables: ANTHROPIC_API_KEY, OPENAI_API_KEY / OPENAI_BASE_URL, OLLAMA_HOST.
|
||||||
Ollama on localhost:11434 is always included as the lowest-cost local fallback.
|
Ollama on localhost:11434 is always included as the lowest-cost local fallback.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import yaml
|
import yaml
|
||||||
|
|
@ -56,8 +57,11 @@ CONFIG_PATH = Path.home() / ".config" / "circuitforge" / "llm.yaml"
|
||||||
|
|
||||||
|
|
||||||
class LLMRouter:
|
class LLMRouter:
|
||||||
def __init__(self, config_path: Path = CONFIG_PATH):
|
def __init__(self, config_path: Path | dict = CONFIG_PATH):
|
||||||
if config_path.exists():
|
self._ollama_tags_cache: dict[str, set[str]] = {}
|
||||||
|
if isinstance(config_path, dict):
|
||||||
|
self.config = config_path
|
||||||
|
elif config_path.exists():
|
||||||
with open(config_path) as f:
|
with open(config_path) as f:
|
||||||
self.config = yaml.safe_load(f)
|
self.config = yaml.safe_load(f)
|
||||||
else:
|
else:
|
||||||
|
|
@ -70,7 +74,8 @@ class LLMRouter:
|
||||||
)
|
)
|
||||||
logger.info(
|
logger.info(
|
||||||
"[LLMRouter] No llm.yaml found — using env-var auto-config "
|
"[LLMRouter] No llm.yaml found — using env-var auto-config "
|
||||||
"(backends: %s)", ", ".join(env_config["fallback_order"])
|
"(backends: %s)",
|
||||||
|
", ".join(env_config["fallback_order"]),
|
||||||
)
|
)
|
||||||
self.config = env_config
|
self.config = env_config
|
||||||
|
|
||||||
|
|
@ -103,7 +108,9 @@ class LLMRouter:
|
||||||
backends["openai"] = {
|
backends["openai"] = {
|
||||||
"type": "openai_compat",
|
"type": "openai_compat",
|
||||||
"enabled": True,
|
"enabled": True,
|
||||||
"base_url": os.environ.get("OPENAI_BASE_URL", "https://api.openai.com/v1"),
|
"base_url": os.environ.get(
|
||||||
|
"OPENAI_BASE_URL", "https://api.openai.com/v1"
|
||||||
|
),
|
||||||
"model": os.environ.get("OPENAI_MODEL", "gpt-4o-mini"),
|
"model": os.environ.get("OPENAI_MODEL", "gpt-4o-mini"),
|
||||||
"api_key": os.environ.get("OPENAI_API_KEY"),
|
"api_key": os.environ.get("OPENAI_API_KEY"),
|
||||||
"supports_images": True,
|
"supports_images": True,
|
||||||
|
|
@ -141,6 +148,37 @@ class LLMRouter:
|
||||||
except Exception:
|
except Exception:
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
def _check_ollama_model_pulled(self, base_url: str, model: str) -> None:
|
||||||
|
"""Raise RuntimeError with actionable message if model is not pulled in Ollama.
|
||||||
|
|
||||||
|
Silently skips the check if the /api/tags endpoint is unavailable (e.g. vLLM).
|
||||||
|
Results are cached per base_url for the lifetime of this router instance.
|
||||||
|
"""
|
||||||
|
tags_url = base_url.rstrip("/").removesuffix("/v1") + "/api/tags"
|
||||||
|
if not hasattr(self, "_ollama_tags_cache"):
|
||||||
|
self._ollama_tags_cache = {}
|
||||||
|
if base_url not in self._ollama_tags_cache:
|
||||||
|
try:
|
||||||
|
resp = requests.get(tags_url, timeout=3)
|
||||||
|
if resp.status_code != 200:
|
||||||
|
return
|
||||||
|
pulled = {
|
||||||
|
m["name"].split(":")[0]
|
||||||
|
for m in resp.json().get("models", [])
|
||||||
|
}
|
||||||
|
self._ollama_tags_cache[base_url] = pulled
|
||||||
|
except Exception:
|
||||||
|
return # can't verify — let the actual embed call fail naturally
|
||||||
|
pulled_models = self._ollama_tags_cache.get(base_url)
|
||||||
|
if pulled_models is None:
|
||||||
|
return
|
||||||
|
model_base = model.split(":")[0]
|
||||||
|
if model_base not in pulled_models:
|
||||||
|
raise RuntimeError(
|
||||||
|
f'Ollama embedding model "{model}" is not pulled.\n'
|
||||||
|
f"Fix: ollama pull {model}"
|
||||||
|
)
|
||||||
|
|
||||||
def _resolve_model(self, client: OpenAI, model: str) -> str:
|
def _resolve_model(self, client: OpenAI, model: str) -> str:
|
||||||
"""Resolve __auto__ to the first model served by vLLM."""
|
"""Resolve __auto__ to the first model served by vLLM."""
|
||||||
if model != "__auto__":
|
if model != "__auto__":
|
||||||
|
|
@ -152,10 +190,19 @@ class LLMRouter:
|
||||||
"""
|
"""
|
||||||
If backend config has a cf_orch block and CF_ORCH_URL is set (env takes
|
If backend config has a cf_orch block and CF_ORCH_URL is set (env takes
|
||||||
precedence over yaml url), allocate via cf-orch and return (ctx, alloc).
|
precedence over yaml url), allocate via cf-orch and return (ctx, alloc).
|
||||||
|
|
||||||
|
Two allocation modes:
|
||||||
|
- task-based (preferred): cf_orch block has `product` + `task` keys.
|
||||||
|
Calls POST /api/inference/task; coordinator resolves model/node from
|
||||||
|
assignments.yaml. No hardcoded model IDs in product config.
|
||||||
|
- service-based (legacy): cf_orch block has `service` key.
|
||||||
|
Calls allocate(service=...) directly.
|
||||||
|
|
||||||
Returns None if not configured or allocation fails.
|
Returns None if not configured or allocation fails.
|
||||||
Caller MUST call ctx.__exit__(None, None, None) in a finally block.
|
Caller MUST call ctx.__exit__(None, None, None) in a finally block.
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
|
|
||||||
orch_cfg = backend.get("cf_orch")
|
orch_cfg = backend.get("cf_orch")
|
||||||
if not orch_cfg:
|
if not orch_cfg:
|
||||||
return None
|
return None
|
||||||
|
|
@ -164,31 +211,46 @@ class LLMRouter:
|
||||||
return None
|
return None
|
||||||
try:
|
try:
|
||||||
from circuitforge_orch.client import CFOrchClient
|
from circuitforge_orch.client import CFOrchClient
|
||||||
|
|
||||||
client = CFOrchClient(orch_url)
|
client = CFOrchClient(orch_url)
|
||||||
service = orch_cfg.get("service", "vllm")
|
|
||||||
candidates = orch_cfg.get("model_candidates", [])
|
|
||||||
ttl_s = float(orch_cfg.get("ttl_s", 3600.0))
|
ttl_s = float(orch_cfg.get("ttl_s", 3600.0))
|
||||||
# CF_APP_NAME identifies the calling product (kiwi, peregrine, etc.)
|
|
||||||
# in coordinator analytics — set in each product's .env.
|
# Task-based allocation: product+task → coordinator resolves model/node.
|
||||||
pipeline = os.environ.get("CF_APP_NAME") or None
|
task = orch_cfg.get("task")
|
||||||
|
product = orch_cfg.get("product") or os.environ.get("CF_APP_NAME") or None
|
||||||
|
if task and product:
|
||||||
|
ctx = client.task_allocate(product, task, ttl_s=ttl_s)
|
||||||
|
alloc = ctx.__enter__()
|
||||||
|
return (ctx, alloc)
|
||||||
|
|
||||||
|
# Service-based allocation (legacy path).
|
||||||
|
cf_app = os.environ.get("CF_APP_NAME") or None
|
||||||
|
caller = f"{cf_app}.llm-router" if cf_app else "llm-router"
|
||||||
ctx = client.allocate(
|
ctx = client.allocate(
|
||||||
service,
|
orch_cfg.get("service", "vllm"),
|
||||||
model_candidates=candidates,
|
model_candidates=orch_cfg.get("model_candidates", []),
|
||||||
ttl_s=ttl_s,
|
ttl_s=ttl_s,
|
||||||
caller="llm-router",
|
caller=caller,
|
||||||
pipeline=pipeline,
|
pipeline=cf_app,
|
||||||
)
|
)
|
||||||
alloc = ctx.__enter__()
|
alloc = ctx.__enter__()
|
||||||
return (ctx, alloc)
|
return (ctx, alloc)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.warning("[LLMRouter] cf_orch allocation failed, using base_url directly: %s", exc)
|
logger.warning(
|
||||||
|
"[LLMRouter] cf_orch allocation failed, using base_url directly: %s",
|
||||||
|
exc,
|
||||||
|
)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def complete(self, prompt: str, system: str | None = None,
|
def complete(
|
||||||
model_override: str | None = None,
|
self,
|
||||||
fallback_order: list[str] | None = None,
|
prompt: str,
|
||||||
images: list[str] | None = None,
|
system: str | None = None,
|
||||||
max_tokens: int | None = None) -> str:
|
model_override: str | None = None,
|
||||||
|
fallback_order: list[str] | None = None,
|
||||||
|
images: list[str] | None = None,
|
||||||
|
max_tokens: int | None = None,
|
||||||
|
) -> str:
|
||||||
"""
|
"""
|
||||||
Generate a completion. Tries each backend in fallback_order.
|
Generate a completion. Tries each backend in fallback_order.
|
||||||
|
|
||||||
|
|
@ -206,7 +268,11 @@ class LLMRouter:
|
||||||
"AI inference is disabled in the public demo. "
|
"AI inference is disabled in the public demo. "
|
||||||
"Run your own instance to use AI features."
|
"Run your own instance to use AI features."
|
||||||
)
|
)
|
||||||
order = fallback_order if fallback_order is not None else self.config["fallback_order"]
|
order = (
|
||||||
|
fallback_order
|
||||||
|
if fallback_order is not None
|
||||||
|
else self.config["fallback_order"]
|
||||||
|
)
|
||||||
for name in order:
|
for name in order:
|
||||||
backend = self.config["backends"][name]
|
backend = self.config["backends"][name]
|
||||||
|
|
||||||
|
|
@ -283,10 +349,14 @@ class LLMRouter:
|
||||||
if images and supports_images:
|
if images and supports_images:
|
||||||
content = [{"type": "text", "text": prompt}]
|
content = [{"type": "text", "text": prompt}]
|
||||||
for img in images:
|
for img in images:
|
||||||
content.append({
|
content.append(
|
||||||
"type": "image_url",
|
{
|
||||||
"image_url": {"url": f"data:image/png;base64,{img}"},
|
"type": "image_url",
|
||||||
})
|
"image_url": {
|
||||||
|
"url": f"data:image/png;base64,{img}"
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
messages.append({"role": "user", "content": content})
|
messages.append({"role": "user", "content": content})
|
||||||
else:
|
else:
|
||||||
messages.append({"role": "user", "content": prompt})
|
messages.append({"role": "user", "content": prompt})
|
||||||
|
|
@ -311,18 +381,27 @@ class LLMRouter:
|
||||||
elif backend["type"] == "anthropic":
|
elif backend["type"] == "anthropic":
|
||||||
api_key = os.environ.get(backend["api_key_env"], "")
|
api_key = os.environ.get(backend["api_key_env"], "")
|
||||||
if not api_key:
|
if not api_key:
|
||||||
print(f"[LLMRouter] {name}: {backend['api_key_env']} not set, skipping")
|
print(
|
||||||
|
f"[LLMRouter] {name}: {backend['api_key_env']} not set, skipping"
|
||||||
|
)
|
||||||
continue
|
continue
|
||||||
try:
|
try:
|
||||||
import anthropic as _anthropic
|
import anthropic as _anthropic
|
||||||
|
|
||||||
client = _anthropic.Anthropic(api_key=api_key)
|
client = _anthropic.Anthropic(api_key=api_key)
|
||||||
if images and supports_images:
|
if images and supports_images:
|
||||||
content = []
|
content = []
|
||||||
for img in images:
|
for img in images:
|
||||||
content.append({
|
content.append(
|
||||||
"type": "image",
|
{
|
||||||
"source": {"type": "base64", "media_type": "image/png", "data": img},
|
"type": "image",
|
||||||
})
|
"source": {
|
||||||
|
"type": "base64",
|
||||||
|
"media_type": "image/png",
|
||||||
|
"data": img,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
content.append({"type": "text", "text": prompt})
|
content.append({"type": "text", "text": prompt})
|
||||||
else:
|
else:
|
||||||
content = prompt
|
content = prompt
|
||||||
|
|
@ -342,6 +421,84 @@ class LLMRouter:
|
||||||
|
|
||||||
raise RuntimeError("All LLM backends exhausted")
|
raise RuntimeError("All LLM backends exhausted")
|
||||||
|
|
||||||
|
def embed(
|
||||||
|
self,
|
||||||
|
texts: list[str],
|
||||||
|
model_override: str | None = None,
|
||||||
|
fallback_order: list[str] | None = None,
|
||||||
|
) -> list[list[float]]:
|
||||||
|
"""
|
||||||
|
Generate embeddings for a list of texts.
|
||||||
|
|
||||||
|
Only openai_compat backends are tried — Ollama and vLLM expose
|
||||||
|
/v1/embeddings; anthropic and vision_service do not.
|
||||||
|
|
||||||
|
Uses ``embedding_model`` from backend config when present;
|
||||||
|
falls back to ``model`` (the chat model) otherwise.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
texts: Texts to embed (batched in a single API call).
|
||||||
|
model_override: Override the embedding model for this call.
|
||||||
|
fallback_order: Override the backend fallback order for this call.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of float vectors, one per input text, in input order.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
RuntimeError: If all eligible backends are exhausted.
|
||||||
|
"""
|
||||||
|
if os.environ.get("DEMO_MODE", "").lower() in ("1", "true", "yes"):
|
||||||
|
raise RuntimeError(
|
||||||
|
"AI inference is disabled in the public demo. "
|
||||||
|
"Run your own instance to use AI features."
|
||||||
|
)
|
||||||
|
order = (
|
||||||
|
fallback_order
|
||||||
|
if fallback_order is not None
|
||||||
|
else self.config["fallback_order"]
|
||||||
|
)
|
||||||
|
for name in order:
|
||||||
|
backend = self.config["backends"][name]
|
||||||
|
if not backend.get("enabled", True):
|
||||||
|
continue
|
||||||
|
if backend["type"] != "openai_compat":
|
||||||
|
continue
|
||||||
|
|
||||||
|
orch_ctx = orch_alloc = None
|
||||||
|
orch_result = self._try_cf_orch_alloc(backend)
|
||||||
|
if orch_result is not None:
|
||||||
|
orch_ctx, orch_alloc = orch_result
|
||||||
|
backend = {**backend, "base_url": orch_alloc.url + "/v1"}
|
||||||
|
elif not self._is_reachable(backend["base_url"]):
|
||||||
|
print(f"[LLMRouter] {name}: unreachable, skipping")
|
||||||
|
continue
|
||||||
|
|
||||||
|
embed_model = model_override or backend.get(
|
||||||
|
"embedding_model", backend["model"]
|
||||||
|
)
|
||||||
|
self._check_ollama_model_pulled(backend["base_url"], embed_model)
|
||||||
|
|
||||||
|
try:
|
||||||
|
client = OpenAI(
|
||||||
|
base_url=backend["base_url"],
|
||||||
|
api_key=backend.get("api_key") or "any",
|
||||||
|
)
|
||||||
|
model = embed_model
|
||||||
|
resp = client.embeddings.create(model=model, input=texts)
|
||||||
|
print(f"[LLMRouter] embed: used backend {name} ({model})")
|
||||||
|
return [item.embedding for item in resp.data]
|
||||||
|
except Exception as e:
|
||||||
|
print(f"[LLMRouter] {name}: embed error — {e}, trying next")
|
||||||
|
continue
|
||||||
|
finally:
|
||||||
|
if orch_ctx is not None:
|
||||||
|
try:
|
||||||
|
orch_ctx.__exit__(None, None, None)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
raise RuntimeError("All LLM backends exhausted for embed()")
|
||||||
|
|
||||||
|
|
||||||
# Module-level singleton for convenience
|
# Module-level singleton for convenience
|
||||||
_router: LLMRouter | None = None
|
_router: LLMRouter | None = None
|
||||||
|
|
|
||||||
4
circuitforge_core/vector/__init__.py
Normal file
4
circuitforge_core/vector/__init__.py
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
from .base import VectorMatch, VectorStore
|
||||||
|
from .sqlite_vec import LocalSQLiteVecStore
|
||||||
|
|
||||||
|
__all__ = ["VectorMatch", "VectorStore", "LocalSQLiteVecStore"]
|
||||||
50
circuitforge_core/vector/base.py
Normal file
50
circuitforge_core/vector/base.py
Normal file
|
|
@ -0,0 +1,50 @@
|
||||||
|
"""
|
||||||
|
circuitforge_core.vector.base — VectorStore ABC and shared types.
|
||||||
|
|
||||||
|
Concrete implementations: LocalSQLiteVecStore (local), QdrantStore (cloud Paid tier).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class VectorMatch:
|
||||||
|
"""A single result from a vector similarity search."""
|
||||||
|
|
||||||
|
entry_id: str
|
||||||
|
score: float # lower is better (L2 / cosine distance)
|
||||||
|
metadata: dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
class VectorStore(ABC):
|
||||||
|
"""Abstract interface for vector storage backends."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def upsert(
|
||||||
|
self, entry_id: str, vector: list[float], metadata: dict[str, Any]
|
||||||
|
) -> None:
|
||||||
|
"""Insert or replace a vector and its metadata."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def query(
|
||||||
|
self,
|
||||||
|
vector: list[float],
|
||||||
|
top_k: int = 10,
|
||||||
|
filter_metadata: dict[str, Any] | None = None,
|
||||||
|
) -> list[VectorMatch]:
|
||||||
|
"""Return the top_k nearest vectors. Optional metadata filter applied post-search."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def delete(self, entry_id: str) -> None:
|
||||||
|
"""Remove a single vector by string ID. No-op if not found."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def delete_where(self, filter_metadata: dict[str, Any]) -> int:
|
||||||
|
"""Remove all vectors whose metadata matches all key-value pairs. Returns count removed.
|
||||||
|
|
||||||
|
Raises ValueError if filter_metadata is empty (would delete entire store).
|
||||||
|
"""
|
||||||
185
circuitforge_core/vector/sqlite_vec.py
Normal file
185
circuitforge_core/vector/sqlite_vec.py
Normal file
|
|
@ -0,0 +1,185 @@
|
||||||
|
# circuitforge_core/vector/sqlite_vec.py
|
||||||
|
"""
|
||||||
|
circuitforge_core.vector.sqlite_vec -- sqlite-vec backed VectorStore.
|
||||||
|
|
||||||
|
Suitable for single-user local deployments. Cloud Paid tier replaces
|
||||||
|
this with QdrantStore via the same VectorStore ABC.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
import sqlite3
|
||||||
|
import struct
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Generator
|
||||||
|
|
||||||
|
import sqlite_vec
|
||||||
|
|
||||||
|
from .base import VectorMatch, VectorStore
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_SAFE_IDENTIFIER = re.compile(r"^[a-zA-Z_][a-zA-Z0-9_]*$")
|
||||||
|
|
||||||
|
|
||||||
|
def _serialize(vector: list[float]) -> bytes:
|
||||||
|
return struct.pack(f"<{len(vector)}f", *vector)
|
||||||
|
|
||||||
|
|
||||||
|
class LocalSQLiteVecStore(VectorStore):
|
||||||
|
"""
|
||||||
|
VectorStore backed by sqlite-vec virtual tables.
|
||||||
|
|
||||||
|
Uses two tables per logical store:
|
||||||
|
- ``<table>_vecs``: vec0 virtual table (rowid-indexed float vectors)
|
||||||
|
- ``<table>_meta``: companion table mapping rowid to string ID + JSON metadata
|
||||||
|
|
||||||
|
Args:
|
||||||
|
db_path: Path to SQLite database file.
|
||||||
|
table: Logical name prefix (default ``"vecs"``).
|
||||||
|
dimensions: Vector length; must match the embedding model (default 768).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
db_path: str | Path,
|
||||||
|
table: str = "vecs",
|
||||||
|
dimensions: int = 768,
|
||||||
|
) -> None:
|
||||||
|
if not _SAFE_IDENTIFIER.match(table):
|
||||||
|
raise ValueError(
|
||||||
|
f"table must be a valid SQL identifier (letters, digits, underscores): {table!r}"
|
||||||
|
)
|
||||||
|
self.db_path = str(db_path)
|
||||||
|
self.table = table
|
||||||
|
self.dimensions = dimensions
|
||||||
|
self._init_tables()
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def _conn(self) -> Generator[sqlite3.Connection, None, None]:
|
||||||
|
conn = sqlite3.connect(self.db_path)
|
||||||
|
conn.enable_load_extension(True)
|
||||||
|
sqlite_vec.load(conn)
|
||||||
|
conn.enable_load_extension(False)
|
||||||
|
conn.row_factory = sqlite3.Row
|
||||||
|
try:
|
||||||
|
yield conn
|
||||||
|
conn.commit()
|
||||||
|
except Exception:
|
||||||
|
conn.rollback()
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
def _init_tables(self) -> None:
|
||||||
|
with self._conn() as conn:
|
||||||
|
conn.execute(f"""
|
||||||
|
CREATE VIRTUAL TABLE IF NOT EXISTS {self.table}_vecs
|
||||||
|
USING vec0(embedding float[{self.dimensions}])
|
||||||
|
""")
|
||||||
|
conn.execute(f"""
|
||||||
|
CREATE TABLE IF NOT EXISTS {self.table}_meta (
|
||||||
|
rowid INTEGER PRIMARY KEY,
|
||||||
|
entry_id TEXT NOT NULL UNIQUE,
|
||||||
|
metadata TEXT NOT NULL DEFAULT '{{}}'
|
||||||
|
)
|
||||||
|
""")
|
||||||
|
|
||||||
|
def upsert(
|
||||||
|
self, entry_id: str, vector: list[float], metadata: dict[str, Any]
|
||||||
|
) -> None:
|
||||||
|
with self._conn() as conn:
|
||||||
|
row = conn.execute(
|
||||||
|
f"SELECT rowid FROM {self.table}_meta WHERE entry_id = ?", [entry_id]
|
||||||
|
).fetchone()
|
||||||
|
|
||||||
|
if row:
|
||||||
|
rowid = row["rowid"]
|
||||||
|
conn.execute(
|
||||||
|
f"UPDATE {self.table}_vecs SET embedding = ? WHERE rowid = ?",
|
||||||
|
[_serialize(vector), rowid],
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
f"UPDATE {self.table}_meta SET metadata = ? WHERE rowid = ?",
|
||||||
|
[json.dumps(metadata), rowid],
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
cursor = conn.execute(
|
||||||
|
f"INSERT INTO {self.table}_meta(entry_id, metadata) VALUES (?, ?)",
|
||||||
|
[entry_id, json.dumps(metadata)],
|
||||||
|
)
|
||||||
|
rowid = cursor.lastrowid
|
||||||
|
conn.execute(
|
||||||
|
f"INSERT INTO {self.table}_vecs(rowid, embedding) VALUES (?, ?)",
|
||||||
|
[rowid, _serialize(vector)],
|
||||||
|
)
|
||||||
|
|
||||||
|
def query(
|
||||||
|
self,
|
||||||
|
vector: list[float],
|
||||||
|
top_k: int = 10,
|
||||||
|
filter_metadata: dict[str, Any] | None = None,
|
||||||
|
) -> list[VectorMatch]:
|
||||||
|
with self._conn() as conn:
|
||||||
|
rows = conn.execute(
|
||||||
|
f"""
|
||||||
|
SELECT m.entry_id, v.distance, m.metadata
|
||||||
|
FROM {self.table}_vecs v
|
||||||
|
JOIN {self.table}_meta m ON m.rowid = v.rowid
|
||||||
|
WHERE v.embedding MATCH ? AND k = ?
|
||||||
|
ORDER BY v.distance
|
||||||
|
""",
|
||||||
|
[_serialize(vector), top_k],
|
||||||
|
).fetchall()
|
||||||
|
results = [
|
||||||
|
VectorMatch(
|
||||||
|
entry_id=r["entry_id"],
|
||||||
|
score=r["distance"],
|
||||||
|
metadata=json.loads(r["metadata"]),
|
||||||
|
)
|
||||||
|
for r in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
if filter_metadata:
|
||||||
|
results = [
|
||||||
|
r
|
||||||
|
for r in results
|
||||||
|
if all(r.metadata.get(k) == v for k, v in filter_metadata.items())
|
||||||
|
]
|
||||||
|
return results
|
||||||
|
|
||||||
|
def delete(self, entry_id: str) -> None:
|
||||||
|
with self._conn() as conn:
|
||||||
|
row = conn.execute(
|
||||||
|
f"SELECT rowid FROM {self.table}_meta WHERE entry_id = ?", [entry_id]
|
||||||
|
).fetchone()
|
||||||
|
if row:
|
||||||
|
rowid = row["rowid"]
|
||||||
|
conn.execute(f"DELETE FROM {self.table}_vecs WHERE rowid = ?", [rowid])
|
||||||
|
conn.execute(f"DELETE FROM {self.table}_meta WHERE rowid = ?", [rowid])
|
||||||
|
|
||||||
|
def delete_where(self, filter_metadata: dict[str, Any]) -> int:
|
||||||
|
if not filter_metadata:
|
||||||
|
raise ValueError(
|
||||||
|
"delete_where requires a non-empty filter; refusing to delete entire store"
|
||||||
|
)
|
||||||
|
with self._conn() as conn:
|
||||||
|
rows = conn.execute(
|
||||||
|
f"SELECT rowid, metadata FROM {self.table}_meta"
|
||||||
|
).fetchall()
|
||||||
|
to_delete = [
|
||||||
|
r["rowid"]
|
||||||
|
for r in rows
|
||||||
|
if all(
|
||||||
|
json.loads(r["metadata"]).get(k) == v
|
||||||
|
for k, v in filter_metadata.items()
|
||||||
|
)
|
||||||
|
]
|
||||||
|
for rowid in to_delete:
|
||||||
|
conn.execute(f"DELETE FROM {self.table}_vecs WHERE rowid = ?", [rowid])
|
||||||
|
conn.execute(f"DELETE FROM {self.table}_meta WHERE rowid = ?", [rowid])
|
||||||
|
return len(to_delete)
|
||||||
11
circuitforge_core/video/__init__.py
Normal file
11
circuitforge_core/video/__init__.py
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
"""
|
||||||
|
circuitforge_core.video — cf-video service: video VLM inference via Marlin-2B.
|
||||||
|
|
||||||
|
Exposes a FastAPI process (managed by cf-orch) with endpoints:
|
||||||
|
GET /health → {"status": "ok", "model": str, "vram_mb": int}
|
||||||
|
POST /caption → CaptionResult (scene description + timestamped events)
|
||||||
|
POST /find → FindResult (temporal grounding span for a natural-language event)
|
||||||
|
|
||||||
|
Run as:
|
||||||
|
python -m circuitforge_core.video.app --model /path/to/NemoStation--Marlin-2B --port 8016 --gpu-id 0
|
||||||
|
"""
|
||||||
191
circuitforge_core/video/app.py
Normal file
191
circuitforge_core/video/app.py
Normal file
|
|
@ -0,0 +1,191 @@
|
||||||
|
"""
|
||||||
|
cf-video FastAPI service — managed by cf-orch.
|
||||||
|
|
||||||
|
Endpoints:
|
||||||
|
GET /health → {"status": "ok", "model": str, "vram_mb": int}
|
||||||
|
POST /caption → CaptionResponse (scene + timestamped events)
|
||||||
|
POST /find → FindResponse (temporal grounding span)
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python -m circuitforge_core.video.app \
|
||||||
|
--model /Library/Assets/LLM/cf-video/models/NemoStation--Marlin-2B \
|
||||||
|
--port 8016 \
|
||||||
|
--gpu-id 0
|
||||||
|
|
||||||
|
The service loads the model once at startup and blocks until it is ready.
|
||||||
|
cf-orch health-polls /health before routing any inference requests.
|
||||||
|
|
||||||
|
Model requirements:
|
||||||
|
transformers >= 5.7.0
|
||||||
|
torch >= 2.11.0
|
||||||
|
torchcodec (installed)
|
||||||
|
qwen-vl-utils >= 0.0.14 (installed)
|
||||||
|
|
||||||
|
Security:
|
||||||
|
Marlin requires trust_remote_code=True. Review the model's
|
||||||
|
modeling_marlin.py before deploying on a production node.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import FastAPI, HTTPException
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from circuitforge_core.video.backends.base import VideoBackend, make_video_backend
|
||||||
|
|
||||||
|
app = FastAPI(title="cf-video", version="0.1.0")
|
||||||
|
_backend: VideoBackend | None = None
|
||||||
|
|
||||||
|
|
||||||
|
# ── Request / response models ─────────────────────────────────────────────────
|
||||||
|
|
||||||
|
class CaptionRequest(BaseModel):
|
||||||
|
video_path: str = Field(..., description="Absolute path to the video file on this node")
|
||||||
|
max_new_tokens: int = Field(2048, ge=64, le=8192)
|
||||||
|
|
||||||
|
|
||||||
|
class VideoEventOut(BaseModel):
|
||||||
|
start: float
|
||||||
|
end: float
|
||||||
|
description: str
|
||||||
|
|
||||||
|
|
||||||
|
class CaptionResponse(BaseModel):
|
||||||
|
scene: str
|
||||||
|
events: list[VideoEventOut]
|
||||||
|
caption: str
|
||||||
|
model: str
|
||||||
|
|
||||||
|
|
||||||
|
class FindRequest(BaseModel):
|
||||||
|
video_path: str = Field(..., description="Absolute path to the video file on this node")
|
||||||
|
event: str = Field(..., min_length=1, description="Natural-language event description to locate")
|
||||||
|
max_new_tokens: int = Field(256, ge=32, le=2048)
|
||||||
|
|
||||||
|
|
||||||
|
class FindResponse(BaseModel):
|
||||||
|
span: list[float] | None = Field(
|
||||||
|
None,
|
||||||
|
description="[start_sec, end_sec] or null when the model could not ground the event",
|
||||||
|
)
|
||||||
|
format_ok: bool
|
||||||
|
raw: str
|
||||||
|
model: str
|
||||||
|
|
||||||
|
|
||||||
|
# ── Endpoints ─────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@app.get("/health")
|
||||||
|
def health() -> dict[str, Any]:
|
||||||
|
if _backend is None:
|
||||||
|
raise HTTPException(503, detail="backend not initialised")
|
||||||
|
return {
|
||||||
|
"status": "ok",
|
||||||
|
"model": _backend.model_name,
|
||||||
|
"vram_mb": _backend.vram_mb,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/caption", response_model=CaptionResponse)
|
||||||
|
def caption(req: CaptionRequest) -> CaptionResponse:
|
||||||
|
if _backend is None:
|
||||||
|
raise HTTPException(503, detail="backend not initialised")
|
||||||
|
try:
|
||||||
|
result = _backend.caption(req.video_path, max_new_tokens=req.max_new_tokens)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(404, detail=str(exc)) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
logging.exception("caption failed for %r", req.video_path)
|
||||||
|
raise HTTPException(500, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
return CaptionResponse(
|
||||||
|
scene=result.scene,
|
||||||
|
events=[
|
||||||
|
VideoEventOut(start=ev.start, end=ev.end, description=ev.description)
|
||||||
|
for ev in result.events
|
||||||
|
],
|
||||||
|
caption=result.caption,
|
||||||
|
model=result.model,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/find", response_model=FindResponse)
|
||||||
|
def find(req: FindRequest) -> FindResponse:
|
||||||
|
if _backend is None:
|
||||||
|
raise HTTPException(503, detail="backend not initialised")
|
||||||
|
try:
|
||||||
|
result = _backend.find(
|
||||||
|
req.video_path,
|
||||||
|
req.event,
|
||||||
|
max_new_tokens=req.max_new_tokens,
|
||||||
|
)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(404, detail=str(exc)) from exc
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(422, detail=str(exc)) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
logging.exception("find failed for %r event=%r", req.video_path, req.event)
|
||||||
|
raise HTTPException(500, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
return FindResponse(
|
||||||
|
span=list(result.span) if result.span is not None else None,
|
||||||
|
format_ok=result.format_ok,
|
||||||
|
raw=result.raw,
|
||||||
|
model=result.model,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ── CLI entry point ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _parse_args() -> argparse.Namespace:
|
||||||
|
p = argparse.ArgumentParser(description="cf-video service (Marlin-2B)")
|
||||||
|
p.add_argument(
|
||||||
|
"--model",
|
||||||
|
required=True,
|
||||||
|
help="Local filesystem path to the Marlin model directory (safetensors)",
|
||||||
|
)
|
||||||
|
p.add_argument("--port", type=int, default=8016)
|
||||||
|
p.add_argument("--host", default="0.0.0.0")
|
||||||
|
p.add_argument(
|
||||||
|
"--gpu-id", type=int, default=0,
|
||||||
|
help="CUDA device index; overridden by CUDA_VISIBLE_DEVICES when set by cf-orch",
|
||||||
|
)
|
||||||
|
p.add_argument("--device", default="cuda", choices=["cuda", "cpu"])
|
||||||
|
p.add_argument(
|
||||||
|
"--mock", action="store_true",
|
||||||
|
help="Run with MockVideoBackend (no GPU, for testing)",
|
||||||
|
)
|
||||||
|
return p.parse_args()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
import uvicorn
|
||||||
|
|
||||||
|
logging.basicConfig(
|
||||||
|
level=logging.INFO,
|
||||||
|
format="%(asctime)s %(levelname)s %(name)s — %(message)s",
|
||||||
|
)
|
||||||
|
args = _parse_args()
|
||||||
|
|
||||||
|
# Pin GPU selection unconditionally — --gpu-id is authoritative.
|
||||||
|
# Force PCI_BUS_ID ordering so --gpu-id matches nvidia-smi (not CUDA's
|
||||||
|
# default FASTEST_FIRST, which can swap indices on multi-GPU nodes).
|
||||||
|
if args.device == "cuda" and not args.mock:
|
||||||
|
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
|
||||||
|
os.environ["CUDA_VISIBLE_DEVICES"] = str(args.gpu_id)
|
||||||
|
|
||||||
|
mock = args.mock or args.model == "mock"
|
||||||
|
device = "cpu" if mock else args.device
|
||||||
|
|
||||||
|
_backend = make_video_backend(
|
||||||
|
model_path=args.model,
|
||||||
|
mock=mock,
|
||||||
|
device=device,
|
||||||
|
gpu_id=args.gpu_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
uvicorn.run(app, host=args.host, port=args.port, log_level="info")
|
||||||
1
circuitforge_core/video/backends/__init__.py
Normal file
1
circuitforge_core/video/backends/__init__.py
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
"""Video backend registry."""
|
||||||
96
circuitforge_core/video/backends/base.py
Normal file
96
circuitforge_core/video/backends/base.py
Normal file
|
|
@ -0,0 +1,96 @@
|
||||||
|
"""
|
||||||
|
VideoBackend Protocol — backend-agnostic interface for video VLM inference.
|
||||||
|
|
||||||
|
Implementations:
|
||||||
|
MarlinBackend — NemoStation/Marlin-2B (dense captioning + temporal grounding)
|
||||||
|
MockVideoBackend — deterministic stub for unit tests
|
||||||
|
|
||||||
|
Both endpoints accept a video_path (local filesystem path) so the service
|
||||||
|
receives pre-staged video files rather than raw byte streams. Large uploads
|
||||||
|
should be staged by the caller before hitting /caption or /find.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Protocol, runtime_checkable
|
||||||
|
|
||||||
|
|
||||||
|
# ── Result types ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class VideoEvent:
|
||||||
|
"""A single timestamped event from a caption pass."""
|
||||||
|
start: float # seconds from video start
|
||||||
|
end: float # seconds from video start
|
||||||
|
description: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class CaptionResult:
|
||||||
|
"""Result from a /caption call."""
|
||||||
|
scene: str # scene-level description paragraph
|
||||||
|
events: list[VideoEvent] # timestamped event list (may be empty)
|
||||||
|
caption: str # full raw caption string from the model
|
||||||
|
model: str # model name / path
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class FindResult:
|
||||||
|
"""Result from a /find call."""
|
||||||
|
span: tuple[float, float] | None # (start_sec, end_sec) or None on parse failure
|
||||||
|
format_ok: bool # True when model output matched expected format
|
||||||
|
raw: str # raw model output for debugging
|
||||||
|
model: str
|
||||||
|
|
||||||
|
|
||||||
|
# ── Backend Protocol ─────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@runtime_checkable
|
||||||
|
class VideoBackend(Protocol):
|
||||||
|
"""Minimal interface all video backends must satisfy."""
|
||||||
|
|
||||||
|
def caption(
|
||||||
|
self,
|
||||||
|
video_path: str,
|
||||||
|
*,
|
||||||
|
max_new_tokens: int = 2048,
|
||||||
|
) -> CaptionResult: ...
|
||||||
|
|
||||||
|
def find(
|
||||||
|
self,
|
||||||
|
video_path: str,
|
||||||
|
event: str,
|
||||||
|
*,
|
||||||
|
max_new_tokens: int = 256,
|
||||||
|
) -> FindResult: ...
|
||||||
|
|
||||||
|
@property
|
||||||
|
def model_name(self) -> str: ...
|
||||||
|
|
||||||
|
@property
|
||||||
|
def vram_mb(self) -> int: ...
|
||||||
|
|
||||||
|
|
||||||
|
# ── Factory ──────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def make_video_backend(
|
||||||
|
model_path: str,
|
||||||
|
*,
|
||||||
|
mock: bool = False,
|
||||||
|
device: str = "cuda",
|
||||||
|
gpu_id: int = 0,
|
||||||
|
) -> VideoBackend:
|
||||||
|
"""Instantiate the appropriate VideoBackend.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
model_path: Local filesystem path to the model directory (safetensors).
|
||||||
|
mock: When True, return MockVideoBackend (no GPU required).
|
||||||
|
device: Torch device string ("cuda" or "cpu").
|
||||||
|
gpu_id: CUDA device index — used only when CUDA_VISIBLE_DEVICES is
|
||||||
|
not already set externally (cf-orch sets it before spawning).
|
||||||
|
"""
|
||||||
|
if mock:
|
||||||
|
from circuitforge_core.video.backends.mock import MockVideoBackend
|
||||||
|
return MockVideoBackend(model_path)
|
||||||
|
from circuitforge_core.video.backends.marlin import MarlinBackend
|
||||||
|
return MarlinBackend(model_path=model_path, device=device)
|
||||||
184
circuitforge_core/video/backends/marlin.py
Normal file
184
circuitforge_core/video/backends/marlin.py
Normal file
|
|
@ -0,0 +1,184 @@
|
||||||
|
"""
|
||||||
|
MarlinBackend — NemoStation/Marlin-2B video VLM via HuggingFace Transformers.
|
||||||
|
|
||||||
|
Marlin-2B is a decoder-only video understanding model that produces:
|
||||||
|
- Dense scene captions with second-precise event timestamps (/caption)
|
||||||
|
- Temporal grounding of natural-language events (/find)
|
||||||
|
|
||||||
|
Requirements (install separately):
|
||||||
|
pip install "transformers>=5.7.0" "torch>=2.11.0" torchcodec "qwen-vl-utils>=0.0.14" av pillow
|
||||||
|
|
||||||
|
Security note:
|
||||||
|
trust_remote_code=True is required. The model ships a custom
|
||||||
|
AutoModelForCausalLM subclass (modeling_marlin.py). Review that file
|
||||||
|
before enabling on any node. The modeling code runs in-process with
|
||||||
|
full filesystem access.
|
||||||
|
|
||||||
|
Environment variables forwarded to the model's preprocessing layer:
|
||||||
|
FORCE_QWENVL_VIDEO_READER default: torchcodec (video decode backend)
|
||||||
|
VIDEO_MAX_PIXELS default: 200704 (max pixels per frame)
|
||||||
|
FPS default: 2.0 (frame sample rate)
|
||||||
|
FPS_MAX_FRAMES default: 240 (frame cap ~2 min video)
|
||||||
|
FPS_MIN_FRAMES default: 4 (minimum frames)
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from circuitforge_core.video.backends.base import CaptionResult, FindResult, VideoEvent
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# Default env overrides so torchcodec is preferred over the slower av/ffmpeg path.
|
||||||
|
_DEFAULT_ENV: dict[str, str] = {
|
||||||
|
"FORCE_QWENVL_VIDEO_READER": "torchcodec",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class MarlinBackend:
|
||||||
|
"""
|
||||||
|
Load Marlin-2B once, expose caption() and find() as synchronous calls.
|
||||||
|
|
||||||
|
The model is loaded eagerly in __init__ — if loading fails (OOM, missing
|
||||||
|
weights, transformers version mismatch) the error propagates immediately
|
||||||
|
rather than on first inference, so cf-orch's 2-second liveness check can
|
||||||
|
catch it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, model_path: str, device: str = "cuda") -> None:
|
||||||
|
self._model_path = model_path
|
||||||
|
self._device = device
|
||||||
|
|
||||||
|
# Apply env defaults before importing transformers — the model's
|
||||||
|
# custom __init__.py reads these at import time.
|
||||||
|
for key, val in _DEFAULT_ENV.items():
|
||||||
|
os.environ.setdefault(key, val)
|
||||||
|
|
||||||
|
self._model = self._load_model(model_path, device)
|
||||||
|
self._vram_mb = self._estimate_vram_mb()
|
||||||
|
logger.info(
|
||||||
|
"MarlinBackend: loaded %r on %s (~%d MB VRAM)",
|
||||||
|
model_path, device, self._vram_mb,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ── Loading ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _load_model(self, model_path: str, device: str):
|
||||||
|
import torch
|
||||||
|
from transformers import AutoModelForCausalLM
|
||||||
|
|
||||||
|
# Verify weights exist before handing to transformers — gives a clear
|
||||||
|
# error instead of a cryptic trust_remote_code failure.
|
||||||
|
path = Path(model_path)
|
||||||
|
if not path.exists():
|
||||||
|
raise FileNotFoundError(
|
||||||
|
f"Marlin model directory not found: {model_path!r}. "
|
||||||
|
"Download via Avocet or: "
|
||||||
|
f"huggingface-cli download NemoStation/Marlin-2B --local-dir {model_path}"
|
||||||
|
)
|
||||||
|
|
||||||
|
logger.info("MarlinBackend: loading model from %r ...", model_path)
|
||||||
|
model = AutoModelForCausalLM.from_pretrained(
|
||||||
|
model_path,
|
||||||
|
trust_remote_code=True, # Required — custom modeling code in repo
|
||||||
|
torch_dtype=torch.bfloat16,
|
||||||
|
device_map={"": device},
|
||||||
|
)
|
||||||
|
model.eval()
|
||||||
|
logger.info("MarlinBackend: model loaded")
|
||||||
|
return model
|
||||||
|
|
||||||
|
def _estimate_vram_mb(self) -> int:
|
||||||
|
"""Read allocated VRAM from torch after load; fall back to catalog estimate."""
|
||||||
|
try:
|
||||||
|
import torch
|
||||||
|
if torch.cuda.is_available():
|
||||||
|
return int(torch.cuda.memory_allocated() / 1024 / 1024)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return 4500 # Catalog estimate for Marlin-2B BF16
|
||||||
|
|
||||||
|
# ── Inference ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def caption(
|
||||||
|
self,
|
||||||
|
video_path: str,
|
||||||
|
*,
|
||||||
|
max_new_tokens: int = 2048,
|
||||||
|
) -> CaptionResult:
|
||||||
|
"""Produce a dense caption with scene description and timestamped events."""
|
||||||
|
if not os.path.exists(video_path):
|
||||||
|
raise FileNotFoundError(f"Video file not found: {video_path!r}")
|
||||||
|
|
||||||
|
raw_result: dict = self._model.caption(
|
||||||
|
video_path,
|
||||||
|
max_new_tokens=max_new_tokens,
|
||||||
|
do_sample=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
events = [
|
||||||
|
VideoEvent(
|
||||||
|
start=float(ev["start"]),
|
||||||
|
end=float(ev["end"]),
|
||||||
|
description=str(ev["description"]),
|
||||||
|
)
|
||||||
|
for ev in raw_result.get("events", [])
|
||||||
|
]
|
||||||
|
|
||||||
|
return CaptionResult(
|
||||||
|
scene=str(raw_result.get("scene", "")),
|
||||||
|
events=events,
|
||||||
|
caption=str(raw_result.get("caption", "")),
|
||||||
|
model=self.model_name,
|
||||||
|
)
|
||||||
|
|
||||||
|
def find(
|
||||||
|
self,
|
||||||
|
video_path: str,
|
||||||
|
event: str,
|
||||||
|
*,
|
||||||
|
max_new_tokens: int = 256,
|
||||||
|
) -> FindResult:
|
||||||
|
"""Ground a natural-language event query to a video time span."""
|
||||||
|
if not os.path.exists(video_path):
|
||||||
|
raise FileNotFoundError(f"Video file not found: {video_path!r}")
|
||||||
|
if not event.strip():
|
||||||
|
raise ValueError("event query must not be empty")
|
||||||
|
|
||||||
|
raw_result: dict = self._model.find(
|
||||||
|
video_path,
|
||||||
|
event=event,
|
||||||
|
max_new_tokens=max_new_tokens,
|
||||||
|
do_sample=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Marlin returns span as a (start, end) tuple or None.
|
||||||
|
raw_span = raw_result.get("span")
|
||||||
|
span: tuple[float, float] | None = None
|
||||||
|
if raw_span is not None:
|
||||||
|
try:
|
||||||
|
span = (float(raw_span[0]), float(raw_span[1]))
|
||||||
|
except (TypeError, IndexError, ValueError):
|
||||||
|
logger.warning(
|
||||||
|
"MarlinBackend.find: could not parse span %r for event %r",
|
||||||
|
raw_span, event,
|
||||||
|
)
|
||||||
|
|
||||||
|
return FindResult(
|
||||||
|
span=span,
|
||||||
|
format_ok=bool(raw_result.get("format_ok", False)),
|
||||||
|
raw=str(raw_result.get("raw", "")),
|
||||||
|
model=self.model_name,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ── Properties ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@property
|
||||||
|
def model_name(self) -> str:
|
||||||
|
return self._model_path
|
||||||
|
|
||||||
|
@property
|
||||||
|
def vram_mb(self) -> int:
|
||||||
|
return self._vram_mb
|
||||||
68
circuitforge_core/video/backends/mock.py
Normal file
68
circuitforge_core/video/backends/mock.py
Normal file
|
|
@ -0,0 +1,68 @@
|
||||||
|
"""
|
||||||
|
MockVideoBackend — deterministic stub for unit tests and CI.
|
||||||
|
|
||||||
|
Returns fixed CaptionResult / FindResult without any model or video I/O.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
from circuitforge_core.video.backends.base import (
|
||||||
|
CaptionResult,
|
||||||
|
FindResult,
|
||||||
|
VideoEvent,
|
||||||
|
)
|
||||||
|
|
||||||
|
_MOCK_SCENE = "A mock scene with placeholder content."
|
||||||
|
_MOCK_EVENTS = [
|
||||||
|
VideoEvent(start=0.0, end=3.0, description="Mock event one"),
|
||||||
|
VideoEvent(start=3.5, end=7.2, description="Mock event two"),
|
||||||
|
]
|
||||||
|
_MOCK_CAPTION = "Scene: A mock scene with placeholder content. Events: [0.0-3.0] Mock event one. [3.5-7.2] Mock event two."
|
||||||
|
_MOCK_FIND_SPAN = (3.5, 7.2)
|
||||||
|
|
||||||
|
|
||||||
|
class MockVideoBackend:
|
||||||
|
"""No-GPU stub. Safe for import on any machine."""
|
||||||
|
|
||||||
|
def __init__(self, model_path: str = "mock") -> None:
|
||||||
|
self._model_path = model_path
|
||||||
|
|
||||||
|
def caption(
|
||||||
|
self,
|
||||||
|
video_path: str,
|
||||||
|
*,
|
||||||
|
max_new_tokens: int = 2048,
|
||||||
|
) -> CaptionResult:
|
||||||
|
if not os.path.exists(video_path):
|
||||||
|
raise FileNotFoundError(f"Video not found: {video_path!r}")
|
||||||
|
return CaptionResult(
|
||||||
|
scene=_MOCK_SCENE,
|
||||||
|
events=list(_MOCK_EVENTS),
|
||||||
|
caption=_MOCK_CAPTION,
|
||||||
|
model=self.model_name,
|
||||||
|
)
|
||||||
|
|
||||||
|
def find(
|
||||||
|
self,
|
||||||
|
video_path: str,
|
||||||
|
event: str,
|
||||||
|
*,
|
||||||
|
max_new_tokens: int = 256,
|
||||||
|
) -> FindResult:
|
||||||
|
if not os.path.exists(video_path):
|
||||||
|
raise FileNotFoundError(f"Video not found: {video_path!r}")
|
||||||
|
return FindResult(
|
||||||
|
span=_MOCK_FIND_SPAN,
|
||||||
|
format_ok=True,
|
||||||
|
raw="From 3.5 to 7.2.",
|
||||||
|
model=self.model_name,
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def model_name(self) -> str:
|
||||||
|
return self._model_path
|
||||||
|
|
||||||
|
@property
|
||||||
|
def vram_mb(self) -> int:
|
||||||
|
return 0
|
||||||
|
|
@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "circuitforge-core"
|
name = "circuitforge-core"
|
||||||
version = "0.17.0"
|
version = "0.21.0"
|
||||||
description = "Shared scaffold for CircuitForge products (MIT)"
|
description = "Shared scaffold for CircuitForge products (MIT)"
|
||||||
requires-python = ">=3.11"
|
requires-python = ">=3.11"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
|
@ -66,6 +66,21 @@ musicgen-service = [
|
||||||
"uvicorn[standard]>=0.29",
|
"uvicorn[standard]>=0.29",
|
||||||
"python-multipart>=0.0.9",
|
"python-multipart>=0.0.9",
|
||||||
]
|
]
|
||||||
|
video-marlin = [
|
||||||
|
"torch>=2.11",
|
||||||
|
"transformers>=5.7.0",
|
||||||
|
"torchvision", # On CUDA 13 nodes install from PyTorch nightly: pip install --index-url https://download.pytorch.org/whl/nightly/cu130 torch torchvision
|
||||||
|
"torchcodec",
|
||||||
|
"qwen-vl-utils>=0.0.14",
|
||||||
|
"av",
|
||||||
|
"Pillow>=10.0",
|
||||||
|
"accelerate>=0.27",
|
||||||
|
]
|
||||||
|
video-service = [
|
||||||
|
"circuitforge-core[video-marlin]",
|
||||||
|
"fastapi>=0.110",
|
||||||
|
"uvicorn[standard]>=0.29",
|
||||||
|
]
|
||||||
vision-siglip = [
|
vision-siglip = [
|
||||||
"torch>=2.0",
|
"torch>=2.0",
|
||||||
"transformers>=4.40",
|
"transformers>=4.40",
|
||||||
|
|
@ -102,6 +117,31 @@ reranker-service = [
|
||||||
"fastapi>=0.110",
|
"fastapi>=0.110",
|
||||||
"uvicorn[standard]>=0.29",
|
"uvicorn[standard]>=0.29",
|
||||||
]
|
]
|
||||||
|
gestures-mediapipe = [
|
||||||
|
"mediapipe>=0.10",
|
||||||
|
"opencv-python>=4.8",
|
||||||
|
"numpy>=1.24",
|
||||||
|
]
|
||||||
|
pdf = [
|
||||||
|
"pdfplumber>=0.11",
|
||||||
|
"pytesseract>=0.3",
|
||||||
|
"Pillow>=10.0",
|
||||||
|
]
|
||||||
|
vector = [
|
||||||
|
"sqlite-vec>=0.1",
|
||||||
|
]
|
||||||
|
mqtt = [
|
||||||
|
"aiomqtt>=2.0",
|
||||||
|
]
|
||||||
|
meshtastic-serial = [
|
||||||
|
"meshtastic>=2.5",
|
||||||
|
"pypubsub>=4.0",
|
||||||
|
]
|
||||||
|
meshtastic-service = [
|
||||||
|
"circuitforge-core[mqtt,meshtastic-serial]",
|
||||||
|
"fastapi>=0.110",
|
||||||
|
"uvicorn[standard]>=0.29",
|
||||||
|
]
|
||||||
dev = [
|
dev = [
|
||||||
"circuitforge-core[manage]",
|
"circuitforge-core[manage]",
|
||||||
"pytest>=8.0",
|
"pytest>=8.0",
|
||||||
|
|
|
||||||
107
tests/test_documents/test_pdf.py
Normal file
107
tests/test_documents/test_pdf.py
Normal file
|
|
@ -0,0 +1,107 @@
|
||||||
|
# tests/test_documents/test_pdf.py
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from circuitforge_core.documents.pdf import PDFExtractor, PageChunk
|
||||||
|
|
||||||
|
|
||||||
|
def _mock_page(text: str) -> MagicMock:
|
||||||
|
page = MagicMock()
|
||||||
|
page.extract_text.return_value = text
|
||||||
|
return page
|
||||||
|
|
||||||
|
|
||||||
|
def _mock_pdf(pages: list[MagicMock]) -> MagicMock:
|
||||||
|
pdf = MagicMock()
|
||||||
|
pdf.__enter__ = MagicMock(return_value=pdf)
|
||||||
|
pdf.__exit__ = MagicMock(return_value=False)
|
||||||
|
pdf.pages = pages
|
||||||
|
return pdf
|
||||||
|
|
||||||
|
|
||||||
|
def test_chunk_pages_single_text_layer_page():
|
||||||
|
page = _mock_page(
|
||||||
|
"Fireball deals 8d6 fire damage on a failed Dexterity saving throw."
|
||||||
|
)
|
||||||
|
with patch("circuitforge_core.documents.pdf.pdfplumber") as mock_pl:
|
||||||
|
mock_pl.open.return_value = _mock_pdf([page])
|
||||||
|
chunks = PDFExtractor().chunk_pages("/fake/book.pdf")
|
||||||
|
assert len(chunks) == 1
|
||||||
|
assert chunks[0].page_number == 1
|
||||||
|
assert chunks[0].source == "text_layer"
|
||||||
|
assert "Fireball" in chunks[0].text
|
||||||
|
assert chunks[0].word_count >= 10
|
||||||
|
|
||||||
|
|
||||||
|
def test_chunk_pages_numbers_from_one():
|
||||||
|
pages = [_mock_page(f"Rule text for page {i} " * 10) for i in range(1, 4)]
|
||||||
|
with patch("circuitforge_core.documents.pdf.pdfplumber") as mock_pl:
|
||||||
|
mock_pl.open.return_value = _mock_pdf(pages)
|
||||||
|
chunks = PDFExtractor().chunk_pages("/fake/book.pdf")
|
||||||
|
assert [c.page_number for c in chunks] == [1, 2, 3]
|
||||||
|
|
||||||
|
|
||||||
|
def test_page_chunk_is_frozen():
|
||||||
|
chunk = PageChunk(page_number=1, text="hello", source="text_layer", word_count=1)
|
||||||
|
with pytest.raises(Exception):
|
||||||
|
chunk.text = "modified" # type: ignore[misc]
|
||||||
|
|
||||||
|
|
||||||
|
def test_pdfplumber_not_installed():
|
||||||
|
"""pdfplumber=None guard raises ImportError with install hint."""
|
||||||
|
import circuitforge_core.documents.pdf as pdf_mod
|
||||||
|
|
||||||
|
with patch.object(pdf_mod, "pdfplumber", None):
|
||||||
|
with pytest.raises(ImportError, match="pdfplumber"):
|
||||||
|
PDFExtractor().chunk_pages("/fake/book.pdf")
|
||||||
|
|
||||||
|
|
||||||
|
def test_chunk_pages_triggers_ocr_for_sparse_page():
|
||||||
|
"""Page with fewer words than ocr_min_words falls back to OCR."""
|
||||||
|
sparse_page = _mock_page("few words only") # 3 words < default 10
|
||||||
|
mock_image = MagicMock()
|
||||||
|
rendered = MagicMock()
|
||||||
|
rendered.original = mock_image
|
||||||
|
|
||||||
|
sparse_page.to_image.return_value = rendered
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch("circuitforge_core.documents.pdf.pdfplumber") as mock_pl,
|
||||||
|
patch("circuitforge_core.documents.pdf.pytesseract") as mock_tess,
|
||||||
|
patch("circuitforge_core.documents.pdf.Image") as mock_pil,
|
||||||
|
):
|
||||||
|
mock_pl.open.return_value = _mock_pdf([sparse_page])
|
||||||
|
mock_pil.open.return_value = mock_image
|
||||||
|
mock_tess.image_to_string.return_value = (
|
||||||
|
"Full OCR extracted rulebook text about saving throws."
|
||||||
|
)
|
||||||
|
|
||||||
|
chunks = PDFExtractor(ocr_min_words=10).chunk_pages("/fake/scan.pdf")
|
||||||
|
|
||||||
|
assert chunks[0].source == "ocr"
|
||||||
|
assert "OCR extracted" in chunks[0].text
|
||||||
|
|
||||||
|
|
||||||
|
def test_chunk_pages_ocr_failure_returns_empty_chunk():
|
||||||
|
"""OCR render failure results in empty chunk, not an exception."""
|
||||||
|
sparse_page = _mock_page("")
|
||||||
|
sparse_page.to_image.side_effect = RuntimeError("render failed")
|
||||||
|
|
||||||
|
with patch("circuitforge_core.documents.pdf.pdfplumber") as mock_pl:
|
||||||
|
mock_pl.open.return_value = _mock_pdf([sparse_page])
|
||||||
|
chunks = PDFExtractor().chunk_pages("/fake/broken.pdf")
|
||||||
|
|
||||||
|
assert len(chunks) == 1
|
||||||
|
assert chunks[0].text == ""
|
||||||
|
assert chunks[0].source == "ocr"
|
||||||
|
assert chunks[0].word_count == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_chunk_pages_empty_pdf_returns_empty_list():
|
||||||
|
with patch("circuitforge_core.documents.pdf.pdfplumber") as mock_pl:
|
||||||
|
mock_pl.open.return_value = _mock_pdf([])
|
||||||
|
chunks = PDFExtractor().chunk_pages("/fake/empty.pdf")
|
||||||
|
assert chunks == []
|
||||||
0
tests/test_input/__init__.py
Normal file
0
tests/test_input/__init__.py
Normal file
0
tests/test_input/test_gestures/__init__.py
Normal file
0
tests/test_input/test_gestures/__init__.py
Normal file
48
tests/test_input/test_gestures/test_camera.py
Normal file
48
tests/test_input/test_gestures/test_camera.py
Normal file
|
|
@ -0,0 +1,48 @@
|
||||||
|
import numpy as np
|
||||||
|
import pytest
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
|
||||||
|
@patch("circuitforge_core.input.gestures.camera.cv2")
|
||||||
|
def test_is_open_reflects_videocapture_state(mock_cv2):
|
||||||
|
from circuitforge_core.input.gestures.camera import CameraCapture
|
||||||
|
|
||||||
|
mock_cv2.VideoCapture.return_value.isOpened.return_value = True
|
||||||
|
cam = CameraCapture()
|
||||||
|
assert cam.is_open is True
|
||||||
|
|
||||||
|
mock_cv2.VideoCapture.return_value.isOpened.return_value = False
|
||||||
|
cam2 = CameraCapture()
|
||||||
|
assert cam2.is_open is False
|
||||||
|
|
||||||
|
|
||||||
|
@patch("circuitforge_core.input.gestures.camera.cv2")
|
||||||
|
def test_frames_yields_until_read_fails(mock_cv2):
|
||||||
|
from circuitforge_core.input.gestures.camera import CameraCapture
|
||||||
|
|
||||||
|
frame = np.zeros((480, 640, 3), dtype=np.uint8)
|
||||||
|
mock_cap = MagicMock()
|
||||||
|
mock_cap.isOpened.return_value = True
|
||||||
|
mock_cap.read.side_effect = [
|
||||||
|
(True, frame),
|
||||||
|
(True, frame),
|
||||||
|
(False, None), # triggers break
|
||||||
|
]
|
||||||
|
mock_cv2.VideoCapture.return_value = mock_cap
|
||||||
|
|
||||||
|
cam = CameraCapture()
|
||||||
|
collected = list(cam.frames())
|
||||||
|
assert len(collected) == 2
|
||||||
|
|
||||||
|
|
||||||
|
@patch("circuitforge_core.input.gestures.camera.cv2")
|
||||||
|
def test_context_manager_calls_release(mock_cv2):
|
||||||
|
from circuitforge_core.input.gestures.camera import CameraCapture
|
||||||
|
|
||||||
|
mock_cap = MagicMock()
|
||||||
|
mock_cv2.VideoCapture.return_value = mock_cap
|
||||||
|
|
||||||
|
with CameraCapture() as cam:
|
||||||
|
pass
|
||||||
|
|
||||||
|
mock_cap.release.assert_called_once()
|
||||||
106
tests/test_input/test_gestures/test_hands.py
Normal file
106
tests/test_input/test_gestures/test_hands.py
Normal file
|
|
@ -0,0 +1,106 @@
|
||||||
|
import numpy as np
|
||||||
|
import pytest
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
from circuitforge_core.input.gestures.hands import HandsDetector, HandLandmarks
|
||||||
|
|
||||||
|
|
||||||
|
def _make_mock_results(n_hands: int = 1):
|
||||||
|
"""Build a fake mediapipe result object with n_hands detected."""
|
||||||
|
mock_results = MagicMock()
|
||||||
|
if n_hands == 0:
|
||||||
|
mock_results.multi_hand_landmarks = None
|
||||||
|
mock_results.multi_handedness = None
|
||||||
|
return mock_results
|
||||||
|
|
||||||
|
hand_landmarks = []
|
||||||
|
handedness_list = []
|
||||||
|
for i in range(n_hands):
|
||||||
|
lm = MagicMock()
|
||||||
|
lm.landmark = [
|
||||||
|
MagicMock(x=float(j) / 100, y=float(j) / 200, z=0.0) for j in range(21)
|
||||||
|
]
|
||||||
|
hand_landmarks.append(lm)
|
||||||
|
|
||||||
|
hand = MagicMock()
|
||||||
|
hand.classification = [
|
||||||
|
MagicMock(label="Right" if i == 0 else "Left", score=0.95)
|
||||||
|
]
|
||||||
|
handedness_list.append(hand)
|
||||||
|
|
||||||
|
mock_results.multi_hand_landmarks = hand_landmarks
|
||||||
|
mock_results.multi_handedness = handedness_list
|
||||||
|
return mock_results
|
||||||
|
|
||||||
|
|
||||||
|
@patch("circuitforge_core.input.gestures.hands.mp")
|
||||||
|
def test_detect_returns_empty_when_no_hands(mock_mp):
|
||||||
|
mock_mp.solutions.hands.Hands.return_value.process.return_value = (
|
||||||
|
_make_mock_results(0)
|
||||||
|
)
|
||||||
|
detector = HandsDetector()
|
||||||
|
frame = np.zeros((480, 640, 3), dtype=np.uint8)
|
||||||
|
results = detector.detect(frame)
|
||||||
|
assert results == []
|
||||||
|
|
||||||
|
|
||||||
|
@patch("circuitforge_core.input.gestures.hands.mp")
|
||||||
|
def test_detect_returns_one_hand(mock_mp):
|
||||||
|
mock_mp.solutions.hands.Hands.return_value.process.return_value = (
|
||||||
|
_make_mock_results(1)
|
||||||
|
)
|
||||||
|
detector = HandsDetector()
|
||||||
|
frame = np.zeros((480, 640, 3), dtype=np.uint8)
|
||||||
|
results = detector.detect(frame)
|
||||||
|
assert len(results) == 1
|
||||||
|
h = results[0]
|
||||||
|
assert isinstance(h, HandLandmarks)
|
||||||
|
assert h.points.shape == (21, 3)
|
||||||
|
assert h.handedness == "Right"
|
||||||
|
assert 0.0 <= h.confidence <= 1.0
|
||||||
|
|
||||||
|
|
||||||
|
@patch("circuitforge_core.input.gestures.hands.mp")
|
||||||
|
def test_detect_returns_two_hands(mock_mp):
|
||||||
|
mock_mp.solutions.hands.Hands.return_value.process.return_value = (
|
||||||
|
_make_mock_results(2)
|
||||||
|
)
|
||||||
|
detector = HandsDetector()
|
||||||
|
frame = np.zeros((480, 640, 3), dtype=np.uint8)
|
||||||
|
results = detector.detect(frame)
|
||||||
|
assert len(results) == 2
|
||||||
|
|
||||||
|
|
||||||
|
@patch("circuitforge_core.input.gestures.hands.mp")
|
||||||
|
def test_handlandmarks_is_immutable(mock_mp):
|
||||||
|
mock_mp.solutions.hands.Hands.return_value.process.return_value = (
|
||||||
|
_make_mock_results(1)
|
||||||
|
)
|
||||||
|
detector = HandsDetector()
|
||||||
|
frame = np.zeros((480, 640, 3), dtype=np.uint8)
|
||||||
|
result = detector.detect(frame)[0]
|
||||||
|
with pytest.raises((AttributeError, TypeError)):
|
||||||
|
result.handedness = (
|
||||||
|
"Left" # frozen dataclass must reject attribute reassignment
|
||||||
|
)
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
result.points[0] = np.array(
|
||||||
|
[1.0, 2.0, 3.0]
|
||||||
|
) # writeable=False must reject in-place mutation
|
||||||
|
|
||||||
|
|
||||||
|
@patch("circuitforge_core.input.gestures.hands.mp")
|
||||||
|
def test_full_pipeline_hands_to_normalized_vector(mock_mp):
|
||||||
|
"""Detect hand → normalize landmarks → get 63-element vector."""
|
||||||
|
from circuitforge_core.input.gestures.normalizer import normalize_hand
|
||||||
|
|
||||||
|
mock_mp.solutions.hands.Hands.return_value.process.return_value = (
|
||||||
|
_make_mock_results(1)
|
||||||
|
)
|
||||||
|
detector = HandsDetector()
|
||||||
|
frame = np.zeros((480, 640, 3), dtype=np.uint8)
|
||||||
|
hands = detector.detect(frame)
|
||||||
|
assert len(hands) == 1
|
||||||
|
vec = normalize_hand(hands[0].points)
|
||||||
|
assert vec.shape == (63,)
|
||||||
|
assert vec.dtype == np.float32
|
||||||
|
assert not np.any(np.isnan(vec))
|
||||||
51
tests/test_input/test_gestures/test_normalizer.py
Normal file
51
tests/test_input/test_gestures/test_normalizer.py
Normal file
|
|
@ -0,0 +1,51 @@
|
||||||
|
import numpy as np
|
||||||
|
import pytest
|
||||||
|
from circuitforge_core.input.gestures.normalizer import normalize_hand
|
||||||
|
|
||||||
|
|
||||||
|
def _synthetic_hand(scale: float = 1.0, offset: float = 0.0) -> np.ndarray:
|
||||||
|
"""21 landmarks, wrist at offset, middle MCP at offset + (scale, 0, 0)."""
|
||||||
|
pts = np.zeros((21, 3), dtype=np.float32)
|
||||||
|
# All landmarks start at the offset (roughly at the wrist)
|
||||||
|
for i in range(21):
|
||||||
|
pts[i] = [offset, 0.0, 0.0]
|
||||||
|
# Then define a few key landmarks relative to wrist
|
||||||
|
pts[0] = [offset, 0.0, 0.0] # wrist
|
||||||
|
pts[9] = [offset + scale, 0.0, 0.0] # middle MCP at distance scale from wrist
|
||||||
|
pts[1] = [offset + 0.1 * scale, 0.05 * scale, 0.0] # thumb
|
||||||
|
pts[5] = [offset + 0.4 * scale, 0.2 * scale, 0.0] # index
|
||||||
|
return pts
|
||||||
|
|
||||||
|
|
||||||
|
def test_output_shape():
|
||||||
|
pts = _synthetic_hand()
|
||||||
|
result = normalize_hand(pts)
|
||||||
|
assert result.shape == (63,)
|
||||||
|
|
||||||
|
|
||||||
|
def test_translation_invariance():
|
||||||
|
pts_a = _synthetic_hand(offset=0.0)
|
||||||
|
pts_b = _synthetic_hand(offset=5.0)
|
||||||
|
np.testing.assert_allclose(normalize_hand(pts_a), normalize_hand(pts_b), atol=1e-5)
|
||||||
|
|
||||||
|
|
||||||
|
def test_scale_invariance():
|
||||||
|
pts_small = _synthetic_hand(scale=0.5)
|
||||||
|
pts_large = _synthetic_hand(scale=2.0)
|
||||||
|
np.testing.assert_allclose(
|
||||||
|
normalize_hand(pts_small), normalize_hand(pts_large), atol=1e-5
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_zero_scale_does_not_crash():
|
||||||
|
"""All landmarks at same point — degenerate hand. Should return zeros, not raise."""
|
||||||
|
pts = np.zeros((21, 3), dtype=np.float32)
|
||||||
|
result = normalize_hand(pts)
|
||||||
|
assert result.shape == (63,)
|
||||||
|
assert not np.any(np.isnan(result))
|
||||||
|
|
||||||
|
|
||||||
|
def test_dtype_is_float32():
|
||||||
|
pts = _synthetic_hand()
|
||||||
|
result = normalize_hand(pts)
|
||||||
|
assert result.dtype == np.float32
|
||||||
|
|
@ -11,69 +11,81 @@ def _make_router(config: dict) -> LLMRouter:
|
||||||
|
|
||||||
|
|
||||||
def test_complete_uses_first_reachable_backend():
|
def test_complete_uses_first_reachable_backend():
|
||||||
router = _make_router({
|
router = _make_router(
|
||||||
"fallback_order": ["local"],
|
{
|
||||||
"backends": {
|
"fallback_order": ["local"],
|
||||||
"local": {
|
"backends": {
|
||||||
"type": "openai_compat",
|
"local": {
|
||||||
"base_url": "http://localhost:11434/v1",
|
"type": "openai_compat",
|
||||||
"model": "llama3",
|
"base_url": "http://localhost:11434/v1",
|
||||||
"supports_images": False,
|
"model": "llama3",
|
||||||
}
|
"supports_images": False,
|
||||||
|
}
|
||||||
|
},
|
||||||
}
|
}
|
||||||
})
|
)
|
||||||
mock_client = MagicMock()
|
mock_client = MagicMock()
|
||||||
mock_client.chat.completions.create.return_value = MagicMock(
|
mock_client.chat.completions.create.return_value = MagicMock(
|
||||||
choices=[MagicMock(message=MagicMock(content="hello"))]
|
choices=[MagicMock(message=MagicMock(content="hello"))]
|
||||||
)
|
)
|
||||||
with patch.object(router, "_is_reachable", return_value=True), \
|
with (
|
||||||
patch("circuitforge_core.llm.router.OpenAI", return_value=mock_client):
|
patch.object(router, "_is_reachable", return_value=True),
|
||||||
|
patch("circuitforge_core.llm.router.OpenAI", return_value=mock_client),
|
||||||
|
):
|
||||||
result = router.complete("say hello")
|
result = router.complete("say hello")
|
||||||
assert result == "hello"
|
assert result == "hello"
|
||||||
|
|
||||||
|
|
||||||
def test_complete_falls_back_on_unreachable_backend():
|
def test_complete_falls_back_on_unreachable_backend():
|
||||||
router = _make_router({
|
router = _make_router(
|
||||||
"fallback_order": ["unreachable", "working"],
|
{
|
||||||
"backends": {
|
"fallback_order": ["unreachable", "working"],
|
||||||
"unreachable": {
|
"backends": {
|
||||||
"type": "openai_compat",
|
"unreachable": {
|
||||||
"base_url": "http://nowhere:1/v1",
|
"type": "openai_compat",
|
||||||
"model": "x",
|
"base_url": "http://nowhere:1/v1",
|
||||||
"supports_images": False,
|
"model": "x",
|
||||||
|
"supports_images": False,
|
||||||
|
},
|
||||||
|
"working": {
|
||||||
|
"type": "openai_compat",
|
||||||
|
"base_url": "http://localhost:11434/v1",
|
||||||
|
"model": "llama3",
|
||||||
|
"supports_images": False,
|
||||||
|
},
|
||||||
},
|
},
|
||||||
"working": {
|
|
||||||
"type": "openai_compat",
|
|
||||||
"base_url": "http://localhost:11434/v1",
|
|
||||||
"model": "llama3",
|
|
||||||
"supports_images": False,
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
})
|
)
|
||||||
mock_client = MagicMock()
|
mock_client = MagicMock()
|
||||||
mock_client.chat.completions.create.return_value = MagicMock(
|
mock_client.chat.completions.create.return_value = MagicMock(
|
||||||
choices=[MagicMock(message=MagicMock(content="fallback"))]
|
choices=[MagicMock(message=MagicMock(content="fallback"))]
|
||||||
)
|
)
|
||||||
|
|
||||||
def reachable(url):
|
def reachable(url):
|
||||||
return "nowhere" not in url
|
return "nowhere" not in url
|
||||||
with patch.object(router, "_is_reachable", side_effect=reachable), \
|
|
||||||
patch("circuitforge_core.llm.router.OpenAI", return_value=mock_client):
|
with (
|
||||||
|
patch.object(router, "_is_reachable", side_effect=reachable),
|
||||||
|
patch("circuitforge_core.llm.router.OpenAI", return_value=mock_client),
|
||||||
|
):
|
||||||
result = router.complete("test")
|
result = router.complete("test")
|
||||||
assert result == "fallback"
|
assert result == "fallback"
|
||||||
|
|
||||||
|
|
||||||
def test_complete_raises_when_all_backends_exhausted():
|
def test_complete_raises_when_all_backends_exhausted():
|
||||||
router = _make_router({
|
router = _make_router(
|
||||||
"fallback_order": ["dead"],
|
{
|
||||||
"backends": {
|
"fallback_order": ["dead"],
|
||||||
"dead": {
|
"backends": {
|
||||||
"type": "openai_compat",
|
"dead": {
|
||||||
"base_url": "http://nowhere:1/v1",
|
"type": "openai_compat",
|
||||||
"model": "x",
|
"base_url": "http://nowhere:1/v1",
|
||||||
"supports_images": False,
|
"model": "x",
|
||||||
}
|
"supports_images": False,
|
||||||
|
}
|
||||||
|
},
|
||||||
}
|
}
|
||||||
})
|
)
|
||||||
with patch.object(router, "_is_reachable", return_value=False):
|
with patch.object(router, "_is_reachable", return_value=False):
|
||||||
with pytest.raises(RuntimeError, match="exhausted"):
|
with pytest.raises(RuntimeError, match="exhausted"):
|
||||||
router.complete("test")
|
router.complete("test")
|
||||||
|
|
@ -83,6 +95,242 @@ def test_try_cf_orch_alloc_import_path():
|
||||||
"""Verify lazy import points to circuitforge_orch, not circuitforge_core.resources."""
|
"""Verify lazy import points to circuitforge_orch, not circuitforge_core.resources."""
|
||||||
import inspect
|
import inspect
|
||||||
from circuitforge_core.llm import router as router_module
|
from circuitforge_core.llm import router as router_module
|
||||||
|
|
||||||
src = inspect.getsource(router_module.LLMRouter._try_cf_orch_alloc)
|
src = inspect.getsource(router_module.LLMRouter._try_cf_orch_alloc)
|
||||||
assert "circuitforge_orch.client" in src
|
assert "circuitforge_orch.client" in src
|
||||||
assert "circuitforge_core.resources.client" not in src
|
assert "circuitforge_core.resources.client" not in src
|
||||||
|
|
||||||
|
|
||||||
|
def test_embed_returns_vectors_from_openai_compat_backend():
|
||||||
|
router = _make_router(
|
||||||
|
{
|
||||||
|
"fallback_order": ["ollama"],
|
||||||
|
"backends": {
|
||||||
|
"ollama": {
|
||||||
|
"type": "openai_compat",
|
||||||
|
"base_url": "http://localhost:11434/v1",
|
||||||
|
"model": "mistral:7b",
|
||||||
|
"embedding_model": "nomic-embed-text",
|
||||||
|
"supports_images": False,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
mock_client = MagicMock()
|
||||||
|
mock_client.embeddings.create.return_value = MagicMock(
|
||||||
|
data=[
|
||||||
|
MagicMock(embedding=[0.1, 0.2, 0.3]),
|
||||||
|
MagicMock(embedding=[0.4, 0.5, 0.6]),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
with (
|
||||||
|
patch.object(router, "_is_reachable", return_value=True),
|
||||||
|
patch("circuitforge_core.llm.router.requests.get", return_value=MagicMock(status_code=404)),
|
||||||
|
patch("circuitforge_core.llm.router.OpenAI", return_value=mock_client),
|
||||||
|
):
|
||||||
|
result = router.embed(["hello world", "fireball rules"])
|
||||||
|
|
||||||
|
assert result == [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]]
|
||||||
|
mock_client.embeddings.create.assert_called_once_with(
|
||||||
|
model="nomic-embed-text",
|
||||||
|
input=["hello world", "fireball rules"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_embed_uses_chat_model_when_no_embedding_model_configured():
|
||||||
|
router = _make_router(
|
||||||
|
{
|
||||||
|
"fallback_order": ["ollama"],
|
||||||
|
"backends": {
|
||||||
|
"ollama": {
|
||||||
|
"type": "openai_compat",
|
||||||
|
"base_url": "http://localhost:11434/v1",
|
||||||
|
"model": "llama3",
|
||||||
|
"supports_images": False,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
mock_client = MagicMock()
|
||||||
|
mock_client.embeddings.create.return_value = MagicMock(
|
||||||
|
data=[MagicMock(embedding=[0.9, 0.8])]
|
||||||
|
)
|
||||||
|
with (
|
||||||
|
patch.object(router, "_is_reachable", return_value=True),
|
||||||
|
patch("circuitforge_core.llm.router.requests.get", return_value=MagicMock(status_code=404)),
|
||||||
|
patch("circuitforge_core.llm.router.OpenAI", return_value=mock_client),
|
||||||
|
):
|
||||||
|
router.embed(["test"])
|
||||||
|
|
||||||
|
call_kwargs = mock_client.embeddings.create.call_args
|
||||||
|
assert call_kwargs.kwargs["model"] == "llama3"
|
||||||
|
|
||||||
|
|
||||||
|
def test_embed_skips_non_openai_compat_backends():
|
||||||
|
router = _make_router(
|
||||||
|
{
|
||||||
|
"fallback_order": ["anthropic", "ollama"],
|
||||||
|
"backends": {
|
||||||
|
"anthropic": {
|
||||||
|
"type": "anthropic",
|
||||||
|
"enabled": True,
|
||||||
|
"model": "claude-haiku-4-5-20251001",
|
||||||
|
"api_key_env": "ANTHROPIC_API_KEY",
|
||||||
|
"supports_images": True,
|
||||||
|
},
|
||||||
|
"ollama": {
|
||||||
|
"type": "openai_compat",
|
||||||
|
"base_url": "http://localhost:11434/v1",
|
||||||
|
"model": "nomic-embed-text",
|
||||||
|
"supports_images": False,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
mock_client = MagicMock()
|
||||||
|
mock_client.embeddings.create.return_value = MagicMock(
|
||||||
|
data=[MagicMock(embedding=[0.1])]
|
||||||
|
)
|
||||||
|
mock_openai = MagicMock(return_value=mock_client)
|
||||||
|
with (
|
||||||
|
patch.object(router, "_is_reachable", return_value=True),
|
||||||
|
patch("circuitforge_core.llm.router.requests.get", return_value=MagicMock(status_code=404)),
|
||||||
|
patch("circuitforge_core.llm.router.OpenAI", mock_openai),
|
||||||
|
):
|
||||||
|
result = router.embed(["hello"])
|
||||||
|
|
||||||
|
assert result == [[0.1]]
|
||||||
|
# Only ollama reached the OpenAI constructor; anthropic was skipped by type check
|
||||||
|
mock_openai.assert_called_once()
|
||||||
|
|
||||||
|
|
||||||
|
def test_embed_raises_when_all_backends_exhausted():
|
||||||
|
router = _make_router(
|
||||||
|
{
|
||||||
|
"fallback_order": ["dead"],
|
||||||
|
"backends": {
|
||||||
|
"dead": {
|
||||||
|
"type": "openai_compat",
|
||||||
|
"base_url": "http://nowhere:1/v1",
|
||||||
|
"model": "x",
|
||||||
|
"supports_images": False,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
with patch.object(router, "_is_reachable", return_value=False):
|
||||||
|
with pytest.raises(RuntimeError, match="exhausted"):
|
||||||
|
router.embed(["test"])
|
||||||
|
|
||||||
|
|
||||||
|
# ── #59: LLMRouter dict init ──────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def test_init_accepts_inline_dict():
|
||||||
|
config = {
|
||||||
|
"fallback_order": ["local"],
|
||||||
|
"backends": {
|
||||||
|
"local": {
|
||||||
|
"type": "openai_compat",
|
||||||
|
"base_url": "http://localhost:11434/v1",
|
||||||
|
"model": "llama3",
|
||||||
|
"supports_images": False,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
router = LLMRouter(config)
|
||||||
|
assert router.config["fallback_order"] == ["local"]
|
||||||
|
assert "local" in router.config["backends"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_init_dict_is_used_directly():
|
||||||
|
config = {"fallback_order": [], "backends": {}}
|
||||||
|
router = LLMRouter(config)
|
||||||
|
assert router.config is config
|
||||||
|
|
||||||
|
|
||||||
|
# ── #60: Ollama embedding model preflight ─────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def _ollama_backend(model: str = "nomic-embed-text") -> dict:
|
||||||
|
return {
|
||||||
|
"fallback_order": ["ollama"],
|
||||||
|
"backends": {
|
||||||
|
"ollama": {
|
||||||
|
"type": "openai_compat",
|
||||||
|
"base_url": "http://localhost:11434/v1",
|
||||||
|
"embedding_model": model,
|
||||||
|
"model": "mistral:7b",
|
||||||
|
"supports_images": False,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_embed_raises_actionable_error_when_model_not_pulled():
|
||||||
|
router = _make_router(_ollama_backend("nomic-embed-text"))
|
||||||
|
tags_resp = MagicMock(status_code=200)
|
||||||
|
tags_resp.json.return_value = {"models": [{"name": "mistral:latest"}]}
|
||||||
|
with (
|
||||||
|
patch.object(router, "_is_reachable", return_value=True),
|
||||||
|
patch("circuitforge_core.llm.router.requests.get", return_value=tags_resp),
|
||||||
|
):
|
||||||
|
with pytest.raises(RuntimeError, match='ollama pull nomic-embed-text'):
|
||||||
|
router.embed(["hello"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_embed_proceeds_when_model_is_pulled():
|
||||||
|
router = _make_router(_ollama_backend("nomic-embed-text"))
|
||||||
|
tags_resp = MagicMock(status_code=200)
|
||||||
|
tags_resp.json.return_value = {
|
||||||
|
"models": [{"name": "nomic-embed-text:latest"}, {"name": "mistral:latest"}]
|
||||||
|
}
|
||||||
|
mock_client = MagicMock()
|
||||||
|
mock_client.embeddings.create.return_value = MagicMock(
|
||||||
|
data=[MagicMock(embedding=[0.1, 0.2])]
|
||||||
|
)
|
||||||
|
with (
|
||||||
|
patch.object(router, "_is_reachable", return_value=True),
|
||||||
|
patch("circuitforge_core.llm.router.requests.get", return_value=tags_resp),
|
||||||
|
patch("circuitforge_core.llm.router.OpenAI", return_value=mock_client),
|
||||||
|
):
|
||||||
|
result = router.embed(["hello"])
|
||||||
|
assert result == [[0.1, 0.2]]
|
||||||
|
|
||||||
|
|
||||||
|
def test_embed_skips_preflight_when_tags_endpoint_unavailable():
|
||||||
|
"""Non-Ollama backends (vLLM, etc.) don't expose /api/tags — check must be silent."""
|
||||||
|
router = _make_router(_ollama_backend("custom-embed"))
|
||||||
|
tags_resp = MagicMock(status_code=404)
|
||||||
|
mock_client = MagicMock()
|
||||||
|
mock_client.embeddings.create.return_value = MagicMock(
|
||||||
|
data=[MagicMock(embedding=[0.5])]
|
||||||
|
)
|
||||||
|
with (
|
||||||
|
patch.object(router, "_is_reachable", return_value=True),
|
||||||
|
patch("circuitforge_core.llm.router.requests.get", return_value=tags_resp),
|
||||||
|
patch("circuitforge_core.llm.router.OpenAI", return_value=mock_client),
|
||||||
|
):
|
||||||
|
result = router.embed(["hello"])
|
||||||
|
assert result == [[0.5]]
|
||||||
|
|
||||||
|
|
||||||
|
def test_ollama_tags_cache_is_hit_only_once():
|
||||||
|
router = _make_router(_ollama_backend("nomic-embed-text"))
|
||||||
|
tags_resp = MagicMock(status_code=200)
|
||||||
|
tags_resp.json.return_value = {"models": [{"name": "nomic-embed-text:latest"}]}
|
||||||
|
mock_client = MagicMock()
|
||||||
|
mock_client.embeddings.create.return_value = MagicMock(
|
||||||
|
data=[MagicMock(embedding=[0.1])]
|
||||||
|
)
|
||||||
|
with (
|
||||||
|
patch.object(router, "_is_reachable", return_value=True),
|
||||||
|
patch("circuitforge_core.llm.router.requests.get", return_value=tags_resp) as mock_get,
|
||||||
|
patch("circuitforge_core.llm.router.OpenAI", return_value=mock_client),
|
||||||
|
):
|
||||||
|
router.embed(["first"])
|
||||||
|
router.embed(["second"])
|
||||||
|
|
||||||
|
# /api/tags is called once (cache hit on second embed)
|
||||||
|
tags_calls = [c for c in mock_get.call_args_list if "api/tags" in str(c)]
|
||||||
|
assert len(tags_calls) == 1
|
||||||
|
|
|
||||||
0
tests/test_vector/__init__.py
Normal file
0
tests/test_vector/__init__.py
Normal file
102
tests/test_vector/test_base.py
Normal file
102
tests/test_vector/test_base.py
Normal file
|
|
@ -0,0 +1,102 @@
|
||||||
|
"""Tests for VectorStore ABC and VectorMatch."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import FrozenInstanceError
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from circuitforge_core.vector.base import VectorMatch, VectorStore
|
||||||
|
|
||||||
|
|
||||||
|
class _ConcreteStore(VectorStore):
|
||||||
|
"""Minimal in-memory implementation for testing the ABC contract."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._data: dict[str, tuple[list[float], dict]] = {}
|
||||||
|
|
||||||
|
def upsert(self, entry_id: str, vector: list[float], metadata: dict) -> None:
|
||||||
|
self._data[entry_id] = (vector, metadata)
|
||||||
|
|
||||||
|
def query(
|
||||||
|
self,
|
||||||
|
vector: list[float],
|
||||||
|
top_k: int = 10,
|
||||||
|
filter_metadata: dict | None = None,
|
||||||
|
) -> list[VectorMatch]:
|
||||||
|
results = [
|
||||||
|
VectorMatch(entry_id=k, score=0.0, metadata=v[1])
|
||||||
|
for k, v in self._data.items()
|
||||||
|
]
|
||||||
|
if filter_metadata:
|
||||||
|
results = [
|
||||||
|
r
|
||||||
|
for r in results
|
||||||
|
if all(r.metadata.get(k) == val for k, val in filter_metadata.items())
|
||||||
|
]
|
||||||
|
return results[:top_k]
|
||||||
|
|
||||||
|
def delete(self, entry_id: str) -> None:
|
||||||
|
self._data.pop(entry_id, None)
|
||||||
|
|
||||||
|
def delete_where(self, filter_metadata: dict) -> int:
|
||||||
|
to_remove = [
|
||||||
|
k
|
||||||
|
for k, (_, meta) in self._data.items()
|
||||||
|
if all(meta.get(fk) == fv for fk, fv in filter_metadata.items())
|
||||||
|
]
|
||||||
|
for k in to_remove:
|
||||||
|
del self._data[k]
|
||||||
|
return len(to_remove)
|
||||||
|
|
||||||
|
|
||||||
|
def test_vector_match_is_frozen():
|
||||||
|
match = VectorMatch(entry_id="a", score=0.1, metadata={})
|
||||||
|
with pytest.raises(FrozenInstanceError):
|
||||||
|
match.score = 0.5 # type: ignore[misc]
|
||||||
|
|
||||||
|
|
||||||
|
def test_vector_match_metadata_is_dict():
|
||||||
|
match = VectorMatch(entry_id="a", score=0.1, metadata={"k": "v"})
|
||||||
|
assert isinstance(match.metadata, dict)
|
||||||
|
assert match.metadata["k"] == "v"
|
||||||
|
|
||||||
|
|
||||||
|
def test_upsert_and_query():
|
||||||
|
store = _ConcreteStore()
|
||||||
|
store.upsert("chunk-1", [0.1, 0.2], {"doc_id": "book-a", "page": 1})
|
||||||
|
results = store.query([0.1, 0.2])
|
||||||
|
assert len(results) == 1
|
||||||
|
assert results[0].entry_id == "chunk-1"
|
||||||
|
assert results[0].metadata["page"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_query_filter_metadata():
|
||||||
|
store = _ConcreteStore()
|
||||||
|
store.upsert("c1", [0.1], {"doc_id": "book-a"})
|
||||||
|
store.upsert("c2", [0.2], {"doc_id": "book-b"})
|
||||||
|
results = store.query([0.1], filter_metadata={"doc_id": "book-a"})
|
||||||
|
assert len(results) == 1
|
||||||
|
assert results[0].entry_id == "c1"
|
||||||
|
|
||||||
|
|
||||||
|
def test_delete():
|
||||||
|
store = _ConcreteStore()
|
||||||
|
store.upsert("x", [0.1], {})
|
||||||
|
store.delete("x")
|
||||||
|
assert store.query([0.1]) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_delete_where():
|
||||||
|
store = _ConcreteStore()
|
||||||
|
store.upsert("c1", [0.1], {"doc_id": "book-a"})
|
||||||
|
store.upsert("c2", [0.2], {"doc_id": "book-a"})
|
||||||
|
store.upsert("c3", [0.3], {"doc_id": "book-b"})
|
||||||
|
count = store.delete_where({"doc_id": "book-a"})
|
||||||
|
assert count == 2
|
||||||
|
assert len(store.query([0.1])) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_cannot_instantiate_abc_directly():
|
||||||
|
with pytest.raises(TypeError):
|
||||||
|
VectorStore() # type: ignore[abstract]
|
||||||
82
tests/test_vector/test_sqlite_vec.py
Normal file
82
tests/test_vector/test_sqlite_vec.py
Normal file
|
|
@ -0,0 +1,82 @@
|
||||||
|
# tests/test_vector/test_sqlite_vec.py
|
||||||
|
"""Integration tests for LocalSQLiteVecStore (uses a real in-memory sqlite-vec DB)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from circuitforge_core.vector.sqlite_vec import LocalSQLiteVecStore
|
||||||
|
|
||||||
|
DIMS = 4 # small dimension for tests
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def store(tmp_path) -> LocalSQLiteVecStore:
|
||||||
|
return LocalSQLiteVecStore(db_path=tmp_path / "vecs.db", dimensions=DIMS)
|
||||||
|
|
||||||
|
|
||||||
|
def _vec(val: float) -> list[float]:
|
||||||
|
return [val] * DIMS
|
||||||
|
|
||||||
|
|
||||||
|
def test_upsert_and_query_returns_match(store):
|
||||||
|
store.upsert("doc-1::p1", _vec(0.1), {"doc_id": "doc-1", "page": 1})
|
||||||
|
results = store.query(_vec(0.1), top_k=5)
|
||||||
|
assert len(results) == 1
|
||||||
|
assert results[0].entry_id == "doc-1::p1"
|
||||||
|
assert results[0].metadata["page"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_upsert_replaces_existing(store):
|
||||||
|
store.upsert("chunk-1", _vec(0.1), {"page": 1})
|
||||||
|
store.upsert("chunk-1", _vec(0.9), {"page": 99})
|
||||||
|
# Metadata check
|
||||||
|
results = store.query(_vec(0.9), top_k=5)
|
||||||
|
assert results[0].metadata["page"] == 99
|
||||||
|
# Vector check: querying with new vector should score better than querying with old
|
||||||
|
old_results = store.query(_vec(0.1), top_k=5)
|
||||||
|
new_results = store.query(_vec(0.9), top_k=5)
|
||||||
|
assert new_results[0].score < old_results[0].score
|
||||||
|
|
||||||
|
|
||||||
|
def test_query_respects_top_k(store):
|
||||||
|
for i in range(5):
|
||||||
|
store.upsert(f"chunk-{i}", _vec(float(i) * 0.1), {"i": i})
|
||||||
|
results = store.query(_vec(0.0), top_k=2)
|
||||||
|
assert len(results) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_filter_metadata(store):
|
||||||
|
store.upsert("c1", _vec(0.1), {"doc_id": "book-a"})
|
||||||
|
store.upsert("c2", _vec(0.2), {"doc_id": "book-b"})
|
||||||
|
results = store.query(_vec(0.1), filter_metadata={"doc_id": "book-a"})
|
||||||
|
assert all(r.metadata["doc_id"] == "book-a" for r in results)
|
||||||
|
|
||||||
|
|
||||||
|
def test_delete(store):
|
||||||
|
store.upsert("x", _vec(0.5), {})
|
||||||
|
store.delete("x")
|
||||||
|
assert store.query(_vec(0.5)) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_delete_where(store):
|
||||||
|
store.upsert("c1", _vec(0.1), {"doc_id": "book-a"})
|
||||||
|
store.upsert("c2", _vec(0.2), {"doc_id": "book-a"})
|
||||||
|
store.upsert("c3", _vec(0.3), {"doc_id": "book-b"})
|
||||||
|
count = store.delete_where({"doc_id": "book-a"})
|
||||||
|
assert count == 2
|
||||||
|
assert len(store.query(_vec(0.1))) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_delete_nonexistent_is_noop(store):
|
||||||
|
store.delete("does-not-exist") # should not raise
|
||||||
|
|
||||||
|
|
||||||
|
def test_empty_query_returns_empty(store):
|
||||||
|
assert store.query(_vec(0.1)) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_delete_where_raises_on_empty_filter(store):
|
||||||
|
store.upsert("c1", _vec(0.1), {"doc_id": "book-a"})
|
||||||
|
with pytest.raises(ValueError, match="empty"):
|
||||||
|
store.delete_where({})
|
||||||
0
tests/test_video/__init__.py
Normal file
0
tests/test_video/__init__.py
Normal file
236
tests/test_video/test_app.py
Normal file
236
tests/test_video/test_app.py
Normal file
|
|
@ -0,0 +1,236 @@
|
||||||
|
"""
|
||||||
|
Tests for the cf-video FastAPI app using mock backend.
|
||||||
|
|
||||||
|
Tests run without GPU, torch, or a real video file.
|
||||||
|
MockVideoBackend checks os.path.exists() but never reads video content,
|
||||||
|
so a zero-byte placeholder is sufficient.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
import circuitforge_core.video.app as video_app
|
||||||
|
from circuitforge_core.video.backends.mock import MockVideoBackend
|
||||||
|
|
||||||
|
|
||||||
|
# ── Fixtures ──────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def inject_mock_backend():
|
||||||
|
"""Replace global backend with mock before each test; restore after."""
|
||||||
|
original = video_app._backend
|
||||||
|
video_app._backend = MockVideoBackend()
|
||||||
|
yield
|
||||||
|
video_app._backend = original
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def client():
|
||||||
|
return TestClient(video_app.app)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def video_file(tmp_path):
|
||||||
|
"""Placeholder file that satisfies os.path.exists() inside the mock."""
|
||||||
|
p = tmp_path / "sample.mp4"
|
||||||
|
p.write_bytes(b"\x00" * 16)
|
||||||
|
return str(p)
|
||||||
|
|
||||||
|
|
||||||
|
# ── /health ───────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def test_health_returns_ok(client):
|
||||||
|
resp = client.get("/health")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.json()
|
||||||
|
assert data["status"] == "ok"
|
||||||
|
assert data["model"] == "mock"
|
||||||
|
assert data["vram_mb"] == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_health_503_when_no_backend(client):
|
||||||
|
video_app._backend = None
|
||||||
|
resp = client.get("/health")
|
||||||
|
assert resp.status_code == 503
|
||||||
|
|
||||||
|
|
||||||
|
# ── /caption ──────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_returns_200(client, video_file):
|
||||||
|
resp = client.post("/caption", json={"video_path": video_file})
|
||||||
|
assert resp.status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_response_has_scene(client, video_file):
|
||||||
|
data = client.post("/caption", json={"video_path": video_file}).json()
|
||||||
|
assert isinstance(data["scene"], str)
|
||||||
|
assert data["scene"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_response_has_events(client, video_file):
|
||||||
|
data = client.post("/caption", json={"video_path": video_file}).json()
|
||||||
|
assert isinstance(data["events"], list)
|
||||||
|
assert len(data["events"]) >= 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_events_have_timestamps(client, video_file):
|
||||||
|
data = client.post("/caption", json={"video_path": video_file}).json()
|
||||||
|
for ev in data["events"]:
|
||||||
|
assert "start" in ev
|
||||||
|
assert "end" in ev
|
||||||
|
assert "description" in ev
|
||||||
|
assert ev["start"] <= ev["end"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_response_has_caption(client, video_file):
|
||||||
|
data = client.post("/caption", json={"video_path": video_file}).json()
|
||||||
|
assert isinstance(data["caption"], str)
|
||||||
|
assert data["caption"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_response_model_field(client, video_file):
|
||||||
|
data = client.post("/caption", json={"video_path": video_file}).json()
|
||||||
|
assert isinstance(data["model"], str)
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_404_on_missing_file(client):
|
||||||
|
resp = client.post("/caption", json={"video_path": "/no/such/file.mp4"})
|
||||||
|
assert resp.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_503_when_no_backend(client, video_file):
|
||||||
|
video_app._backend = None
|
||||||
|
resp = client.post("/caption", json={"video_path": video_file})
|
||||||
|
assert resp.status_code == 503
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_custom_max_new_tokens(client, video_file):
|
||||||
|
resp = client.post(
|
||||||
|
"/caption",
|
||||||
|
json={"video_path": video_file, "max_new_tokens": 512},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_rejects_max_new_tokens_below_min(client, video_file):
|
||||||
|
resp = client.post(
|
||||||
|
"/caption",
|
||||||
|
json={"video_path": video_file, "max_new_tokens": 10},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_rejects_max_new_tokens_above_max(client, video_file):
|
||||||
|
resp = client.post(
|
||||||
|
"/caption",
|
||||||
|
json={"video_path": video_file, "max_new_tokens": 99999},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
# ── /find ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_returns_200(client, video_file):
|
||||||
|
resp = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": "someone waves"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_response_has_span(client, video_file):
|
||||||
|
data = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": "mock event"},
|
||||||
|
).json()
|
||||||
|
# MockVideoBackend always returns a non-null span
|
||||||
|
assert data["span"] is not None
|
||||||
|
assert len(data["span"]) == 2
|
||||||
|
assert data["span"][0] <= data["span"][1]
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_span_is_list_of_floats(client, video_file):
|
||||||
|
data = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": "mock event"},
|
||||||
|
).json()
|
||||||
|
span = data["span"]
|
||||||
|
assert all(isinstance(v, float) for v in span)
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_format_ok_field(client, video_file):
|
||||||
|
data = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": "mock event"},
|
||||||
|
).json()
|
||||||
|
assert data["format_ok"] is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_raw_field(client, video_file):
|
||||||
|
data = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": "mock event"},
|
||||||
|
).json()
|
||||||
|
assert isinstance(data["raw"], str)
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_model_field(client, video_file):
|
||||||
|
data = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": "mock event"},
|
||||||
|
).json()
|
||||||
|
assert isinstance(data["model"], str)
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_404_on_missing_file(client):
|
||||||
|
resp = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": "/no/such/file.mp4", "event": "wave"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_503_when_no_backend(client, video_file):
|
||||||
|
video_app._backend = None
|
||||||
|
resp = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": "wave"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 503
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_rejects_empty_event(client, video_file):
|
||||||
|
resp = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": ""},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_custom_max_new_tokens(client, video_file):
|
||||||
|
resp = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": "wave", "max_new_tokens": 128},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_rejects_max_new_tokens_below_min(client, video_file):
|
||||||
|
resp = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": "wave", "max_new_tokens": 10},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_rejects_max_new_tokens_above_max(client, video_file):
|
||||||
|
resp = client.post(
|
||||||
|
"/find",
|
||||||
|
json={"video_path": video_file, "event": "wave", "max_new_tokens": 99999},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 422
|
||||||
157
tests/test_video/test_mock_backend.py
Normal file
157
tests/test_video/test_mock_backend.py
Normal file
|
|
@ -0,0 +1,157 @@
|
||||||
|
"""
|
||||||
|
Tests for MockVideoBackend and the VideoBackend protocol.
|
||||||
|
|
||||||
|
All tests run without a GPU, torch install, or any real video file
|
||||||
|
(MockVideoBackend only checks os.path.exists, not video validity).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from circuitforge_core.video.backends.base import (
|
||||||
|
CaptionResult,
|
||||||
|
FindResult,
|
||||||
|
VideoBackend,
|
||||||
|
VideoEvent,
|
||||||
|
make_video_backend,
|
||||||
|
)
|
||||||
|
from circuitforge_core.video.backends.mock import MockVideoBackend
|
||||||
|
|
||||||
|
|
||||||
|
# ── Fixtures ──────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def video_file(tmp_path):
|
||||||
|
"""Create a temporary file that satisfies os.path.exists() checks."""
|
||||||
|
p = tmp_path / "test.mp4"
|
||||||
|
p.write_bytes(b"\x00" * 16) # placeholder bytes; mock never reads content
|
||||||
|
return str(p)
|
||||||
|
|
||||||
|
|
||||||
|
# ── Protocol conformance ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def test_mock_satisfies_protocol():
|
||||||
|
backend = MockVideoBackend()
|
||||||
|
assert isinstance(backend, VideoBackend)
|
||||||
|
|
||||||
|
|
||||||
|
def test_mock_model_name_default():
|
||||||
|
assert MockVideoBackend().model_name == "mock"
|
||||||
|
|
||||||
|
|
||||||
|
def test_mock_model_name_custom():
|
||||||
|
assert MockVideoBackend(model_path="custom-path").model_name == "custom-path"
|
||||||
|
|
||||||
|
|
||||||
|
def test_mock_vram_mb():
|
||||||
|
assert MockVideoBackend().vram_mb == 0
|
||||||
|
|
||||||
|
|
||||||
|
# ── caption() ─────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_returns_caption_result(video_file):
|
||||||
|
result = MockVideoBackend().caption(video_file)
|
||||||
|
assert isinstance(result, CaptionResult)
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_scene_is_str(video_file):
|
||||||
|
result = MockVideoBackend().caption(video_file)
|
||||||
|
assert isinstance(result.scene, str)
|
||||||
|
assert result.scene # non-empty
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_events_are_video_events(video_file):
|
||||||
|
result = MockVideoBackend().caption(video_file)
|
||||||
|
assert isinstance(result.events, list)
|
||||||
|
for ev in result.events:
|
||||||
|
assert isinstance(ev, VideoEvent)
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_events_have_numeric_timestamps(video_file):
|
||||||
|
result = MockVideoBackend().caption(video_file)
|
||||||
|
for ev in result.events:
|
||||||
|
assert isinstance(ev.start, float)
|
||||||
|
assert isinstance(ev.end, float)
|
||||||
|
assert ev.start <= ev.end
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_caption_str(video_file):
|
||||||
|
result = MockVideoBackend().caption(video_file)
|
||||||
|
assert isinstance(result.caption, str)
|
||||||
|
assert result.caption
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_model_matches_path(video_file):
|
||||||
|
result = MockVideoBackend(model_path="test-model").caption(video_file)
|
||||||
|
assert result.model == "test-model"
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_raises_on_missing_file():
|
||||||
|
with pytest.raises(FileNotFoundError):
|
||||||
|
MockVideoBackend().caption("/nonexistent/video.mp4")
|
||||||
|
|
||||||
|
|
||||||
|
def test_caption_max_new_tokens_accepted(video_file):
|
||||||
|
"""max_new_tokens kwarg must be accepted without error."""
|
||||||
|
result = MockVideoBackend().caption(video_file, max_new_tokens=512)
|
||||||
|
assert isinstance(result, CaptionResult)
|
||||||
|
|
||||||
|
|
||||||
|
# ── find() ────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_returns_find_result(video_file):
|
||||||
|
result = MockVideoBackend().find(video_file, "someone waves")
|
||||||
|
assert isinstance(result, FindResult)
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_span_is_tuple_or_none(video_file):
|
||||||
|
result = MockVideoBackend().find(video_file, "mock event")
|
||||||
|
# MockVideoBackend always returns a span
|
||||||
|
assert result.span is not None
|
||||||
|
assert len(result.span) == 2
|
||||||
|
assert result.span[0] <= result.span[1]
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_format_ok_true(video_file):
|
||||||
|
result = MockVideoBackend().find(video_file, "mock event")
|
||||||
|
assert result.format_ok is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_raw_is_str(video_file):
|
||||||
|
result = MockVideoBackend().find(video_file, "mock event")
|
||||||
|
assert isinstance(result.raw, str)
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_model_matches_path(video_file):
|
||||||
|
result = MockVideoBackend(model_path="my-model").find(video_file, "event")
|
||||||
|
assert result.model == "my-model"
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_raises_on_missing_file():
|
||||||
|
with pytest.raises(FileNotFoundError):
|
||||||
|
MockVideoBackend().find("/nonexistent/video.mp4", "event")
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_max_new_tokens_accepted(video_file):
|
||||||
|
result = MockVideoBackend().find(video_file, "event", max_new_tokens=128)
|
||||||
|
assert isinstance(result, FindResult)
|
||||||
|
|
||||||
|
|
||||||
|
# ── make_video_backend factory ────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def test_factory_returns_mock_when_flag_set():
|
||||||
|
backend = make_video_backend(model_path="mock", mock=True)
|
||||||
|
assert isinstance(backend, MockVideoBackend)
|
||||||
|
|
||||||
|
|
||||||
|
def test_factory_mock_uses_model_path():
|
||||||
|
backend = make_video_backend(model_path="some-path", mock=True)
|
||||||
|
assert backend.model_name == "some-path"
|
||||||
Loading…
Reference in a new issue