RFC: cf-vision module in cf-core (YOLO26-backed shared vision pipeline) #51

New issue

Open

opened 2026-06-23 09:29:10 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-06-23 09:29:10 -07:00

Owner

Motivation

Three CircuitForge products need real-time computer vision:

Kiwi: Barcode detection, food item recognition, receipt region detection
Nuthatch: Object detection + pose for robot navigation
Raven: Pose estimation for gesture-based alternative input

Building vision pipelines independently in each product wastes effort and
fragments license management. A shared cf-vision module in circuitforge-core
would give all three a common interface.

Catalyst

Ultralytics YOLO26 (https://arxiv.org/abs/2606.03748) ships a unified model
covering detection, segmentation, pose, oriented detection, and classification in
one pipeline. NMS-free end-to-end inference. Exports to TensorRT, ONNX, CoreML,
TFLite, OpenVINO, NCNN, ExecuTorch.

YOLOE-26 variant adds open-vocabulary (zero-shot, text-prompted) detection —
enabling use cases like "find all food items" without task-specific training.

Proposed module: `circuitforge_core.vision`

cf-core/
  circuitforge_core/
    vision/
      __init__.py
      detector.py      # Unified detect/segment/pose interface
      barcode.py       # Kiwi-specific: barcode region extraction
      pose.py          # Raven/Nuthatch: keypoint pipeline
      models.py        # Model registry + weight management

Interface sketch:

from circuitforge_core.vision import VisionPipeline

pipe = VisionPipeline(model="yolo26n", task="detect", device="cpu")
results = pipe.run(image)  # returns structured detections

License design constraint

YOLO26 code/weights: AGPL-3.0 (open source) or Ultralytics commercial license.

Free/local tier: AGPL-3.0 acceptable (local inference, no SaaS distribution).
Cloud/paid tier: AGPL requires source disclosure for network services —
need Ultralytics Enterprise license OR design inference-only wrapper around
weights without distributing modified Ultralytics training framework.

cf-vision module should abstract the backend so a non-AGPL-licensed alternative
(e.g., custom ONNX runtime wrapping bare weights) can be swapped in for cloud.

Priority

Backlog — activate when Kiwi Phase 2 (photo-add flow) or Nuthatch hardware
integration begins. Kiwi is the likely first consumer.

kiwi#X: YOLO26 for barcode detection and food item recognition
nuthatch#X: YOLO26 for robot vision
raven#X: YOLO26 pose estimation for gesture input

## Motivation Three CircuitForge products need real-time computer vision: - **Kiwi:** Barcode detection, food item recognition, receipt region detection - **Nuthatch:** Object detection + pose for robot navigation - **Raven:** Pose estimation for gesture-based alternative input Building vision pipelines independently in each product wastes effort and fragments license management. A shared `cf-vision` module in `circuitforge-core` would give all three a common interface. ## Catalyst Ultralytics YOLO26 (https://arxiv.org/abs/2606.03748) ships a unified model covering detection, segmentation, pose, oriented detection, and classification in one pipeline. NMS-free end-to-end inference. Exports to TensorRT, ONNX, CoreML, TFLite, OpenVINO, NCNN, ExecuTorch. YOLOE-26 variant adds open-vocabulary (zero-shot, text-prompted) detection — enabling use cases like "find all food items" without task-specific training. ## Proposed module: `circuitforge_core.vision` ``` cf-core/ circuitforge_core/ vision/ __init__.py detector.py # Unified detect/segment/pose interface barcode.py # Kiwi-specific: barcode region extraction pose.py # Raven/Nuthatch: keypoint pipeline models.py # Model registry + weight management ``` Interface sketch: ```python from circuitforge_core.vision import VisionPipeline pipe = VisionPipeline(model="yolo26n", task="detect", device="cpu") results = pipe.run(image) # returns structured detections ``` ## License design constraint YOLO26 code/weights: AGPL-3.0 (open source) or Ultralytics commercial license. - Free/local tier: AGPL-3.0 acceptable (local inference, no SaaS distribution). - Cloud/paid tier: AGPL requires source disclosure for network services — need Ultralytics Enterprise license OR design inference-only wrapper around weights without distributing modified Ultralytics training framework. cf-vision module should abstract the backend so a non-AGPL-licensed alternative (e.g., custom ONNX runtime wrapping bare weights) can be swapped in for cloud. ## Priority Backlog — activate when Kiwi Phase 2 (photo-add flow) or Nuthatch hardware integration begins. Kiwi is the likely first consumer. ## Related product tickets - kiwi#X: YOLO26 for barcode detection and food item recognition - nuthatch#X: YOLO26 for robot vision - raven#X: YOLO26 pose estimation for gesture input