DepthFusion layer — source-agnostic depth adapter interface (cf-core) #10

New issue

Open

opened 2026-04-26 21:46:21 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-04-26 21:46:21 -07:00

Owner

Extract a DepthFusion adapter layer into cf_input.gestures in circuitforge-core. All downstream code (feature extractors, classifiers, action mapper) receives normalized 3D landmarks regardless of depth source.

Interface:

class DepthSource(Protocol):
    def get_depth_frame(self) -> np.ndarray: ...  # (H, W) float32 depth map
    def get_3d_landmarks(self, landmarks_2d: np.ndarray) -> np.ndarray: ...  # lifts 2D → 3D
    def is_available(self) -> bool: ...

Auto-detection priority at startup:

StructuredLightSource (OpenNI2 / librealsense)
StereoSource (OpenCV StereoSGBM)
MonocularDepthSource (PyTorch Depth Anything v2)
Fallback: RGB-only (Phase A, Z-coordinate = 0)

Adding a new backend requires only a new DepthSource adapter class. Nothing in the classifier or action layer changes.

Part of cf-core (MIT). Spec: circuitforge-plans/merlin/superpowers/specs/2026-04-26-merlin-bci-design.md § Phase B.

Extract a `DepthFusion` adapter layer into `cf_input.gestures` in circuitforge-core. All downstream code (feature extractors, classifiers, action mapper) receives normalized 3D landmarks regardless of depth source. **Interface:** ```python class DepthSource(Protocol): def get_depth_frame(self) -> np.ndarray: ... # (H, W) float32 depth map def get_3d_landmarks(self, landmarks_2d: np.ndarray) -> np.ndarray: ... # lifts 2D → 3D def is_available(self) -> bool: ... ``` **Auto-detection priority at startup:** 1. StructuredLightSource (OpenNI2 / librealsense) 2. StereoSource (OpenCV StereoSGBM) 3. MonocularDepthSource (PyTorch Depth Anything v2) 4. Fallback: RGB-only (Phase A, Z-coordinate = 0) Adding a new backend requires only a new `DepthSource` adapter class. Nothing in the classifier or action layer changes. Part of cf-core (MIT). Spec: `circuitforge-plans/merlin/superpowers/specs/2026-04-26-merlin-bci-design.md` § Phase B.

pyr0ball added this to the Phase B — Depth and High-Fidelity Spatial Input milestone 2026-04-26 21:46:21 -07:00

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Circuit-Forge/raven#10

No description provided.