Circuit-Forge/raven
3
0
Fork 0
Code Issues 24 Pull requests Projects Releases Packages Wiki Activity Actions

StructuredLight source adapter — OpenNI2 (Xtion/Kinect) and librealsense (RealSense) #11

New issue
Open
opened 2026-04-26 21:46:21 -07:00 by pyr0ball · 0 comments
pyr0ball commented 2026-04-26 21:46:21 -07:00
Owner
Copy link

Implement StructuredLightSource adapter for the DepthFusion layer.

Backends:

  • OpenNI2: ASUS Xtion Pro Live, Microsoft Kinect v1 (via openni2 Python bindings)
  • librealsense: Intel RealSense D-series (structured light models) via pyrealsense2

Probe logic: try openni2.initialize() first, then pyrealsense2.pipeline(). If neither imports or no device found, is_available() returns False.

Output: aligned RGB + depth frames at 30fps. Depth map in mm. get_3d_landmarks() uses the depth map to lift MediaPipe 2D landmarks to 3D using the camera intrinsics.

Optional dependency group in pyproject.toml: merlin[depth-structured] → openni2>=0.2, pyrealsense2>=2.50.

Sub-centimeter accuracy at hand distances. Best source when hardware present.

Implement `StructuredLightSource` adapter for the DepthFusion layer. **Backends:** - OpenNI2: ASUS Xtion Pro Live, Microsoft Kinect v1 (via `openni2` Python bindings) - librealsense: Intel RealSense D-series (structured light models) via `pyrealsense2` **Probe logic:** try `openni2.initialize()` first, then `pyrealsense2.pipeline()`. If neither imports or no device found, `is_available()` returns False. **Output:** aligned RGB + depth frames at 30fps. Depth map in mm. `get_3d_landmarks()` uses the depth map to lift MediaPipe 2D landmarks to 3D using the camera intrinsics. Optional dependency group in pyproject.toml: `merlin[depth-structured]` → `openni2>=0.2`, `pyrealsense2>=2.50`. Sub-centimeter accuracy at hand distances. Best source when hardware present.
pyr0ball added this to the Phase B — Depth and High-Fidelity Spatial Input milestone 2026-04-26 21:46:21 -07:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels
No items
No labels
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Circuit-Forge/raven#11
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?