Multi-modal fusion — camera + EEG/EMG combined classification #22

New issue

Open

opened 2026-04-26 21:47:46 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-04-26 21:47:46 -07:00

Owner

Combine camera-based gesture confidence and EEG/EMG signal confidence into a single fused decision.

Architecture: weighted voting across active sources. Each source produces a (class, confidence) pair per frame. Fuser computes weighted sum; fires action only when aggregate confidence > threshold.

Weights: tunable per profile. Default: camera 0.6, EEG 0.3, EMG 0.1 (camera is most reliable baseline). User can adjust in config.

Fusion modes:

any — fire when any single source reaches threshold (low latency, higher false-positive rate)
majority — fire when weighted majority agrees (default, balanced)
unanimous — all active sources must agree (lowest false-positive, higher latency)

Benefit: camera confirms EEG intent for ambiguous epochs. EEG confirms camera gesture for low-light or partial occlusion. Combined accuracy significantly higher than either source alone.

This is the Phase C flagship feature — the full Merlin stack operating simultaneously.

Combine camera-based gesture confidence and EEG/EMG signal confidence into a single fused decision. **Architecture:** weighted voting across active sources. Each source produces a `(class, confidence)` pair per frame. Fuser computes weighted sum; fires action only when aggregate confidence > threshold. **Weights:** tunable per profile. Default: camera 0.6, EEG 0.3, EMG 0.1 (camera is most reliable baseline). User can adjust in config. **Fusion modes:** - `any` — fire when any single source reaches threshold (low latency, higher false-positive rate) - `majority` — fire when weighted majority agrees (default, balanced) - `unanimous` — all active sources must agree (lowest false-positive, higher latency) **Benefit:** camera confirms EEG intent for ambiguous epochs. EEG confirms camera gesture for low-light or partial occlusion. Combined accuracy significantly higher than either source alone. This is the Phase C flagship feature — the full Merlin stack operating simultaneously.

pyr0ball added this to the Phase C — EEG and EMG Hardware milestone 2026-04-26 21:47:46 -07:00

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Circuit-Forge/raven#22

No description provided.