Merlin — Hands-Free Computer Control

Part of the Circuit Forge LLC "AI for the tasks you hate most" suite.

Alternative input, trained to you. Merlin is a BCI and alternative input training harness — letting you control your computer via eye gaze, blink patterns, and hand gestures, with models trained on your own input so they work the way you actually move.

What it does

Merlin uses your webcam to detect gaze direction, blink events, and hand gestures, then maps them to configurable actions (clicks, keystrokes, macros). A built-in training harness lets you record samples and fine-tune the models for your specific movement patterns.

Why it matters

Commercial alternative input tools are expensive, inflexible, and assume a normative movement profile. Merlin is local-first and trainable — your models stay on your hardware and improve with use.

Core pipeline

Camera feed → MediaPipe detection (gaze / blink / gesture)
→ Feature extraction → Action mapper → OS input event
↑
Training loop: record samples → fit scikit-learn model → reload

Features

Gaze estimation — map where you look to cursor position or zone actions
Blink detection — single, double, and long-blink events → configurable actions
Hand gesture recognition — MediaPipe Hands → custom gesture vocabulary
Training harness — record labeled samples, retrain, hot-reload without restart
FastAPI control API — start/stop detection loop, reload config, check status (localhost:8522)
Local-first — no cloud inference, no data leaves your machine

Setup

conda run -n cf pip install -e .
conda run -n cf python -m merlin.daemon

Config: ~/.merlin/config.yaml

Product code (license key)

CFG-MRLN-XXXX-XXXX-XXXX

License

BSL 1.1 — free for personal non-commercial self-hosting; converts to MIT after 4 years. Commercial use requires a paid license.

2.2 KiB Raw Blame History