52 lines
2.2 KiB
Markdown
52 lines
2.2 KiB
Markdown
# Merlin — Hands-Free Computer Control
|
|
|
|
> *Part of the Circuit Forge LLC "AI for the tasks you hate most" suite.*
|
|
|
|
[](https://circuitforge.tech/software/merlin)
|
|
[](https://circuitforge.tech)
|
|
[](LICENSE)
|
|
|
|
**Alternative input, trained to you.** Merlin is a BCI and alternative input training harness — letting you control your computer via eye gaze, blink patterns, and hand gestures, with models trained on your own input so they work the way you actually move.
|
|
|
|
## What it does
|
|
|
|
Merlin uses your webcam to detect gaze direction, blink events, and hand gestures, then maps them to configurable actions (clicks, keystrokes, macros). A built-in training harness lets you record samples and fine-tune the models for your specific movement patterns.
|
|
|
|
## Why it matters
|
|
|
|
Commercial alternative input tools are expensive, inflexible, and assume a normative movement profile. Merlin is local-first and trainable — your models stay on your hardware and improve with use.
|
|
|
|
## Core pipeline
|
|
|
|
```
|
|
Camera feed → MediaPipe detection (gaze / blink / gesture)
|
|
→ Feature extraction → Action mapper → OS input event
|
|
↑
|
|
Training loop: record samples → fit scikit-learn model → reload
|
|
```
|
|
|
|
## Features
|
|
|
|
- **Gaze estimation** — map where you look to cursor position or zone actions
|
|
- **Blink detection** — single, double, and long-blink events → configurable actions
|
|
- **Hand gesture recognition** — MediaPipe Hands → custom gesture vocabulary
|
|
- **Training harness** — record labeled samples, retrain, hot-reload without restart
|
|
- **FastAPI control API** — start/stop detection loop, reload config, check status (`localhost:8522`)
|
|
- **Local-first** — no cloud inference, no data leaves your machine
|
|
|
|
## Setup
|
|
|
|
```bash
|
|
conda run -n cf pip install -e .
|
|
conda run -n cf python -m merlin.daemon
|
|
```
|
|
|
|
Config: `~/.merlin/config.yaml`
|
|
|
|
## Product code (license key)
|
|
|
|
`CFG-MRLN-XXXX-XXXX-XXXX`
|
|
|
|
## License
|
|
|
|
**BSL 1.1** — free for personal non-commercial self-hosting; converts to MIT after 4 years. Commercial use requires a paid license.
|