Evaluate Vocalinux as dictation input layer + cf-orch as annotation backend #35

New issue

Open

opened 2026-07-19 15:49:44 -07:00 by pyr0ball · 1 comment

pyr0ball commented

2026-07-19 15:49:44 -07:00

Owner

Background

Vocalinux is a local-only Linux voice dictation tool (589 stars, actively maintained, v0.14.2 shipped 2026-07-18) that transcribes speech and injects text into any X11/Wayland application.

Stack: Python + TypeScript, whisper.cpp / OpenAI Whisper / VOSK as swappable STT backends, Silero VAD, Vulkan GPU acceleration, GTK tray UI.

Proposed integration

Vocalinux and Linnet are complementary layers:

[User speaks]
  → Vocalinux (local STT: whisper.cpp / VOSK)
  → raw transcript
  → cf-orch worker (Linnet tone annotation pipeline)
  → tone-annotated output (Elcor tags, sentiment, context)
  → injected/displayed by Vocalinux or Linnet UI

This means:

Vocalinux owns the audio capture, VAD, and text injection layer — proven, local, no round-trip
Linnet owns the meaning layer — tone classification, Elcor annotation, ND-aware context
cf-orch routes annotation tasks to available GPU workers, enabling cloud fallback when local VRAM is insufficient

cf-orch backend angle

For users without local GPU headroom, cf-orch can serve as the annotation backend:

Vocalinux transcript POSTs to cf-orch task queue
cf-orch schedules on available worker (local Sif/Strahl/Muninn, or cloud)
Result returned to Linnet UI / injected back via Vocalinux

This fits the CF architecture pattern: local inference first, cf-orch as smart fallback.

License constraint

Vocalinux is GPL-3.0 — Linnet (BSL 1.1) cannot bundle or statically link it. Integration must be process-boundary / IPC:

Linnet registers as a Vocalinux plugin via its plugin API (if one exists or can be proposed upstream)
Or: Vocalinux pipes transcripts to a local Linnet socket/HTTP endpoint
Or: shared DBus signal that both apps listen to

Next steps

Audit Vocalinux plugin/extension points (check plugins/ dir and README)
Prototype: Vocalinux → local HTTP POST → Linnet tone endpoint → annotated response
Design cf-orch task type for Linnet annotation jobs
Confirm IPC boundary satisfies GPL isolation requirement (no linking)
Check whether upstream would accept a "pipe to external annotator" plugin hook

References

Vocalinux repo: https://github.com/jatinkrmalik/vocalinux
cf-orch architecture: circuitforge-plans/shared/ (cf-orch docs)
Linnet product context: circuitforge-plans/linnet/

## Background [Vocalinux](https://github.com/jatinkrmalik/vocalinux) is a local-only Linux voice dictation tool (589 stars, actively maintained, v0.14.2 shipped 2026-07-18) that transcribes speech and injects text into any X11/Wayland application. **Stack:** Python + TypeScript, whisper.cpp / OpenAI Whisper / VOSK as swappable STT backends, Silero VAD, Vulkan GPU acceleration, GTK tray UI. ## Proposed integration Vocalinux and Linnet are complementary layers: ``` [User speaks] → Vocalinux (local STT: whisper.cpp / VOSK) → raw transcript → cf-orch worker (Linnet tone annotation pipeline) → tone-annotated output (Elcor tags, sentiment, context) → injected/displayed by Vocalinux or Linnet UI ``` This means: - Vocalinux owns the audio capture, VAD, and text injection layer — proven, local, no round-trip - Linnet owns the meaning layer — tone classification, Elcor annotation, ND-aware context - cf-orch routes annotation tasks to available GPU workers, enabling cloud fallback when local VRAM is insufficient ## cf-orch backend angle For users without local GPU headroom, cf-orch can serve as the annotation backend: - Vocalinux transcript POSTs to cf-orch task queue - cf-orch schedules on available worker (local Sif/Strahl/Muninn, or cloud) - Result returned to Linnet UI / injected back via Vocalinux This fits the CF architecture pattern: local inference first, cf-orch as smart fallback. ## License constraint Vocalinux is **GPL-3.0** — Linnet (BSL 1.1) cannot bundle or statically link it. Integration must be process-boundary / IPC: - Linnet registers as a Vocalinux plugin via its plugin API (if one exists or can be proposed upstream) - Or: Vocalinux pipes transcripts to a local Linnet socket/HTTP endpoint - Or: shared DBus signal that both apps listen to ## Next steps - [ ] Audit Vocalinux plugin/extension points (check `plugins/` dir and README) - [ ] Prototype: Vocalinux → local HTTP POST → Linnet tone endpoint → annotated response - [ ] Design cf-orch task type for Linnet annotation jobs - [ ] Confirm IPC boundary satisfies GPL isolation requirement (no linking) - [ ] Check whether upstream would accept a "pipe to external annotator" plugin hook ## References - Vocalinux repo: https://github.com/jatinkrmalik/vocalinux - cf-orch architecture: `circuitforge-plans/shared/` (cf-orch docs) - Linnet product context: `circuitforge-plans/linnet/`

pyr0ball commented

2026-07-19 16:23:12 -07:00

Author

Owner

Extension point audit results

No plugin system exists today. Source: recognition_manager.py.

Internal callbacks present but in-process only:

register_text_callback(callback: Callable[[str], None])    # fired on every transcription
register_state_callback(callback: Callable[[RecognitionState], None])
register_action_callback(callback: Callable[[str], None])  # voice commands
register_audio_level_callback(callback: Callable[[float], None])

The existing HTTP Remote API (docs/HTTP_REMOTE.md) is inbound only — Vocalinux as STT client posting audio to a remote server. No outbound webhook, no DBus, no plugin registry.

Recommended IPC path: upstream webhook PR

Add post_transcription_webhook_url config key to recognition_manager.py. After _notify_text_callbacks() fires, POST the transcript JSON to the configured URL. Estimated ~15 lines. GPL-isolated at the HTTP boundary (no linking).

Linnet runs a local HTTP listener (e.g. localhost:8650/annotate), receives the transcript, annotates, returns tone data.

Vocalinux (local STT)
  → POST http://localhost:8650/annotate  {"text": "...", "timestamp": ...}
  → Linnet local listener
  → cf-orch task (if local VRAM insufficient)
  → annotated response {"tone": "...", "elcor": [...], "confidence": 0.9}
  → Linnet UI / injected back

This is general-purpose enough to propose upstream — any post-processing tool (translation, logging, accessibility) could use it, not just Linnet.

Fallback (prototype only)

Parse ~/.local/share/vocalinux/vocalinux.log for transcription lines. Zero upstream changes needed. Fragile — use only to validate the annotation pipeline before investing in the upstream PR.

Next step

Draft the upstream PR to Vocalinux adding post_transcription_webhook_url. Frame it as a general post-processing hook, not a Linnet-specific feature.

## Extension point audit results **No plugin system exists today.** Source: `recognition_manager.py`. Internal callbacks present but in-process only: ```python register_text_callback(callback: Callable[[str], None]) # fired on every transcription register_state_callback(callback: Callable[[RecognitionState], None]) register_action_callback(callback: Callable[[str], None]) # voice commands register_audio_level_callback(callback: Callable[[float], None]) ``` The existing HTTP Remote API (`docs/HTTP_REMOTE.md`) is **inbound only** — Vocalinux as STT client posting audio to a remote server. No outbound webhook, no DBus, no plugin registry. ## Recommended IPC path: upstream webhook PR Add `post_transcription_webhook_url` config key to `recognition_manager.py`. After `_notify_text_callbacks()` fires, POST the transcript JSON to the configured URL. Estimated ~15 lines. GPL-isolated at the HTTP boundary (no linking). Linnet runs a local HTTP listener (e.g. `localhost:8650/annotate`), receives the transcript, annotates, returns tone data. ``` Vocalinux (local STT) → POST http://localhost:8650/annotate {"text": "...", "timestamp": ...} → Linnet local listener → cf-orch task (if local VRAM insufficient) → annotated response {"tone": "...", "elcor": [...], "confidence": 0.9} → Linnet UI / injected back ``` This is general-purpose enough to propose upstream — any post-processing tool (translation, logging, accessibility) could use it, not just Linnet. ## Fallback (prototype only) Parse `~/.local/share/vocalinux/vocalinux.log` for transcription lines. Zero upstream changes needed. Fragile — use only to validate the annotation pipeline before investing in the upstream PR. ## Next step Draft the upstream PR to Vocalinux adding `post_transcription_webhook_url`. Frame it as a general post-processing hook, not a Linnet-specific feature.