linnet/docs/getting-started/llm-setup.md

1.5 KiB

Local Model Setup

Linnet runs fully locally on the Free tier. The transcription pipeline (Whisper) and tone annotation engine (wav2vec2-based SER, or speech emotion recognition) are bundled and download automatically on first start. No additional setup is required for core functionality.

What runs locally

Component Model Purpose
Transcription Whisper (base or small) Speech-to-text
Tone annotation wav2vec2 SER Emotional/social register classification

Audio never leaves your machine. Both models run on-device via the cf-voice inference layer.

Optional: translation via DeepL

Translation is the only feature that requires an external service. Two paths are available:

Free tier: BYOK (bring your own key)

DeepL offers a free API tier (500,000 characters/month). Sign up, copy your API key, and add it to your .env:

DEEPL_API_KEY=your-deepl-api-key-here

Restart Linnet after saving the file.

Paid tier

The Paid tier includes a CF-managed DeepL key. No environment variable is needed. Translation appears automatically once your license key is active.

Choosing a Whisper model size

The default is whisper-base. If your machine has more VRAM or you want higher accuracy:

WHISPER_MODEL=small   # default: base

Available sizes: tiny, base, small, medium, large. Larger models are slower but more accurate, especially for accented speech.