spec(M2): LLM Chat — Ollama-backed streaming chat with migration context #6

New issue

Open

opened 2026-05-19 07:46:44 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-05-19 07:46:44 -07:00

Owner

What M2 built

Robin's chat panel is now wired to a real local LLM via Ollama. Type a question and get a streaming response, character by character, with your migration context baked in.

How it works

You type a question and hit Enter or click send
Robin builds a system prompt that includes your source OS (macOS/Windows) and current distro (e.g. CachyOS)
Your message goes to Ollama (/api/chat, streaming)
Tokens stream back into the chat panel in real time
Input is locked while Robin is thinking — unlocks on completion or error

System prompt shape

Robin tells the LLM: "You are Robin, a friendly Linux migration companion. The user migrated from {source_os} and is currently using {distro}. Help them with Linux questions. When concepts differ from their previous OS, explain the Linux equivalent clearly. Be concise, practical, and warm."

Error handling

If Ollama is not running, the chat panel shows a friendly error message instead of a spinner that never resolves.

Configuration

~/.config/robin/config.toml:

ollama.base_url — default http://localhost:11434
ollama.model — default llama3.2

Limitations in M2

Single-turn only: Robin sees your current message + the system prompt, not previous messages in the conversation
No knowledge base retrieval yet — Robin answers from the LLM's training data only
Multi-turn memory and RAG retrieval are M4

Questions for Xander

Does Robin's tone feel right for a migration companion, or does it need tuning?
Are there questions Robin answers badly where you can tell it lacks the migration context?
Is llama3.2 the right default model for CachyOS users, or is there a better local option?
Does the streaming UX feel responsive enough, or does it feel slow?

Implementation: PR #3

## What M2 built Robin's chat panel is now wired to a real local LLM via Ollama. Type a question and get a streaming response, character by character, with your migration context baked in. ### How it works 1. You type a question and hit Enter or click send 2. Robin builds a system prompt that includes your source OS (macOS/Windows) and current distro (e.g. CachyOS) 3. Your message goes to Ollama (`/api/chat`, streaming) 4. Tokens stream back into the chat panel in real time 5. Input is locked while Robin is thinking — unlocks on completion or error ### System prompt shape Robin tells the LLM: "You are Robin, a friendly Linux migration companion. The user migrated from {source_os} and is currently using {distro}. Help them with Linux questions. When concepts differ from their previous OS, explain the Linux equivalent clearly. Be concise, practical, and warm." ### Error handling If Ollama is not running, the chat panel shows a friendly error message instead of a spinner that never resolves. ### Configuration `~/.config/robin/config.toml`: - `ollama.base_url` — default `http://localhost:11434` - `ollama.model` — default `llama3.2` ### Limitations in M2 - Single-turn only: Robin sees your current message + the system prompt, not previous messages in the conversation - No knowledge base retrieval yet — Robin answers from the LLM's training data only - Multi-turn memory and RAG retrieval are M4 ## Questions for Xander - Does Robin's tone feel right for a migration companion, or does it need tuning? - Are there questions Robin answers badly where you can tell it lacks the migration context? - Is `llama3.2` the right default model for CachyOS users, or is there a better local option? - Does the streaming UX feel responsive enough, or does it feel slow? **Implementation:** PR #3

pyr0ball added this to the M2: LLM Chat milestone 2026-05-19 07:46:44 -07:00

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Circuit-Forge/robin#6

No description provided.