arch: Rust rewrite evaluation — decided against (two exceptions noted) #33

New issue

Open

opened 2026-04-12 21:17:44 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-04-12 21:17:44 -07:00

Owner

Decision

Evaluated a full Rust rewrite of the menagerie (including via Python→Rust transpiler). Decision: no rewrite, no transpiler.

Why not

I/O-bound, not CPU-bound. Products are waiting on LLM inference, API round-trips, and DB writes. Rust's primary wins (no GC pauses, zero-cost abstractions, memory safety) apply to CPU-bound or latency-critical systems code. FastAPI + asyncio is not the bottleneck anywhere in the stack.

ML/audio layer is irreplaceable in Python. faster-whisper, pyannote, transformers, YAMNet, librosa have no Rust equivalents. Rust code in the hot path would call back into Python (via pyo3) anyway, adding a language boundary with no benefit.

Transpilers are not ready. py2many and similar tools handle only a narrow, idiomatic-Python subset and produce unidiomatic Rust requiring heavy hand-editing. Maintaining transpiler output is the worst of both worlds.

Velocity cost is too high. Products are alpha/beta. A rewrite would delay shipping significantly with no user-facing benefit.

Two legitimate exceptions — revisit later

Component	Why Rust makes sense	When
Minerva firmware (ESP32-S3)	`no_std` Rust is a real option over MicroPython for production embedded firmware	After MicroPython prototype is proven
cf-voice acoustic classifier inner loop	Real-time MFCC/mel spectrogram extraction could benefit from a `pyo3` Rust extension	Only if profiling shows it as a CPU bottleneck

Standing rule

Do not raise Rust as a solution for menagerie performance without profiling evidence of a CPU-bound bottleneck first. If something is slow, profile it — the answer is almost always the LLM call, network round-trip, or DB write.

## Decision Evaluated a full Rust rewrite of the menagerie (including via Python→Rust transpiler). Decision: no rewrite, no transpiler. ## Why not **I/O-bound, not CPU-bound.** Products are waiting on LLM inference, API round-trips, and DB writes. Rust's primary wins (no GC pauses, zero-cost abstractions, memory safety) apply to CPU-bound or latency-critical systems code. FastAPI + asyncio is not the bottleneck anywhere in the stack. **ML/audio layer is irreplaceable in Python.** `faster-whisper`, `pyannote`, `transformers`, `YAMNet`, `librosa` have no Rust equivalents. Rust code in the hot path would call back into Python (via `pyo3`) anyway, adding a language boundary with no benefit. **Transpilers are not ready.** `py2many` and similar tools handle only a narrow, idiomatic-Python subset and produce unidiomatic Rust requiring heavy hand-editing. Maintaining transpiler output is the worst of both worlds. **Velocity cost is too high.** Products are alpha/beta. A rewrite would delay shipping significantly with no user-facing benefit. ## Two legitimate exceptions — revisit later | Component | Why Rust makes sense | When | |---|---|---| | Minerva firmware (ESP32-S3) | `no_std` Rust is a real option over MicroPython for production embedded firmware | After MicroPython prototype is proven | | cf-voice acoustic classifier inner loop | Real-time MFCC/mel spectrogram extraction could benefit from a `pyo3` Rust extension | Only if profiling shows it as a CPU bottleneck | ## Standing rule Do not raise Rust as a solution for menagerie performance without profiling evidence of a CPU-bound bottleneck first. If something is slow, profile it — the answer is almost always the LLM call, network round-trip, or DB write.