# Octo · Welvet model shell

> Octo downloads, converts, and runs .entity models on the Welvet engine, with a benchmarked sweep across 21 quantization formats so you can pick the right speed, size, and quality trade-off.

Canonical: https://openfluke.com/octo

---

The app
Octo
Octo is the easy way to actually use AI models on your own computer. Grab a model, get it
ready in one step, then chat with it, make images, or generate speech, all running locally through Welvet.
Get Octo on GitHub
The engine behind it
How it works
Three simple steps
No machine-learning setup, no cloud account. You point Octo at a model, it does the prep, and then you use it.
⬇️
1. Download
Paste the name of any model from Hugging Face (the popular model library) and Octo fetches it.
📦
2. Convert
Octo packs the model into a compact .entity file so it runs well on everyday hardware.
▶️
3. Run
Chat, generate an image, or create speech, offline. You can even host it as a local API.
What you can make
More than a chatbot
This is the big upgrade from our earlier tooling: Octo runs several kinds of model, all on your device.
💬
Text & chat
Language models like SmolLM2 and Qwen3 for conversation, drafting, and answering questions.
🎨
Images
Describe a picture in words and generate it locally with image models (Flux2 / Bonsai).
🔊
Speech
Turn text into natural-sounding audio with an on-device voice model (MOSS-TTS-Nano).
Benchmarked, not promised
Every quant format, measured
Octo's bench runs the same model (SmolLM2-135M-Instruct) through 21 quantization formats on two fused
profiles, so you can see the real speed, size, and quality trade-offs before you pick one.
Quant GPU-fused tok/s SIMD-fused tok/s Entity size
Q8_0 108 33 253 MB
Q4_0 112 37 188 MB
Q4_K 73 8 192 MB
IQ4_XS 106 8 204 MB
TernaryPacked 107 19 172 MB
BinaryPacked 88 21 140 MB
SmolLM2-135M-Instruct · linux/amd64 · GTX 1650 SUPER · 80 runs, 76 ok. Higher-bit formats (Q8–Q4) stay
coherent; sub-2-bit formats trade quality for size. Full report ships with the book.
For developers
Quick start
cd welvet/apps/octo
go run .
# menu: download HF → convert → run .entity (simd_fuse / gpu_fuse)
./octo image "a red bicycle" # → octo_outputs/*.png
./octo speak "Hello." # → octo_outputs/*.wav
Octo can also host a single model over HTTP with a simple /v1/generate endpoint. Nothing is
hardcoded as "the" model; presets are just prompts. The engine never imports apps; Octo imports Welvet.
github.com/openfluke/octo
download · convert · run
chat · image · speech