OpenFluke
The app
Octo
Octo is the easy way to actually use AI models on your own computer. Grab a model, get it ready in one step, then chat with it, make images, or generate speech, all running locally through Welvet.
Three simple steps
No machine-learning setup, no cloud account. You point Octo at a model, it does the prep, and then you use it.
1. Download
Paste the name of any model from Hugging Face (the popular model library) and Octo fetches it.
2. Convert
Octo packs the model into a compact .entity file so it runs well on everyday hardware.
3. Run
Chat, generate an image, or create speech, offline. You can even host it as a local API.
More than a chatbot
This is the big upgrade from our earlier tooling: Octo runs several kinds of model, all on your device.
Text & chat
Language models like SmolLM2 and Qwen3 for conversation, drafting, and answering questions.
Images
Describe a picture in words and generate it locally with image models (Flux2 / Bonsai).
Speech
Turn text into natural-sounding audio with an on-device voice model (MOSS-TTS-Nano).
Every quant format, measured
Octo's bench runs the same model (SmolLM2-135M-Instruct) through 21 quantization formats on two fused profiles, so you can see the real speed, size, and quality trade-offs before you pick one.
| Quant | GPU-fused tok/s | SIMD-fused tok/s | Entity size |
|---|---|---|---|
| Q8_0 | 108 | 33 | 253 MB |
| Q4_0 | 112 | 37 | 188 MB |
| Q4_K | 73 | 8 | 192 MB |
| IQ4_XS | 106 | 8 | 204 MB |
| TernaryPacked | 107 | 19 | 172 MB |
| BinaryPacked | 88 | 21 | 140 MB |
Quick start
cd welvet/apps/octo go run . # menu: download HF → convert → run .entity (simd_fuse / gpu_fuse) ./octo image "a red bicycle" # → octo_outputs/*.png ./octo speak "Hello." # → octo_outputs/*.wav
Octo can also host a single model over HTTP with a simple /v1/generate endpoint. Nothing is
hardcoded as "the" model; presets are just prompts. The engine never imports apps; Octo imports Welvet.