THE OPEN NOTEBOOK

A book for the curious.

Name: The Welvet feature book
Author: Samuel Watson

70 chapters inside the Welvet AI engine. Start with why it exists, then follow the ideas into layers, learning, and the systems that connect them.

New to the work? Start with Welvet in plain language, then read What Welvet is. Code examples and reported results below are preserved from the source edition. Read the complete book on one page →

Explore:Everything I · Orientation II · Foundation III · Layers IV · Runtime V · Systems VI · Model IO VII · Apps VIII · Stubs IX · Validate

70 entries

I · Orientation

What Welvet is

Loom’s flat poly/ package hit import-cycle and honesty walls (QAT morph, silent fallbacks, god-layer). Welvet is the rewrite: one feature per folder, storage-truth dtypes/quants, Dense as th

I · Orientation

Repository map

Readers need a single map of what is engine vs harness vs app vs stub.

II · Foundation

core — types & backends

Every polymorphic path needs one place for DType, LayerType, Activation, Backend, Tensor[T], and slim Layer metadata — without QAT morph defaults.

II · Foundation

weights — FormatNone MatVec

Unquantized matrices still need a typed store that streams MatVec and SGD without forcing a float32 master or Morph-as-training.

II · Foundation

quant — 20 pack formats

Inference, storage, and train need classic Q-packs, k-quants, IQ, Ternary/Binary, and Affine without a separate QAT mode or retained f32 master. Format is storage truth.

II · Foundation

simd — Plan 9 kernels

CPU peak needs hand-written AVX2/NEON without a silent Go fallback that pretends SIMD ran.

II · Foundation

webgpu — device GEMV & shaders

GPU paths must bind a real adapter. Host “fake GPU” was banned so suites cannot stamp WebGPU done when ALU ran on CPU.

II · Foundation

tiling — SC/MC & workgroups

MatVec throughput depends on tile size and when to go multi-core vs GPU workgroups. Centralizing caps keeps Dense and friends consistent.

II · Foundation

architecture — volumetric grid

Networks are spatial (Depth×Rows×Cols×LayersPerCell), not only linear stacks. Topology lives here; compute lives in layer packages.

II · Foundation

fusedgpu — decoder on device

Token-by-token host round-trips kill decode. A fused engine keeps weights and scratch resident for Q4_0 and BinaryG128 hybrid paths.

III · Layers

layers/dense — MatVec microkernel

Most FLOPs are W@x. One Dense stack owns FormatNone×34 and all quants × three backends so every composite proj shares one correctness surface — including native in-dtype SGD.

III · Layers

layers/mha — attention

Transformers need multi-head attention with masks, RoPE/ALiBi, GQA/MQA, and cross-attn — without forking MatVec for every projection.

III · Layers

layers/swiglu — gated FFN

Modern decoder FFNs are SiLU(gate)⊙up → down. Projections must share Dense’s quant/backend matrix.

III · Layers

layers/rmsnorm

Llama-style blocks normalize by RMS, not mean+var. Needs native fwd/bwd and WebGPU shaders.

III · Layers

layers/layernorm

Classic mean+var normalization with γ/β — still required for many HF architectures.

III · Layers

layers/embedding

Token IDs must gather rows from a table — not a Dense MatVec — with scatter grads on backward.

III · Layers

layers/softmax

Classification heads and attention need stable softmax variants, including sparse/Gumbel/Entmax for research paths.

III · Layers

layers/sequential

Some cells need an ordered Dense chain without burning grid hops.

III · Layers

layers/residual

Skip connections stabilize deep stacks: y = F(x) + x with correct skip grads.

III · Layers

layers/cnn1 · cnn2 · cnn3

Conv nets must sit on the same dtype×quant×backend matrix as Dense. im2col → Dense GEMV is the intentional first cut.

III · Layers

layers/rnn · lstm

Sequence models before transformers still need vanilla RNN and LSTM with BPTT on the shared MatVec stack.

III · Layers

layers/seqmix — mixer contract

Attention, SSM, linear attn, and conv mixers must not be accidental forks of mha. Naming the contract keeps packages honest.

III · Layers

layers/gdn — gated delta net

Linear attention / decode-first mixers (Gated DeltaNet) need a first-class package under KindLinearAttn.

III · Layers

layers/mamba — selective SSM

SSM mixers (KindSSM) are not MHA clones — they need their own selective-scan path.

III · Layers

layers/convt1 · convt2 · convt3

Generators and U-Nets need transposed convolution as a peer of CNN, on the same Dense proj surface.

III · Layers

layers/kmeans

Soft clustering as a differentiable layer lets topology experiments sit inside the same train loop.

III · Layers

layers/parallel — MoE + cameral

Mixture-of-experts and multi-path cells need concat/add/avg/filter combines. Cameral graphs need sibling hemispheres that share input, merge outputs, and optionally train under distinct mode

III · Layers

layers/metacognition

Observed layers can apply heuristic stability rules (gate/scale/reset) without dtype morph/QAT.

IV · Runtime

runtime/forward

A grid of heterogeneous ops needs one walker that dispatches by concrete type and fails loudly on unknowns.

IV · Runtime

runtime/backward

Training needs a reverse tape over the same ops forward used — no separate graph framework.

IV · Runtime

runtime/training

Suites and small nets need MSE+SGD and tween hooks without inventing an external trainer or a retained float32 master beside storage.

IV · Runtime

runtime/step — step mesh

Spatial feedback (remote links) needs a discrete-time mesh where every cell updates from a double buffer — different from a decoder wavefront. Cross-numeric train also needs the same mesh wi

V · Systems

systems/dna

Quant and train must be measurable as topology/weight fingerprints — DNA detects logic shifts.

V · Systems

systems/evolution

Topology search and weight crossover need first-class splice + NEAT on CPU-resident grids.

V · Systems

systems/tween

Target propagation (chain-rule or Hebbian layerwise gaps) is an alternative credit-assignment path. Not the same package as TrainMode Tween / TweenChain on a Sandwich (those are layers/paral

V · Systems

systems/tanhi — TANHI · UDP HUD

Training visualization must never block the engine — best-effort UDP JSON-lines to a HUD.

V · Systems

systems/telemetry

Static structural blueprints (sizes, op kinds) differ from live TANHI events.

VI · Model IO

model/entity — .entity files

HF safetensors are awkward for native topology + packed weights. ENTITY is the Welvet checkpoint.

VI · Model IO

model/hf — snapshots

Import starts with probing HF/MLX layouts before packing ENTITY.

VI · Model IO

model/tokenizer

Generate needs encode/decode of HF tokenizer.json without pulling Python.

VI · Model IO

model/sampling

Logits → token ID needs ArgMax, TopK+temperature, penalties, and chat hygiene in one place.

VI · Model IO

model/transformer — generate

ENTITY packs must run as Llama-style decoders with KV cache, profiles (SIMD/WebGPU/fused), and chat templates.

VII · Apps

apps — octo · flux2 · mosstts

Products must not pollute engine packages. Octo is the model shell; flux2/mosstts are domain apps. Lucy races (AAI test41 / test48 / test50) are benches, not Welvet packages.

VII · Apps

Octo — model shell

A model is only useful with a shell around it: pull weights from Hugging Face, convert them to a Welvet .entity, then chat, serve, or benchmark. Octo is that shell, kept in its own module so

VIII · Stubs

stub/seed

Ship topology recipes (layer seeds → He-init) without weight blobs.

VIII · Stubs

stub/serialization

Volumetric grids need JSON/ENTITY persist beyond transformer packs.

VIII · Stubs

stub/memory

HF→ENTITY and GPU upload need footprint accounting and optional history charts.

VIII · Stubs

stub/donate

LAN donors should accept framed JSON jobs without embedding HTTP in the engine.

VIII · Stubs

stub/fountain

Recover specialist weight blobs over lossy links via LT fountain codes, then ensemble.

VIII · Stubs

stub/hardware

Dispatchers and UIs need a portable host audit (OS/CPU/RAM/GPU).

VIII · Stubs

stub/accel — NPU/Metal/QNN

Vendor accelerators (Intel NPU, Qualcomm QNN, Apple Metal) will plug beside WebGPU — not replace it.

VIII · Stubs

stub/clustering

Offline clustering helpers on tensors without inventing a second math stack.

VIII · Stubs

stub/ensemble

Combine multiple model votes and find complementary specialists.

VIII · Stubs

stub/evaluation

Benchmark grids through runtime/forward with deviation metrics.

VIII · Stubs

stub/grafting

Merge grids into Parallel/Residual structures for topology experiments.

VIII · Stubs

stub/grouping

Detect layer archetypes from safetensor-style names before mounting.

VIII · Stubs

stub/introspection

UIs and FFI need to list Grid methods without hardcoding every export.

VIII · Stubs

stub/observer

Attach forward/backward observers for debugging without coupling to tanhi UDP.

VIII · Stubs

stub/pipeline

Decoder wavefront stats helpers — not a full Lucy-style pipeline runner yet.

VIII · Stubs

stub/templates

Chat prompts must match model families (ChatML, Llama3, BitNet) without app-specific string glue.

VIII · Stubs

stub/universal

Probe unknown safetensor geometry and mount placeholder grids until full weight import lands.

IX · Validate

w2a — validation harness

Engine packages must stay free of tests. w2a owns timed 34×20×3 matrices, gap census, honesty stamps, and the train-mode permutation smoke (Test49). See §63 for a live full-suite run.

IX · Validate

Validation report — full suite

Claims are cheap; a stamped matrix is not. This is the actual output of one full w2a [0] Run ALL so the book's ✅ marks are backed by numbers you can reproduce, not asserted.

IX · Validate

Scorecard → v1.0 / minors

Version is earned from a weighted board, not marketing. v1.0 is 100/100 on the engine board. Minor/patch tags (v1.1.0, v1.1.1, …) pack features without a new board. Apps, stubs, and NPU sit

IV · Runtime

Cross-numeric train + down-the-dem

Weight storage dtype and activation Tensor[T] are independent axes. Proving train without a retained float32 master means sweeping W×A — not only matched float32 acts.

V · Systems

lucy — SoftAcc / Score measuring

Adaptation benches (test41-w, tide, live_gpt) need one shared measuring math — SoftAcc, Availability, AdaptPct, Score — not three copies of the formulas.

IV · Runtime

TrainMode — 29 named updates

Backprop is one update, not the only one. Credit assignment (broadcast gap, head proxy, sparse duty clock) has to be a named axis you can race — not a comment in a notebook. Cameral Mix also

VII · Apps

Cameral sandwiches + AAI Lucy

A single Dense chain cannot host two independent weight copies that share an input, merge, and optionally train under different updates. That is the cameral graph: hemispheres, not screen-sp

V · Systems

Lucy density — synthetic organism

A new host (char LM, MNIST, a layer sprint) should not copy tide's goldilocks math. The question is always the same: can the net run and train at the same time in a small box, then how far t

VII · Apps

CamSync — inter-cameral / cross-mesh weight blend

Mix BranchModes and different inits let cams diverge on purpose. Sometimes you want them to share — gently (1% pull) or hard (full average) — within a Parallel, across Stack children, or eve