Chapter 65

Cross-numeric train + down-the-dem

github.com/openfluke/welvet/runtime/training✅

Why it exists

Weight storage dtype and activation Tensor[T] are independent axes. Proving train without a retained float32 master means sweeping W×A — not only matched float32 acts.

What it is

W2A Step Cross-Numeric Train: polyops.AllKinds() × FormatNone weight dtype × Go Numeric act host (smoke ~735; full ~10.7k) via StepMesh, then assert no retained f32 master. Public Dense volumetric showcase: down-the-dem — dtype demotion ladder, packed quants, and the full 34×15×3 perm matrix with charts/PDF.

down-the-dem — same 5-layer Dense stack on cubes 1³ / 2³ / 3³; only the number story changes:

• dtype — FormatNone demotion (wide → 1-bit), in-dtype SGD
• quant — packed formats; unpack → update → re-Pack
• perm — every weight dtype × every act host (1,530 cells on three grids)

Repo + report: github.com/openfluke/down-the-dem · PDF: report/down-the-dem.pdf in that repo. Honesty: runnable storage-truth train ≠ task accuracy or native low-precision GEMV ALU.

package main

import (
	"fmt"

	"github.com/openfluke/welvet/architecture"
	"github.com/openfluke/welvet/core"
	"github.com/openfluke/welvet/layers/dense"
	"github.com/openfluke/welvet/quant"
	"github.com/openfluke/welvet/runtime/training"
)

func main() {
	// Weight storage: int8 FormatNone. Activations/grads: Tensor[int8].
	g := architecture.NewGrid(1, 1, 1, 1)
	init := make([]float32, 4*4)
	for i := range init {
		init[i] = 0.1
	}
	l, err := dense.NewConfigured(4, 4, core.ActivationLinear, core.DTypeInt8, quant.FormatNone, init)
	if err != nil {
		panic(err)
	}
	if err := dense.Place(g, 0, 0, 0, 0, l); err != nil {
		panic(err)
	}
	x := core.NewTensor[int8](1, 4)
	y := core.NewTensor[int8](1, 4)
	for i := 0; i < 4; i++ {
		x.Data[i] = int8(i + 1)
		y.Data[i] = int8(i)
	}
	loss, _, err := training.StepMesh(g, x, y, 1, 0.05)
	fmt.Println(loss, err, l.Weights.RetainsF32Master())
}

1.5 <nil> false