Apple Metal: Mamba results and stress runner

Samuel Watson · Welvet feature bookOriginal source ↗

captured snapshot

Why it exists

Measure real end-to-end Go bindings on sufficiently large models.

What it is

Display a labelled captured Mamba forward excerpt, then run the reproducible Go/C++/Metal stress matrix on your machine.

Captured forward excerpt

These user-supplied measurements were recorded on 2026-10-27. They are a historical snapshot, not a benchmark performed by loading this page. The five rows below are also printed by the example.

Dtype	Go ms	C++ ms	Metal ms	Metal speedup vs Go	Metal speedup vs C++
float64	89.314	236.242	28.959	3.084×	8.158×
float32	69.019	225.813	29.010	2.379×	7.784×
float16	190.545	1155.902	28.947	6.583×	39.932×
bfloat16	149.351	1049.180	29.003	5.149×	36.174×
fp8e4m3	265.882	1519.874	29.159	9.118×	52.124×

All 12 supplied forward rows reported Go/Metal exact-bit and tolerance agreement of 100%/100%. C++ unsigned storage differed in exact bits (uint64 21.2%, uint32 45.4%) while tolerance agreement remained 100%. This is numerical agreement, not prediction accuracy or a guarantee for future training runs.

Run the matrix

The bundled stress runner documentation describes all metrics and the full excerpt. It compares Mamba across 34 parameter dtypes and 29 training modes: 34 forward rows plus 986 training rows, or 1,020 rows, each covering three backends. Use the normal cgo build with both native libraries installed.

cd welvet/examples/apple_stress_test
./run.sh -profile smoke -samples 1 -iters 1 -steps 8
./run.sh

The default stress profile uses DModel 512, batch 8, sequence 32, three samples, three iterations, eight training steps and one warm-up. Reports include median wall time, setup and final weight-pull costs, speed ratios, time saved, exact-bit and tolerance agreement, and repeatability. CSV/JSONL/summary JSON are written progressively under results/.

Small-shape verification covered the full matrix; a separate stress-size Float32 NormalBP check measured Metal gains. That evidence does not claim the full large-shape matrix has already been timed. CPU/GPU numerical differences can grow over long training horizons. Best effort aims at exact agreement; project acceptance labels use 60/80/100 tolerance bands, not percentages of correct tensor elements or model accuracy.

Requirements and library installation: chapter 82.

Go example

examples/88-apple-mamba-results/main.go

Run:cd welvet/examples/88-apple-mamba-results && source ../env.sh && go run .

// A labelled snapshot of user-provided forward results; no GPU run is implied.
package main

import "fmt"

func main() {
	fmt.Println("Captured Mamba forward excerpt, supplied 2026-10-27 (not a live benchmark)")
	fmt.Printf("%-10s %12s %12s %12s %13s %14s\n", "dtype", "Go ms", "C++ ms", "Metal ms", "Metal/Go x", "Metal/C++ x")
	rows := []struct {
		dtype                string
		goUS, cppUS, metalUS float64
	}{
		{"float64", 89314.5, 236241.7, 28958.6}, {"float32", 69018.5, 225812.9, 29010.4},
		{"float16", 190545.0, 1155902.2, 28946.8}, {"bfloat16", 149351.3, 1049179.8, 29003.3},
		{"fp8e4m3", 265881.6, 1519874.3, 29159.0},
	}
	for _, r := range rows {
		fmt.Printf("%-10s %12.4f %12.4f %12.4f %13.3f %14.3f\n", r.dtype, r.goUS/1000, r.cppUS/1000, r.metalUS/1000, r.goUS/r.metalUS, r.cppUS/r.metalUS)
	}
	fmt.Println("Go/Metal exact bits and within-tolerance agreement: 100%/100% across all 12 supplied forward rows.")
	fmt.Println("Full excerpt and metric definitions: ../apple_stress_test/README.md")
	fmt.Println("Run your own 34-dtype / 29-mode matrix: cd ../apple_stress_test && ./run.sh")
	fmt.Println("Small verification: ./run.sh -profile smoke -samples 1 -iters 1 -steps 8")
}

Output

exit 0 · last run via go run .

Captured Mamba forward excerpt, supplied 2026-10-27 (not a live benchmark)
dtype             Go ms       C++ ms     Metal ms    Metal/Go x    Metal/C++ x
float64         89.3145     236.2417      28.9586         3.084          8.158
float32         69.0185     225.8129      29.0104         2.379          7.784
float16        190.5450    1155.9022      28.9468         6.583         39.932
bfloat16       149.3513    1049.1798      29.0033         5.149         36.174
fp8e4m3        265.8816    1519.8743      29.1590         9.118         52.124
Go/Metal exact bits and within-tolerance agreement: 100%/100% across all 12 supplied forward rows.
Full excerpt and metric definitions: ../apple_stress_test/README.md
Run your own 34-dtype / 29-mode matrix: cd ../apple_stress_test && ./run.sh
Small verification: ./run.sh -profile smoke -samples 1 -iters 1 -steps 8

Validate (harness)

cd welvet/examples/88-apple-mamba-results && source ../env.sh && go run .

Explore the companion example and source code →

← Apple Metal: mixed Go/C++/Metal