layers/gdn — gated delta net

github.com/openfluke/welvet/layers/gdn✅

Why it exists

Linear attention / decode-first mixers (Gated DeltaNet) need a first-class package under KindLinearAttn.

What it is

Exec CPU/SIMD/WebGPU; ForwardDecode; truncated BPTT. Timed matrix is Float32-primary: PermutationOK is f32 × FormatNone/BinaryPacked × CPU/SIMD/WebGPU; other cells GAP (declared), not FAIL.

Honesty: GDN GAP ≠ fail PermutationOK is Float32 × FormatNone/BinaryPacked × CPU/SIMD/WebGPU only. Exotic dtypes and other packed formats are declared GAP in the timed matrix. Truncated BPTT; blobs are not on the Dense Store dtype axis. Do not read GAP as “GDN is broken.”

Go example

examples/23-gdn/main.go

Run:cd welvet/examples/23-gdn && source ../env.sh && go run .

package main

import (
	"fmt"

	"github.com/openfluke/welvet/layers/gdn"
)

func main() {
	l, err := gdn.New(gdn.Config{
		HiddenSize: 64, NumKeyHeads: 4, NumValueHeads: 4,
		KeyHeadDim: 16, ValueHeadDim: 16, ConvKernel: 4,
	})
	if err != nil {
		fmt.Println("config rejected:", err)
		return
	}
	l.Reset()
	x := make([]float32, 64)
	y := make([]float32, 64)
	fmt.Println(l.ForwardDecode(x, y))
}

Output

exit 0 · last run via go run .

<nil>