16. layers/embedding

Part: III · Layers
Package: github.com/openfluke/welvet/layers/embedding
Status: ok — ✅

When

Building or training a net that needs the layers/embedding Op (also usable as a Parallel cam).

Where

import "github.com/openfluke/welvet/layers/embedding"

cd 16-embedding && source ../env.sh && go run .

Why

Token IDs must gather rows from a table — not a Dense MatVec — with scatter grads on backward.

What

Config{VocabSize, EmbeddingDim, SeqLen}; table on weights.Store; host gather on SIMD/WebGPU today.

Sample output (captured)

[1 4 16] <nil>

Live capture from go run ./cmd/runall on local ../../welvet (exit 0).

Source

Copied from the Welvet feature book examples (openfluke.github.io/welvet/examples/16-embedding).

package main

import (
	"fmt"

	"github.com/openfluke/welvet/core"
	"github.com/openfluke/welvet/layers/embedding"
)

func main() {
	l, err := embedding.New(embedding.Config{VocabSize: 32, EmbeddingDim: 16, SeqLen: 4})
	if err != nil {
		panic(err)
	}
	ids := core.NewTensor[float32](1, 4)
	ids.Data[0], ids.Data[1] = 3, 7
	_, y, err := embedding.Forward(l, ids)
	fmt.Println(y.Shape, err)
}