27. layers/parallel — MoE + cameral

Part: III · Layers
Package: github.com/openfluke/welvet/layers/parallel
Status: ok — ✅

When

Building or training a net that needs the layers/parallel Op (also usable as a Parallel cam).

Where

import "github.com/openfluke/welvet/layers/parallel"

cd 27-parallel && source ../env.sh && go run .

Why

Mixture-of-experts and multi-path cells need concat/add/avg/filter combines. Cameral graphs need sibling hemispheres that share input, merge outputs, and optionally train under distinct modes.

What

Parallel branches + Stack sandwiches. Hemispheres / Bicameral / Sandwich build nested multi-cameral nets. SetBranchModes + TrainStackMSE let each hemi use a different TrainMode (all 29 named updates — BP, Tween, Split, FastProxy, Sparse, Step*, Mesh*, …).

Sample output (captured)

loss 0 err <nil>

Live capture from go run ./cmd/runall on local ../../welvet (exit 0).

Source

Copied from the Welvet feature book examples (openfluke.github.io/welvet/examples/27-parallel).

package main

import (
	"fmt"

	"github.com/openfluke/welvet/core"
	"github.com/openfluke/welvet/layers/parallel"
	"github.com/openfluke/welvet/quant"
)

func main() {
	// Dense stem → 2 hemispheres (add) → Dense head
	s, err := parallel.Bicameral(8, 16, 1, core.ActivationLeakyReLU,
		core.DTypeFloat32, quant.FormatNone)
	if err != nil {
		panic(err)
	}
	// Mix: left hemi StepBP, right hemi StepTweenChain
	hemi := s.Children[1].(*parallel.Layer)
	hemi.SetBranchModes(parallel.ModeStepBP, parallel.ModeStepTweenChain)

	x := core.NewTensor[float32](1, 8)
	t := core.NewTensor[float32](1, 1)
	t.Data[0] = 0.5
	loss, err := parallel.TrainStackMSE(s, x, t, parallel.ModeStepBP, 0.01)
	fmt.Println("loss", loss, "err", err)
}