layers/parallel — MoE + cameral

github.com/openfluke/welvet/layers/parallel✅

Why it exists

Mixture-of-experts and multi-path cells need concat/add/avg/filter combines. Cameral graphs need sibling hemispheres that share input, merge outputs, and optionally train under distinct modes.

What it is

Parallel branches + Stack sandwiches. Hemispheres / Bicameral / Sandwich build nested multi-cameral nets. SetBranchModes + TrainStackMSE let each hemi use a different TrainMode (all 29 named updates — BP, Tween, Split, FastProxy, Sparse, Step*, Mesh*, …).

input x
   │
   ▼
 Dense stem
   │
   ├──────────┐
   ▼          ▼
 Hemi L     Hemi R     ← own Dense weights
 StepBP     TweenChain ← BranchModes
   │          │
   └──── add ─┘
         │
         ▼
    Dense head → ŷ → MSE

Cameral API (v1.0)

• Hemispheres / HemispheresFrom — n twins (Dense or mixed Ops), merged by add/avg/concat/filter
• Bicameral / Sandwich / BicameralFrom — stem → Parallel → head Stack
• SetBranchModes(...) — stamp per-hemi TrainMode
• TrainStackMSE — forward → MSE → per-branch update (honours BranchModes)
• ResidualGraft — skip around a Parallel F without Residual importing this package
• WriteCameralFile / LoadCameral — cameral .entity (in model/entity)
• CamSyncConfig / SetCamSync / SyncNow — soft/hard inter-cameral weight blend + cross-layer same-shape pairs → §70

Uniform Bi/Tri/Quad can train via Grid training.Step (one mode for the net). Mix jobs need the Stack path so each cameral actually uses its own mode.

Why hemispheres exist, and how AAI Lucy benches use them: §68. Inter-cameral weight sync (crazy cross-mesh same-shape blends): §70. All 29 named updates and their equations: §67.

package main

import (
	"fmt"

	"github.com/openfluke/welvet/core"
	"github.com/openfluke/welvet/layers/parallel"
	"github.com/openfluke/welvet/quant"
)

func main() {
	// Dense stem → 2 hemispheres (add) → Dense head
	s, err := parallel.Bicameral(8, 16, 1, core.ActivationLeakyReLU,
		core.DTypeFloat32, quant.FormatNone)
	if err != nil {
		panic(err)
	}
	// Mix: left hemi StepBP, right hemi StepTweenChain
	hemi := s.Children[1].(*parallel.Layer)
	hemi.SetBranchModes(parallel.ModeStepBP, parallel.ModeStepTweenChain)

	x := core.NewTensor[float32](1, 8)
	t := core.NewTensor[float32](1, 1)
	t.Data[0] = 0.5
	loss, err := parallel.TrainStackMSE(s, x, t, parallel.ModeStepBP, 0.01)
	fmt.Println("loss", loss, "err", err)
}

loss 0 err <nil>