Neural Fountain — shard specialists · LT peel · Master ensemble

Neural Fountain is a training / assembly paradigm in poly/ that mirrors LT fountain codes on network weights instead of data bytes.

Pixel fountain recovers K source blocks byte-exact via spray / peel.
Neural Fountain recovers K specialist weight blobs byte-exact the same way, then exposes a Master ensemble.

Learning happens in poly.Train on each shard. Fountain is transport / reassembly, not the optimizer. No layer_seed search — contrast seed_manifests.md.

Companion MNIST demo: chaosglue/loom_neural_fountain. Related (data-only LT): chaosglue/loom_fountain_codes.

Pipeline

1. Partition training batches into K shards (every sample covered).
2. Specialize — NetworkFactory(i) builds any architecture; Train on shard i (optional UseExactDType / UniformDType).
3. Pack — recursive FP32 Master (+ aux + Scale) over the full layer tree → []byte.
4. Fountain — LT XOR spray (lossy OK) → peel until K/K recovered.
5. Unpack → Masters restored → ForceMorph(layer.DType) for each layer’s numerical type → Master ensemble.

Specialists must share an identical parameter layout (same walk order / lengths). Architecture is free via NetworkFactory (dense, CNN, residual, MHA, parallel/sequential nests, …).

Any layer type / any numerical type

Layers

PackNetworkWeights / UnpackNetworkWeights recursively visit:

• top-level Layers
• ParallelBranches, SequentialLayers
• FilterGateConfig, MetaObservedLayer

and pack, per layer:

• WeightStore.Master (FP32 persistence space — Loom’s SoT)
• WeightStore.Scale
• QNormWeight, KNormWeight, InnerNormWeight (MHA / BitNet-style aux)

Any layer that stores trainable state there participates. Nested stacks included.

Numerical types

Loom persists through FP32 Masters, then morphs to the layer’s DType (any of the supported storage types):

• After unpack: MorphNetworkToLayerDTypes → ForceMorph(layer.DType)
• ApplyUniformDType(net, dtype) forces one dtype on the whole tree before train
• UseExactDType: true enables native-dtype train/forward paths during specialize

So fountain cargo is bit-exact FP32 Masters; runtime dtype is whatever each layer is set to (float16, int8, …), including exact-dtype training.

Activations / TrainingBatch remain float32 (standard Loom train API); weights can be exact-dtype.

Core API

// Dense example with mixed layer dtypes:
factory := poly.DenseSpecialistFactory("net",
    []int{784, 128, 64, 10},
    []string{"float16", "float16", "float32"})

// Or any architecture:
factory := func(i int) (*poly.VolumetricNetwork, error) {
    return buildMyCNNOrMHA(i) // identical layout across i
}

cfg := poly.DefaultNeuralFountainConfig()
cfg.K = 16
cfg.UseExactDType = true
// cfg.UniformDType = poly.DTypeFloat16  // optional blanket morph

master, err := poly.NeuralFountain(factory, batches, cfg)
out, err := master.Forward(input)

Config knobs

Master semantics

• Forward / ForwardArgmax — average specialist outputs (deployable).
• OracleForward / OracleArgmax — shard expert owns sample i (coverage check).
• Prefer ensemble Master; do not average differently trained weight blobs into one net.

Honest scope

• Fountain does not invent weights without specialist Train.
• Pack path is Loom’s FP32 Master SoT (+ aux); natives are rematerialized via ForceMorph.
• Layout equality across specialists is required (same recursive float count).
• LT recover is probabilistic; large blobs need enough spray overhead.