Cameral learning-rate sweeps

What I was poking at

This was the first big “does cameral actually matter under live load?” play session. Welvet can stack sibling hemispheres that share an input, merge, and optionally train under different update rules — that sandwich is the cameral graph. I wanted to see what happens when you change how many cams you give the net and how aggressively you step the learning rate, while Lucy keeps score on whether the model can still serve while it learns.

The harness is Tide’s GPT-char race: live_gpt on tinyshakespeare, 4096 windows, sequence 32, causal MHA, A→B→A2, SIMD. Each finished report is one epoch that completed all 986 cells in the permute tree (dtypes × train modes × formats for that host). The PDFs below are the Tide Lucy reports for those finishes — not polished papers, just the scoreboards that fell out of the runs.

The setup, in plain terms

Lucy Score is live-fit: Throughput × Availability × Accuracy / 10,000. A cell that trains hard but blocks serving dies on Availability. SoftAcc is serve-confidence only. LPD (Lucy Pareto density) asks how much of that live-fit you keep when you shrink the weight box versus the accuracy champion. The reports also flag a “mobile score (trap)” — tiny dtypes that look dense and cheap without real live-fit quality.

Across the series I varied cameral width (1 on desktop, 3 on M4, 5 on M5, 8 and 15 on the Nitros) and learning rate from a polite 0.02 up through 2, 20, 200, and even 1000 on the M4. Same character-level Shakespeare toy; different machines doing the sweeping.

What showed up

On the Mac M4 tricameral sweeps, Lucy Score climbed as the learning rate left the textbook zone. At lr 0.02 the champion sat around 21 with bfloat16 Tween/Split/Linear. At lr 2 it jumped past 31 on int4. Peak of this series was lr 20: Lucy Score 35.3 on int8 with Tween/Split/Linear Cache. Higher still (200 and 1000) stayed strong in the low-thirties but did not beat that mid-high sweet spot. Hard accuracy champions often wore different clothes than the Lucy champions — accuracy alone kept missing the serve-while-learn story.

Mac M5 with five cams told a similar story at a slightly lower ceiling: lr 0.02 topped out near 18 on bfloat16, then lr 2–200 clustered around 29–30 with int/binary sandwich modes. More cams did not automatically win. Nitro 51 with fifteen cams peaked at 22.8 (lr 2), and Nitro 52 with eight cams at lr 0.02 only reached about 6 — closer to the desktop single-cam baseline (~6) than to the M4 tricameral peak.

Low learning rates favoured familiar float32 / bfloat16 step and tween modes. Once the rate got spicy, low-bit integer sandwich modes (especially Tween/Split/Linear and its Cache variant) took the Lucy crown. Binary and other tiny dtypes kept lighting up the mobile-score trap: they can look “efficient” on paper while Lucy’s live-fit read is telling a different story.

What I take from it

Cameral width and learning rate are a real joint surface, not two independent knobs. Three cams on a fast Apple Silicon box, with mid-high learning rates and sandwich credit, produced the liveliest Lucy winners in this archive. Fifteen cams on a Nitro did not magically outscore that. Polite SGD at 0.02 is not the interesting region for this live-fit metric — the interesting region is where aggressive steps meet train modes that keep Availability alive.

Treat each PDF as a finished Tide snapshot for that host / cam / lr combo. The table below is the Lucy Score champion from page one of each report; open the PDF for the full axis champions, per-mode boards, and the LPD / gold-standard columns.