openflukeWhat I was poking at
Experiments 01 and 02 were mostly “what wins under Lucy?” on machines I controlled. Experiment 03 asks the uglier question: what happens if you leave the whole Tide stack online, on cheap public cloud, at ~100% CPU, for days or weeks — reboots, noisy neighbors, flaky HTTP, the lot?
The farm was three Linode ~$5 boxes (peers c1 / c2 / c3), each running a Dockerized test51 Tide slice of the think-race tree. Ocean sat above them, polling boards and stitching a master report. This was never “finish every cell.” A 110k–122k leaf matrix on tiny VMs was always going to crawl. The experiment was longevity under load.
What “volatile online” meant
Public cloud VMs are not a lab bench. Disk stalls, neighbor noise, restarts, and dashboards fighting the training loop all count. Tide was configured to resume (`TEST51_RESUME`), so finished cell IDs skip after a bounce. Dashboards stayed up on `:8080` / `:5151` while the loop hammered the CPU. On c3 you could even see intermittent `/api/*` stalls while training logs kept going — the process did not die; the HTTP path just got crowded.
Welvet still mattered underneath: storage-truth dtypes, 29 TrainModes, Lucy live-fit (Throughput × Availability × Acc). A soak that “survives” by silently falling back to float32 would be a fake. The point was honest cells still ticking under chaos.
What showed up
Nothing broke. That is the headline. The peers kept serving, training, checkpointing, and emitting Lucy boards through a sustained full-CPU soak. Finishing only ~1–2% of the tree after a long run on $5 boxes is the expected price of a huge live matrix — not a failed experiment.
At the September extract, Ocean saw 2/3 Tides answering and ~3.9k cells across the farm. Holistic lean: StepTweenSplit + float64 + singlex1. Peer boards: c1 (~2.2%, 2675/122400) led on NormalBP float64 chase cells (~Lucy 679); c2 (~1.0%, 1207/122400) was hottest with StepTweenSplit float64 (~Lucy 1554, Availability ~95%); c3 was early after a restart (~54/110160) on MeshTweenAlt float64 (~Lucy 376) but still alive and scoring. Challenge cells (chase / flee / collect / shock) and grids up to 3×3×3 kept moving under live load.
So the science mid-run agrees with Experiments 01–02 — Tween/Split families keep Availability when serve and train share a box — while the ops result is simpler: cheap volatile cloud did not take the stack down.
What I take from it
Welvet + Tide + Lucy can live as a long-running online organism on budget VMs, not only as a laptop overnight job. Checkpoint resume and Ocean consolidation are part of the product story, not afterthoughts. Early leaders are signals, not finals — do not overclaim SOTA from a few percent of a giant tree.
Read EXPERIMENT.md and the research briefs for the stack map, then Ocean’s master PDF for cross-peer votes, then each c1/c2/c3 Tide report + board JSON for the raw soak snapshots.
Champion snapshot
Mid-run Lucy Score champions from the Sep 2026 extract. Progress is cells done vs plan — not “model converged.” Ocean holistic lean: Step[T][S] · float64 · singlex1.
| Machine | Cams | LR | Lucy Score | Winning cell |
|---|---|---|---|---|
| Linode c1 | slice A | 0.05 host | 679.42 | NormalBP · float64 · chase 3×3×3 · ~2.2% (2675/122400) |
| Linode c2 | slice B | 0.05 host | 1553.61 | Step[T][S] · float64 · chase 2×2×2 · Avail ~95% · hottest peer |
| Linode c3 | slice C | 0.05 host | 376.30 | Mesh[T]Alt · float64 · early after restart · ~54/110160 · still up |
| Ocean | 2/3 up | farm | 1553.61 | Master lean Step[T][S] · float64 · ~3922 cells across answering peers |