Apple (Metal GPU / MPSGraph) bridge

Version: Loom v0.83.0 "Apple Bridge" — experimental (same maturity bar as the Intel & Qualcomm NPU bridges) Status: macOS on Apple silicon · Metal GPU (MPSGraph) + Accelerate/CPU reference · forward-only Plugin: libloom_accel_apple.dylib (Apple Metal Performance Shaders Graph inside) Lucy menu: [13] — log: lucy/lucy_testing_output/apple.txt

This is the Apple mirror of the Intel path in accelerators.md and the Qualcomm path in snapdragon_npu.md. It shares the identical vendor-neutral C ABI (accel/apple/include/loom_accel.h), the same poly/accel loader, the same SyncToAccel → DispatchLayer flow. Only the vendor backend differs: Metal / MPSGraph instead of OpenVINO or QNN.

TL;DR

The Apple plugin is the cleanest of the three vendor bridges to stand up, because there is no external SDK to vendor — Metal, MetalPerformanceShaders, and MetalPerformanceShadersGraph ship with macOS. There's no 1.3 GB download, no signed DSP skel, no LD_LIBRARY_PATH dance.

It exposes two devices behind the same C ABI:

• CPU — a portable, deterministic C++ reference forward for all 15 layers. This is the parity anchor (it plays the role QnnCpu plays for Qualcomm).
• GPU — a Metal / MPSGraph backend for the ops MPSGraph handles well (MatMul, MHA, ReLU, Sigmoid, Softmax, Add, Multiply). Anything it doesn't build silently falls back to the CPU reference, so every cell still returns a correct value.

The result: a real Loom → Apple GPU dispatch bridge with byte-perfect run-to-run determinism (180/180 on both devices) and multi-× speedups on large MatMul/MHA on the GPU and on elementwise ops on the CPU. It is not production-complete — Conv and the norms are still CPU-reference-only and slower than Loom's own CPU, and the usual low-bit MatMul drift is present. Treat it exactly like the Intel/Qualcomm bridges: proven plumbing to build on, not a "flip the GPU on for any network" feature.

What actually runs today

Measured from apple.txt (Lucy [13] → [5], the 10-layer DispatchLayer matrix × 6 dtypes × 3 sizes = 180 cells, SyncToAccel once per device). The raw CABI bench (menu [0]) additionally covers DepthwiseConv / AvgPool / MaxPool for the full 15-layer plugin surface.

✅ = compiles + infers on that device. ⚠️ CPU fallback = the GPU device transparently runs the CPU reference (correct result, no GPU speedup).

Every dtype runs on every layer — the matrix is FP32 / FP16 / BF16 / INT16 / INT8 / INT4. Compute is FP32 internally; the dtype only changes the wire byte layout (FP16/BF16 are 2 bytes, the rest hand over FP32 values, matching poly/accel_intel.go). BF16 is the Apple-native addition to the bridge — Apple silicon has native bfloat16.

Numbers that matter (from apple.txt)

Determinism — production-grade

Repeat-forward drift (same input twice) is 💎 EXACT on every cell, both devices:

Byte-identical outputs run to run — the single most important property for using either device as a real execution target.

Loom ↔ Apple parity

Full GPU bucket spread: 💎 6 · ✅ 126 · 🟨 36 (LOWBIT) · ❌ 12 (BROKE). Full CPU bucket spread: 💎 6 · ✅ 72 · 🟨 54 · 🟤 27 (H-DRIFT) · ❌ 21.

The ≤ INDUS count is higher on the GPU than the CPU because the GPU target carries a looser parity tolerance (GPU math legitimately reorders reductions) — the raw drift magnitudes for the two devices are nearly identical. So "GPU parity looks better" is a tolerance-bucket effect, not the GPU being more precise than the CPU reference.

Parity highlights:

• Softmax is effectively exact (~1e-9 FP32, ~1e-6 FP16/BF16) — the standout.
• MatMul / MHA / Conv sit at 🟨 LOWBIT → 🟤 H-DRIFT as tiers grow (FP32/FP16/BF16 accumulation order vs Loom's CPU matmul).
• LayerNorm / RMSNorm are ❌ BROKE at ~1.8 for FP32/BF16 — the CPU reference fills its own constant scale/bias instead of Loom's weights (identical root cause to the Intel bridge norms).
• INT8 MatMul/Conv/MHA drift is large (3 → 31) and goes ❌ BROKE on the large tier — the same Loom-dequant-matmul vs plugin-FP32 mismatch seen on Intel/Qualcomm.

Speed — GPU for big MAC, CPU for elementwise

Two clear winners depending on op class (Spd = Loom ÷ Apple, higher = Apple faster):

Metal GPU wins on large MatMul / MHA (the GPU has a ~0.5 ms launch floor, so it only pays off once the tensor is big enough):

Apple CPU reference wins big on elementwise (tight native loop, no launch cost):

Where both Apple devices lose:

• Small tier, GPU — the ~0.5 ms floor swamps sub-0.01 ms compute (0.02× on MatMul/ReLU).
• Conv1D / Conv2D, every tier — both devices run the CPU reference conv, which is ~0.24–0.27× Loom's own optimized CPU conv. Don't offload Conv yet.
• Norms — tiny ops, CPU-reference-only, ~0.3–0.6× (and parity broken anyway).

One-time compile cost

SyncToAccel compiles each layer once. The CPU reference is ~free; the MPSGraph build is the cost on the GPU device — and it is cheap compared to Hexagon/OpenVINO:

Compile is paid once per layer/device; steady inference is the sub-ms numbers above.

How it plugs into Loom (unchanged dispatch model)

reg, err := poly.DiscoverAppleAccel(accel.AccelConfig{
    AppleSO: accel.DefaultApplePath(),
})
if err != nil { /* no plugin — stay on Loom CPU */ }
defer reg.Close()

net, _ := poly.BuildNetworkFromJSON(spec)
net.Accel = reg

net.Layers[0].ExecTarget = accel.ExecAppleGPU // or ExecAppleCPU
_ = net.SyncToAccel("medium")                 // compile once + upload weights

out, _, _ := poly.ForwardPolymorphic(net, input)

ExecTarget values

Accelerator devices

Known gaps — what's left

Honestly experimental. What's still missing or weak:

When to offload

Rule of thumb: big matmuls → Metal GPU; heavy elementwise → Apple CPU; convolutions → keep on Loom CPU until the GPU conv path lands.

Build & run

No SDK download — just Xcode command-line tools (for the Metal frameworks). Full notes in accel/apple/README.md. Short version:

cd accel/apple
./build.sh                 # CMake → build/libloom_accel_apple.dylib

cd ../../lucy
CGO_ENABLED=1 go run .      # → [13] Apple GPU bridge
#   [4] medium DispatchLayer suite   (fast)
#   [5] full 10×6×3 matrix           (the apple.txt tables above)
#   [0] raw CABI matrix (all 15 layers, direct plugin)

accel.DefaultApplePath() walks up from cwd for accel/apple/build/libloom_accel_apple.dylib, or set LOOM_ACCEL_APPLE_DYLIB.

See also

• accelerators.md — the accel model + Intel NPU bridge + vendor support matrix
• snapdragon_npu.md — the Qualcomm/Hexagon mirror
• accel/apple/README.md — build / dtypes / layout
• numerical_types.md — Loom's 21 DTypes vs vendor bench dtypes
• dispatch.md — DispatchLayer routing