Snapdragon (Hexagon NPU) bridge

Version: Loom v0.82.0 "Snapdragon Bridge" — experimental (same maturity bar as the Intel NPU bridge) Status: Windows on Snapdragon X · Hexagon v73 HTP + Kryo CPU reference · forward-only Plugin: loom_accel_qualcomm.dll (Qualcomm QNN AI Engine Direct inside) Lucy menu: [12] — log: lucy/lucy_testing_output/snapdragon.txt

This is the Snapdragon mirror of the Intel path in accelerators.md. It shares the identical vendor-neutral C ABI (accel/qualcomm/include/loom_accel.h), the same poly/accel loader, the same SyncToAccel → DispatchLayer flow. Only the vendor backend differs: QNN/Hexagon instead of OpenVINO/Intel.

TL;DR — this was a lot of work, and it works enough to demo

Getting Loom talking to the Hexagon NPU on Windows/ARM64 required clearing a long chain of blockers, none of which were "the neural net" itself:

• ARM64 SIMD kernels (neon_arm64.go) so the Loom CPU baseline runs natively.
• webgpu go.mod replace — the published static archive has an ABI mismatch on windows/arm64; both loom/go.mod and lucy/go.mod must point at a local build.
• QAIRT SDK acquisition — ~1.3 GB Qualcomm SDK behind a login, vendored under accel/qualcomm/deps/, plus the free HTP runtime with signed Hexagon skel/cat.
• A C++ QNN plugin built with llvm-mingw (no CMake/MSVC on the box) that loads QnnHtp.dll / QnnCpu.dll, builds a single-op graph per layer, quantizes weights, finalizes and executes it.
• Windows cgo loader (LoadLibraryExA + GetProcAddress) for the plugin.
• Machine-wide env so QNN_SDK_ROOT survives a fresh SSH login / any user.
• Bug hunts: duplicate QNN graph names failing every dtype after FP32; a graph leak that slowed the full matrix and could fault (0xc0000005) on Ctrl+C; a wall of DSP-transport log spam.

The result: a real Loom → Hexagon NPU dispatch bridge with deterministic inference and multi-× speedups on larger layers. It is not production-complete — plenty of layers/dtypes still fall back or error. Treat it exactly like the Intel bridge: proven plumbing to build on, not a "flip the NPU on for any network" feature.

What actually runs today

Measured from snapdragon.txt (Lucy [12] → [5], full 15-layer × 5-dtype × 3-size matrix, SyncToAccel once per device):

✅ = compiles + infers on Hexagon HTP (and QnnCpu where shown). ✗ = errors at graph build today (see Known gaps).

Elementwise/activation ops are the strong suit (FP32 + INT16/INT8/INT4). MatMul and MHA run FP32 on the NPU. Everything else is still on the to-do pile.

Numbers that matter (from snapdragon.txt)

Determinism — production-grade

Repeat-forward drift (same input twice) is 💎 EXACT on every cell that ran:

The NPU gives byte-identical outputs run to run — the single most important property for using it as a real execution target.

Loom ↔ Hexagon parity

NPU parity is tighter than CPU here because the QnnCpu reference backend is FP32-only and drifts more on the quantized activation ops. Softmax is near-exact (~1e-6); MatMul/MHA FP32 sit at 🟨 LOWBIT on HTP (H-DRIFT vs the CPU anchor).

Speed — HTP wins as layers grow

The Hexagon infer floor is ~0.5 ms (fixed launch cost), so small tensors lose but large tiers win big (Loom ÷ Hexagon, higher = NPU faster):

Manifest totals: Hexagon beat Loom on 25 / 150 cells, mostly the large tier. QnnCpu beat Loom on only 1 / 150 — it's an accuracy anchor, not a speed play.

One-time compile cost

SyncToAccel compiles each graph once. HTP graph prepare is the expensive part:

Compile is paid once per layer/device; steady inference is the sub-ms numbers above.

How it plugs into Loom (unchanged dispatch model)

reg, err := poly.DiscoverQualcommAccel(accel.AccelConfig{
    QualcommSO: accel.DefaultQualcommPath(),
})
if err != nil { /* no plugin — stay on Loom CPU */ }
defer reg.Close()

net, _ := poly.BuildNetworkFromJSON(spec)
net.Accel = reg

net.Layers[0].ExecTarget = accel.ExecQualcommNPU // or ExecQualcommCPU
_ = net.SyncToAccel("medium")                    // compile once + upload weights

out, _, _ := poly.ForwardPolymorphic(net, input)

ExecTarget values

Accelerator cores

* FP16 is native HTP silicon but currently errors in our graph build — see gaps.

Known gaps — a lot of work left

This is honestly experimental. What's still broken or missing:

Environment robustness

• QNN on Windows/Snapdragon usually can't load the signed Hexagon skel and falls back to the user driver path — this is normal, inference still works (Hexagon NPU: available). The chatter is filtered in Lucy [12] unless LOOM_QNN_VERBOSE=1.
• QNN_SDK_ROOT must be set machine-wide for fresh shells / other users; install_qairt.ps1 -Persist (elevated) does this.

Build & run

Full setup (SDK download, env persistence, llvm-mingw build, webgpu note) lives in accel/qualcomm/README.md. Short version:

cd accel/qualcomm
pwsh -File .\install_qairt.ps1 -Persist   # runtime + SDK detect + machine env (elevated)
. .\setup_env.ps1
.\build_clang.ps1                         # llvm-mingw → build/loom_accel_qualcomm.dll

cd ..\..\lucy
go run .                                  # → [12] Snapdragon NPU bridge
#   [4] medium DispatchLayer suite  (fast, ~9 s)
#   [5] full 15×5×3 matrix          (minutes — HTP compiles per graph)

Avoid Ctrl+C mid-compile on the full matrix — QNN can fault (0xc0000005).

See also

• accelerators.md — the accel model + Intel NPU bridge
• accel/qualcomm/README.md — install / build / env / webgpu
• numerical_types.md — Loom's 21 DTypes vs vendor bench dtypes
• dispatch.md — DispatchLayer routing
• windows_arm64.md — Windows on ARM build notes