Book contents +

XVII · Python C-ABI (welvet on PyPI) / CHAPTER 29

Python C-ABI: MHA place and prepare_grid

Samuel Watson · Welvet feature bookOriginal source ↗

welvet (PyPI)cpu · accel prepare


Why it exists

Not every layer is Dense. Python place takes the same kind strings as Go; some accelerators need an explicit prepare/Sync before forward.

What it is

Place an MHA layer with a shaped forward, list layer kinds, and show the prepare_grid call used for qnn/mlx mounts.

place kinds

place(handle, kind, spec) accepts Dense, Softmax, RMSNorm, LayerNorm, MHA, SwiGLU, CNN*, LSTM, Embedding, Residual, Sequential, and friends: see list_layer_types(). Shaped tensors use the shape= argument on forward.

prepare_grid runs SyncTo* for backends that need an explicit mount (notably qnn / mlx). Routed CUDA/Metal/ROCm/Vulkan usually do not need it after create_grid(backend=…). Release assets: GitHub cabi-v1.1.2 · PyPI welvet==1.1.2.

Python example

examples/117-python-layers/main.py

Run:cd welvet/examples/117-python-layers && python3 main.py
"""117. Python C-ABI: MHA place and prepare_grid."""
from __future__ import annotations

import welvet as w


def main() -> None:
    w.assert_engine_version()
    kinds = w.list_layer_types()
    print("layer_types", len(kinds), "sample", kinds[:8])

    g = w.create_grid(backend="cpu_tiled")
    try:
        w.place(
            g,
            "mha",
            {
                "dim": 8,
                "DModel": 8,
                "NumHeads": 2,
                "SeqLen": 4,
                "dtype": w.DType.FLOAT32,
            },
        )
        # batch=1, seq=4, dim=8
        out = w.forward(g, [0.1] * 32, shape=[1, 4, 8])
        print("mha_out_len", len(out))

        # For qnn / mlx grids, call prepare_grid after place_* before forward:
        #   info = w.prepare_grid(g)
        #   print("prepare", info)
        print("prepare_grid", "call after place_* when backend is qnn or mlx")
    finally:
        w.free_grid(g)


if __name__ == "__main__":
    main()

Output

exit 0 · last run via python3 main.py

layer_types 23 sample ['Dense', 'MultiHeadAttention', 'SwiGLU', 'RMSNorm', 'LayerNorm', 'Embedding', 'Softmax', 'Sequential']
mha_out_len 32
prepare_grid call after place_* when backend is qnn or mlx

Validate (harness)

cd welvet/examples/117-python-layers && PYTHONPATH=../../../w2a/python/src python3 main.py