89. NVIDIA CUDA: mount and forward

Part: XIII · NVIDIA acceleration (welvet.cuda)
Package: nativecuda
Status: ok: needs Linux/Windows amd64 + NVIDIA runtime

When

Mount a Go layer and execute its forward pass on NVIDIA GPU.

Where

From example/welvet/89-nvidia-cuda:

go run .
go run . -library /path/to/libwelvet_cuda.so

Why

See the backend version and CUDA runtime identity; release the session explicitly.

What

Creates an identity Dense layer, mounts it, forwards a Float32 tensor, and closes the session.

Requirements

Linux x86_64 or Windows amd64, Go with cgo, NVIDIA driver/runtime, and the installed independent CUDA shared library. Set WELVET_CUDA_LIBRARY or install from the CUDA checkout:

cd ../welvet.cuda   # sibling of welvet
./scripts/linux_install_go.sh
# Windows: ./scripts/windows_install_go.ps1

Use the Go v1.4.0 checkout containing nativecuda. Examples 89-93 can also use go run -tags welvet_cuda . to exclude the independent C++ binding. Example 94 needs the normal build to keep both native backends. Unsupported platforms print SKIP. Missing libraries / no GPU on a supported host also print SKIP.

These are usage examples with small shapes. For speed matrices, use w2a go_v_cuda / the fat online suite. Results are numerical CPU/GPU comparisons, not model prediction accuracy.

Backend labels in routed examples: cpu-tiled / go_cpu is Go CPU, native_cpu is welvet.cpp, and cuda is the independent NVIDIA GPU backend.

Captured sample

backend=cuda version=0.79
device=cuda:…
input=[0.5 -0.25] output=[0.5 -0.25] shape=[1 2]

// Mount one layer, execute its forward pass on NVIDIA CUDA, and release it.
package main

import (
	"flag"
	"fmt"
	"log"
	"runtime"

	"github.com/openfluke/welvet/core"
	"github.com/openfluke/welvet/layers/dense"
	"github.com/openfluke/welvet/nativecuda"
	"github.com/openfluke/welvet/quant"
)

func main() {
	if e := run(); e != nil {
		log.Fatal(e)
	}
}

func run() error {
	library := flag.String("library", "", "optional libwelvet_cuda.so/.dll path")
	flag.Parse()
	if !cudaPlatformOK() {
		fmt.Println("SKIP: NVIDIA CUDA needs linux/amd64 or windows/amd64 and cgo")
		return nil
	}
	l, e := dense.NewConfigured(2, 2, core.ActivationLinear, core.DTypeFloat32, quant.FormatNone, []float32{1, 0, 0, 1})
	if e != nil {
		return e
	}
	session, e := nativecuda.Mount(l, nativecuda.Options{LibraryPath: *library})
	if e != nil {
		fmt.Printf("SKIP: CUDA unavailable (%v)\n", e)
		return nil
	}
	defer session.Close()
	x := &core.Tensor[float32]{Shape: []int{1, 2}, Data: []float32{.5, -.25}}
	post, e := session.Forward(x)
	if e != nil {
		return e
	}
	fmt.Printf("backend=cuda version=%s\ndevice=%s\ninput=%v output=%v shape=%v\n", session.Version(), session.Identity(), x.Data, post.Data, post.Shape)
	return nil
}

func cudaPlatformOK() bool {
	return runtime.GOARCH == "amd64" && (runtime.GOOS == "linux" || runtime.GOOS == "windows")
}