NVIDIA CUDA: mount and forward

github.com/openfluke/welvet/nativecudalinux/windows amd64

Why it exists

Run a Go layer on NVIDIA GPU with an explicit lifetime.

What it is

Mount an identity Dense layer, inspect the CUDA version and device identity, and forward a Float32 tensor.

Session ownership

Create a session with nativecuda.Mount, call Forward repeatedly and close it when finished. Mount accepts supported layers, Stack, Parallel and Grid layouts. The library is independent of welvet.cpp and welvet.metal.

Requirements and backend boundaries

Go Welvet v1.4.0 uses the independent welvet.cuda 0.79 library (mounted model ABI 1). Install from its checkout:

cd ../welvet.cuda
./scripts/linux_install_go.sh
# Windows amd64 (MSVC + CUDA Toolkit):
./scripts/windows_install_go.ps1

Execution requires Linux x86_64 or Windows amd64, Go with cgo, and an NVIDIA driver/runtime. Set WELVET_CUDA_LIBRARY to override library discovery. On Windows, ensure the CUDA runtime (cudart) resolves via PATH. Ordinary Go CPU use does not require an NVIDIA GPU. Unsupported hosts print SKIP; missing libraries or no GPU on a supported host also soft-SKIP in these book examples.

Go schedules work and manages memory; numerical CUDA forward, backward and weight updates execute device kernels. Activations are Float32; FormatNone parameter storage follows the CUDA dtype matrix. Small examples demonstrate APIs; GPU launch costs can outweigh useful work at small shapes.

cpu-tiled / go_cpu means Go CPU; native_cpu means welvet.cpp; cuda means the independent NVIDIA GPU library. Examples 89-93 can use go run -tags welvet_cuda . to exclude C++ linkage. Chapter 94 requires the normal cgo build for both native engines. JSON routing (§93-§94) is Linux amd64 today; explicit Mount works on Windows amd64 as well.

// Mount one layer, execute its forward pass on NVIDIA CUDA, and release it.
package main

import (
	"flag"
	"fmt"
	"log"
	"runtime"

	"github.com/openfluke/welvet/core"
	"github.com/openfluke/welvet/layers/dense"
	"github.com/openfluke/welvet/nativecuda"
	"github.com/openfluke/welvet/quant"
)

func main() {
	if e := run(); e != nil {
		log.Fatal(e)
	}
}

func run() error {
	library := flag.String("library", "", "optional libwelvet_cuda.so/.dll path")
	flag.Parse()
	if !cudaPlatformOK() {
		fmt.Println("SKIP: NVIDIA CUDA needs linux/amd64 or windows/amd64 and cgo")
		return nil
	}
	l, e := dense.NewConfigured(2, 2, core.ActivationLinear, core.DTypeFloat32, quant.FormatNone, []float32{1, 0, 0, 1})
	if e != nil {
		return e
	}
	session, e := nativecuda.Mount(l, nativecuda.Options{LibraryPath: *library})
	if e != nil {
		fmt.Printf("SKIP: CUDA unavailable (%v)\n", e)
		return nil
	}
	defer session.Close()
	x := &core.Tensor[float32]{Shape: []int{1, 2}, Data: []float32{.5, -.25}}
	post, e := session.Forward(x)
	if e != nil {
		return e
	}
	fmt.Printf("backend=cuda version=%s\ndevice=%s\ninput=%v output=%v shape=%v\n", session.Version(), session.Identity(), x.Data, post.Data, post.Shape)
	return nil
}

func cudaPlatformOK() bool {
	return runtime.GOARCH == "amd64" && (runtime.GOOS == "linux" || runtime.GOOS == "windows")
}

backend=cuda version=0.79
device=cuda:NVIDIA GeForce GTX 1650 SUPER:idx=0:sm=7.5:mem=3902799872
input=[0.5 -0.25] output=[0.5 -0.25] shape=[1 2]

cd welvet/examples/89-nvidia-cuda && source ../env.sh && go run .