Regenerate `cust_raw`for newer CUDA / support multiple versions

[LegNeato]

Now that we are adding support for newer CUDA (#100), we need a way to select the version for cust_raw's bindgen (and elsewhere?) See:

Rust-CUDA/crates/cust_raw/README.md

Line 6 in 400f816

Current version is based on CUDA 11.2

[jorge-ortega]

Currently we have a checked-in generated bindings that seem to be for windows based on the repr(i32) value for enums. That probably won't break linux, and we could update current binding and also generate linux bindings, but neither seems ideal.

In the usual bindgen flow, the user selects the version through whatever they have installed on their machine, and cust_raw (typically -sys) crate would generate bindings for that version. The cust crate can then configure itself through reading the version used to generate the bindings.

Say we have support up to 12.8, and the user has 12.0 installed. cust should be able to support APIs up to 12.0 and disable newer once through its configuration. If the user has a newer version, then at best cust could still compile without any support for newer features/APIs, but the user should still be able to access them through the sys crate. At worst, we don't compile due to some backwards incompatible change with the newer version.

[LegNeato]

Right, we need to decide if we generate and check in plus the hide the variants behind targets + features or we generate on the fly via build.rs and what is installed. The former is sort of gross but we see the complete picture and changes for each version and have more visibility when bugs are filed. The later means cross compilation is unsupported.

[jorge-ortega]

I gave generating bindings at build time a shoot and have a WIP branch focusing on cuda 12 and up for now on Windows. I built with versions 12.0, 12.1, 12.3, and 12.8. The only breaking changes were in the graph api. 12.3 introduced conditional nodes, so cust checks support for that and sets a cfg so the wrapper enum can account for it. With these changes, cust is able to build for on every version I tested. 12.0 can't codegen thought, as libnvvm doesn't like our data layout for some reason, but 12.1 and up were ok with it.

[adamcavendish]

I gave generating bindings at build time a shoot and have a WIP branch focusing on cuda 12 and up for now on Windows. I built with versions 12.0, 12.1, 12.3, and 12.8. The only breaking changes were in the graph api. 12.3 introduced conditional nodes, so cust checks support for that and sets a cfg so the wrapper enum can account for it. With these changes, cust is able to build for on every version I tested. 12.0 can't codegen thought, as libnvvm doesn't like our data layout for some reason, but 12.1 and up were ok with it.

Hey @jorge-ortega , I'm also working on this and I have just saw your changes. I can help write the rest if something else is keeping your hands full or I can help test on Linux distributions on CUDA 12.8 if it's ready.

[jorge-ortega]

Hey @adamcavendish, would be great if you could give the changes a try on linux.

[adamcavendish]

I gave generating bindings at build time a shoot and have a WIP branch focusing on cuda 12 and up for now on Windows. I built with versions 12.0, 12.1, 12.3, and 12.8. The only breaking changes were in the graph api. 12.3 introduced conditional nodes, so cust checks support for that and sets a cfg so the wrapper enum can account for it. With these changes, cust is able to build for on every version I tested. 12.0 can't codegen thought, as libnvvm doesn't like our data layout for some reason, but 12.1 and up were ok with it.

Hi @jorge-ortega, I've looked into your branch and I'd like to discuss a few considerations to refine the implementation of the bindgen-generated bindings. I know things are not finalized yet so there might be ideas that we think in common and you just didn't finish. I'm just raising these out for some discussions ahead of time.

1. Recommend Moving Bindings to OUT_DIR

Generating bindings directly into the Git workspace introduces unstaged files, which complicates git status tracking and risks conflicting with intentional changes. Additionally, this approach limits us to a single binding version (e.g., cust_raw/nvvm), conflicting with our goal of supporting multiple CUDA versions.

The bindgen documentation recommends placing generated files in the OUT_DIR to ensure they are treated as build artifacts. This approach keeps the repository clean and allows version-specific bindings to be managed dynamically during build time.

2. Suggest Renaming Crates to Follow -sys Convention

To align with Rust's ecosystem conventions for system bindings (e.g., *-sys naming), renaming:

• cust_raw → cust-sys
• nvvm → nvvm-sys

This improves discoverability and adheres to crate naming standards for FFI crates. The current cust crate already aliases cust_raw as sys (see here), which supports this transition.

3. Enable non_exhaustive Enums for Backward Compatibility

To ensure compatibility across CUDA versions, we should mark generated enums (e.g., GraphNodeType) as non_exhaustive. This prevents breaking changes when new variants are introduced in future CUDA headers.

Example configuration for bindgen:

let bindings = bindgen::Builder::default()
    .header("wrapper.h")
    // ... allowlist_types ...
    .default_enum_style(bindgen::EnumVariation::Rust {
        non_exhaustive: true,
    });

The from_raw implementation should handle unknown variants gracefully:

#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
#[repr(u8)]
#[non_exhaustive]
pub enum GraphNodeType {
    // ... existing variants ...
}

impl GraphNodeType {
    pub fn from_raw(raw: cuda::CUgraphNodeType) -> Self {
        match raw {
            // ... known variants ...
            _ => GraphNodeType::Empty, // Fallback for unrecognized values
        }
    }
}

4. Simplify Version Checks via Direct CUDA_VERSION Usage

Instead of parsing CUDA versions from a JSON file, leverage the cust_raw::CUDA_VERSION constant in cust's build.rs. This reduces parsing errors and simplifies conditional compilation.

Additionally, use version-based feature flags (e.g., cuda_12_3_0) rather than feature flags tied to individual API changes (e.g., conditional_node). This approach encapsulates all version-specific changes under a single gate, improving maintainability.

Example build.rs code:

use cust_raw::CUDA_VERSION;

fn main() {
    if CUDA_VERSION >= 12030 {
        println!("cargo:rustc-cfg=cuda_12_3_0");
    }
}

[jorge-ortega]

Thanks @adamcavendish for providing this feedback, really appreciate it. I'm mostly in alignment with your suggestions.

1. my changes are aligned here, so I read this as mostly being in favor of of generating bindings at build.

Technically, we're not limited to a single version if we do pre-gen bindings. See cudarc (more on this below).

2. I wanted to suggest this as well. Ideal we would name this 'cuda-sys', but that crate name has already been taken by a completely different now archived project. Maybe there's a chance we can take over that?

Since the SDK contains several different libraries, and different abstractions (driver vs runtime api) I was also considering consolidating bindings to a single crate (cuda-sdk-sys), and using features flags to allow users/downstream crates to pick which libraries to generate and link against. There might be some benefits with this (consolidated build logic, maybe better build times?) and I'm sure there are downsides I just haven't thought of yet.

3. I'm good with this approach, so long as we can agree on what graceful means depending on the context. Sometimes the most graceful thing you can do is panic.

4. I was hesitant to link against the sys crate in the build script as it seemed a bit heavy handed just to get the version constant. I was also considering cases where users might build for a different target than the host, which would require building the sys crate twice. Though this isn't the 99% case, and linking sys in the build doesn't prevent the 1% case, so this could just be me overthinking it.

I don't necessarily agree that using version based flags would be more maintainable. It's straightforward when the range is open ended, but tricky if we need to gate on a closed range. We would either need to add not cfgs for every version above the max supported version, or define greater than/less than cfgs for every version stop. We'd need something like this if we want to also support version 11. The SDK has a fairly large surface area that could evolve in different ways, and the idea with the api cfgs was keep that version range gating in the build script, and applying a single cfg on the relevant api surfaces.

cudarc

@LegNeato has mentioned interest in collaborating with cudarc, which also wraps the cuda SDK libraries. Their abstractions are more mature then cust's and they support multiple versions of the SDK through multiple checked in bindings. I'm still surprised that all prior work I've found in this space seems to go that direction. Users have reported issues where the bindings fail layout tests on their platform, and the fix has been to disable layout tests, possibly hiding UB. For the moment this is the aspect I'd like to see changed if we do collaborate.

Multiple LLVMs

Since bindgen requires libclang 9+, we need users to have a more recent llvm install in addition to llvm 7 required by the codegen. This shouldn't be a major hurdle for most users, but does increase the complexity. We might also want to consider renaming environment variables to better distinguish which version should be set and for what.

[adamcavendish]

@jorge-ortega Thanks for providing more info about the roadmap.

1. cudarc

If we do plan to switch over to cudarc, I can help make some changes to see how much work is needed to switch over. cc @LegNeato

About the multiple checked-in bindings, yeah, I never thought of this path previously as many community crates are using the standard bindgen recommended ways.

2. Multiple LLVMs

I can further improve the current Dockerfiles for various Linux distribution supports when we have the exact environment variables settled down. Hope the distribution provided official LLVM versions are good enough for bindgen (Most of the time, yes).

[jorge-ortega]

I just became aware of how to use cargo metadata to pass information to dependent packages.¹ We could use this to pass the version information to higher level crates. This approach could potentially supersede most of what the find_cuda_helper crate provides.

Footnotes

1. https://doc.rust-lang.org/cargo/reference/build-script-examples.html#using-another-sys-crate ↩

[adamcavendish]

I just became aware of how to use cargo metadata to pass information to dependent packages.1 We could use this to pass the version information to higher level crates. This approach could potentially supersede most of what the find_cuda_helper crate provides.

Footnotes

1. https://doc.rust-lang.org/cargo/reference/build-script-examples.html#using-another-sys-crate ↩

Wow, that's awesome. This is what we really need. I can work on a refactor of this to make it cleaner.
It also shows a way to specify multiple conditional compilation version support in the openssl-sys crate.

[jorge-ortega]

Sure, give it a try. I was looking at how openssl-sys passes version info as an example here. They hex encoded it and then parse it out to set their version cfgs. Haven't figured out exactly how they get the version information out of the headers yet. Has something to do with expanding version macros. It might be easier to read the version.json file instead?

[adamcavendish]

Sure, give it a try. I was looking at how openssl-sys passes version info as an example here. They hex encoded it and then parse it out to set their version cfgs. Haven't figured out exactly how they get the version information out of the headers yet. Has something to do with expanding version macros. It might be easier to read the version.json file instead?

For the version.json file, it's not available in the Linux container. I guess it's available in your Windows setup but it's not cross platform?

[jorge-ortega]

I should have known that wouldn't be cross-platform... I guess you could just bindgen and parse the version out from the bindings?

[adamcavendish]

I should have known that wouldn't be cross-platform... I guess you could just bindgen and parse the version out from the bindings?

Yeah, that's ok. May I cherry-pick your changes (so it'll show your Author information) and start with that? Before the merge, I'll write that we co-authored?

[adamcavendish]

I should have known that wouldn't be cross-platform... I guess you could just bindgen and parse the version out from the bindings?

Yeah, that's ok. May I cherry-pick your changes (so it'll show your Author information) and start with that? Before the merge, I'll write that we co-authored?

Will be drafting in the branch feature/refactor-cust-raw-and-find-cuda and aim to be ready before next Wednesday.