Compilation fails on build dependency, Cannot bind to C library.

[Tooney712]

I'm not sure if this should be here, in Regressions, or Library Tracking Issue, but regardless... I can't get a package dependency (dav1d-rs) to compile when running cargo check in Rust nightly. If I try to build it in Rust stable, it works perfectly fine and builds with no errors. As far as I can tell, there's no issue with the package's build script, it seems to be Bindgen that is throwing a fit(?).

I'm including the relevant code from the package in question, but there doesn't seem to be any problems with it.

Code

Link to dav1d-sys build.rs

build.rs

use std::env;
use std::path::PathBuf;

mod build {
    use super::*;
    use std::path::Path;
    use std::process::{Command, Stdio};

    const REPO: &str = "https://code.videolan.org/videolan/dav1d.git";
    const TAG: &str = "1.3.0";

    macro_rules! runner {
        ($cmd:expr, $($arg:expr),*) => {
            Command::new($cmd)
                $(.arg($arg))*
                .stderr(Stdio::inherit())
                .stdout(Stdio::inherit())
                .output()
                .expect(concat!($cmd, " failed"));

        };
    }

    pub fn build_from_src(
        lib: &str,
        _version: &str,
    ) -> Result<system_deps::Library, system_deps::BuildInternalClosureError> {
        let build_dir = "build";
        let release_dir = "release";

        let source = PathBuf::from(env::var("OUT_DIR").unwrap()).join("dav1d");
        let build_path = source.join(build_dir);
        let release_path = source.join(release_dir);

        if !Path::new(&source.join(".git")).exists() {
            runner!("git", "clone", "--depth", "1", "-b", TAG, REPO, &source);
        } else {
            runner!(
                "git",
                "-C",
                source.to_str().unwrap(),
                "fetch",
                "--depth",
                "1",
                "origin",
                TAG
            );
            runner!(
                "git",
                "-C",
                source.to_str().unwrap(),
                "checkout",
                "FETCH_HEAD"
            );
        }

        runner!(
            "meson",
            "setup",
            "-Ddefault_library=static",
            "--prefix",
            release_path.to_str().unwrap(),
            build_path.to_str().unwrap(),
            source.to_str().unwrap()
        );
        runner!("ninja", "-C", build_path.to_str().unwrap());
        runner!("meson", "install", "-C", build_path.to_str().unwrap());

        let pkg_dir = build_path.join("meson-private");
        system_deps::Library::from_internal_pkg_config(&pkg_dir, lib, TAG)
    }
}

fn main() {
    if std::env::var("DOCS_RS").is_ok() {
        return;
    }

    system_deps::Config::new()
        .add_build_internal("dav1d", build::build_from_src)
        .probe()
        .unwrap();
}

From lib.rs

#[repr(C)]
#[derive(Copy, Clone)]
pub struct Dav1dWarpedMotionParams {
    pub type_: Dav1dWarpedMotionType,
    pub matrix: [i32; 6usize],
    pub u: Dav1dWarpedMotionParamsU,
}

#[repr(C)]
#[derive(Copy, Clone)]
pub union Dav1dWarpedMotionParamsU {
    pub p: Dav1dWarpedMotionParamsUP,
    pub abcd: [i16; 4usize],
}

#[repr(C)]
#[derive(Debug, Copy, Clone)]
pub struct Dav1dWarpedMotionParamsUP {
    pub alpha: i16,
    pub beta: i16,
    pub gamma: i16,
    pub delta: i16,
}

From dav1d/include/dav1d/headers.h

enum Dav1dWarpedMotionType {
    DAV1D_WM_TYPE_IDENTITY,
    DAV1D_WM_TYPE_TRANSLATION,
    DAV1D_WM_TYPE_ROT_ZOOM,
    DAV1D_WM_TYPE_AFFINE,
};

typedef struct Dav1dWarpedMotionParams {
    enum Dav1dWarpedMotionType type;
    int32_t matrix[6];
    union {
        struct {
            int16_t alpha, beta, gamma, delta;
        } p;
        int16_t abcd[4];
    } u;
} Dav1dWarpedMotionParams;

Meta

rustc --version --verbose:

rustc 1.77.0-nightly (d78329b92 2024-01-13)
binary: rustc
commit-hash: d78329b92e8d141d19505e7c1527181c4ab87ed4
commit-date: 2024-01-13
host: x86-64-unknown-linux-gnu
release: 1.77.0-nightly
LLVM version: 17.0.6

I checked this in both nightly 2024-01-11 and 2024-01-13, the result was the same for both.

Error output

  Compiling dav1d-sys v0.3.5
error: failed to run custom build command for `dav1d-sys v0.3.5`

Caused by:
  process didn't exit successfully: `$(PROJECTDIR)/target/debug/build/dav1d-sys-2befe1be2c51b161/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=DAV1D_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=SYSROOT
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  cargo:rustc-link-lib=dav1d
  cargo:include=/usr/include
  cargo:rerun-if-env-changed=SYSTEM_DEPS_BUILD_INTERNAL
  cargo:rerun-if-env-changed=SYSTEM_DEPS_LINK
  cargo:rerun-if-env-changed=SYSTEM_DEPS_DAV1D_LIB
  cargo:rerun-if-env-changed=SYSTEM_DEPS_DAV1D_LIB_FRAMEWORK
  cargo:rerun-if-env-changed=SYSTEM_DEPS_DAV1D_SEARCH_NATIVE
  cargo:rerun-if-env-changed=SYSTEM_DEPS_DAV1D_SEARCH_FRAMEWORK
  cargo:rerun-if-env-changed=SYSTEM_DEPS_DAV1D_INCLUDE
  cargo:rerun-if-env-changed=SYSTEM_DEPS_DAV1D_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=SYSTEM_DEPS_DAV1D_BUILD_INTERNAL
  cargo:rerun-if-env-changed=SYSTEM_DEPS_DAV1D_LINK

  cargo:rustc-cfg=system_deps_have_dav1d

  --- stderr
  thread 'main' panicked at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/ir/context.rs:878:9:
  "Dav1dWarpedMotionParams_union_(unnamed_at_/usr/include/dav1d/headers_h_100_5)" is not a valid Ident
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Included CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true as dav1d-sys is a build dependency for dav1d-rs.

Backtrace

  stack backtrace:
     0: rust_begin_unwind
               at /rustc/d78329b92e8d141d19505e7c1527181c4ab87ed4/library/std/src/panicking.rs:647:5
     1: core::panicking::panic_fmt
               at /rustc/d78329b92e8d141d19505e7c1527181c4ab87ed4/library/core/src/panicking.rs:72:14
     2: proc_macro2::fallback::validate_ident
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.71/src/fallback.rs:828:9
     3: proc_macro2::fallback::Ident::new_checked
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.71/src/fallback.rs:760:9
     4: proc_macro2::imp::Ident::new_checked
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.71/src/wrapper.rs:646:50
     5: proc_macro2::Ident::new
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.71/src/lib.rs:956:21
     6: bindgen::ir::context::BindgenContext::rust_ident_raw
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/ir/context.rs:878:9
     7: bindgen::ir::context::BindgenContext::rust_ident
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/ir/context.rs:870:9
     8: <bindgen::ir::comp::CompInfo as bindgen::codegen::CodeGenerator>::codegen
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:1731:31
     9: <bindgen::ir::ty::Type as bindgen::codegen::CodeGenerator>::codegen
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:806:39
    10: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:492:17
    11: <bindgen::ir::comp::CompInfo as bindgen::codegen::CodeGenerator>::codegen
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:2062:13
    12: <bindgen::ir::ty::Type as bindgen::codegen::CodeGenerator>::codegen
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:806:39
    13: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:492:17
    14: <bindgen::ir::module::Module as bindgen::codegen::CodeGenerator>::codegen::{{closure}}
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:515:21
    15: <bindgen::ir::module::Module as bindgen::codegen::CodeGenerator>::codegen
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:545:13
    16: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:483:17
    17: bindgen::codegen::codegen::{{closure}}
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:4287:9
    18: bindgen::ir::context::BindgenContext::gen
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/ir/context.rs:1190:19
    19: bindgen::codegen::codegen
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/codegen/mod.rs:4251:5
    20: bindgen::Bindings::generate
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/lib.rs:2374:32
    21: bindgen::Builder::generate
               at $(HOME)/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.59.2/src/lib.rs:1478:9
    22: build_script_build::main
               at ./build.rs:77:13
    23: core::ops::function::FnOnce::call_once
               at /rustc/d78329b92e8d141d19505e7c1527181c4ab87ed4/library/core/src/ops/function.rs:250:5
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[Noratrieb]

Thank you for the report! It would be useful to bisect the regression using cargo-bisect-rustc to make it easier to figure out what the problem could be.

[Noratrieb]

I tried to reproduce this with the following script in a fresh podman run -it docker.io/rust /bin/bash but was not able to reproduce any issues, it all worked and checked successfully. (copied the setup code from dav1d-rs's CI)

rustup toolchain install nightly-2024-01-13
rustup default nightly-2024-01-13

apt update
apt install -y meson ninja-build nasm
git clone https://github.com/rust-av/dav1d-rs.git
cd dav1d-rs
git checkout a73672d040b6a8702115fcb00c697104576ee8f5

export DAV1D_DIR=dav1d_dir
export LIB_PATH=lib/x86_64-linux-gnu

git clone --branch 1.3.0 --depth 1 https://code.videolan.org/videolan/dav1d.git
cd dav1d
git checkout 48035599cdd4e4415732e408c407e0c1cd1c7444 # If this fails, do more depth
meson build -Dprefix=$HOME/$DAV1D_DIR -Denable_tools=false -Denable_examples=false --buildtype release

ninja -C build
ninja -C build install

export PKG_CONFIG_PATH="$HOME/$DAV1D_DIR/$LIB_PATH/pkgconfig"
export LD_LIBRARY_PATH="$HOME/$DAV1D_DIR/$LIB_PATH"

cd ..
cargo check

[Noratrieb]

What's your bindgen version in your lockfile?
While I cannot reproduce this, the error message looks like it might be a result of someone relying on token pretty printing which was changed in #114571. cc @nnethercote are you aware of any known issues here with bindgen?

[Tooney712]

The bindgen version mentioned in the lockfile is 0.59.2. I also have a separate bindgen installed in .cargo/bin/ with version 0.66.1. I ran cargo-bisect-rustc up to nightly-2023-03-04 before stopping it, it failed every time.

That having been said, I found that the dav1d-rs crate is being called as a dependency of the image crate, which I didn't realize when writing up the issue. Looking into image's features shows that the dav1d dependency only works on stable rust. I don't particularly know why it only works on stable rust, but regardless this seems to be more of an issue of me being stupid and not fully investigating the error. I apologize for the disruption, hope you have a good day.

[Noratrieb]

Looking into image's features shows that the dav1d dependency only works on stable rust.

That does not sound good and is definitely a bug somewhere..., I'll reopen the issue until we know that it's a bug in a crate and not in rustc.