Skip to content

Insert checks for enum discriminants when debug assertions are enabled #141759

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

1c3t3a
Copy link
Member

@1c3t3a 1c3t3a commented May 30, 2025

Similar to the existing null-pointer and alignment checks, this checks for valid enum discriminants on creation of enums through unsafe transmutes. Essentially this sanitizes patterns like the following:

let val: MyEnum = unsafe { std::mem::transmute<u32, MyEnum>(42) };

An extension of this check will be done in a follow-up that explicitly sanitizes for extern enum values that come into Rust from e.g. C/C++.

This check is similar to Miri's capabilities of checking for valid construction of enum values.

This PR is inspired by saethlin@'s PR
#104862. Thank you so much for keeping this code up and the detailed comments!

I also pair-programmed large parts of this together with vabr-g@.

r? @saethlin

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels May 30, 2025
@rustbot
Copy link
Collaborator

rustbot commented May 30, 2025

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

This PR changes MIR

cc @oli-obk, @RalfJung, @JakobDegen, @davidtwco, @vakaras

Some changes occurred in compiler/rustc_codegen_ssa

cc @WaffleLapkin

Some changes occurred in compiler/rustc_codegen_cranelift

cc @bjorn3

Some changes occurred to the CTFE machinery

cc @RalfJung, @oli-obk, @lcnr

rust-analyzer is developed in its own repository. If possible, consider making this change to rust-lang/rust-analyzer instead.

cc @rust-lang/rust-analyzer

@rustbot

This comment has been minimized.

@1c3t3a 1c3t3a force-pushed the discriminants-query branch from 6d3fe75 to a7dd718 Compare May 30, 2025 09:46
@rustbot

This comment has been minimized.

@1c3t3a 1c3t3a force-pushed the discriminants-query branch from a7dd718 to 4f3342e Compare May 30, 2025 09:59
@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@1c3t3a 1c3t3a force-pushed the discriminants-query branch from 54b6e74 to b03960e Compare May 30, 2025 13:33
@rust-log-analyzer

This comment has been minimized.

@1c3t3a 1c3t3a force-pushed the discriminants-query branch from b03960e to 228b656 Compare May 30, 2025 13:59
@rust-log-analyzer

This comment has been minimized.

@1c3t3a 1c3t3a force-pushed the discriminants-query branch from 228b656 to d1d8f88 Compare June 2, 2025 14:34
@rust-log-analyzer

This comment has been minimized.

@1c3t3a 1c3t3a force-pushed the discriminants-query branch from d1d8f88 to 93b24d7 Compare June 2, 2025 20:23
@rust-log-analyzer

This comment has been minimized.

Similar to the existing nullpointer and alignment checks, this checks
for valid enum discriminants on creation of enums through unsafe
transmutes. Essentially this sanitizes patterns like the following:
```rust
let val: MyEnum = unsafe { std::mem::transmute<u32, MyEnum>(42) };
```
An extension of this check will be done in a follow-up that explicitly
sanitizes for extern enum values that come into Rust from e.g. C/C++.

This check is similar to Miri's capabilities of checking for valid
construction of enum values.

This PR is inspired by saethlin@'s PR
rust-lang#104862. Thank you so much for
keeping this code up and the detailed comments!

I also pair-programmed large parts of this together with vabr-g@.
@1c3t3a 1c3t3a force-pushed the discriminants-query branch from 93b24d7 to c2a8415 Compare June 3, 2025 12:31
@rust-log-analyzer
Copy link
Collaborator

The job aarch64-gnu-llvm-19-2 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

---- collections::btree::map::tests::test_check_ord_chaos stdout ----
---- collections::btree::map::tests::test_check_ord_chaos stderr ----

thread 'main' panicked at alloctests/../alloc/src/collections/btree/map/tests.rs:101:17:
Governed(1, Governor { flipped: Cell { value: true } }) >= Governed(2, Governor { flipped: Cell { value: true } })
stack backtrace:
note: child process exited with signal 5
---- collections::btree::map::tests::test_range_backwards_1 stdout ----
---- collections::btree::map::tests::test_range_backwards_1 stderr ----

---
---- collections::vec_deque::tests::test_rotate_left_panic stdout ----
---- collections::vec_deque::tests::test_rotate_left_panic stderr ----

thread 'main' panicked at alloctests/../alloc/src/collections/vec_deque/mod.rs:2590:9:
assertion failed: n <= self.len()
stack backtrace:
note: child process exited with signal 5
---- collections::vec_deque::tests::test_rotate_right_panic stdout ----
---- collections::vec_deque::tests::test_rotate_right_panic stderr ----

thread 'main' panicked at alloctests/../alloc/src/collections/vec_deque/mod.rs:2633:9:
assertion failed: n <= self.len()
stack backtrace:
note: child process exited with signal 5
---- collections::vec_deque::tests::test_swap_panic stdout ----
---- collections::vec_deque::tests::test_swap_panic stderr ----

thread 'main' panicked at alloctests/../alloc/src/collections/vec_deque/mod.rs:751:9:
assertion failed: j < self.len()
stack backtrace:
note: child process exited with signal 5
---- raw_vec::tests::zst_reserve_exact_panic stdout ----
---- raw_vec::tests::zst_reserve_exact_panic stderr ----

---
    raw_vec::tests::zst_reserve_panic

test result: FAILED. 214 passed; 17 failed; 16 ignored; 0 measured; 0 filtered out; finished in 2.57s

error: test failed, to rerun pass `-p alloctests --lib`
env -u RUSTC_WRAPPER CARGO_ENCODED_RUSTDOCFLAGS="-Csymbol-mangling-version=v0\u{1f}-Zrandomize-layout\u{1f}-Zunstable-options\u{1f}--check-cfg=cfg(bootstrap)\u{1f}--check-cfg=cfg(llvm_enzyme)\u{1f}-Dwarnings\u{1f}-Wrustdoc::invalid_codeblock_attributes\u{1f}--crate-version\u{1f}1.89.0-nightly\t(fc5ccf4fa\t2025-06-03)" CARGO_ENCODED_RUSTFLAGS="-Csymbol-mangling-version=v0\u{1f}-Zrandomize-layout\u{1f}-Zunstable-options\u{1f}--check-cfg=cfg(bootstrap)\u{1f}--check-cfg=cfg(llvm_enzyme)\u{1f}-Zmacro-backtrace\u{1f}-Csplit-debuginfo=off\u{1f}-Clink-arg=-L/usr/lib/llvm-19/lib\u{1f}-Cllvm-args=-import-instr-limit=10\u{1f}-Clink-args=-Wl,-z,origin\u{1f}-Clink-args=-Wl,-rpath,$ORIGIN/../lib\u{1f}-Alinker-messages\u{1f}--cap-lints=allow\u{1f}--cfg\u{1f}randomized_layouts" RUSTC="/checkout/obj/build/aarch64-unknown-linux-gnu/stage1-tools/cg_clif/dist/rustc-clif" RUSTDOC="/checkout/obj/build/aarch64-unknown-linux-gnu/stage1-tools/cg_clif/dist/rustdoc-clif" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage0/bin/cargo" "test" "--manifest-path" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage1-tools/cg_clif/build/sysroot_tests/Cargo.toml" "--target-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage1-tools/cg_clif/build/sysroot_tests_target" "--locked" "--target" "aarch64-unknown-linux-gnu" "-p" "coretests" "-p" "alloctests" "--tests" "--" "-q" exited with status ExitStatus(unix_wait_status(25856))
Command has failed. Rerun with -v to see more details.
Build completed unsuccessfully in 0:11:16
  local time: Tue Jun  3 12:54:08 UTC 2025
  network time: Tue, 03 Jun 2025 12:54:08 GMT
##[error]Process completed with exit code 1.
Post job cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants