Description
During this issue all examples will be done using x86_64.
When you do 0 as *const T
the type of the integer literal is inferred to be usize
. This is fine.
When you do !0 as *const T
you can observe that the resulting address is 0xffffffffffffffff
which is fine.
When doing FFI sometimes a handle type will be represented as a raw pointer type and sometimes there will be constants of that handle type initialized to negative values, which is entirely normal.
The user's first choice is to try -1 as *const T
which will of course fail because the type of the integer literal is inferred to be usize
which cannot be negated because Rust hates me.
Ideally Rust would be able to infer the type as isize
instead of usize
in this case, but I know better than to expect Rust to be actually helpful.
The issue is if the user tries to be clever and do (0-1) as *const T
which somehow seems to work but in reality they are triggering a combination of two horrible design decisions (or maybe bugs, it can be hard to tell sometimes) resulting in a giant footgun.
- The type of those integer literals is being inferred as
i32
instead ofusize
. Rust earlier refused to infer the type asisize
instead ofusize
, so why is it now inferring the type asi32
? - The conversion from
i32
to*const T
does zero extension instead of sign extension which is different than what C++ does. Add lint for u8 as *mut cast #42915
As a result, if you check the resulting address of (0-1) as *const T
you'll see that it gives 0xffffffff
which is wrong.
The correct answer is achieved by doing -1isize as *const T
which results in the correct address of 0xffffffffffffffff
.
Ideally both of those two points above would be fixed, but at the very least there should be a very loud warning that you are doing something very horribly wrong.
And yes, I found examples of this in winapi so some care will be needed when fixing this in order to not break the ecosystem.
Possible solutions
- Add a lint to forbid casts between pointer types and integer types other than usize/isize.
- Change casts from signed integer types to pointer types to perform signed conversion.
- Fix the inference issue causing the literals to be inferred as
i32
instead ofusize
.