miri function argument passing should use FnAbi

[RalfJung]

In the miri engine, when evaluating a function call, it can happen that caller and callee do not agree on the type of an argument. In this case, the argument effectively gets transmuted from the caller type to the callee type. However, this is not legal for all types, and whether it is legal depends on horrible details of the ABI. This logic is implemented for codegen, where it is called FnType. miri should probably use that same infrastructure.

The relevant code in miri that would need changing is here. Currently, we only allow argument type punning for the Rust ABI, and we only allow the valid range of a Scalar/ScalarPair to change -- effectively, we allow one side to have &T while the other side has *const T.

For the FnType side, I do not know much, but @eddyb offered to mentor someone trying to do this cleanup. :) He also left the following hints:

move some code from rustc_codegen_ssa and then compare ArgType pairwise.

The main issue to moving that stuff around is pointee_info_at (it would have to be moved to rustc::ty::layout, as an additional method in TyLayoutMethods):

rust/src/librustc_codegen_llvm/abi.rs

Line 487 in dae6c93

if let Some(pointee) = layout.pointee_info_at(cx, offset) {

Everything else is mostly relying on rustc_target doing the heavy lifting, already.

Cc @oli-obk

[wildarch]

It seems this issue has been open for a while, is it still relevant? If so, I would like to take a shot at it 😄.

[RalfJung]

Awesome! Nobody has been working on this so far, so you are free to grab it. :)

@eddyb your mentoring is needed :D

[wildarch]

I have started with the refactoring by moving the pointee_info_at function to TyLayoutMethods. For now I still need to figure out how to translate some of the calls that were previously using the CodegenCx reference to work with the different context type available in TyLayoutMethods.
I'm quite confident I can get that to work, the only thing I'm struggling with at the moment are the MaybeResult<TyLayout>s I get when calling layout_of on the context. It's the first time I encounter this trait, and I am not quite sure how to work with it, given that I eventually want to end up with an Option<PointeeInfo> instead of a MaybeResult<PointeeInfo>.

@eddyb do you have some pointers on this? I might also ask around on IRC. Perhaps I should change the signature to be MaybeResult<PointeeInfo>?

[wildarch]

Figured out the situation with the MaybeResult, will post another update when pointee_info_at has been moved

[wildarch]

I have created a duplicate implementation of pointee_info_at in TyLayoutMethods as suggested by @eddyb (I will remove the original in a later commit). I think the method by itself is okay now, except that I want to move the type_is_freeze method in src/librustc_codegen_ssa/common.rs to somewhere in the rustc crate, right now I have simply copied to implementation into pointee_info_at, which is far from ideal. Any ideas on where to put that function @eddyb or @RalfJung?

Current changes are in my fork, should I already turn it into a WIP pull request?

[oli-obk]

should I already turn it into a WIP pull request?

That's a good idea, because it makes commenting on the impl much easier

[RalfJung]

I want to move the type_is_freeze method in src/librustc_codegen_ssa/common.rs

I wonder why it even has such a method, can't it use is_freeze?

But anyway, @eddyb is much more qualified to answer these questions. ;)

[wildarch]

@RalfJung type_is_freeze actually calls out to is_freeze, providing the param_env and span arguments. The following two are equivalent:

cx.type_is_freeze(ty)
ty.is_freeze(cx, ParamEnv::reveal_all(), DUMMY_SPAN)

Instead of moving type_is_freeze I could also change the occurrences to use is_freeze, how do you feel about that?

[wildarch]

should I already turn it into a WIP pull request?

That's a good idea, because it makes commenting on the impl much easier

Done!

[RalfJung]

Instead of moving type_is_freeze I could also change the occurrences to use is_freeze, how do you feel about that?

Depends on how often it gets called. But really I have very little experience in rustc outside miri, so I'll leave this to @eddyb.