Rust<->C by-val ABI issues on 32bit

[olsonjeffery]

In my latest bit of work on libuv bindings in rust, I've come into some issues with passing structs, by-val, from Rust->C in windows (64bit host, 32bit rust build).

I've found a few things:

1. The data passed, by-val, from rust->C comes out as garbage in C. C->Rust via return and C->Rust via crust don't appear to be affected (this is in the context of my uv_tcp branch work on http://github.com/olsonjeffery/rust)
2. when a by-val struct is including in a function signature in C, it seems that all of the parameters that come after become null/garbage, as well (I encountered with a callback fn ptr that was null when it was passed into C. But moving the ptr to be before the by-val struct in question caused it to be unbroken)

This is happening after #1402 was closed, and I know it was expected that there's be some bogeymen hiding out in 32bit edge cases, so hopefully this helps.

[olsonjeffery]

So, I've already worked around this issue in my uv_tcp branch (the sockaddr_in is passed by-ref into C from rust, huzzah).

But you can repro the behavior I'm describing by:

• be on windows (I wonder if this is an issue in linux, as well?)
• git clone git://github.com/olsonjeffery/rust
• checking out the uv_tcp branch
• and checking out back to 80f1d676 , which is the commit immediately preceeding my workaround.
• If then you do make check-stage1-std TESTNAME=test_uv_tcp , you'll see the test hang because the on_connect_cb
  is null and/or because the provided sockaddr_in to rustrt::rust_uv_tcp_connect() is garbage, once we get into C. The current printf debugging in that commit only shows the bad port val (it'll be different than the previous printfs showing its
  value after the sockaddr_in is created.. the value of sin_port is usually 20480 for when uv_ip4_addr is provided 80 for its port arg) .. but an additional printf will show that the provided uv_connect_cb is, in fact, equal to 0.

[erickt]

Another issue, rust recs with i64 values does not seem to match c structures with int64_t types.

[olsonjeffery]

some more additions, based on work to get libuv stuff passing on 32bit linux:

Consider the following:

unsafe fn buf_init(++input: *u8, len: uint) -> uv_buf_t {
    io::println(#fmt("ll::buf_init - input %u", input as uint));
    let result = rustrt::rust_uv_buf_init(input, len);
    let res_base = get_base_from_buf(result);
    io::println(#fmt("ll::buf_init - result %u", res_base as uint));
    ret result;
}

Here's some output:

running 1 tests
before receiving on server continue_port
resp_msg ptr: 164350200
ll::buf_init - input 164350200
uv_buf_init: base ptr 0x3
after uv_buf_init: result->base 0x3
make: *** [check-stage1-T-i686-unknown-linux-gnu-H-x86_64-unknown-lin
ux-gnu-std-dummy] Segmentation fault           

I'm doing a 32bit build on my 64bit host (ubuntu oneiric) and I couldn't get valgrind working. thankfully linux2 is barfing on the same issue, I'm pretty sure (it originally lead me to put the debug println()s on this section of code).

Here's its output (you can see the whole deal here )

==55900== Thread 7: ==55900== Invalid write of size 4 
==55900== at 0x4DAFFB8: uv_buf_init (uv-common.c:75) 
==55900== by 0x8065FF0: rust_uv_buf_init__c_stack_shim (in /home/rustbuild/src/rustbot/workspace-try-x86_64-unknown-linux-gnu/obj/x86_64-unknown-linux-gnu/test/stdtest.stage2-i686-unknown-linux-gnu) 
==55900== by 0x4DAFEBE: ??? (in /home/rustbuild/src/rustbot/workspace-try-x86_64-unknown-linux-gnu/obj/x86_64-unknown-linux-gnu/stage2/lib/rustc/i686-unknown-linux-gnu/lib/librustrt.so) 
==55900== Address 0x8a42efc is 0 bytes after a block of size 12 alloc'd 
==55900== at 0x4D70E8A: malloc (vg_replace_malloc.c:236) 
==55900== by 0x4DAFA6B: memory_region::malloc(unsigned int, char const*, bool) (memory_region.cpp:108) 
==55900== by 0x4DA3D32: rust_kernel::malloc(unsigned int, char const*) (rust_kernel.cpp:47) 
==55900== by 0x4D9E0F6: upcall_s_shared_malloc (rust_upcall.cpp:232) 
==55900== by 0x4DAFEBE: ??? (in /home/rustbuild/src/rustbot/workspace-try-x86_64-unknown-linux-gnu/obj/x86_64-unknown-linux-gnu/stage2/lib/rustc/i686-unknown-linux-gnu/lib/librustrt.so) 
==55900== vex x86->IR: unhandled instruction bytes: 0x2E 0xA4 0x8 0x3 
==55900== valgrind: Unrecognised instruction at address 0x8cac991. 
==55900== Your program just tried to execute an instruction that Valgrind 
==55900== did not recognise. There are two possible reasons for this. 
==55900== 1. Your program has a bug and erroneously jumped to a non-code 
==55900== location. If you are running Memcheck and you just saw a 
==55900== warning about a bad jump, it's probably your program's fault. 
==55900== 2. The instruction is legitimate but Valgrind doesn't handle it, 
==55900== i.e. it's Valgrind's fault. If you think this is the case or 
==55900== you are not sure, please let us know and we'll try to fix it. 
==55900== Either way, Valgrind will now raise a SIGILL signal which will 
==55900== probably kill your program. 
==55900== 
==55900== Process terminating with default action of signal 4 (SIGILL) 
==55900== Illegal opcode at address 0x8CAC991 
==55900== at 0x8CAC991: ??? make: *** [check-stage2-T-i686-unknown-linux-gnu-H-x86_64-unknown-linux-gnu-std-dummy] Killed

looking at the above failure, a few things of note:

1. It is completing the entire C func (both printf's are fire).. but we don't successfully return to Rust (the println() immediately after the call to rust_uv_buf_init() is never processed)
2. The char* base, once in C, is clobbered (notice that it went from an actual value to being 0x3 .. hm). I'm curious if this isn't something of my doing, as it seems like enough of a problem that it'd appear in other bindings and cause much havoc.

[olsonjeffery]

I have a branch with a test that exposes the possible stack smashing issue alluded to above (per graydon's analysis of the valgrind output).

the test is named test_abi_byval_passing_c_into_rust and is located in src/libcore/os.rs.

I think this is probably the only useful test. the others are probably related to mistakes on my part.

olsonjeffery@8a9a672#L0R686