Large types cause linker failure

[benwis]

So I'm one of the maintainers of Leptos, and we're experimenting with a new static type system for our next release. The issue we're having is that for people porting existing websites/apps to the beta, experience a linker issue with link.exe, lld, mold, and ld. Our theory is that once the types reach a certain size, it will crash the linker.

In this version we're encoding the HTML tree into the type system by constructing View<> types that contain a tuple of it's descendants.
You can see what that looks like in the below code, which does not crash.

#Cargo.toml
[package]
name = "linker_issue"
version = "0.1.0"
edition = "2021"

[dependencies]
leptos = { git = "https://github.com/leptos-rs/leptos", features = ["ssr"] }

// main.rs
use leptos::html::{div, p, HtmlElement};
use leptos::prelude::*;

fn main() {
    let view: HtmlElement<_, _, _, Dom> = div().child((
        div().child((div(),)),
        p().child((div(), div())),
        div().child((div(), div(), div())),
        p().child((div(), div(), div(), div())),
        div().child((div(), div(), div(), div())),
        p().child((div(), p(), div(), div())),
        div().child((div(), div(), p(), div())),
        p().child((div(), p(), div(), p())),
    ));
    println!("type: {:?}", std::any::type_name_of_val(&view));
    println!("\n\nsize: {:?}", std::mem::size_of_val(&view));
    let html = view.to_html();
    println!("\n\noutput: {html}");
}

So far we haven't nailed down a reasonable reproduction I can post, but I can reproduce the issue while trying to build one of our user's private projects. Running cargo leptos build, which runs cargo build --no-default-features --features=ssr, generates this error. I've put it in a gist because it is huge: https://gist.github.com/benwis/fe3a8010243c0f6b338f6aef0b0e7ad2

I'm not quite sure how to debug this, we'd love to use this system as it offers a number of benefits, but if it's going to break compilation at larger app sizes we'll have to defer. Any thoughts?

[saethlin]

This looks similar to #83060. How big are the rlibs in question?

[benwis]

This looks similar to #83060. How big are the rlibs in question?

Funny to see you here
The ones listed appear to be 311k, 97k and 1.8k respectively. There are much larger rlibs in here, the main app rlib is 4G

[dmgolembiowski]

I might be way off here (since I'm not currently compiling and checking anything) but I'd be curious if there are a bunch of monstrously large and monomorphized enums at play, @benwis

[benwis]

I might be way off here (since I'm not currently compiling and checking anything) but I'd be curious if there are a bunch of monstrously large and monomorphized enums at play, @benwis

Enums? I don't think so. But we do have those large types that contain a tuple that contains all the children

[saethlin]

The ones listed appear to be 311k, 97k and 1.8k respectively. There are much larger rlibs in here, the main app rlib is 4G

4 GB would do it. R_X86_64_32 means a 32-bit offset. I'm not sure the particular point at which the linker starts emitting errors is relevant, because that might just depend on the order of the functions in object files, which isn't stable.

You should be able to set RUSTFLAGS=-Zprint-type-sizes and see the sizes of all types in the computation, perhaps that will point you toward the problem.

[workingjubilee]

So far we haven't nailed down a reasonable reproduction I can post,

If this is about exceeding the 4GB limit, the most-minimal reproduction will be unreasonable. Nonetheless, we would like it.

[benwis]

So far we haven't nailed down a reasonable reproduction I can post,

If this is about exceeding the 4GB limit, the most-minimal reproduction will be unreasonable. Nonetheless, we would like it.

I'm working on getting y'all a repro. There are potentially multiple issues here and I need to look at it

[gbj]

monstrously large and monomorphized enums

This is entirely possible. Routing between different pages is handled by enums. A site that has, say, 16 pages, would have its view defined by something like EitherOf16<A, B, C, D, ...>, where each of those generics would themselves be a fairly large and complex view type like the above.

I've reopened a branch over in our repo that uses type erasure instead of that enum, and asked people to test against it. It had not solved some of our other issues with overall compile time on some of those larger repos, but it's possible it helps with this particular linker issue.

[vincentberthier]

I’m also having linking issues on my side.
I’m not sure it’s directly related to large types, but since it might…
Here’s the error’s gist: https://gist.github.com/VincentBerthier/63d3e1187aeb291499fd755dcbbfba36
Note that I’ve no idea why, but cargo/the linker spends a suuuuuuuuuuper long time (as in, 20+ minutes…) on the last step…