Don't leak non-exported symbols from staticlibs

[bjorn3]

When compiling a cdylib only #[no_mangle] symbols are exported. #[rustc_std_internal_symbol] and mangled symbols are not exported. This prevents symbol conflicts and avoids overriding symbols in ways that causes UB when using a rust cdylib in a rust program. For staticlibs however all symbols leak out of the staticlib. Causing symbol overrides that are potentially UB and symbols conflicts. For example when statically linking spidermonkey, you will get symbol conflicts if the rust version you compile your program with doesn't match the one the rust parts of spidermonkey were compiled with.

cc rust-lang/wg-allocators#108 (comment)
cc https://bytecodealliance.zulipchat.com/#narrow/stream/206238-general/topic/.E2.9C.94.20spidermonkey-wasm-rs/near/309604553

[bjorn3]

cc @chorman0773

[chorman0773]

FTR, I'm unsure there is a way to limit exported symbols from a staticlib, without both limiting the number of CGUs within the crate to 1 and preventing any upstream crates from using the #[global_allocator]. There is further limited ways to prevent upstream crate symbols from being exported. My Original Post was made with that in mind.

[bjorn3]

Maybe partial linking would work? If not we will at least need to version all #[rustc_std_internal_symbol]s for each rustc version to prevent UB if different rustc versions implement them in backwards incompatible ways. And maybe we can edit every object file to rename symbols to be unique? By the way note that for cdylib and staticlib all rust crate dependencies are included directly in the cdylib/staticlib. No dynamic linking supported.

[chorman0773]

Maybe partial linking would work?

IDK whether all linkers support partial linking (in particular, I don't know about microsoft's link.exe). Though, personally, I don't particularily want to touch ld -r with a 10' pole anyways if I don't have to. The versioning would work, if done within the mangling (for mangled symbols).

By the way note that for cdylib and staticlib all rust crate dependencies are included directly in the cdylib/staticlib. No dynamic linking supported.

I plan to do it slightly differently when possible - for staticlibs resolve the dependencies and produce a links.o "object" that is just a linker script. For cdylibs, just link as normal and when dynamically linking, add DT_RPATH (to the stdlib directory)+DT_NEEDED as needed.

[bjorn3]

for staticlibs resolve the dependencies and produce a links.o "object" that is just a linker script.

That is a backward incompatible change for rustc

For cdylibs, just link as normal and when dynamically linking, add DT_RPATH (to the stdlib directory)+DT_NEEDED as needed.

That will share and thus leak the global state of libstd across the cdylib boundary. This among other things will break the mitigation of #102721 to prevent catching foreign rust panics.

[chorman0773]

On Tue, Nov 22, 2022 at 07:28 bjorn3 ***@***.***> wrote: for staticlibs resolve the dependencies and produce a links.o "object" that is just a linker script. That is a backward incompatible change for rustc

rustc currently just emits the library directly into the archive, right? I'm not particularly sure what the difference here is, except in terms of file size, and the fact that using libstd.so.0.1 is an option in addition to libstd.rlib.0.1.

For cdylibs, just link as normal and when dynamically linking, add DT_RPATH (to the stdlib directory)+DT_NEEDED as needed. That will share and thus leak the global state of libstd across the cdylib boundary.

That is true, though I'm unsure how it is avoidable - even when statically linking the symbols from libstd et. al need to have default visibility so that when linking into a dylib, the symbols can be used via the dylib. It would not make a difference on ELF.

…

— Reply to this email directly, view it on GitHub <#104707 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGLD2667YDJXO3NIQ42553WJS4AXANCNFSM6AAAAAASHT3BQM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bjorn3]

rustc currently just emits the library directly into the archive, right?
I'm not particularly sure what the difference here is, except in terms of
file size, and the fact that using libstd.so.0.1 is an option in addition
to libstd.rlib.0.1.

The difference is that currently you can ship a staticlib as a standalone file and expect linking to succeed, but with your proposal you also need all dependencies to be available at exactly the same place.

That is true, though I'm unsure how it is avoidable - even when statically
linking the symbols from libstd et. al need to have default visibility so
that when linking into a dylib, the symbols can be used via the dylib. It
would not make a difference on ELF.

When linking a cdylib, libstd is statically linked and none of it's symbols are exported from the cdylib. When linking a rust dylib, sharing state is just fine. In fact you aren't allowed to duplicate crates in that case.

[chorman0773]

The difference is that currently you can ship a staticlib as a standalone file and expect linking to succeed, but with your proposal you also need all dependencies to be available at exactly the same place.

Fair enough, though the result is potentially shipping GiB for an API surface that should take MiB.

When linking a cdylib, libstd is statically linked and none of it's symbols are exported from the cdylib. When linking a rust dylib, sharing state is just fine. In fact you aren't allowed to duplicate crates in that case.

On ELF I'm unsure how this would be achieved. Symbol visibilty is controlled when the symbol is defined (in the object file), and I just send everything on to ld (post processing? What is that? I only know "add head and tail libraries to link line"). ELF shared objects don't have an "Export List", the dynamic symbol list is just built from the static symbol list usually excluding internal and hidden symbols. Every GLOBAL or WEAK symbol in the list with PROTECTED or DEFAULT visibility can be imported from the cdylib and every symbol with DEFAULT visibilty can additonally be replaced. These are functions of the link editor producing the files and the dynamic linker-loader resolving runtime relocations, and are far from under the control of rust as a language or any particular implementation.

[bjorn3]

Rustc passes a version script to the linker specifying exactly which symbols to export and making it to hide everything else. This is but one of the reasons rustc is in change of invoking the linker.

[chorman0773]

Rustc passes a version script to the linker specifying exactly which symbols to export and making it to hide everything else. This is but one of the reasons rustc is in change of invoking the linker.

Ah. My problem is that I have to support link editors that are like "Version Script? What is this? Expected PHDRS, MEMORY, or SECTIONS"

[bjorn3]

Linker scripts and versions scripts are different. Linker scripts tell what should be put where in the linked artifact. Version scripts only list which symbols are exported and which aren't amd optionally provide a version for the purpose of symbol versioning. The format of version scripts is trivial in comparison to linker scripts. See

rust/compiler/rustc_codegen_ssa/src/back/linker.rs

Lines 666 to 724 in b7463e8

	if self.sess.target.is_like_osx {
	// Write a plain, newline-separated list of symbols
	let res: io::Result<()> = try {
	let mut f = BufWriter::new(File::create(&path)?);
	for sym in symbols {
	debug!(" _{}", sym);
	writeln!(f, "_{}", sym)?;
	}
	};
	if let Err(error) = res {
	self.sess.emit_fatal(errors::LibDefWriteFailure { error });
	}
	} else if is_windows {
	let res: io::Result<()> = try {
	let mut f = BufWriter::new(File::create(&path)?);
	// .def file similar to MSVC one but without LIBRARY section
	// because LD doesn't like when it's empty
	writeln!(f, "EXPORTS")?;
	for symbol in symbols {
	debug!(" _{}", symbol);
	writeln!(f, " {}", symbol)?;
	}
	};
	if let Err(error) = res {
	self.sess.emit_fatal(errors::LibDefWriteFailure { error });
	}
	} else {
	// Write an LD version script
	let res: io::Result<()> = try {
	let mut f = BufWriter::new(File::create(&path)?);
	writeln!(f, "{{")?;
	if !symbols.is_empty() {
	writeln!(f, " global:")?;
	for sym in symbols {
	debug!(" {};", sym);
	writeln!(f, " {};", sym)?;
	}
	}
	writeln!(f, "\n local:\n *;\n}};")?;
	};
	if let Err(error) = res {
	self.sess.emit_fatal(errors::VersionScriptWriteFailure { error });
	}
	}
	if self.sess.target.is_like_osx {
	self.linker_args(&[OsString::from("-exported_symbols_list"), path.into()]);
	} else if self.sess.target.is_like_solaris {
	self.linker_args(&[OsString::from("-M"), path.into()]);
	} else {
	if is_windows {
	self.linker_arg(path);
	} else {
	let mut arg = OsString::from("--version-script=");
	arg.push(path);
	self.linker_arg(arg);
	}
	}

[chorman0773]

I am aware of version scripts. The simplicity is not the problem. The problem is if I'm faced with a link editor that doesn't support them, which I cannot always assume.

[bjorn3]

Which linker doesn't support it? AFAIK every platform targeted by rustc has a linker supporting them or some other way to hide symbols.