WIP (do not merge): Performance test for zlib-ng-compat #75732

joshtriplett · 2020-08-20T05:11:08Z

Test PR for a perf run with zlib-ng-compat. rustc uses flate2 to
compress and decompress metadata (and in some cases LLVM bytecode), and
it's worth evaluating the potential performance improvement here. This
would require more care (and a config.toml setting) to do properly.

rust-highfive · 2020-08-20T05:11:11Z

r? @Mark-Simulacrum

(rust_highfive has picked a reviewer for you, use r? to override)

joshtriplett · 2020-08-20T05:12:10Z

@bors try @rust-timer queue

rust-timer · 2020-08-20T05:12:11Z

Awaiting bors try build completion

bors · 2020-08-20T05:12:21Z

⌛ Trying commit 08e06af56c96cb72fc04627437c6a278396c8a74 with merge 86b2c01be5672069ae4742dc7330133e9acb017c...

rust-log-analyzer · 2020-08-20T05:17:58Z

The job dist-x86_64-linux of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

   Compiling compiler_builtins v0.1.32
   Compiling unwind v0.0.0 (/checkout/library/unwind)
   Compiling profiler_builtins v0.0.0 (/checkout/library/profiler_builtins)
   Compiling rustc-std-workspace-core v1.99.0 (/checkout/library/rustc-std-workspace-core)
error: cannot find derive macro `Copy` in this scope
   --> /cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.75/src/unix/linux_like/linux/gnu/mod.rs:320:18
    |
320 |         #[derive(Copy,Clone)]


error: cannot find derive macro `Clone` in this scope
   --> /cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.75/src/unix/linux_like/linux/gnu/mod.rs:320:23
    |
320 |         #[derive(Copy,Clone)]


error[E0658]: unions with non-`Copy` fields are unstable
   --> /cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.75/src/unix/linux_like/linux/gnu/mod.rs:331:9
331 | /         union sifields {
331 | /         union sifields {
332 | |             _align_pointer: *mut ::c_void,
333 | |             sigchld: sifields_sigchld,
    | |_________^
    |
    |
    = note: see issue #55149 <https://github.com/rust-lang/rust/issues/55149> for more information
    = help: add `#![feature(untagged_unions)]` to the crate attributes to enable
error: aborting due to 3 previous errors

For more information about this error, try `rustc --explain E0658`.
For more information about this error, try `rustc --explain E0658`.
error: could not compile `libc`.
To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed
error: build failed
command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "16" "--release" "--locked" "--color" "always" "--features" "panic-unwind backtrace profiler compiler-builtins-c" "--manifest-path" "/checkout/library/test/Cargo.toml" "--message-format" "json-render-diagnostics"
failed to run: /checkout/obj/build/bootstrap/debug/bootstrap dist --host x86_64-unknown-linux-gnu --target x86_64-unknown-linux-gnu
Build completed unsuccessfully in 0:01:35
== clock drift check ==
  local time: Thu Aug 20 05:17:50 UTC 2020
  local time: Thu Aug 20 05:17:50 UTC 2020
  network time: Thu, 20 Aug 2020 05:17:50 GMT
== end clock drift check ==
##[error]Process completed with exit code 1.
Terminate orphan process: pid (6225) (python)

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

bors · 2020-08-20T05:18:12Z

💔 Test failed - checks-actions

joshtriplett · 2020-08-20T05:22:47Z

@bors try @rust-timer queue

rust-timer · 2020-08-20T05:22:48Z

Awaiting bors try build completion

bors · 2020-08-20T05:22:57Z

⌛ Trying commit b86161b1e88045064177caeeb71cb1117a329517 with merge 0991e982415f7bc29d102f3268e50132d3367d84...

rust-log-analyzer · 2020-08-20T05:28:39Z

The job dist-x86_64-linux of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

   Compiling compiler_builtins v0.1.32
   Compiling unwind v0.0.0 (/checkout/library/unwind)
   Compiling profiler_builtins v0.0.0 (/checkout/library/profiler_builtins)
   Compiling rustc-std-workspace-core v1.99.0 (/checkout/library/rustc-std-workspace-core)
error: cannot find derive macro `Copy` in this scope
   --> /cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.75/src/unix/linux_like/linux/gnu/mod.rs:320:18
    |
320 |         #[derive(Copy,Clone)]


error: cannot find derive macro `Clone` in this scope
   --> /cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.75/src/unix/linux_like/linux/gnu/mod.rs:320:23
    |
320 |         #[derive(Copy,Clone)]


error[E0658]: unions with non-`Copy` fields are unstable
   --> /cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.75/src/unix/linux_like/linux/gnu/mod.rs:331:9
331 | /         union sifields {
331 | /         union sifields {
332 | |             _align_pointer: *mut ::c_void,
333 | |             sigchld: sifields_sigchld,
    | |_________^
    |
    |
    = note: see issue #55149 <https://github.com/rust-lang/rust/issues/55149> for more information
    = help: add `#![feature(untagged_unions)]` to the crate attributes to enable
error: aborting due to 3 previous errors

For more information about this error, try `rustc --explain E0658`.
For more information about this error, try `rustc --explain E0658`.
error: could not compile `libc`.
To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed
error: build failed
command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "16" "--release" "--locked" "--color" "always" "--features" "panic-unwind backtrace profiler compiler-builtins-c" "--manifest-path" "/checkout/library/test/Cargo.toml" "--message-format" "json-render-diagnostics"
failed to run: /checkout/obj/build/bootstrap/debug/bootstrap dist --host x86_64-unknown-linux-gnu --target x86_64-unknown-linux-gnu
Build completed unsuccessfully in 0:01:42
== clock drift check ==
  local time: Thu Aug 20 05:28:30 UTC 2020
  local time: Thu Aug 20 05:28:30 UTC 2020
  network time: Thu, 20 Aug 2020 05:28:30 GMT
== end clock drift check ==
##[error]Process completed with exit code 1.
Terminate orphan process: pid (6461) (python)

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

bors · 2020-08-20T05:28:55Z

💔 Test failed - checks-actions

joshtriplett · 2020-08-20T07:42:22Z

Evidently zlib-ng requires a newer cmake than the one from the version of mingw we're using. This experiment will need to wait until we update mingw cmake.

mati865 · 2020-08-20T08:50:25Z

MinGW does not provide CMake. The builder used CMake 3.10.2 because that's what Ubuntu 18.04 provides as cmake package.

mati865 · 2020-08-20T11:49:40Z

@joshtriplett I've reported that build error to the upstream: zlib-ng/zlib-ng#729
This bug will affect only cross-compiling images like mingw-check and some of the Docker jobs but you can still get the performance numbers in this PR.

joshtriplett · 2020-08-20T17:50:47Z

@mati865 Thanks! Worth a try, then.

joshtriplett · 2020-08-20T17:59:10Z

@bors try @rust-timer queue

bors · 2020-08-20T17:59:21Z

⌛ Trying commit b8e3e0b with merge 69677eb315b30092708001efd2ee7aa01465e749...

rust-timer · 2020-08-20T17:59:22Z

Awaiting bors try build completion

bors · 2020-08-20T17:59:30Z

⌛ Trying commit b8e3e0b with merge 101b1b679086e6a473cf43f6b3876b2376bc83f9...

rust-log-analyzer · 2020-08-20T18:00:22Z

Your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

bors · 2020-08-20T18:44:19Z

☀️ Try build successful - checks-actions, checks-azure
Build commit: 101b1b679086e6a473cf43f6b3876b2376bc83f9 (101b1b679086e6a473cf43f6b3876b2376bc83f9)

rust-timer · 2020-08-20T18:44:20Z

Queued 101b1b679086e6a473cf43f6b3876b2376bc83f9 with parent 5fff382, future comparison URL.

joshtriplett · 2020-08-20T18:51:14Z

Looking at the binaries, it looks like they're still getting linked to system libz through LLVM, so this may not give accurate results.

rust-timer · 2020-08-20T20:38:21Z

Finished benchmarking try commit (101b1b679086e6a473cf43f6b3876b2376bc83f9): comparison url.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. Please note that if the perf results are neutral, you should likely undo the rollup=never given below by specifying rollup- to bors.

Importantly, though, if the results of this run are non-neutral do not roll this PR up -- it will mask other regressions or improvements in the roll up.

@bors rollup=never

joshtriplett · 2020-08-20T21:23:35Z

Not bad! Zero regressions, tiny improvements across the board, a few larger improvements.

We'd likely get even more improvement by switching to an even faster compression format, like lz4 or snappy. But this test shows that there's performance to be gained.

rust-highfive assigned Mark-Simulacrum Aug 20, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 20, 2020

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 20, 2020

joshtriplett force-pushed the wip-perf-test-zlib-ng branch from 08e06af to b86161b Compare August 20, 2020 05:22

WIP (do not merge): Performance test for zlib-ng-compat

b8e3e0b

joshtriplett force-pushed the wip-perf-test-zlib-ng branch from b86161b to b8e3e0b Compare August 20, 2020 07:24

joshtriplett closed this Aug 20, 2020

joshtriplett reopened this Aug 20, 2020

joshtriplett closed this Aug 20, 2020

joshtriplett mentioned this pull request Aug 20, 2020

Switch to Snappy compression for metadata #75754

Merged

joshtriplett deleted the wip-perf-test-zlib-ng branch April 29, 2021 18:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP (do not merge): Performance test for zlib-ng-compat #75732

WIP (do not merge): Performance test for zlib-ng-compat #75732

joshtriplett commented Aug 20, 2020

rust-highfive commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

rust-timer commented Aug 20, 2020

bors commented Aug 20, 2020

rust-log-analyzer commented Aug 20, 2020

bors commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

rust-timer commented Aug 20, 2020

bors commented Aug 20, 2020

rust-log-analyzer commented Aug 20, 2020

bors commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

mati865 commented Aug 20, 2020

mati865 commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

joshtriplett commented Aug 20, 2020 •

edited

Loading

bors commented Aug 20, 2020

rust-timer commented Aug 20, 2020

bors commented Aug 20, 2020

rust-log-analyzer commented Aug 20, 2020

bors commented Aug 20, 2020

rust-timer commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

rust-timer commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

WIP (do not merge): Performance test for zlib-ng-compat #75732

WIP (do not merge): Performance test for zlib-ng-compat #75732

Conversation

joshtriplett commented Aug 20, 2020

rust-highfive commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

rust-timer commented Aug 20, 2020

bors commented Aug 20, 2020

rust-log-analyzer commented Aug 20, 2020

bors commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

rust-timer commented Aug 20, 2020

bors commented Aug 20, 2020

rust-log-analyzer commented Aug 20, 2020

bors commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

mati865 commented Aug 20, 2020

mati865 commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

joshtriplett commented Aug 20, 2020 • edited Loading

bors commented Aug 20, 2020

rust-timer commented Aug 20, 2020

bors commented Aug 20, 2020

rust-log-analyzer commented Aug 20, 2020

bors commented Aug 20, 2020

rust-timer commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

rust-timer commented Aug 20, 2020

joshtriplett commented Aug 20, 2020

joshtriplett commented Aug 20, 2020 •

edited

Loading