Optimize the compiler on the no-opt bots

[brson]

These bots take a very long time, surely in large part because they are using an unoptimized compiler. Running the test suite unoptimized seems sufficient for testing Rust without optimizations, and we can do that with a fast compiler.

[graydon]

I think we should do a no-opt bootstrap. Once, somewhere. Maybe on a special builder. I agree that the no-opt compiler should not be building the testsuite. It's extremely pessimistic as it runs the many-megabytes deserialize-the-whole-metadata routine once per test through a non-optimized compiler and libstd/libextra. It takes 1.5s-per-test at least to compile (vs. more like 200ms per when optimized, i.e. 7 times faster).

Here's a profile of it unoptimized:

  4.51%  rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] ebml::reader::vuint_at::_18cd4fed191a82c7::_0$x2e8$x2dpre
  4.32%  rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] vec::__extensions__::as_imm_buf_9677::_fda9e92e81201d18::_0$x2e8$x2dpre
  3.73%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] num::strconv::int_to_str_bytes_common_13657::_1161b2e29d696be2::_0$x2e8$x2dpre
  2.78%  rustc  libc-2.15.so                          [.] __memcpy_ssse3_back
  2.49%  rustc  [kernel.kallsyms]                     [k] 0xffffffff81044ffa
  2.41%  rustc  librustrt.so                          [.] tinfl_decompress
  2.35%  rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] vec::__extensions__::as_imm_buf_22952::_81e027eeb3dd3ed::_0$x2e8$x2dpre
  1.79%  rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] cast::transmute_copy_22959::_bcc64bde5a34be3::_0$x2e8$x2dpre
  1.78%  rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] cast::transmute_copy_9577::_8ecce79de860424::_0$x2e8$x2dpre
  1.70%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] io::u64_from_be_bytes::_a6ae987ff224d4ce::_0$x2e8$x2dpre
  1.55%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] vec::__extensions__::push_12845::_e41e39f2a5e3ea3::_0$x2e8$x2dpre
  1.49%  rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] vec::__extensions__::as_imm_buf_41526::_fa5b4e6de46e501c::_0$x2e8$x2dpre
  1.39%  rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] hash::__extensions__::meth_22947::write::_2bbc6fdf571a12f3::_0$x2e8$x2dpre
  1.16%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] cast::transmute_copy_12693::_66cf21d2f4e7e921::_0$x2e8$x2dpre
  1.12%  rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] cast::transmute_22965::_4f62c5819971f461::_0$x2e8$x2dpre
  1.06%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] cast::transmute_12699::_5c2077cf4be69ebb::_0$x2e8$x2dpre
  1.06%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] vec::__extensions__::as_imm_buf_12687::_ec5a5e5f17ce0bd::_0$x2e8$x2dpre
  0.94%  rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] metadata::decoder::lookup_hash::anon::expr_fn_91752
  0.94%  rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] sys::nonzero_size_of_22979::_3db6da11d4d5cc1c::_0$x2e8$x2dpre
  0.94%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] vec::__extensions__::push_fast_12873::_f69e95958bf99a7::_0$x2e8$x2dpre
  0.88%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] uint::generated::__extensions__::meth_13571::div::_7c85cb84fcc0f326::_0$x2e8$x2dpre
  0.83%  rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] num::strconv::from_str_bytes_common_91086::_eb35d3835a47889::_0$x2e8$x2dpre
  0.78%  rustc  libc-2.15.so                          [.] _int_free
  0.74%  rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] cast::transmute_9543::_2dbbfe1d77f1429a::_0$x2e8$x2dpre
  0.74%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] str::__extensions__::meth_17218::char_range_at::_4ebce56bd59271d::_0$x2e8$x2dpre
  0.73%  rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] sys::nonzero_size_of_9591::_d049d39a158acdd4::_0$x2e8$x2dpre
  0.73%  rustc  libc-2.15.so                          [.] _int_malloc
  0.73%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] str::__extensions__::meth_17215::next::_3a94af25b756f3::_0$x2e8$x2dpre
  0.71%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] sys::nonzero_size_of_12708::_806083deb827ec::_0$x2e8$x2dpre
  0.66%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] uint::generated::__extensions__::meth_13574::rem::_7c85cb84fcc0f326::_0$x2e8$x2dpre
  0.63%  rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] hash::__extensions__::meth_22989::result_u64::_35b0a054bc66faac::_0$x2e8$x2dpre
  0.60%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] uint::next_power_of_two::_3ed57dc8152f94a6::_0$x2e8$x2dpre
  0.58%  rustc  libc-2.15.so                          [.] realloc
  0.58%  rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] ebml::reader::tagged_docs::_77f45b5746d3ce3::_0$x2e8$x2dpre
  0.58%  rustc  libc-2.15.so                          [.] malloc
  0.56%  rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] ebml::reader::maybe_get_doc::_46884e6f9ab5f33::_0$x2e8$x2dpre
  0.56%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] iterator::fold_17566::_5cb678841c7131a4::_0$x2e8$x2dpre
  0.53%  rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] iterator::__extensions__::do_map_17482::_95a56f958e63ac3::_0$x2e8$x2dpre
  0.53%  rustc  libc-2.15.so                          [.] __memmove_ssse3_back
  0.51%  rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] cast::transmute_15871::_2675e2e451b103a::_0$x2e8$x2dpre

Compare with optimized:

 12.71%     rustc  librustrt.so                          [.] tinfl_decompress
 12.11%     rustc  [kernel.kallsyms]                     [k] 0xffffffff81044ffa
  5.65%     rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] ebml::reader::vuint_at::_112d46e1a3c7d2e::_0$x2e8$x2dpre
  3.59%        ld  libbfd-2.22-system.so                 [.] bfd_link_hash_traverse
  3.01%     rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] io::u64_from_be_bytes::_a6ae987ff224d4ce::_0$x2e8$x2dpre
  2.80%        ld  libbfd-2.22-system.so                 [.] bfd_hash_lookup
  2.63%     rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] metadata::decoder::lookup_hash::anon::expr_fn_91466
  2.46%        ld  libbfd-2.22-system.so                 [.] 0x0000000000054c70
  2.41%     rustc  libc-2.15.so                          [.] __memmove_ssse3_back
  1.95%     rustc  libc-2.15.so                          [.] _int_free
  1.94%     rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] ebml::reader::tagged_docs::_f3d79846144c6234::_0$x2e8$x2dpre
  1.85%     rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] metadata::decoder::maybe_find_item::anon::expr_fn_91481
  1.67%        ld  [kernel.kallsyms]                     [k] 0xffffffff81044ffa
  1.57%     rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] hash::__extensions__::meth_22527::write::_2bbc6fdf571a12f3::_0$x2e8$x2dpre
  1.56%        ld  libc-2.15.so                          [.] __memcpy_ssse3_back
  1.40%     rustc  libstd-6c65cf4b443341b1-0.8-pre.so    [.] cleanup::each_live_alloc::_b9a8f420b91ad622::_0$x2e8$x2dpre
  1.37%     rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] hash::__extensions__::meth_22569::result_u64::_35b0a054bc66faac::_0$x2e8$x2dpre
  1.24%        cc  [kernel.kallsyms]                     [k] 0xffffffff81044ffa
  1.17%     rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] num::strconv::from_str_bytes_common_90798::_25931c29ce4e3c55::_0$x2e8$x2dpre
  1.17%     rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] ebml::reader::maybe_get_doc::_3f955cb7f9569d::_0$x2e8$x2dpre
  1.01%        ld  libbfd-2.22-system.so                 [.] bfd_elf_link_add_symbols
  0.97%     rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] vec::__extensions__::slice_73314::_e73310fe6cef7c2c::_0$x2e8$x2dpre
  0.89%     rustc  libextra-a7c050cfd46b2c9a-0.8-pre.so  [.] ebml::reader::get_doc::_e6da4d6eaa59445::_0$x2e8$x2dpre
  0.88%     rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] metadata::decoder::item_name::_10164553c4fefcce::_0$x2e8$x2dpre
  0.88%     rustc  libc-2.15.so                          [.] _int_malloc
  0.87%     rustc  ld-2.15.so                            [.] _dl_lookup_symbol_x
  0.79%  collect2  [kernel.kallsyms]                     [k] 0xffffffff81044ffa
  0.69%        ld  libc-2.15.so                          [.] __memset_sse2
  0.68%     rustc  libc-2.15.so                          [.] malloc
  0.68%     rustc  libsyntax-64629f7f0c6a9bc-0.8-pre.so  [.] util::interner::__extensions__::meth_8795::intern::_6f4abfe5617d84bc::_0$x2e8$x2dpre
  0.59%     rustc  librustc-d3cb8c2ccd84a7a7-0.8-pre.so  [.] metadata::decoder::__extensions__::each_child_of_module_or_crate::anon::expr_fn_91625
  0.58%

[graydon]

In concert with this, of course, I still think (have thought for like a year?) that #4572 / changing metadata to store compressed info and ASTs per-item, and load them lazily, is our biggest potential cycle-time win.

[alexcrichton]

Now that #8450 has landed, I believe that the bots can be configured in this way now, and once cycles times are confirmed down this should be close-able.

[graydon]

I have now reconfigured the bots this way.

[graydon]

This is now active. Reopen if you see a reason to.