Description
There seems to be a bug in the way that Rust compiles Iterator::flat_map()
, in release mode, with complex iterators.
It also triggers on other "type" vs "type as trait" comparisons in the compiler, for example unsized-vec fails with:
error: internal compiler error: broken MIR in DropGlue(DefId(2:2141 ~ core[0ec9]::ptr::drop_in_place), Some(unsized_vec::UnsizedVec<dyn std::fmt::Debug>)) (after phase change to runtime-optimized) at bb0[0]:
Field projection `(*_1).field[0]` specified type `unsized_vec::inner::unaligned::UnalignedVecInner<dyn std::fmt::Debug>`, but actual type is `<dyn std::fmt::Debug as unsized_vec::inner::UnsizedVecImpl>::Impl`
--> /home/dev/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:490:1
|
490 | pub unsafe fn drop_in_place<T: ?Sized>(to_drop: *mut T) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: delayed at compiler/rustc_const_eval/src/transform/validate.rs:229:30
error: internal compiler error: broken MIR in DropGlue(DefId(2:2141 ~ core[0ec9]::ptr::drop_in_place), Some(unsized_vec::UnsizedVec<[i32]>)) (after phase change to runtime-optimized) at bb0[0]:
Field projection `(*_1).field[0]` specified type `unsized_vec::inner::aligned::AlignedVecInner<[i32]>`, but actual type is `<[i32] as unsized_vec::inner::UnsizedVecImpl>::Impl`
--> /home/dev/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:490:1
Code
Related crate code:
pub enum Transaction {
V5 {
orchard_shielded_data: Option<ShieldedData>,
},
}
impl Transaction {
pub fn orchard_shielded_data(&self) -> Option<&ShieldedData> {
match self {
// Maybe Orchard shielded data
Transaction::V5 {
orchard_shielded_data,
..
} => orchard_shielded_data.as_ref(),
}
}
pub fn orchard_nullifiers(&self) -> impl Iterator<Item = &Nullifier> {
self.orchard_shielded_data()
.into_iter()
.flat_map(ShieldedData::nullifiers)
}
}
pub struct ShieldedData {
pub actions: AtLeastOne<AuthorizedAction>,
}
impl ShieldedData {
pub fn actions(&self) -> impl Iterator<Item = &Action> {
self.actions.actions()
}
pub fn nullifiers(&self) -> impl Iterator<Item = &Nullifier> {
self.actions().map(|action| &action.nullifier)
}
}
impl AtLeastOne<AuthorizedAction> {
pub fn actions(&self) -> impl Iterator<Item = &Action> {
self.iter()
.map(|authorized_action| &authorized_action.action)
}
}
pub struct AuthorizedAction {
pub action: Action,
}
pub struct Action {
pub nullifier: (),
}
pub struct AtLeastOne<T> {
inner: Vec<T>,
}
impl<T> Deref for AtLeastOne<T> {
type Target = Vec<T>;
fn deref(&self) -> &Self::Target {
&self.inner
}
}
impl<T> AsRef<[T]> for AtLeastOne<T> {
fn as_ref(&self) -> &[T] {
self.inner.as_ref()
}
}
impl<T> IntoIterator for AtLeastOne<T> {
type Item = T;
type IntoIter = std::vec::IntoIter<T>;
fn into_iter(self) -> std::vec::IntoIter<T> {
self.inner.into_iter()
}
}
Failing code:
impl Transaction {
pub fn orchard_nullifiers(&self) -> impl Iterator<Item = &Nullifier> {
self.orchard_shielded_data()
.into_iter()
.flat_map(ShieldedData::nullifiers)
}
}
let orchard_nullifier_count = block
.transactions
.iter()
.flat_map(|t| t.orchard_nullifiers())
.count();
for nullifier in prepared.block.orchard_nullifiers() { }
Workaround:
pub fn orchard_nullifiers(&self) -> impl Iterator<Item = &orchard::Nullifier> {
let nullifiers: Vec<_> = self
.transactions
.iter()
.flat_map(|transaction| transaction.orchard_nullifiers())
.collect();
nullifiers.into_iter()
}
let orchard_nullifier_count: usize = block
.transactions
.iter()
.map(|t| t.orchard_nullifiers().into_iter().count())
.sum();
Meta
Fails from nightly nightly-2022-11-20 onwards, in commit 2f8d804.
rustc --version --verbose
:
rustc 1.67.0-nightly (2585bcea0 2022-11-28)
binary: rustc
commit-hash: 2585bcea0bc2a9c42a4be2c1eba5c61137f2b167
commit-date: 2022-11-28
host: x86_64-unknown-linux-gnu
release: 1.67.0-nightly
LLVM version: 15.0.4
I used these commands to bisect the nightly Rust version:
git clone https://github.com/ZcashFoundation/zebra.git
git checkout a763eec9f3
cd zebra-state
cat << EOF > cargo-build-release.sh
#!/usr/bin/env bash
cargo build --release
EOF
chmod +x cargo-build-release.sh
cargo-bisect-rustc --start=2022-11-07 --regress=ice --script=./cargo-build-release.sh
Zebra needs some build dependencies:
https://github.com/ZcashFoundation/zebra#build-instructions
Regression log:
searched nightlies: from nightly-2022-11-07 to nightly-2022-11-29
regressed nightly: nightly-2022-11-20
searched commit range: b833ad5...c5d82ed
regressed commit: 2f8d804
bisected with cargo-bisect-rustc v0.6.4
Host triple: x86_64-unknown-linux-gnu
Reproduce with:
cargo bisect-rustc --regress=ice --script=./cargo-build-release.sh
Analysis
This might be caused by -C linker-plugin-lto
or release builds, but I couldn't find an exact set of flags or a minimal example to reproduce it.
cargo build
does not fail, either from the individual crate or workspace directory. But cargo build --release
and cargo install --git https://github.com/ZcashFoundation/zebra.git zebrad
fail.
I tried commenting out my whole .cargo/config.toml
, and it didn't make a difference.
Error output
error: internal compiler error: no errors encountered even though `delay_span_bug` issued
error: internal compiler error: broken MIR in Item(WithOptConstParam { did: DefId(0:1718 ~ zebra_state[4d
f4]::service::finalized_state::zebra_db::metrics::block_precommit_metrics), const_param_did: None }) (aft
er phase change to runtime-optimized) at bb128[5]:
Field projection `_37.field[0]` specified type `std::iter::adapters::flat
ten::FlattenCompat<std::iter::Map<std::slice::Iter<'_, std::sync::Arc<zebra_chain::transaction::Transacti
on>>, [closure@zebra-state/src/service/finalized_state/zebra_db/metrics.rs:39:19: 39:22]>, std::iter::Fla
tMap<std::option::IntoIter<&zebra_chain::orchard::ShieldedData>, std::iter::Map<std::iter::Map<std::slice
::Iter<'_, zebra_chain::orchard::AuthorizedAction>, [closure@zebra_chain::orchard::shielded_data::<impl z
ebra_chain::serialization::AtLeastOne<zebra_chain::orchard::AuthorizedAction>>::actions::{closure#0}]>, [
closure@zebra_chain::orchard::ShieldedData::nullifiers::{closure#0}]>, for<'a> fn(&'a zebra_chain::orchar
d::ShieldedData) -> impl std::iter::Iterator<Item = &'a zebra_chain::orchard::Nullifier> {zebra_chain::or
chard::ShieldedData::nullifiers}>>`, but actual type is `std::iter::adapters::flatten::FlattenCompat<std:
:iter::Map<std::slice::Iter<'_, std::sync::Arc<zebra_chain::transaction::Transaction>>, [closure@zebra-st
ate/src/service/finalized_state/zebra_db/metrics.rs:39:19: 39:22]>, <std::iter::FlatMap<std::option::Into
Iter<&zebra_chain::orchard::ShieldedData>, std::iter::Map<std::iter::Map<std::slice::Iter<'_, zebra_chain
::orchard::AuthorizedAction>, [closure@zebra_chain::orchard::shielded_data::<impl zebra_chain::serializat
ion::AtLeastOne<zebra_chain::orchard::AuthorizedAction>>::actions::{closure#0}]>, [closure@zebra_chain::o
rchard::ShieldedData::nullifiers::{closure#0}]>, for<'a> fn(&'a zebra_chain::orchard::ShieldedData) -> im
pl std::iter::Iterator<Item = &'a zebra_chain::orchard::Nullifier> {zebra_chain::orchard::ShieldedData::n
ullifiers}> as std::iter::IntoIterator>::IntoIter>`
--> zebra-state/src/service/finalized_state/zebra_db/metrics.rs:36:35
|
36 | let orchard_nullifier_count = block
| ___________________________________^
37 | | .transactions
38 | | .iter()
39 | | .flat_map(|t| t.orchard_nullifiers())
40 | | .count();
| |________________^
|
= note: delayed at compiler/rustc_const_eval/src/transform/validate.rs:229:30
error: internal compiler error: broken MIR in Item(WithOptConstParam { did: DefId(0:4282 ~ zebra_state[4d
f4]::service::check::nullifier::no_duplicates_in_finalized_chain), const_param_did: None }) (after phase
change to runtime-optimized) at bb60[6]:
Field projection `(*_128).field[0]` specified type `std::iter::adapters::
flatten::FlattenCompat<std::iter::Map<std::slice::Iter<'_, std::sync::Arc<zebra_chain::transaction::Trans
action>>, [closure@zebra_chain::block::Block::orchard_nullifiers::{closure#0}]>, std::iter::FlatMap<std::
option::IntoIter<&zebra_chain::orchard::ShieldedData>, std::iter::Map<std::iter::Map<std::slice::Iter<'_,
zebra_chain::orchard::AuthorizedAction>, [closure@zebra_chain::orchard::shielded_data::<impl zebra_chain
::serialization::AtLeastOne<zebra_chain::orchard::AuthorizedAction>>::actions::{closure#0}]>, [closure@ze
bra_chain::orchard::ShieldedData::nullifiers::{closure#0}]>, for<'a> fn(&'a zebra_chain::orchard::Shielde
dData) -> impl std::iter::Iterator<Item = &'a zebra_chain::orchard::Nullifier> {zebra_chain::orchard::Shi
eldedData::nullifiers}>>`, but actual type is `std::iter::adapters::flatten::FlattenCompat<std::iter::Map
<std::slice::Iter<'_, std::sync::Arc<zebra_chain::transaction::Transaction>>, [closure@zebra_chain::block
::Block::orchard_nullifiers::{closure#0}]>, <std::iter::FlatMap<std::option::IntoIter<&zebra_chain::orcha
rd::ShieldedData>, std::iter::Map<std::iter::Map<std::slice::Iter<'_, zebra_chain::orchard::AuthorizedAct
ion>, [closure@zebra_chain::orchard::shielded_data::<impl zebra_chain::serialization::AtLeastOne<zebra_ch
ain::orchard::AuthorizedAction>>::actions::{closure#0}]>, [closure@zebra_chain::orchard::ShieldedData::nu
llifiers::{closure#0}]>, for<'a> fn(&'a zebra_chain::orchard::ShieldedData) -> impl std::iter::Iterator<I
tem = &'a zebra_chain::orchard::Nullifier> {zebra_chain::orchard::ShieldedData::nullifiers}> as std::iter
::IntoIterator>::IntoIter>`
--> zebra-state/src/service/check/nullifier.rs:48:22
|
48 | for nullifier in prepared.block.orchard_nullifiers() {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: delayed at compiler/rustc_const_eval/src/transform/validate.rs:229:30]
thread 'rustc' panicked at 'Box<dyn Any>', compiler/rustc_errors/src/lib.rs:1609:13
Backtrace
stack backtrace:
0: std::panicking::begin_panic::<rustc_errors::ExplicitBug>
1: std::panic::panic_any::<rustc_errors::ExplicitBug>
2: <rustc_errors::HandlerInner>::flush_delayed::<alloc::vec::Vec<rustc_errors::diagnostic::Diagnostic>
, &str>
3: <rustc_errors::HandlerInner as core::ops::drop::Drop>::drop
4: core::ptr::drop_in_place::<rustc_session::parse::ParseSess>
5: core::ptr::drop_in_place::<rustc_session::session::Session>
6: core::ptr::drop_in_place::<rustc_interface::interface::Compiler>
7: rustc_span::with_source_map::<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_interf
ace::interface::run_compiler<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_driver::run_c
ompiler::{closure#1}>::{closure#0}::{closure#0}>
8: <scoped_tls::ScopedKey<rustc_span::SessionGlobals>>::set::<rustc_interface::interface::run_compiler
<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_driver::run_compiler::{closure#1}>::{clos
ure#0}, core::result::Result<(), rustc_errors::ErrorGuaranteed>>
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
note: compiler flags: --crate-type lib -C opt-level=3 -C panic=abort -C linker-plugin-lto -C debuginfo=1
-C debug-assertions=on
note: some of the compiler flags provided by cargo are hidden
query stack during panic:
end of query stack
error: could not compile `zebra-state`