Skip to content

Commit e498b5d

Browse files
committed
Make TokenStream::from_iter less general and more efficient.
The current code has this impl: ``` impl<T: Into<TokenStream>> iter::FromIterator<T> for TokenStream ``` If given an `IntoIterator<Item = TokenTree>`, it will convert each individual `TokenTree` to a `TokenStream` (at the cost of two allocations: a `Vec` and an `Lrc`). It will then merge those `TokenStream`s into a single `TokenStream`. This is inefficient. This commit changes the impl to this less general one: ``` impl iter::FromIterator<TokenTree> for TokenStream ``` It collects the `TokenTree`s into a single `Vec` first and then converts that to a `TokenStream` by wrapping it in a single `Lrc`. The previous generality was unnecessary; no other code needs changing. This change speeds up several benchmarks by up to 4%.
1 parent 237d54f commit e498b5d

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

src/libsyntax/tokenstream.rs

+3-3
Original file line numberDiff line numberDiff line change
@@ -202,9 +202,9 @@ impl From<TokenTree> for TreeAndJoint {
202202
}
203203
}
204204

205-
impl<T: Into<TokenStream>> iter::FromIterator<T> for TokenStream {
206-
fn from_iter<I: IntoIterator<Item = T>>(iter: I) -> Self {
207-
TokenStream::from_streams(iter.into_iter().map(Into::into).collect::<SmallVec<_>>())
205+
impl iter::FromIterator<TokenTree> for TokenStream {
206+
fn from_iter<I: IntoIterator<Item = TokenTree>>(iter: I) -> Self {
207+
TokenStream::new(iter.into_iter().map(Into::into).collect::<Vec<TreeAndJoint>>())
208208
}
209209
}
210210

0 commit comments

Comments
 (0)