Skip to content

Some parser and AST cleanups #114353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Aug 3, 2023
Merged

Some parser and AST cleanups #114353

merged 8 commits into from
Aug 3, 2023

Conversation

nnethercote
Copy link
Contributor

Things I found while looking closely at this code.

r? @petrochenkov

@nnethercote
parse_all_token_trees cannot fail.
d9f2fdf
@nnethercote
Inline and remove parse_all_token_trees.
db7b07a 
It has a single call site.
@nnethercote
Only call parse_token_tree once.
203cba7 
This code currently uses a `while` loop and gathers token trees into a
vector, but it only succeeds in the case where there is a single token
tree. We can instead just call `parse_token_tree` once and then look for
`Eof` to detect the success case.
@nnethercote
Move TokenCursor::num_next_calls into Parser and rename it.
54eb6bc 
It's more of a `Parser`-level concern than a `TokenCursor`-level
concern. Also, `num_bump_calls` is a more accurate name, because it's
incremented in `Parser::bump`.
@nnethercote
Move TokenCursor::break_last_token into Parser.
6fc2c48 
Similar to the last commit, it's more of a `Parser`-level concern than a
`TokenCursor`-level concern. And the struct size reductions are nice.

After this change, `TokenCursor` is as minimal as possible (two fields
and two methods) which is nice.
@nnethercote
Avoid an unnecessary local variable.
ac64a53
@nnethercote
Fix an erroneous comment.
ba294a8 
The `None` variant has already been renamed `Invisible`.
@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Aug 2, 2023
@rustbot
Copy link
Collaborator

rustbot commented Aug 2, 2023

Some changes occurred in src/tools/rustfmt

cc @rust-lang/rustfmt

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@petrochenkov
Copy link
Contributor

r=me after addressing #114353 (comment) and fixing doc comments.
@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 2, 2023
@nnethercote nnethercote force-pushed the parser-ast-cleanups branch from 69f8ece to 60231d6 Compare August 2, 2023 09:03
@nnethercote
Copy link
Contributor Author

Thank you for the fast review. I addressed the review comments and fixed the doc comments.

@bors r=petrochenkov rollup

@bors
Copy link
Collaborator

bors commented Aug 2, 2023

📌 Commit 60231d6 has been approved by petrochenkov

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 2, 2023
@rust-log-analyzer

This comment has been minimized.

Sign in to view
Noratrieb added a commit to Noratrieb/rust that referenced this pull request Aug 2, 2023
@Noratrieb
Rollup merge of rust-lang#114353 - nnethercote:parser-ast-cleanups, r…
afb15c4 
…=petrochenkov

Some parser and AST cleanups

Things I found while looking closely at this code.

r? `@petrochenkov`
@Noratrieb Noratrieb mentioned this pull request Aug 2, 2023
Closed
@Noratrieb
Copy link
Member

@bors r- CI failure

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 2, 2023
@nnethercote
Remove MacDelimiter.
d75ee2a 
It's the same as `Delimiter`, minus the `Invisible` variant. I'm
generally in favour of using types to make impossible states
unrepresentable, but this one feels very low-value, and the conversions
between the two types are annoying and confusing.

Look at the change in `src/tools/rustfmt/src/expr.rs` for an example:
the old code converted from `MacDelimiter` to `Delimiter` and back
again, for no good reason. This suggests the author was confused about
the types.
@nnethercote nnethercote force-pushed the parser-ast-cleanups branch from 60231d6 to d75ee2a Compare August 2, 2023 23:03
@nnethercote
Copy link
Contributor Author

I fixed the rustfmt formatting error.

@bors r=petrochenkov

@bors
Copy link
Collaborator

bors commented Aug 2, 2023

📌 Commit d75ee2a has been approved by petrochenkov

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 2, 2023
@bors
Copy link
Collaborator

bors commented Aug 3, 2023

⌛ Testing commit d75ee2a with merge fb31b3c...

@bors
Copy link
Collaborator

bors commented Aug 3, 2023

☀️ Test successful - checks-actions
Approved by: petrochenkov
Pushing fb31b3c to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 3, 2023
@bors bors merged commit fb31b3c into rust-lang:master Aug 3, 2023
@rustbot rustbot added this to the 1.73.0 milestone Aug 3, 2023
@bors bors mentioned this pull request Aug 3, 2023
Closed
@nnethercote nnethercote deleted the parser-ast-cleanups branch August 3, 2023 08:53
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (fb31b3c): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 -1.0% [-1.7%, -0.6%] 5
All ❌✅ (primary) - - 0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 4.3% [1.5%, 7.0%] 7
Improvements ✅
(primary)		 -2.5% [-2.5%, -2.5%] 1
Improvements ✅
(secondary)		 -2.3% [-2.3%, -2.3%] 1
All ❌✅ (primary) -2.5% [-2.5%, -2.5%] 1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 650.497s -> 647.579s (-0.45%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@petrochenkov petrochenkov petrochenkov left review comments

Assignees

@petrochenkov petrochenkov

Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
1.73.0
Development

Successfully merging this pull request may close these issues.
7 participants
@nnethercote @rustbot @rust-log-analyzer @petrochenkov @bors @Noratrieb @rust-timer