Skip to content

Commit 9a4ff92

Browse files
committed
Some edits to address review comments
1 parent ea6f6b5 commit 9a4ff92

File tree

1 file changed

+114
-60
lines changed

1 file changed

+114
-60
lines changed

src/closure.md

+114-60
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,24 @@
11
# Closure Expansion in rustc
22

3-
Let's start with a few examples
3+
This section describes how rustc handles closures. Closures in Rust are
4+
effectively "desugared" into structs that contain the values they use (or
5+
references to the values they use) from their creator's stack frame. rustc has
6+
the job of figuring out which values a closure uses and how, so it can decide
7+
whether to capture a given variable by shared reference, mutable reference, or
8+
by move. rustc also has to figure out which the closure traits ([`Fn`][fn],
9+
[`FnMut`][fn_mut], or [`FnOnce`][fn_once]) a closure is capable of
10+
implementing.
11+
12+
[fn]: https://doc.rust-lang.org/std/ops/trait.Fn.html
13+
[fn_mut]:https://doc.rust-lang.org/std/ops/trait.FnMut.html
14+
[fn_once]: https://doc.rust-lang.org/std/ops/trait.FnOnce.html
15+
16+
Let's start with a few examples:
417

518
### Example 1
19+
20+
To start, let's take a look at how the closure in the following example is desugared:
21+
622
```rust
723
fn closure(f: impl Fn()) {
824
f();
@@ -14,22 +30,37 @@ fn main() {
1430
println!("Value of x after return {}", x);
1531
}
1632
```
17-
Let's say the above is the content of a file called immut.rs. If we compile immut.rs using the command
18-
```
19-
rustc +stage1 immut.rs -Zdump-mir=all
33+
34+
Let's say the above is the content of a file called `immut.rs`. If we compile
35+
`immut.rs` using the following command. The [`-Zdump-mir=all`][dump-mir] flag will cause
36+
`rustc` to generate and dump the [MIR][mir] to a directory called `mir_dump`.
37+
```console
38+
> rustc +stage1 immut.rs -Zdump-mir=all
2039
```
21-
we will see a newly generated directory in our current working directory called mir_dump, which will
22-
contain several files. If we look at file `rustc.main.-------.mir_map.0.mir`, we will find, among
40+
41+
[mir]: ./mir/index.md
42+
[dump-mir]: ./mir/passes.md
43+
44+
After we run this command, we will see a newly generated directory in our
45+
current working directory called `mir_dump`, which will contain several files.
46+
If we look at file `rustc.main.-------.mir_map.0.mir`, we will find, among
2347
other things, it also contains this line:
2448

2549
```rust,ignore
26-
_4 = &_1; // bb0[6]: scope 1 at immut.rs:7:13: 7:36
27-
_3 = [[email protected]:7:13: 7:36] { x: move _4 }; // bb0[7]: scope 1 at immut.rs:7:13: 7:36
50+
_4 = &_1;
51+
_3 = [[email protected]:7:13: 7:36] { x: move _4 };
2852
```
29-
Here in first line `_4 = &_1;`, the mir_dump tells us that x was borrowed as an immutable reference.
30-
This is what we would hope as our closure just reads x.
53+
54+
Note that in the MIR examples in this chapter, `_1` is `x`.
55+
56+
Here in first line `_4 = &_1;`, the `mir_dump` tells us that `x` was borrowed
57+
as an immutable reference. This is what we would hope as our closure just
58+
reads `x`.
3159

3260
### Example 2
61+
62+
Here is another example:
63+
3364
```rust
3465
fn closure(mut f: impl FnMut()) {
3566
f();
@@ -46,13 +77,16 @@ fn main() {
4677
```
4778

4879
```rust,ignore
49-
_4 = &mut _1; // bb0[6]: scope 1 at mut.rs:7:13: 10:6
50-
_3 = [[email protected]:7:13: 10:6] { x: move _4 }; // bb0[7]: scope 1 at mut.rs:7:13: 10:6
80+
_4 = &mut _1;
81+
_3 = [[email protected]:7:13: 10:6] { x: move _4 };
5182
```
5283
This time along, in the line `_4 = &mut _1;`, we see that the borrow is changed to mutable borrow.
53-
fair enough as the closure increments x by 10.
84+
Fair enough! The closure increments `x` by 10.
5485

5586
### Example 3
87+
88+
One more example:
89+
5690
```rust
5791
fn closure(f: impl FnOnce()) {
5892
f();
@@ -70,57 +104,66 @@ fn main() {
70104
```rust,ignore
71105
_6 = [[email protected]:7:13: 9:6] { x: move _1 }; // bb16[3]: scope 1 at move.rs:7:13: 9:6
72106
```
73-
Here, x is directly moved into the closure and the access to it will not be permitted after the
107+
Here, `x` is directly moved into the closure and the access to it will not be permitted after the
74108
closure.
75109

110+
## Inferences in the compiler
76111

77112
Now let's dive into rustc code and see how all these inferences are done by the compiler.
78113

79114
Let's start with defining a term that we will be using quite a bit in the rest of the discussion -
80-
*upvar*. An **upvar** is a variable that is local to the function, where the closure is defined. So,
115+
*upvar*. An **upvar** is a variable that is local to the function where the closure is defined. So,
81116
in the above examples, **x** will be an upvar to the closure. They are also sometimes referred to as
82117
the *free variables* meaning they are not bound to the context of the closure.
83-
`src/librustc/ty/query/mod.rs` defines a query called *freevars* for this purpose.
118+
[`src/librustc/ty/query/mod.rs`][freevars] defines a query called *freevars* for this purpose.
84119

85-
So, we know that other than lazy invocation, one other thing that the distinguishes a closure from a
86-
normal function is that it can use the upvars. Because, it borrows these upvars from its surrounding
87-
context, therfore the compiler has to determine the upvar's borrow type. The compiler starts with
120+
[freevars]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/query/queries/struct.freevars.html
121+
122+
Other than lazy invocation, one other thing that the distinguishes a closure from a
123+
normal function is that it can use the upvars. It borrows these upvars from its surrounding
124+
context; therfore the compiler has to determine the upvar's borrow type. The compiler starts with
88125
assigning an immutable borrow type and lowers the restriction (that is, changes it from
89126
**immutable** to **mutable** to **move**) as needed, based on the usage. In the Example 1 above, the
90127
closure only uses the variable for printing but does not modify it in any way and therefore, in the
91-
mir_dump, we find the borrow type for the upvar x to be immutable. In example 2, however the
92-
closure modifies x and increments it by some value. Because of this mutation, the compiler, which
93-
started off assigning x as an immutable reference type, has to adjust it as mutable reference.
128+
`mir_dump`, we find the borrow type for the upvar `x` to be immutable. In example 2, however, the
129+
closure modifies `x` and increments it by some value. Because of this mutation, the compiler, which
130+
started off assigning `x` as an immutable reference type, has to adjust it as a mutable reference.
94131
Likewise in the third example, the closure drops the vector and therefore this requires the variable
95-
x to be moved into the closure. Depending on the borrow kind, the closure has to implement the
96-
appropriate trait. Fn trait for immutable borrow, FnMut for mutable borrow and FnOnce for move
97-
semantics.
132+
`x` to be moved into the closure. Depending on the borrow kind, the closure has to implement the
133+
appropriate trait: `Fn` trait for immutable borrow, `FnMut` for mutable borrow,
134+
and `FnOnce` for move semantics.
135+
136+
Most of the code related to the closure is in the
137+
[`src/librustc_typeck/check/upvar.rs`][upvar] file and the data structures are
138+
declared in the file [`src/librustc/ty/mod.rs`][ty].
98139

99-
Most of the code related to the closure is in the src/librustc_typeck/check/upvar.rs file and the
100-
data structures are declared in the file src/librustc/ty/mod.rs.
140+
[upvar]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/check/upvar/index.html
141+
[ty]:https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/index.html
101142

102143
Before we go any further, let's discuss how we can examine the flow of coontrol through the rustc
103-
codebase. For the closure part specifically, I would set the RUST_LOG as under and collect the
104-
output in a file
144+
codebase. For closures specifically, set the `RUST_LOG` env variable as below and collect the
145+
output in a file:
105146

106-
```
107-
RUST_LOG=rustc_typeck::check::upvar rustc +stage1 -Zdump-mir=all <.rs file to compile> 2> <file
108-
where the output will be dumped>
147+
```console
148+
> RUST_LOG=rustc_typeck::check::upvar rustc +stage1 -Zdump-mir=all \
149+
<.rs file to compile> 2> <file where the output will be dumped>
109150
```
110151

111-
This uses the stage1 compiler.
152+
This uses the stage1 compiler and enables `debug!` logging for the
153+
`rustc_typeck::check::upvar` module.
112154

113155
The other option is to step through the code using lldb or gdb.
114156

115-
```
116-
1. rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs
117-
2. b upvar.rs:134 // Setting the breakpoint on a certain line in the upvar.rs file
118-
3. r // Run the program until it hits the breakpoint
119-
```
157+
1. `rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs`
158+
2. In lldb:
159+
1. `b upvar.rs:134` // Setting the breakpoint on a certain line in the upvar.rs file`
160+
2. `r` // Run the program until it hits the breakpoint
161+
162+
Let's start with [`upvar.rs`][upvar]. This file has something called
163+
the [`euv::ExprUseVisitor`][euv] which walks the source of the closure and
164+
invokes a callbackfor each upvar that is borrowed, mutated, or moved.
120165

121-
Let's start with the file: `upvar.rs`. This file has something called the euv::ExprUseVisitor which
122-
walks the source of the closure and it gets called back for each upvar that is borrowed, mutated or
123-
moved.
166+
[euv]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/expr_use_visitor/struct.ExprUseVisitor.html
124167

125168
```rust
126169
fn main() {
@@ -132,28 +175,39 @@ fn main() {
132175
}
133176
```
134177

135-
In the above example, our visitor will be called twice, for the lines marked 1 and 2, once as a
136-
shared borrow and another one as a mutable borrow. It will also tell as what was borrowed. The
137-
callbacks get invoked at the delegate. The delegate is of type `struct InferBorrowKind` which has a
138-
few fields but the one we are interested in is the `adjust_upvar_captures` which is of type
139-
`FxHashMap<UpvarId, UpvarCapture<'tcx>>` which tells us for each upvar, which mode of borrow did we
140-
require. The modes of borrow can be ByValue (moved) or ByRef (borrowed) and for ByRef borrows, it
141-
can be one among shared, shallow, unique or mut as defined in the `src/librustc/mir/mod.rs`
142-
143-
The method callbacks are the method implementations of the euv::Delegate trait for InferBorrowKind.
144-
**consume** callback is for *move* of a variable, **borrow** callback if there is a *borrow* of some
145-
kind, shared or mutable and **mutate** when we see an *assignment* of something. We will see that
146-
all these callbacks have a common argument *cmt* which stands for category, Mutability and Type and
147-
is defined in *src/librustc/middle/mem_categorization.rs*. Borrowing from the code comments *cmt *is
148-
a complete categorization of a value indicating where it originated and how it is located, as well
149-
as the mutability of the memory in which the value is stored.** Based on the callback (consume,
150-
borrow etc.), we will call the relevant *adjust_upvar_borrow_kind_for_<something>* and pass the cmt
151-
along. Once the borrow type is adjusted, we store it in the table, which basically says for this
152-
closure, these set of borrows were made.
178+
In the above example, our visitor will be called twice, for the lines marked 1 and 2, once for a
179+
shared borrow and another one for a mutable borrow. It will also tell us what was borrowed.
153180

154-
```
181+
The callbacks are defined by implementing the [`Delegate`][delegate] trait. The
182+
[`InferBorrowKind`][ibk] type implements `Delegate` and keeps a map that
183+
records for each upvar which mode of borrow was required. The modes of borrow
184+
can be `ByValue` (moved) or `ByRef` (borrowed). For `ByRef` borrows, it can be
185+
`shared`, `shallow`, `unique` or `mut` as defined in the
186+
[`src/librustc/mir/mod.rs`][mir_mod].
187+
188+
[mir_mod]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/mir/index.html
189+
190+
`Delegate` defines a few different methods (the different callbacks):
191+
**consume**: for *move* of a variable, **borrow** for a *borrow* of some kind
192+
(shared or mutable), and **mutate** when we see an *assignment* of something.
193+
194+
All of these callbacks have a common argument *cmt* which stands for Category,
195+
Mutability and Type and is defined in
196+
[`src/librustc/middle/mem_categorization.rs`][cmt]. Borrowing from the code
197+
comments, "`cmt` is a complete categorization of a value indicating where it
198+
originated and how it is located, as well as the mutability of the memory in
199+
which the value is stored". Based on the callback (consume, borrow etc.), we
200+
will call the relevant *adjust_upvar_borrow_kind_for_<something>* and pass the
201+
`cmt` along. Once the borrow type is adjusted, we store it in the table, which
202+
basically says what borrows were made for each closure.
203+
204+
```rust,ignore
155205
self.tables
156206
.borrow_mut()
157207
.upvar_capture_map
158208
.extend(delegate.adjust_upvar_captures);
159209
```
210+
211+
[delegate]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/expr_use_visitor/trait.Delegate.html
212+
[ibk]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/check/upvar/struct.InferBorrowKind.html
213+
[cmt]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/mem_categorization/index.html

0 commit comments

Comments
 (0)