1
1
# Closure Expansion in rustc
2
2
3
- Let's start with a few examples
3
+ This section describes how rustc handles closures. Closures in Rust are
4
+ effectively "desugared" into structs that contain the values they use (or
5
+ references to the values they use) from their creator's stack frame. rustc has
6
+ the job of figuring out which values a closure uses and how, so it can decide
7
+ whether to capture a given variable by shared reference, mutable reference, or
8
+ by move. rustc also has to figure out which the closure traits ([ ` Fn ` ] [ fn ] ,
9
+ [ ` FnMut ` ] [ fn_mut ] , or [ ` FnOnce ` ] [ fn_once ] ) a closure is capable of
10
+ implementing.
11
+
12
+ [ fn ] : https://doc.rust-lang.org/std/ops/trait.Fn.html
13
+ [ fn_mut ] :https://doc.rust-lang.org/std/ops/trait.FnMut.html
14
+ [ fn_once ] : https://doc.rust-lang.org/std/ops/trait.FnOnce.html
15
+
16
+ Let's start with a few examples:
4
17
5
18
### Example 1
19
+
20
+ To start, let's take a look at how the closure in the following example is desugared:
21
+
6
22
``` rust
7
23
fn closure (f : impl Fn ()) {
8
24
f ();
@@ -14,22 +30,37 @@ fn main() {
14
30
println! (" Value of x after return {}" , x );
15
31
}
16
32
```
17
- Let's say the above is the content of a file called immut.rs. If we compile immut.rs using the command
18
- ```
19
- rustc +stage1 immut.rs -Zdump-mir=all
33
+
34
+ Let's say the above is the content of a file called ` immut.rs ` . If we compile
35
+ ` immut.rs ` using the following command. The [ ` -Zdump-mir=all ` ] [ dump-mir ] flag will cause
36
+ ` rustc ` to generate and dump the [ MIR] [ mir ] to a directory called ` mir_dump ` .
37
+ ``` console
38
+ > rustc +stage1 immut.rs -Zdump-mir=all
20
39
```
21
- we will see a newly generated directory in our current working directory called mir_dump, which will
22
- contain several files. If we look at file ` rustc.main.-------.mir_map.0.mir ` , we will find, among
40
+
41
+ [ mir ] : ./mir/index.md
42
+ [ dump-mir ] : ./mir/passes.md
43
+
44
+ After we run this command, we will see a newly generated directory in our
45
+ current working directory called ` mir_dump ` , which will contain several files.
46
+ If we look at file ` rustc.main.-------.mir_map.0.mir ` , we will find, among
23
47
other things, it also contains this line:
24
48
25
49
``` rust,ignore
26
- _4 = &_1; // bb0[6]: scope 1 at immut.rs:7:13: 7:36
27
- _3 = [[email protected] :7:13: 7:36] { x: move _4 }; // bb0[7]: scope 1 at immut.rs:7:13: 7:36
50
+ _4 = &_1;
51
+ _3 = [[email protected] :7:13: 7:36] { x: move _4 };
28
52
```
29
- Here in first line ` _4 = &_1; ` , the mir_dump tells us that x was borrowed as an immutable reference.
30
- This is what we would hope as our closure just reads x.
53
+
54
+ Note that in the MIR examples in this chapter, ` _1 ` is ` x ` .
55
+
56
+ Here in first line ` _4 = &_1; ` , the ` mir_dump ` tells us that ` x ` was borrowed
57
+ as an immutable reference. This is what we would hope as our closure just
58
+ reads ` x ` .
31
59
32
60
### Example 2
61
+
62
+ Here is another example:
63
+
33
64
``` rust
34
65
fn closure (mut f : impl FnMut ()) {
35
66
f ();
@@ -46,13 +77,16 @@ fn main() {
46
77
```
47
78
48
79
``` rust,ignore
49
- _4 = &mut _1; // bb0[6]: scope 1 at mut.rs:7:13: 10:6
50
- _3 = [[email protected] :7:13: 10:6] { x: move _4 }; // bb0[7]: scope 1 at mut.rs:7:13: 10:6
80
+ _4 = &mut _1;
81
+ _3 = [[email protected] :7:13: 10:6] { x: move _4 };
51
82
```
52
83
This time along, in the line ` _4 = &mut _1; ` , we see that the borrow is changed to mutable borrow.
53
- fair enough as the closure increments x by 10.
84
+ Fair enough! The closure increments ` x ` by 10.
54
85
55
86
### Example 3
87
+
88
+ One more example:
89
+
56
90
``` rust
57
91
fn closure (f : impl FnOnce ()) {
58
92
f ();
@@ -70,57 +104,66 @@ fn main() {
70
104
``` rust,ignore
71
105
_6 = [[email protected] :7:13: 9:6] { x: move _1 }; // bb16[3]: scope 1 at move.rs:7:13: 9:6
72
106
```
73
- Here, x is directly moved into the closure and the access to it will not be permitted after the
107
+ Here, ` x ` is directly moved into the closure and the access to it will not be permitted after the
74
108
closure.
75
109
110
+ ## Inferences in the compiler
76
111
77
112
Now let's dive into rustc code and see how all these inferences are done by the compiler.
78
113
79
114
Let's start with defining a term that we will be using quite a bit in the rest of the discussion -
80
- * upvar* . An ** upvar** is a variable that is local to the function, where the closure is defined. So,
115
+ * upvar* . An ** upvar** is a variable that is local to the function where the closure is defined. So,
81
116
in the above examples, ** x** will be an upvar to the closure. They are also sometimes referred to as
82
117
the * free variables* meaning they are not bound to the context of the closure.
83
- ` src/librustc/ty/query/mod.rs ` defines a query called * freevars* for this purpose.
118
+ [ ` src/librustc/ty/query/mod.rs ` ] [ freevars ] defines a query called * freevars* for this purpose.
84
119
85
- So, we know that other than lazy invocation, one other thing that the distinguishes a closure from a
86
- normal function is that it can use the upvars. Because, it borrows these upvars from its surrounding
87
- context, therfore the compiler has to determine the upvar's borrow type. The compiler starts with
120
+ [ freevars ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/query/queries/struct.freevars.html
121
+
122
+ Other than lazy invocation, one other thing that the distinguishes a closure from a
123
+ normal function is that it can use the upvars. It borrows these upvars from its surrounding
124
+ context; therfore the compiler has to determine the upvar's borrow type. The compiler starts with
88
125
assigning an immutable borrow type and lowers the restriction (that is, changes it from
89
126
** immutable** to ** mutable** to ** move** ) as needed, based on the usage. In the Example 1 above, the
90
127
closure only uses the variable for printing but does not modify it in any way and therefore, in the
91
- mir_dump, we find the borrow type for the upvar x to be immutable. In example 2, however the
92
- closure modifies x and increments it by some value. Because of this mutation, the compiler, which
93
- started off assigning x as an immutable reference type, has to adjust it as mutable reference.
128
+ ` mir_dump ` , we find the borrow type for the upvar ` x ` to be immutable. In example 2, however, the
129
+ closure modifies ` x ` and increments it by some value. Because of this mutation, the compiler, which
130
+ started off assigning ` x ` as an immutable reference type, has to adjust it as a mutable reference.
94
131
Likewise in the third example, the closure drops the vector and therefore this requires the variable
95
- x to be moved into the closure. Depending on the borrow kind, the closure has to implement the
96
- appropriate trait. Fn trait for immutable borrow, FnMut for mutable borrow and FnOnce for move
97
- semantics.
132
+ ` x ` to be moved into the closure. Depending on the borrow kind, the closure has to implement the
133
+ appropriate trait: ` Fn ` trait for immutable borrow, ` FnMut ` for mutable borrow,
134
+ and ` FnOnce ` for move semantics.
135
+
136
+ Most of the code related to the closure is in the
137
+ [ ` src/librustc_typeck/check/upvar.rs ` ] [ upvar ] file and the data structures are
138
+ declared in the file [ ` src/librustc/ty/mod.rs ` ] [ ty ] .
98
139
99
- Most of the code related to the closure is in the src/librustc_typeck/ check/upvar.rs file and the
100
- data structures are declared in the file src/librustc/ ty/mod.rs.
140
+ [ upvar ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/ check/upvar/index.html
141
+ [ ty ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ ty/index.html
101
142
102
143
Before we go any further, let's discuss how we can examine the flow of coontrol through the rustc
103
- codebase. For the closure part specifically, I would set the RUST_LOG as under and collect the
104
- output in a file
144
+ codebase. For closures specifically, set the ` RUST_LOG ` env variable as below and collect the
145
+ output in a file:
105
146
106
- ```
107
- RUST_LOG=rustc_typeck::check::upvar rustc +stage1 -Zdump-mir=all <.rs file to compile> 2> <file
108
- where the output will be dumped>
147
+ ``` console
148
+ > RUST_LOG=rustc_typeck::check::upvar rustc +stage1 -Zdump-mir=all \
149
+ <.rs file to compile> 2> <file where the output will be dumped>
109
150
```
110
151
111
- This uses the stage1 compiler.
152
+ This uses the stage1 compiler and enables ` debug! ` logging for the
153
+ ` rustc_typeck::check::upvar ` module.
112
154
113
155
The other option is to step through the code using lldb or gdb.
114
156
115
- ```
116
- 1. rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs
117
- 2. b upvar.rs:134 // Setting the breakpoint on a certain line in the upvar.rs file
118
- 3. r // Run the program until it hits the breakpoint
119
- ```
157
+ 1 . ` rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs `
158
+ 2 . In lldb:
159
+ 1 . ` b upvar.rs:134 ` // Setting the breakpoint on a certain line in the upvar.rs file`
160
+ 2 . ` r ` // Run the program until it hits the breakpoint
161
+
162
+ Let's start with [ ` upvar.rs ` ] [ upvar ] . This file has something called
163
+ the [ ` euv::ExprUseVisitor ` ] [ euv ] which walks the source of the closure and
164
+ invokes a callbackfor each upvar that is borrowed, mutated, or moved.
120
165
121
- Let's start with the file: ` upvar.rs ` . This file has something called the euv::ExprUseVisitor which
122
- walks the source of the closure and it gets called back for each upvar that is borrowed, mutated or
123
- moved.
166
+ [ euv ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/expr_use_visitor/struct.ExprUseVisitor.html
124
167
125
168
``` rust
126
169
fn main () {
@@ -132,28 +175,39 @@ fn main() {
132
175
}
133
176
```
134
177
135
- In the above example, our visitor will be called twice, for the lines marked 1 and 2, once as a
136
- shared borrow and another one as a mutable borrow. It will also tell as what was borrowed. The
137
- callbacks get invoked at the delegate. The delegate is of type ` struct InferBorrowKind ` which has a
138
- few fields but the one we are interested in is the ` adjust_upvar_captures ` which is of type
139
- ` FxHashMap<UpvarId, UpvarCapture<'tcx>> ` which tells us for each upvar, which mode of borrow did we
140
- require. The modes of borrow can be ByValue (moved) or ByRef (borrowed) and for ByRef borrows, it
141
- can be one among shared, shallow, unique or mut as defined in the ` src/librustc/mir/mod.rs `
142
-
143
- The method callbacks are the method implementations of the euv::Delegate trait for InferBorrowKind.
144
- ** consume** callback is for * move* of a variable, ** borrow** callback if there is a * borrow* of some
145
- kind, shared or mutable and ** mutate** when we see an * assignment* of something. We will see that
146
- all these callbacks have a common argument * cmt* which stands for category, Mutability and Type and
147
- is defined in * src/librustc/middle/mem_categorization.rs* . Borrowing from the code comments * cmt * is
148
- a complete categorization of a value indicating where it originated and how it is located, as well
149
- as the mutability of the memory in which the value is stored.** Based on the callback (consume,
150
- borrow etc.), we will call the relevant * adjust_upvar_borrow_kind_for_ <something >* and pass the cmt
151
- along. Once the borrow type is adjusted, we store it in the table, which basically says for this
152
- closure, these set of borrows were made.
178
+ In the above example, our visitor will be called twice, for the lines marked 1 and 2, once for a
179
+ shared borrow and another one for a mutable borrow. It will also tell us what was borrowed.
153
180
154
- ```
181
+ The callbacks are defined by implementing the [ ` Delegate ` ] [ delegate ] trait. The
182
+ [ ` InferBorrowKind ` ] [ ibk ] type implements ` Delegate ` and keeps a map that
183
+ records for each upvar which mode of borrow was required. The modes of borrow
184
+ can be ` ByValue ` (moved) or ` ByRef ` (borrowed). For ` ByRef ` borrows, it can be
185
+ ` shared ` , ` shallow ` , ` unique ` or ` mut ` as defined in the
186
+ [ ` src/librustc/mir/mod.rs ` ] [ mir_mod ] .
187
+
188
+ [ mir_mod ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/mir/index.html
189
+
190
+ ` Delegate ` defines a few different methods (the different callbacks):
191
+ ** consume** : for * move* of a variable, ** borrow** for a * borrow* of some kind
192
+ (shared or mutable), and ** mutate** when we see an * assignment* of something.
193
+
194
+ All of these callbacks have a common argument * cmt* which stands for Category,
195
+ Mutability and Type and is defined in
196
+ [ ` src/librustc/middle/mem_categorization.rs ` ] [ cmt ] . Borrowing from the code
197
+ comments, "` cmt ` is a complete categorization of a value indicating where it
198
+ originated and how it is located, as well as the mutability of the memory in
199
+ which the value is stored". Based on the callback (consume, borrow etc.), we
200
+ will call the relevant * adjust_upvar_borrow_kind_for_ <something >* and pass the
201
+ ` cmt ` along. Once the borrow type is adjusted, we store it in the table, which
202
+ basically says what borrows were made for each closure.
203
+
204
+ ``` rust,ignore
155
205
self.tables
156
206
.borrow_mut()
157
207
.upvar_capture_map
158
208
.extend(delegate.adjust_upvar_captures);
159
209
```
210
+
211
+ [ delegate ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/expr_use_visitor/trait.Delegate.html
212
+ [ ibk ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/check/upvar/struct.InferBorrowKind.html
213
+ [ cmt ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/mem_categorization/index.html
0 commit comments