|
20 | 20 | #
|
21 | 21 | # Using MaskedTensor
|
22 | 22 | # ++++++++++++++++++
|
| 23 | +# |
| 24 | +# In this section we discuss how to use MaskedTensor including how to construct, access, the data |
| 25 | +# and mask, as well as indexing and slicing. |
23 | 26 | #
|
24 | 27 | # Construction
|
25 | 28 | # ------------
|
|
174 | 177 | x = torch.tensor([1., 1.], requires_grad=True)
|
175 | 178 | div = torch.tensor([0., 1.])
|
176 | 179 | y = x/div # => y is [inf, 1]
|
177 |
| - >>> |
178 | 180 | mask = (div != 0) # => mask is [0, 1]
|
179 | 181 | loss = as_masked_tensor(y, mask)
|
180 | 182 | loss.sum().backward()
|
|
213 | 215 | # Safe Softmax
|
214 | 216 | # ------------
|
215 | 217 | #
|
216 |
| -# Safe softmax is another great example of `an issue <https://github.com/pytorch/pytorch/issues/55056>`_ |
| 218 | +# Safe softmax is another great example of `an issue <https://github.com/pytorch/pytorch/issues/55056>`__ |
217 | 219 | # that arises frequently. In a nutshell, if there is an entire batch that is "masked out"
|
218 | 220 | # or consists entirely of padding (which, in the softmax case, translates to being set `-inf`),
|
219 | 221 | # then this will result in NaNs, which can lead to training divergence.
|
|
247 | 249 |
|
248 | 250 | ######################################################################
|
249 | 251 | # Implementing missing torch.nan* operators
|
250 |
| -# -------------------------------------------------------------------------------------------------------------- |
| 252 | +# ----------------------------------------- |
251 | 253 | #
|
252 | 254 | # In `Issue 61474 <<https://github.com/pytorch/pytorch/issues/61474>`__,
|
253 | 255 | # there is a request to add additional operators to cover the various `torch.nan*` applications,
|
254 | 256 | # such as ``torch.nanmax``, ``torch.nanmin``, etc.
|
255 | 257 | #
|
256 | 258 | # In general, these problems lend themselves more naturally to masked semantics, so instead of introducing additional
|
257 |
| -# operators, we propose using :class:`MaskedTensor`s instead. Since |
258 |
| -# `nanmean has already landed <https://github.com/pytorch/pytorch/issues/21987>`_, we can use it as a comparison point: |
| 259 | +# operators, we propose using :class:`MaskedTensor`s instead. |
| 260 | +# Since `nanmean has already landed <https://github.com/pytorch/pytorch/issues/21987>`__, |
| 261 | +# we can use it as a comparison point: |
259 | 262 | #
|
260 | 263 |
|
261 | 264 | x = torch.arange(16).float()
|
|
0 commit comments