Container -> Module

Author: soumith

Creating Extensions using FFI.md

@@ -98,7 +98,7 @@ import torch.nn as nn
 from torch.autograd import Variable
 from modules.add import MyAddModule
 
-class MyNetwork(nn.Container):
+class MyNetwork(nn.Module):
     def __init__(self):
         super(MyNetwork, self).__init__(
             add=MyAddModule(),

Deep Learning with PyTorch.ipynb

@@ -14,14 +14,15 @@
     "*This tutorial assumes that you have a basic familiarity of numpy*\n",
     "\n",
     "\n",
+    "**Note:** Make sure you have the [torch](https://github.com/pytorch/pytorch) and [torchvision](https://github.com/pytorch/vision) packages installed.\n",
+    "\n",
+    "\n",
     "### What is PyTorch?\n",
     "\n",
     "It's a Python based scientific computing package targeted at two sets of audiences:\n",
     "\n",
     "- A replacement for numpy to use the power of GPUs\n",
-    "- a deep learning research platform that provides maximum flexibility and speed\n",
-    "\n",
-    "**If you want to complete the full tutorial, including training a neural network for image classification, you have to install the `torchvision` package.**"
+    "- a deep learning research platform that provides maximum flexibility and speed\n"
    ]
   },
   {
@@ -38,7 +39,7 @@
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
-    "collapsed": true
+    "collapsed": false
    },
    "outputs": [],
    "source": [
@@ -61,7 +62,7 @@
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
-    "collapsed": true
+    "collapsed": false
    },
    "outputs": [],
    "source": [
@@ -402,7 +403,9 @@
    "outputs": [],
    "source": [
     "# let's backprop now\n",
-    "out.backward()"
+    "out.backward()\n",
+    "\n",
+    "# out.backward() is equivalent to doing out.backward(torch.Tensor([1.0]))"
    ]
   },
   {
@@ -501,7 +504,7 @@
     "\n",
     "Now that you had a glimpse of `autograd`, `nn` depends on `autograd` to define models and differentiate them.\n",
     "\n",
-    "An `nn.Container` contains layers, and a method `forward(input)`that returns the `output`.\n",
+    "An `nn.Module` contains layers, and a method `forward(input)`that returns the `output`.\n",
     "\n",
     "For example, look at this network that classfies digit images:\n",
     "\n",
@@ -533,11 +536,8 @@
    "source": [
     "import torch.nn as nn\n",
     "import torch.nn.functional as F\n",
-    "# Some more python helpers\n",
-    "import functools\n",
-    "import operator\n",
     "\n",
-    "class Net(nn.Container):\n",
+    "class Net(nn.Module):\n",
     "    def __init__(self):\n",
     "        super(Net, self).__init__()\n",
     "        self.conv1 = nn.Conv2d(1, 6, 5) # 1 input image channel, 6 output channels, 5x5 square convolution kernel\n",
@@ -556,7 +556,11 @@
     "        return x\n",
     "    \n",
     "    def num_flat_features(self, x):\n",
-    "        return functools.reduce(operator.mul, x.size()[1:])\n",
+    "        size = x.size()[1:] # all dimensions except the batch dimension\n",
+    "        num_features = 1\n",
+    "        for s in size:\n",
+    "            num_features *= s\n",
+    "        return num_features\n",
     "\n",
     "net = Net()\n",
     "net"
@@ -638,7 +642,6 @@
     "* `torch.Tensor` - A **multi-dimensional array**.\n",
     "* `autograd.Variable` - **Wraps a Tensor and records the history of operations** applied to it. Has the same API as a `Tensor`, with some additions like `backward()`. Also **holds the gradient** w.r.t. the tensor.\n",
     "* `nn.Module` - Neural network module. **Convenient way of encapsulating parameters**, with helpers for moving them to GPU, exporting, loading, etc.\n",
-    "* `nn.Container` - `Module` that is a **container for other Modules**.\n",
     "* `nn.Parameter` - A kind of Variable, that is **automatically registered as a parameter when assigned as an attribute to a `Module`**.\n",
     "* `autograd.Function` - Implements **forward and backward definitions of an autograd operation**. Every `Variable` operation, creates at least a single `Function` node, that connects to functions that created a `Variable` and **encodes its history**.\n",
     "\n",
@@ -659,7 +662,7 @@
    "source": [
     "A loss function takes the (output, target) pair of inputs, and computes a value that estimates how far away the output is from the target.\n",
     "\n",
-    "There are several different loss functions under the nn package.\n",
+    "There are [several different loss functions under the nn package](http://pytorch.org/docs/nn.html#loss-functions).\n",
     "\n",
     "A simple loss is: `nn.MSELoss` which computes the mean-squared error between the input and the target.\n",
     "\n",
@@ -955,7 +958,7 @@
    },
    "outputs": [],
    "source": [
-    "class Net(nn.Container):\n",
+    "class Net(nn.Module):\n",
     "    def __init__(self):\n",
     "        super(Net, self).__init__()\n",
     "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
@@ -964,14 +967,13 @@
     "        self.fc1   = nn.Linear(16*5*5, 120)\n",
     "        self.fc2   = nn.Linear(120, 84)\n",
     "        self.fc3   = nn.Linear(84, 10)\n",
-    "        self.relu  = nn.ReLU()\n",
     "\n",
     "    def forward(self, x):\n",
-    "        x = self.pool(self.relu(self.conv1(x)))\n",
-    "        x = self.pool(self.relu(self.conv2(x)))\n",
+    "        x = self.pool(F.relu(self.conv1(x)))\n",
+    "        x = self.pool(F.relu(self.conv2(x)))\n",
     "        x = x.view(-1, 16*5*5)\n",
-    "        x = self.relu(self.fc1(x))\n",
-    "        x = self.relu(self.fc2(x))\n",
+    "        x = F.relu(self.fc1(x))\n",
+    "        x = F.relu(self.fc2(x))\n",
     "        x = self.fc3(x)\n",
     "        return x\n",
     "\n",
@@ -1041,7 +1043,8 @@
     "        running_loss += loss.data[0]\n",
     "        if i % 2000 == 1999: # print every 2000 mini-batches\n",
     "            print('[%d, %5d] loss: %.3f' % (epoch+1, i+1, running_loss / 2000))\n",
-    "            running_loss = 0.0"
+    "            running_loss = 0.0\n",
+    "print('Finished Training')"
    ]
   },
   {
@@ -1186,6 +1189,7 @@
    "source": [
     "#### Training on the GPU\n",
     "Just like how you transfer a Tensor on to the GPU, you transfer the neural net onto the GPU.\n",
+    "\n",
     "This will recursively go over all modules and convert their parameters and buffers to CUDA tensors."
    ]
   },
@@ -1207,7 +1211,13 @@
     "Why dont I notice MASSIVE speedup compared to CPU? Because your network is realllly small.\n",
     "\n",
     "**Exercise:** Try increasing the width of your network \n",
-    "(argument 1 and 2 of `nn.Conv2d`, see what kind of speedup you get.\n",
+    "(argument 1 and 2 of `nn.Conv2d`, see what kind of speedup you get."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
     "\n",
     "#### Goals achieved:\n",
     "\n",
@@ -1221,33 +1231,43 @@
    "source": [
     "## Where do I go next?\n",
     "\n",
-    "- [Train neural nets to play video games](https://github.com/pytorch/tutorials/blob/master/Reinforcement%20(Q-)Learning%20with%20PyTorch.ipynb)\n",
+    "- [Train neural nets to play video games](https://goo.gl/uGOksc)\n",
     "- [Train a state-of-the-art ResNet network on imagenet](https://github.com/pytorch/examples/tree/master/imagenet)\n",
     "- [Train an face generator using Generative Adversarial Networks](https://github.com/pytorch/examples/tree/master/dcgan)\n",
     "- [Train a word-level language model using Recurrent LSTM networks](https://github.com/pytorch/examples/tree/master/word_language_model)\n",
     "- [More examples](https://github.com/pytorch/examples)\n",
     "- [More tutorials](https://github.com/pytorch/tutorials)\n",
+    "- [Discuss PyTorch on the Forums](https://discuss.pytorch.org/)\n",
     "- [Chat with other users on Slack](pytorch.slack.com/messages/beginner/)"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 2",
    "language": "python",
-   "name": "python3"
+   "name": "python2"
   },
   "language_info": {
    "codemirror_mode": {
     "name": "ipython",
-    "version": 3
+    "version": 2
    },
    "file_extension": ".py",
    "mimetype": "text/x-python",
    "name": "python",
    "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
+   "pygments_lexer": "ipython2",
+   "version": "2.7.12"
   }
  },
  "nbformat": 4,