TF_Fundamentals_45_Mints.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensors\n",
    "\n",
    "A **tensor** is a container for *numeric* data. Tensors can contain data within an arbitrary number of dimensions. That is, it can be a zero-dimensional (0D), one-dimensional (1D), two-dimensional (2D), three-dimensional (3D), and so on. Within the context of tensors, a dimension is often called an **axis**.\n",
    "\n",
    "So, tensors are a generalization of matrices represented by n-dimensional arrays. The dimensionality of a tensor is often described by its number of axes. The number of axes represented by a tensor is called its **rank**. Tensors are defined by how many axes they have in total.\n",
    "\n",
    "![Imgur](https://imgur.com/342G6Hk.png) \n",
    "\n",
    "#### [source](https://www.guru99.com/tensor-tensorflow.html)\n",
    "\n",
    "\n",
    "![Imgur](https://imgur.com/duJyL5S.png)\n",
    "\n",
    "[source](https://www.tensorflow.org/guide/tensor)\n",
    "\n",
    "## Scalars (0D tensors)\n",
    "\n",
    "A **scalar** is a tensor of only one number. For example, a Numpy float32 or float64 number is a scalar tensor (or scalar array).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.6.0\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(5, shape=(), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "tensor = tf.constant(5)\n",
    "print(tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor([1 2 3], shape=(3,), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "tensor = tf.constant([1, 2, 3])\n",
    "print(tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n"
     ]
    }
   ],
   "source": [
    "tensor = tf.constant([[1, 2, 3], [2,3,9]])\n",
    "# print(tensor)\n",
    "print(tf.rank(tensor).numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "It is easy to dispay the number of axes (or dimensionality) of a Numpy tensor with the *ndim* attribute. Let's look at an example of a Numpy scalar:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n"
     ]
    }
   ],
   "source": [
    "print(tensor.ndim)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 2, 3), dtype=int32, numpy=\n",
       "array([[[ 1,  2,  3],\n",
       "        [ 4,  5,  6]],\n",
       "\n",
       "       [[ 7,  8,  9],\n",
       "        [10, 11, 12]],\n",
       "\n",
       "       [[13, 14, 15],\n",
       "        [16, 17, 18]]], dtype=int32)>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor = tf.constant([[[1, 2, 3],\n",
    "                       [4, 5, 6]],\n",
    "                      \n",
    "                      [[7, 8, 9],\n",
    "                       [10, 11, 12]],\n",
    "                      \n",
    "                      [[13, 14, 15],\n",
    "                       [16, 17, 18]]])\n",
    "tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 2), dtype=float16, numpy=\n",
       "array([[10.,  7.],\n",
       "       [14., 15.]], dtype=float16)>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor_float16 = tf.constant([[10, 7], [14, 15]], dtype=tf.float16)\n",
    "tensor_float16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([4, 5], dtype=int32)>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mutable_tensor = tf.Variable([4, 5])\n",
    "\n",
    "immuable_tensor = tf.constant([5, 6])\n",
    "\n",
    "mutable_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([12,  5], dtype=int32)>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# mutable_tensor[0] = 12\n",
    "mutable_tensor[0].assign(12)\n",
    "mutable_tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating Random Tensors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=float32, numpy=\n",
       "array([[-0.7565803 , -0.06854702,  0.07595026],\n",
       "       [-1.2573844 , -0.23193763, -1.8107855 ]], dtype=float32)>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random_gen = tf.random.Generator.from_seed(42)\n",
    "\n",
    "random_normal = random_gen.normal(shape=(2, 3))\n",
    "random_normal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=float32, numpy=\n",
       "array([[0.7647915 , 0.03845465, 0.8506975 ],\n",
       "       [0.20781887, 0.711869  , 0.8843919 ]], dtype=float32)>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random_uniform = random_gen.uniform(shape=(2, 3))\n",
    "random_uniform"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[False False False False False]\n"
     ]
    }
   ],
   "source": [
    "a = tf.random.shuffle([1,2,3,4,5], seed=2345).numpy()\n",
    "b = tf.random.shuffle([1,2,3,4,5], seed=2345).numpy()\n",
    "print(a == b)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Special Notes on tf.random.set_seed()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ True  True  True  True  True]\n"
     ]
    }
   ],
   "source": [
    "tf.random.set_seed(2345)\n",
    "a = tf.random.shuffle([1,2,3,4,5], seed=2345).numpy()\n",
    "tf.random.set_seed(2345)\n",
    "b = tf.random.shuffle([1,2,3,4,5], seed=2345).numpy()\n",
    "print(a == b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Official Doc](https://www.tensorflow.org/api_docs/python/tf/random/set_seed)\n",
    "\n",
    "\n",
    "Operations that rely on a random seed actually derive it from two seeds: the global and operation-level seeds. This sets the global seed.\n",
    "\n",
    "Its interactions with operation-level seeds is as follows:\n",
    "\n",
    "* If neither the global seed nor the operation seed is set: A randomly picked seed is used for this op.\n",
    "* If the global seed is set, but the operation seed is not: The system deterministically picks an operation seed in conjunction with the global seed so that it gets a unique random sequence. Within the same version of tensorflow and user code, this sequence is deterministic. However across different versions, this sequence might change. If the code depends on particular seeds to work, specify both global and operation-level seeds explicitly.\n",
    "* If the operation seed is set, but the global seed is not set: A default global seed and the specified operation seed are used to determine the random sequence.\n",
    "* If both the global and the operation seed are set: Both seeds are used in conjunction to determine the random sequence.\n",
    "\n",
    "To illustrate the user-visible effects, consider these examples:\n",
    "\n",
    "If neither the global seed nor the operation seed is set, we get different results for every call to the random op and every re-run of the program:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The tf.data API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(5,), dtype=int32, numpy=array([0, 1, 2, 3, 4], dtype=int32)>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = tf.range(5)\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2, 3, 4], dtype=int32)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<TensorSliceDataset shapes: (), types: tf.int32>\n"
     ]
    }
   ],
   "source": [
    "dataset = tf.data.Dataset.from_tensor_slices(X)\n",
    "print(dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(0, shape=(), dtype=int32)\n",
      "tf.Tensor(1, shape=(), dtype=int32)\n",
      "tf.Tensor(2, shape=(), dtype=int32)\n",
      "tf.Tensor(3, shape=(), dtype=int32)\n",
      "tf.Tensor(4, shape=(), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "for item in dataset:\n",
    "  print(item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Depending on the data position and format, the `tf.data.Dataset` class offers quite a few\n",
    "static methods to use to create a dataset easily:\n",
    "\n",
    "* Tensors in memory: `tf.data.Dataset.from_tensors` or `tf.data.Dataset.from_tensor_slices`. In this case, the tensors can be NumPy arrays or tf.Tensor objects.\n",
    "\n",
    "* From a Python generator: tf.data.Dataset.from_generator.\n",
    "\n",
    "* From a list of files that matches a pattern: tf.data.Dataset.list_files.\n",
    "\n",
    "Also, there are two specializations of the `tf.data.Dataset` object created for working \n",
    "with two commonly used file formats:\n",
    "\n",
    "* tf.data.TFRecordDataset - to work with the TFRecord files\n",
    "\n",
    "* tf.data.TextLineDataset - to work with text files, reading them line by line\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<FlatMapDataset shapes: <unknown>, types: tf.float32>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def noise():\n",
    "  while True:\n",
    "    yield tf.random.uniform((100, 0))\n",
    "    \n",
    "dataset = tf.data.Dataset.from_generator(noise, (tf.float32))\n",
    "dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The only peculiarity of the from_generator method is the need to pass the type of\n",
    "the parameters (tf.float32, in this case) as the second parameter; this is required\n",
    "since to build a graph we need to know the type of the parameters in advance.\n",
    "\n",
    "Using method chaining, it is possible to create new dataset objects, transforming\n",
    "the one just built to get the data our machine learning model expects as input.\n",
    "\n",
    "For example, if we want to \n",
    "\n",
    "- sum 10 to every component of the noise vector,\n",
    "- shuffle the dataset content, and \n",
    "- create batches of 32 vectors each, we can do so\n",
    "\n",
    "by calling just three methods chained together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "(32, 100, 0)\n",
      "1\n",
      "(32, 100, 0)\n"
     ]
    }
   ],
   "source": [
    "buffer_size = 10\n",
    "batch_size = 32\n",
    "\n",
    "dataset = dataset.map(lambda x: x + 10).shuffle(buffer_size).batch(batch_size)\n",
    "\n",
    "for idx, noise in enumerate(dataset):\n",
    "  if idx == 2:\n",
    "    break\n",
    "  print(idx)\n",
    "  print(noise.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `map` method is the most widely used method of the tf.data.Dataset object since\n",
    "it allows us to apply a function to every element of the input dataset, producing a\n",
    "new, transformed dataset.\n",
    "\n",
    "The `shuffle` method is used in every training pipeline since this transformation\n",
    "randomly shuffles the input dataset using a fixed-sized buffer; this means that\n",
    "the shuffled data first fetches the buffer_size element from its input, then shuffles\n",
    "them and produces the output.\n",
    "\n",
    "The bat`c`h method gathers the batch_size elements from its input and creates a\n",
    "batch as output. The only constraint of this transformation is that all elements of\n",
    "the batch must have the same shape."
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac"
  },
  "kernelspec": {
   "display_name": "Python 3.9.6 64-bit",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}