week_4/training-the-cbow-model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Word Embeddings: Training the CBOW model\n",
    "\n",
    "In previous lecture notebooks you saw how to prepare data before feeding it to a continuous bag-of-words model, the model itself, its architecture and activation functions. This notebook will walk you through:\n",
    "\n",
    "- Forward propagation.\n",
    "\n",
    "- Cross-entropy loss.\n",
    "\n",
    "- Backpropagation.\n",
    "\n",
    "- Gradient descent.\n",
    "\n",
    "Which are concepts necessary to understand how the training of the model works.\n",
    "\n",
    "Let's dive into it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from utils2 import get_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Forward propagation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's dive into the neural network itself, which is shown below with all the dimensions and formulas you'll need.\n",
    "\n",
    "<div style=\"width:image width px; font-size:100%; text-align:center;\"><img src='cbow_model_dimensions_single_input.png' alt=\"alternate text\" width=\"width\" height=\"height\" style=\"width:839;height:349;\" /> Figure 2 </div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set $N$ equal to 3. Remember that $N$ is a hyperparameter of the CBOW model that represents the size of the word embedding vectors, as well as the size of the hidden layer.\n",
    "\n",
    "Also set $V$ equal to 5, which is the size of the vocabulary we have used so far."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the size of the word embedding vectors and save it in the variable 'N'\n",
    "N = 3\n",
    "\n",
    "# Define V. Remember this was the size of the vocabulary in the previous lecture notebooks\n",
    "V = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialization of the weights and biases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before you start training the neural network, you need to initialize the weight matrices and bias vectors with random values.\n",
    "\n",
    "In the assignment you will implement a function to do this yourself using `numpy.random.rand`. In this notebook, we've pre-populated these matrices and vectors for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define first matrix of weights\n",
    "W1 = np.array([[ 0.41687358,  0.08854191, -0.23495225,  0.28320538,  0.41800106],\n",
    "               [ 0.32735501,  0.22795148, -0.23951958,  0.4117634 , -0.23924344],\n",
    "               [ 0.26637602, -0.23846886, -0.37770863, -0.11399446,  0.34008124]])\n",
    "\n",
    "# Define second matrix of weights\n",
    "W2 = np.array([[-0.22182064, -0.43008631,  0.13310965],\n",
    "               [ 0.08476603,  0.08123194,  0.1772054 ],\n",
    "               [ 0.1871551 , -0.06107263, -0.1790735 ],\n",
    "               [ 0.07055222, -0.02015138,  0.36107434],\n",
    "               [ 0.33480474, -0.39423389, -0.43959196]])\n",
    "\n",
    "# Define first vector of biases\n",
    "b1 = np.array([[ 0.09688219],\n",
    "               [ 0.29239497],\n",
    "               [-0.27364426]])\n",
    "\n",
    "# Define second vector of biases\n",
    "b2 = np.array([[ 0.0352008 ],\n",
    "               [-0.36393384],\n",
    "               [-0.12775555],\n",
    "               [-0.34802326],\n",
    "               [-0.07017815]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Check that the dimensions of these matrices match those shown in the figure above.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "V (vocabulary size): 5\n",
      "N (embedding size / size of the hidden layer): 3\n",
      "size of W1: (3, 5) (NxV)\n",
      "size of b1: (3, 1) (Nx1)\n",
      "size of W2: (5, 3) (VxN)\n",
      "size of b2: (5, 1) (Vx1)\n"
     ]
    }
   ],
   "source": [
    "print(f'V (vocabulary size): {V}')\n",
    "print(f'N (embedding size / size of the hidden layer): {N}')\n",
    "print(f'size of W1: {W1.shape} (NxV)')\n",
    "print(f'size of b1: {b1.shape} (Nx1)')\n",
    "print(f'size of W2: {W2.shape} (VxN)')\n",
    "print(f'size of b2: {b2.shape} (Vx1)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Before moving forward, you will need some functions and variables defined in previous notebooks. They can be found next. Be sure you understand everything that is going on in the next cell, if not consider doing a refresh of the first lecture notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the tokenized version of the corpus\n",
    "words = ['i', 'am', 'happy', 'because', 'i', 'am', 'learning']\n",
    "\n",
    "# Get 'word2Ind' and 'Ind2word' dictionaries for the tokenized corpus\n",
    "word2Ind, Ind2word = get_dict(words)\n",
    "\n",
    "# Define the 'get_windows' function as seen in a previous notebook\n",
    "def get_windows(words, C):\n",
    "    i = C\n",
    "    while i < len(words) - C:\n",
    "        center_word = words[i]\n",
    "        context_words = words[(i - C):i] + words[(i+1):(i+C+1)]\n",
    "        yield context_words, center_word\n",
    "        i += 1\n",
    "\n",
    "# Define the 'word_to_one_hot_vector' function as seen in a previous notebook\n",
    "def word_to_one_hot_vector(word, word2Ind, V):\n",
    "    one_hot_vector = np.zeros(V)\n",
    "    one_hot_vector[word2Ind[word]] = 1\n",
    "    return one_hot_vector\n",
    "\n",
    "# Define the 'context_words_to_vector' function as seen in a previous notebook\n",
    "def context_words_to_vector(context_words, word2Ind, V):\n",
    "    context_words_vectors = [word_to_one_hot_vector(w, word2Ind, V) for w in context_words]\n",
    "    context_words_vectors = np.mean(context_words_vectors, axis=0)\n",
    "    return context_words_vectors\n",
    "\n",
    "# Define the generator function 'get_training_example' as seen in a previous notebook\n",
    "def get_training_example(words, C, word2Ind, V):\n",
    "    for context_words, center_word in get_windows(words, C):\n",
    "        yield context_words_to_vector(context_words, word2Ind, V), word_to_one_hot_vector(center_word, word2Ind, V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the next cells to get the first training example, made of the vector representing the context words \"i am because i\", and the target which is the one-hot vector representing the center word \"happy\".\n",
    "\n",
    "> You don't need to worry about the Python syntax, but there are some explanations below if you want to know what's happening behind the scenes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<generator object get_training_example at 0x000001F8940DD048>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save generator object in the 'training_examples' variable with the desired arguments\n",
    "training_examples = get_training_example(words, 2, word2Ind, V)\n",
    "training_examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> `get_training_examples`, which uses the `yield` keyword, is known as a generator. When run, it builds an iterator, which is a special type of object that... you can iterate on (using a `for` loop for instance), to retrieve the successive values that the function generates.\n",
    ">\n",
    "> In this case `get_training_examples` `yield`s training examples, and iterating on `training_examples` will return the successive training examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get first values from generator\n",
    "x_array, y_array = next(training_examples)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> `next` is another special keyword, which gets the next available value from an iterator. Here, you'll get the very first value, which is the first training example. If you run this cell again, you'll get the next value, and so on until the iterator runs out of values to return.\n",
    ">\n",
    "> In this notebook `next` is used because you will only be performing one iteration of training. In this week's assignment with the full training over several iterations you'll use regular `for` loops with the iterator that supplies the training examples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The vector representing the context words, which will be fed into the neural network, is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.25, 0.25, 0.  , 0.5 , 0.  ])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print context words vector\n",
    "x_array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The one-hot vector representing the center word to be predicted is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 1., 0., 0.])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print one hot vector of center word\n",
    "y_array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now convert these vectors into matrices (or 2D arrays) to be able to perform matrix multiplication on the right types of objects, as explained in a previous notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x:\n",
      "[[0.25]\n",
      " [0.25]\n",
      " [0.  ]\n",
      " [0.5 ]\n",
      " [0.  ]]\n",
      "\n",
      "y:\n",
      "[[0.]\n",
      " [0.]\n",
      " [1.]\n",
      " [0.]\n",
      " [0.]]\n"
     ]
    }
   ],
   "source": [
    "# Copy vector\n",
    "x = x_array.copy()\n",
    "\n",
    "# Reshape it\n",
    "x.shape = (V, 1)\n",
    "\n",
    "# Print it\n",
    "print(f'x:\\n{x}\\n')\n",
    "\n",
    "# Copy vector\n",
    "y = y_array.copy()\n",
    "\n",
    "# Reshape it\n",
    "y.shape = (V, 1)\n",
    "\n",
    "# Print it\n",
    "print(f'y:\\n{y}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you will need the activation functions seen before. Again, if this feel unfamiliar consider checking the previous lecture notebook. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the 'relu' function as seen in the previous lecture notebook\n",
    "def relu(z):\n",
    "    result = z.copy()\n",
    "    result[result < 0] = 0\n",
    "    return result\n",
    "\n",
    "# Define the 'softmax' function as seen in the previous lecture notebook\n",
    "def softmax(z):\n",
    "    e_z = np.exp(z)\n",
    "    sum_e_z = np.sum(e_z)\n",
    "    return e_z / sum_e_z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Values of the hidden layer\n",
    "\n",
    "Now that you have initialized all the variables that you need for forward propagation, you can calculate the values of the hidden layer using the following formulas:\n",
    "\n",
    "\\begin{align}\n",
    " \\mathbf{z_1} = \\mathbf{W_1}\\mathbf{x} + \\mathbf{b_1}  \\tag{1} \\\\\n",
    " \\mathbf{h} = \\mathrm{ReLU}(\\mathbf{z_1})  \\tag{2} \\\\\n",
    "\\end{align}\n",
    "\n",
    "First, you can calculate the value of $\\mathbf{z_1}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute z1 (values of first hidden layer before applying the ReLU function)\n",
    "z1 = W1.dot(x) + b1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> `np.dot` is numpy's function for matrix multiplication.\n",
    "\n",
    "As expected you get an $N$ by 1 matrix, or column vector with $N$ elements, where $N$ is equal to the embedding size, which is 3 in this example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.36483875],\n",
       "       [ 0.63710329],\n",
       "       [-0.3236647 ]])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print z1\n",
    "z1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can now take the ReLU of $\\mathbf{z_1}$ to get $\\mathbf{h}$, the vector with the values of the hidden layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.36483875],\n",
       "       [0.63710329],\n",
       "       [0.        ]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compute h (z1 after applying ReLU function)\n",
    "h = relu(z1)\n",
    "\n",
    "# Print h\n",
    "h"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Applying ReLU means that the negative element of $\\mathbf{z_1}$ has been replaced with a zero."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Values of the output layer\n",
    "\n",
    "Here are the formulas you need to calculate the values of the output layer, represented by the vector $\\mathbf{\\hat y}$:\n",
    "\n",
    "\\begin{align}\n",
    " \\mathbf{z_2} &= \\mathbf{W_2}\\mathbf{h} + \\mathbf{b_2}   \\tag{3} \\\\\n",
    " \\mathbf{\\hat y} &= \\mathrm{softmax}(\\mathbf{z_2})   \\tag{4} \\\\\n",
    "\\end{align}\n",
    "\n",
    "**First, calculate $\\mathbf{z_2}$.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.31973737],\n",
       "       [-0.28125477],\n",
       "       [-0.09838369],\n",
       "       [-0.33512159],\n",
       "       [-0.19919612]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compute z2 (values of the output layer before applying the softmax function)\n",
    "z2 = W2.dot(h) + b2\n",
    "\n",
    "# Print z2\n",
    "z2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[-0.31973737],\n",
    "           [-0.28125477],\n",
    "           [-0.09838369],\n",
    "           [-0.33512159],\n",
    "           [-0.19919612]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a $V$ by 1 matrix, where $V$ is the size of the vocabulary, which is 5 in this example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now calculate the value of $\\mathbf{\\hat y}$.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.18519074],\n",
       "       [0.19245626],\n",
       "       [0.23107446],\n",
       "       [0.18236353],\n",
       "       [0.20891502]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compute y_hat (z2 after applying softmax function)\n",
    "y_hat = softmax(z2)\n",
    "\n",
    "# Print y_hat\n",
    "y_hat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[0.18519074],\n",
    "           [0.19245626],\n",
    "           [0.23107446],\n",
    "           [0.18236353],\n",
    "           [0.20891502]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you've performed the calculations with random matrices and vectors (apart from the input vector), the output of the neural network is essentially random at this point. The learning process will adjust the weights and biases to match the actual targets better.\n",
    "\n",
    "**That being said, what word did the neural network predict?**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>    \n",
    "<summary>\n",
    "    <font size=\"3\" color=\"darkgreen\"><b>Solution</b></font>\n",
    "</summary>\n",
    "<p>The neural network predicted the word \"happy\": the largest element of $\\mathbf{\\hat y}$ is the third one, and the third word of the vocabulary is \"happy\".</p>\n",
    "<p>Here's how you could implement this in Python:</p>\n",
    "<p><code>print(Ind2word[np.argmax(y_hat)])</code></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Well done, you've completed the forward propagation phase!\n",
    "\n",
    "## Cross-entropy loss\n",
    "\n",
    "Now that you have the network's prediction, you can calculate the cross-entropy loss to determine how accurate the prediction was compared to the actual target.\n",
    "\n",
    "> Remember that you are working on a single training example, not on a batch of examples, which is why you are using *loss* and not *cost*, which is the generalized form of loss.\n",
    "\n",
    "First let's recall what the prediction was."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.18519074],\n",
       "       [0.19245626],\n",
       "       [0.23107446],\n",
       "       [0.18236353],\n",
       "       [0.20891502]])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print prediction\n",
    "y_hat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the actual target value is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.],\n",
       "       [0.],\n",
       "       [1.],\n",
       "       [0.],\n",
       "       [0.]])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print target value\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The formula for cross-entropy loss is:\n",
    "\n",
    "$$ J=-\\sum\\limits_{k=1}^{V}y_k\\log{\\hat{y}_k} \\tag{6}$$\n",
    "\n",
    "**Try implementing the cross-entropy loss function so you get more familiar working with numpy**\n",
    "\n",
    "Here are a some hints if you're stuck."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cross_entropy_loss(y_predicted, y_actual):\n",
    "    # Fill the loss variable with your code\n",
    "    loss = - (y * np.log(y_hat)).sum()\n",
    "    return loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>    \n",
    "<summary>\n",
    "    <font size=\"3\" color=\"darkgreen\"><b>Hint 1</b></font>\n",
    "</summary>\n",
    "    <p>To multiply two numpy matrices (such as <code>y</code> and <code>y_hat</code>) element-wise, you can simply use the <code>*</code> operator.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>    \n",
    "<summary>\n",
    "    <font size=\"3\" color=\"darkgreen\"><b>Hint 2</b></font>\n",
    "</summary>\n",
    "<p>Once you have a vector equal to the element-wise multiplication of <code>y</code> and <code>y_hat</code>, you can use <code>np.sum</code> to calculate the sum of the elements of this vector.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>    \n",
    "<summary>\n",
    "    <font size=\"3\" color=\"darkgreen\"><b>Solution</b></font>\n",
    "</summary>\n",
    "<p><code>loss = np.sum(-np.log(y_hat)*y)</code></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Don't forget to run the cell containing the `cross_entropy_loss` function once it is solved."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now use this function to calculate the loss with the actual values of $\\mathbf{y}$ and $\\mathbf{\\hat y}$.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.4650152923611106"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print value of cross entropy loss for prediction and target value\n",
    "cross_entropy_loss(y_hat, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    1.4650152923611106"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This value is neither good nor bad, which is expected as the neural network hasn't learned anything yet.\n",
    "\n",
    "The actual learning will start during the next phase: backpropagation.\n",
    "\n",
    "## Backpropagation\n",
    "\n",
    "The formulas that you will implement for backpropagation are the following.\n",
    "\n",
    "\\begin{align}\n",
    " \\frac{\\partial J}{\\partial \\mathbf{W_1}} &= \\rm{ReLU}\\left ( \\mathbf{W_2^\\top} (\\mathbf{\\hat{y}} - \\mathbf{y})\\right )\\mathbf{x}^\\top \\tag{7}\\\\\n",
    " \\frac{\\partial J}{\\partial \\mathbf{W_2}} &= (\\mathbf{\\hat{y}} - \\mathbf{y})\\mathbf{h^\\top} \\tag{8}\\\\\n",
    " \\frac{\\partial J}{\\partial \\mathbf{b_1}} &= \\rm{ReLU}\\left ( \\mathbf{W_2^\\top} (\\mathbf{\\hat{y}} - \\mathbf{y})\\right ) \\tag{9}\\\\\n",
    " \\frac{\\partial J}{\\partial \\mathbf{b_2}} &= \\mathbf{\\hat{y}} - \\mathbf{y} \\tag{10}\n",
    "\\end{align}\n",
    "\n",
    "> Note: these formulas are slightly simplified compared to the ones in the lecture as you're working on a single training example, whereas the lecture provided the formulas for a batch of examples. In the assignment you'll be implementing the latter.\n",
    "\n",
    "Let's start with an easy one.\n",
    "\n",
    "**Calculate the partial derivative of the loss function with respect to $\\mathbf{b_2}$, and store the result in `grad_b2`.**\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial \\mathbf{b_2}} = \\mathbf{\\hat{y}} - \\mathbf{y} \\tag{10}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.18519074],\n",
       "       [ 0.19245626],\n",
       "       [-0.76892554],\n",
       "       [ 0.18236353],\n",
       "       [ 0.20891502]])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compute vector with partial derivatives of loss function with respect to b2\n",
    "grad_b2 = y_hat - y\n",
    "\n",
    "# Print this vector\n",
    "grad_b2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[ 0.18519074],\n",
    "           [ 0.19245626],\n",
    "           [-0.76892554],\n",
    "           [ 0.18236353],\n",
    "           [ 0.20891502]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Next, calculate the partial derivative of the loss function with respect to $\\mathbf{W_2}$, and store the result in `grad_W2`.**\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial \\mathbf{W_2}} = (\\mathbf{\\hat{y}} - \\mathbf{y})\\mathbf{h^\\top} \\tag{8}$$\n",
    "\n",
    "> Hint: use `.T` to get a transposed matrix, e.g. `h.T` returns $\\mathbf{h^\\top}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.06756476,  0.11798563,  0.        ],\n",
       "       [ 0.0702155 ,  0.12261452,  0.        ],\n",
       "       [-0.28053384, -0.48988499,  0.        ],\n",
       "       [ 0.06653328,  0.1161844 ,  0.        ],\n",
       "       [ 0.07622029,  0.13310045,  0.        ]])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compute matrix with partial derivatives of loss function with respect to W2\n",
    "grad_W2 = np.dot(y_hat - y, h.T)\n",
    "\n",
    "# Print matrix\n",
    "grad_W2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[ 0.06756476,  0.11798563,  0.        ],\n",
    "           [ 0.0702155 ,  0.12261452,  0.        ],\n",
    "           [-0.28053384, -0.48988499,  0.        ],\n",
    "           [ 0.06653328,  0.1161844 ,  0.        ],\n",
    "           [ 0.07622029,  0.13310045,  0.        ]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now calculate the partial derivative with respect to $\\mathbf{b_1}$ and store the result in `grad_b1`.**\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial \\mathbf{b_1}} = \\rm{ReLU}\\left ( \\mathbf{W_2^\\top} (\\mathbf{\\hat{y}} - \\mathbf{y})\\right ) \\tag{9}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.        ],\n",
       "       [0.        ],\n",
       "       [0.17045858]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compute vector with partial derivatives of loss function with respect to b1\n",
    "grad_b1 = relu(W2.T.dot(y_hat - y))\n",
    "\n",
    "# Print vector\n",
    "grad_b1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[0.        ],\n",
    "           [0.        ],\n",
    "           [0.17045858]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Finally, calculate the partial derivative of the loss with respect to $\\mathbf{W_1}$, and store it in `grad_W1`.**\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial \\mathbf{W_1}} = \\rm{ReLU}\\left ( \\mathbf{W_2^\\top} (\\mathbf{\\hat{y}} - \\mathbf{y})\\right )\\mathbf{x}^\\top \\tag{7}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.        , 0.        , 0.        , 0.        , 0.        ],\n",
       "       [0.        , 0.        , 0.        , 0.        , 0.        ],\n",
       "       [0.04261464, 0.04261464, 0.        , 0.08522929, 0.        ]])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compute matrix with partial derivatives of loss function with respect to W1\n",
    "grad_W1 = relu(W2.T.dot(y_hat - y)).dot(x.T)\n",
    "\n",
    "# Print matrix\n",
    "grad_W1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[0.        , 0.        , 0.        , 0.        , 0.        ],\n",
    "           [0.        , 0.        , 0.        , 0.        , 0.        ],\n",
    "           [0.04261464, 0.04261464, 0.        , 0.08522929, 0.        ]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before moving on to gradient descent, double-check that all the matrices have the expected dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "V (vocabulary size): 5\n",
      "N (embedding size / size of the hidden layer): 3\n",
      "size of grad_W1: (3, 5) (NxV)\n",
      "size of grad_b1: (3, 1) (Nx1)\n",
      "size of grad_W2: (5, 3) (VxN)\n",
      "size of grad_b2: (5, 1) (Vx1)\n"
     ]
    }
   ],
   "source": [
    "print(f'V (vocabulary size): {V}')\n",
    "print(f'N (embedding size / size of the hidden layer): {N}')\n",
    "print(f'size of grad_W1: {grad_W1.shape} (NxV)')\n",
    "print(f'size of grad_b1: {grad_b1.shape} (Nx1)')\n",
    "print(f'size of grad_W2: {grad_W2.shape} (VxN)')\n",
    "print(f'size of grad_b2: {grad_b2.shape} (Vx1)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient descent\n",
    "\n",
    "During the gradient descent phase, you will update the weights and biases by subtracting $\\alpha$ times the gradient from the original matrices and vectors, using the following formulas.\n",
    "\n",
    "\\begin{align}\n",
    " \\mathbf{W_1} &:= \\mathbf{W_1} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{W_1}} \\tag{11}\\\\\n",
    " \\mathbf{W_2} &:= \\mathbf{W_2} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{W_2}} \\tag{12}\\\\\n",
    " \\mathbf{b_1} &:= \\mathbf{b_1} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{b_1}} \\tag{13}\\\\\n",
    " \\mathbf{b_2} &:= \\mathbf{b_2} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{b_2}} \\tag{14}\\\\\n",
    "\\end{align}\n",
    "\n",
    "First, let set a value for $\\alpha$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define alpha\n",
    "alpha = 0.03"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The updated weight matrix $\\mathbf{W_1}$ will be:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute updated W1\n",
    "W1_new = W1 - alpha * grad_W1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's compare the previous and new values of $\\mathbf{W_1}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "old value of W1:\n",
      "[[ 0.41687358  0.08854191 -0.23495225  0.28320538  0.41800106]\n",
      " [ 0.32735501  0.22795148 -0.23951958  0.4117634  -0.23924344]\n",
      " [ 0.26637602 -0.23846886 -0.37770863 -0.11399446  0.34008124]]\n",
      "\n",
      "new value of W1:\n",
      "[[ 0.41687358  0.08854191 -0.23495225  0.28320538  0.41800106]\n",
      " [ 0.32735501  0.22795148 -0.23951958  0.4117634  -0.23924344]\n",
      " [ 0.26509758 -0.2397473  -0.37770863 -0.11655134  0.34008124]]\n"
     ]
    }
   ],
   "source": [
    "print('old value of W1:')\n",
    "print(W1)\n",
    "print()\n",
    "print('new value of W1:')\n",
    "print(W1_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The difference is very subtle (hint: take a closer look at the last row), which is why it takes a fair amount of iterations to train the neural network until it reaches optimal weights and biases starting from random values.\n",
    "\n",
    "**Now calculate the new values of $\\mathbf{W_2}$ (to be stored in `W2_new`), $\\mathbf{b_1}$ (in `b1_new`), and $\\mathbf{b_2}$ (in `b2_new`).**\n",
    "\n",
    "\\begin{align}\n",
    " \\mathbf{W_2} &:= \\mathbf{W_2} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{W_2}} \\tag{12}\\\\\n",
    " \\mathbf{b_1} &:= \\mathbf{b_1} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{b_1}} \\tag{13}\\\\\n",
    " \\mathbf{b_2} &:= \\mathbf{b_2} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{b_2}} \\tag{14}\\\\\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "W2_new\n",
      "[[-0.22384758 -0.43362588  0.13310965]\n",
      " [ 0.08265956  0.0775535   0.1772054 ]\n",
      " [ 0.19557112 -0.04637608 -0.1790735 ]\n",
      " [ 0.06855622 -0.02363691  0.36107434]\n",
      " [ 0.33251813 -0.3982269  -0.43959196]]\n",
      "\n",
      "b1_new\n",
      "[[ 0.09688219]\n",
      " [ 0.29239497]\n",
      " [-0.27875802]]\n",
      "\n",
      "b2_new\n",
      "[[ 0.02964508]\n",
      " [-0.36970753]\n",
      " [-0.10468778]\n",
      " [-0.35349417]\n",
      " [-0.0764456 ]]\n"
     ]
    }
   ],
   "source": [
    "# Compute updated W2\n",
    "W2_new = W2 - alpha * grad_W2\n",
    "\n",
    "# Compute updated b1\n",
    "b1_new = b1 - alpha * grad_b1\n",
    "\n",
    "# Compute updated b2\n",
    "b2_new = b2 - alpha * grad_b2\n",
    "\n",
    "\n",
    "print('W2_new')\n",
    "print(W2_new)\n",
    "print()\n",
    "print('b1_new')\n",
    "print(b1_new)\n",
    "print()\n",
    "print('b2_new')\n",
    "print(b2_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    W2_new\n",
    "    [[-0.22384758 -0.43362588  0.13310965]\n",
    "     [ 0.08265956  0.0775535   0.1772054 ]\n",
    "     [ 0.19557112 -0.04637608 -0.1790735 ]\n",
    "     [ 0.06855622 -0.02363691  0.36107434]\n",
    "     [ 0.33251813 -0.3982269  -0.43959196]]\n",
    "\n",
    "    b1_new\n",
    "    [[ 0.09688219]\n",
    "     [ 0.29239497]\n",
    "     [-0.27875802]]\n",
    "\n",
    "    b2_new\n",
    "    [[ 0.02964508]\n",
    "     [-0.36970753]\n",
    "     [-0.10468778]\n",
    "     [-0.35349417]\n",
    "     [-0.0764456 ]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congratulations, you have completed one iteration of training using one training example!\n",
    "\n",
    "You'll need many more iterations to fully train the neural network, and you can optimize the learning process by training on batches of examples, as described in the lecture. You will get to do this during this week's assignment.\n",
    "\n",
    "### How this practice relates to and differs from the upcoming graded assignment\n",
    "\n",
    "- In the assignment, for each iteration of training you will use batches of examples instead of a single example. The formulas for forward propagation and backpropagation will be modified accordingly, and you will use cross-entropy cost instead of cross-entropy loss.\n",
    "- You will also complete several iterations of training, until you reach an acceptably low cross-entropy cost, at which point you can extract good word embeddings from the weight matrices."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}