W04_01_MLP_for_Images.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/dkaratzas/XNAP_Profs/blob/main/W04_01_MLP_for_Images.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MLP for image classification\n",
    "\n",
    "In this notebook we are going to learn how to use a Multi-Layer Perceptron (MLP) for clasifing images. \n",
    "\n",
    "A MLP like the ones you used in the notebook of last week can be used with any kind of input data if we can represent it with a vector of real numbers. The shape of the input vector determines the size of the first layer in the model. In the case of images (2d arrays of pixel values) we can get fixed-lenght vectors by: (1) using images of the same size in all our dataset, and (2) flatenning the images into a 1d array. The flatten operation collapses an array into one dimension. For example, if we have a grayscale image of 28*28 pixels, its flattened version is a 1d array of 784 pixel values. Now we can fed these 784 values into a MLP for classifiying the image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import the necessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "YIr2ty0tFA4C"
   },
   "outputs": [],
   "source": [
    "import torch #should be installed by default in any colab notebook\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython import display\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "from torchvision import datasets, transforms\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use GPU if available"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gWLIxo9Oigfo"
   },
   "outputs": [],
   "source": [
    "# If this cell fails you need to change the runtime of your colab notebook to GPU\n",
    "# Go to Runtime -> Change Runtime Type and select GPU\n",
    "assert torch.cuda.is_available(), \"GPU is not enabled\"\n",
    "\n",
    "# use gpu if available\n",
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eI_YXyigdTUC"
   },
   "source": [
    "## The Fashion-MNIST dataset\n",
    "\n",
    "[**Fashion-MNIST**](https://pytorch.org/vision/0.8/datasets.html#fashion-mnist) is a dataset consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. It was poroposed as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms. It shares the same image size and structure of training and testing splits.\n",
    "\n",
    "Each training and test example is assigned to one of the following labels: 0 T-shirt/top, 1 Trouser, 2 Pullover, 3 Dress, 4 Coat, 5 Sandal, 6 Shirt, 7 Sneaker, 8 Bag, 9 Ankle boot.\n",
    "\n",
    "The Fashion-MNIST dataset is available in [torchvision](https://pytorch.org/vision/stable/index.html) and can be loaded with the following code:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "m7kr-LrSZK7h"
   },
   "outputs": [],
   "source": [
    "train_set = datasets.FashionMNIST(\"data\", train=True, download=True, transform=transforms.Compose([transforms.ToTensor()]))\n",
    "val_set = datasets.FashionMNIST(\"data\", train=False, download=True, transform=transforms.Compose([transforms.ToTensor()]))\n",
    "\n",
    "print(train_set.data.size(), val_set.data.size())\n",
    "print(train_set.targets.size(), val_set.targets.size())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# show some images\n",
    "plt.figure(figsize=(16, 6))\n",
    "for i in range(10):\n",
    "    plt.subplot(2, 5, i + 1)\n",
    "    image = train_set.data[i,...]\n",
    "    plt.imshow(image.squeeze().numpy(), cmap=\"gray\")\n",
    "    plt.axis('off');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The dataloaders\n",
    "\n",
    "Now we introduce a **critical piece in any deep learning training process**: the dataloader. In Pytorch we can create a dataloader for a given dataset as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_loader = torch.utils.data.DataLoader(train_set, batch_size=1, shuffle=False, num_workers=0)\n",
    "val_loader = torch.utils.data.DataLoader(val_set, batch_size=1, shuffle=False, num_workers=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8KiB18s1a1e0"
   },
   "source": [
    "# Exercise 1\n",
    "\n",
    "1.1. Take a look at the documentation of [torch.utils.data.DataLoader](https://pytorch.org/docs/stable/data.html?highlight=dataloader#torch.utils.data.DataLoader) and answer the following questions:\n",
    "\n",
    "- What are the benefits of a dataloader?\n",
    "- How can we make the dataloaders defined above better? \n",
    "\n",
    "1.2. When we loaded the Fashion-MNIST dataset we used the method `transforms.Compose`. Take a look at the documentation of [torchvision.transforms](https://pytorch.org/vision/0.8/transforms.html?highlight=transforms) and answer the following question:\n",
    "\n",
    "- Is there another transform that we can add to make our classification problem easier?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e-WSDzgqcsY4"
   },
   "source": [
    "# Excersice 2\n",
    "\n",
    "Modify your best performing architecture from last week notebook and create a class model that can work for any input and output size (using the `__init__()` class constructor parameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VyEjLgfjrWzs"
   },
   "source": [
    "### Fully connected model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "uvfwZqcPoAdT"
   },
   "outputs": [],
   "source": [
    "\n",
    "class FCModel(nn.Module):\n",
    "    def __init__(self, input_size, n_hidden, output_size):\n",
    "        super(FCModel, self).__init__()\n",
    "        self.input_size = input_size\n",
    "        self.network = nn.Sequential(\n",
    "            # Your code here \n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = x.view(-1, self.input_size)\n",
    "        return self.network(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parameter Initialization\n",
    "\n",
    "In PyTorch the default parameter initialization depends on the layer type. For example, for the Linear layer the default initialization is defined [here](https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/linear.py#L87). Take a look and see if you recognize the initialization method.\n",
    "\n",
    "You can find more initialization methods in the [`torch.nn.init`](https://pytorch.org/docs/stable/nn.init.html?highlight=init) module.\n",
    "\n",
    "If necessary, you can change the default initialization of a layer as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "HLSfT6eZcXTI"
   },
   "outputs": [],
   "source": [
    "def initialize_parameters(model):\n",
    "    for name, w in model.named_parameters():\n",
    "        if \"weight\" in name:\n",
    "            nn.init.ones_(w)\n",
    "        \n",
    "        if \"bias\" in name:\n",
    "            nn.init.zeros_(w)   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the model and initialize its parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "oAoS_kiRZPYT"
   },
   "outputs": [],
   "source": [
    "learning_rate = 1e-3\n",
    "lambda_l2 = 1e-5\n",
    "torch.manual_seed(0) # seed for reproductibility\n",
    "\n",
    "input_size  = 28*28   # images are 28x28 pixels\n",
    "output_size = 10      # there are 10 classes\n",
    "\n",
    "model = FCModel(input_size, 128, output_size)\n",
    "\n",
    "# utility function to count number of parameters in a model\n",
    "def get_n_params(model):\n",
    "    np=0\n",
    "    for p in list(model.parameters()):\n",
    "        np += p.numel()\n",
    "    return np\n",
    "\n",
    "print(f\"Number of parameters {get_n_params(model)}:\")\n",
    "\n",
    "# move model to gpu if available\n",
    "model.to(device)\n",
    "\n",
    "# nn package also has different loss functions.\n",
    "# we use cross entropy loss for our classification task\n",
    "criterion = torch.nn.CrossEntropyLoss() # https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss\n",
    "\n",
    "# we use the optim package to apply\n",
    "# stochastic gradient descent for our parameter updates\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, momentum=0.9, weight_decay=lambda_l2) # built-in L2\n",
    "\n",
    "# WARNING! What are we doing here?\n",
    "initialize_parameters(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the train and validation methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "voDADNIzj4wV"
   },
   "outputs": [],
   "source": [
    "@torch.no_grad()  # prevent this function from computing gradients \n",
    "def validate(criterion, model, loader):\n",
    "\n",
    "    val_loss = 0\n",
    "    correct = 0\n",
    "\n",
    "    model.eval()\n",
    "\n",
    "    for data, target in loader:\n",
    "\n",
    "        data, target = data.to(device), target.to(device)\n",
    "\n",
    "        data = data.view(-1, 28*28)\n",
    "        output = model(data)\n",
    "        loss = criterion(output, target)\n",
    "        val_loss += loss.item()                                                              \n",
    "        pred = output.data.max(1, keepdim=True)[1] # get the index of the max log-probability                                                                 \n",
    "        correct += pred.eq(target.view_as(pred)).sum().item()\n",
    "\n",
    "    val_loss /= len(loader.dataset)\n",
    "    accuracy = 100. * correct / len(loader.dataset)\n",
    "    print('\\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
    "        val_loss, correct, len(loader.dataset),\n",
    "        accuracy))\n",
    "\n",
    "\n",
    "    return val_loss\n",
    "\n",
    "\n",
    "def train(epoch, criterion, model, optimizer, loader):\n",
    "    \n",
    "    total_loss = 0.0\n",
    "\n",
    "    model.train()\n",
    "\n",
    "    for batch_idx, (data, target) in enumerate(loader):\n",
    "        \n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        data, target = data.to(device), target.to(device)\n",
    "\n",
    "        data = data.view(-1, 28*28)\n",
    "\n",
    "        output = model(data)\n",
    "        loss = criterion(output, target)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        \n",
    "        # print loss every N iterations\n",
    "        if batch_idx % 100 == 0:\n",
    "            print('Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n",
    "                epoch, batch_idx * len(data), len(loader.dataset),\n",
    "                100. * batch_idx / len(loader), loss.item()))\n",
    "\n",
    "\n",
    "        total_loss += loss.item()  #.item() is very important here? Why?\n",
    "\n",
    "    return total_loss / len(loader.dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_Prc0qFO0SR5"
   },
   "source": [
    "### The training loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "psBKsIFAtkXQ"
   },
   "outputs": [],
   "source": [
    "losses = {\"train\": [], \"val\": []}\n",
    "for epoch in range(10):\n",
    "\n",
    "    train_loss = train(epoch, criterion, model, optimizer, train_loader)\n",
    "    val_loss = validate(criterion, model, val_loader)\n",
    "    losses[\"train\"].append(train_loss)\n",
    "    losses[\"val\"].append(val_loss)\n",
    "    \n",
    "    display.clear_output(wait=True)\n",
    "\n",
    "    plt.plot(losses[\"train\"], label=\"training loss\")\n",
    "    plt.plot(losses[\"val\"], label=\"validation loss\")\n",
    "\n",
    "    plt.legend()\n",
    "    plt.pause(0.000001)\n",
    "    plt.show()   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EcgjBWogo1N1"
   },
   "source": [
    "# Homework\n",
    "\n",
    "A) Change the initialization of the model parameters (this will help a great deal) and train your model on the Fashion-MNIST dataset.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "B) Try to improve the Accuracy of your model on the validation set by adding more layers and/or more hidden units in you model. For example you can use a MLP with 2 hidden layers with 512 and 256 units respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "C) Try at least two different [optimizers](https://pytorch.org/docs/stable/optim.html#algorithms) (e.g. SGD with momentum, RMSProp, Adam, etc.) and plot **in a single matplotlib figure** the loss curves for training the model with them. We want them in a single figure to be able to easily compare the three learning curves."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "D) Calculate the Accuracy for each individual class in the dataset and plot the [confusion matrix](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.plot_confusion_matrix.html#sklearn.metrics.plot_confusion_matrix) of your trained models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "authorship_tag": "ABX9TyMtjWCsaZkieM/xNNLaY9ws",
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "MLP for Images.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}