CA3.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# execute this cell before you start\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras import layers\n",
    "\n",
    "# Helper libraries\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from tensorflow import keras\n",
    "\n",
    "print(tf.VERSION)\n",
    "print(tf.keras.__version__)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  CA3\n",
    "## due on 22/03/2019\n",
    "\n",
    "to submit the assignment, please do the following:\n",
    "\n",
    "- do `Cell -> All output -> Clear` to clear all your output\n",
    "- save the notebook (CA3.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Cifar 10 dataset\n",
    "\n",
    "Consider the data in  `keras.datasets.cifar10` and train a network which reliably categorizes the data. You can get some inspiartion from the following worked out example:\n",
    "\n",
    "https://keras.io/examples/cifar10_cnn/\n",
    "\n",
    "Try to understand tradeoffs:\n",
    "\n",
    "- What increases computing time?\n",
    "- What increases accuracy?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Load the data and create the class labels :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-62578aed3a4137c9",
     "locked": false,
     "points": 10,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = keras.datasets.cifar10.load_data()\n",
    "\n",
    "# Class labels from https://www.cs.toronto.edu/~kriz/cifar.html\n",
    "class_labels = [    \n",
    "    'airplane', \n",
    "    'automobile',\n",
    "    'bird',\n",
    "    'cat',\n",
    "    'deer',\n",
    "    'dog',\n",
    "    'frog',\n",
    "    'horse',\n",
    "    'ship',\n",
    "    'truck'\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Get an overview of the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train.shape[1:]\n",
    "\n",
    "plt.figure(figsize=(10,10))\n",
    "for i in range(0,25):\n",
    "    plt.subplot(5,5, i+1)\n",
    "    plt.imshow(x_train[i])\n",
    "    y_train[i]    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Normalize the images :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train/255.0\n",
    "x_test = x_test/255.0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Build and fit the model :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "model = keras.models.Sequential()\n",
    "\n",
    "model.add(keras.layers.Conv2D(filters=32, kernel_size=(3,3), strides=(2,2),\n",
    "                              padding='same', input_shape=x_train.shape[1:],\n",
    "                             activation=tf.nn.relu))\n",
    "model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "model.add(keras.layers.Dropout(0.2))\n",
    "\n",
    "# Using strides=2 instead of strides=1 significantly decreases \n",
    "# computing time (down to 5 minutes from about 20), with\n",
    "# an acceptable decrease in accuracy of about 3% (down to 72% from 75%)\n",
    "model.add(keras.layers.Conv2D(filters=64, kernel_size=(3,3), strides=(2,2),\n",
    "                              padding='same', input_shape=x_train.shape[1:],\n",
    "                             activation=tf.nn.relu))\n",
    "model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "model.add(keras.layers.Dropout(0.2))\n",
    "\n",
    "model.add(keras.layers.Flatten())\n",
    "model.add(keras.layers.Dense(512, activation=tf.nn.relu))\n",
    "model.add(keras.layers.Dense(len(class_labels), activation=tf.nn.softmax))\n",
    "\n",
    "model.compile(optimizer='adam',loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
    "\n",
    "# Fit the model, with 30% of the data as validation data\n",
    "fit_result = model.fit(x_train, y_train, epochs=25, validation_split=0.3)\n",
    "history = fit_result.history\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Plot Epochs vs Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(fit_result.epoch, history['acc'], 'b', label='Training acc')\n",
    "plt.plot(fit_result.epoch, history['val_acc'], 'r', label='Validation acc')\n",
    "plt.title('Accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Plot Epochs vs Loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(fit_result.epoch, history['loss'], 'b', label='Training loss')\n",
    "plt.plot(fit_result.epoch, history['val_loss'], 'r', label='Validation loss')\n",
    "plt.title('Loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Plot evaluation:\n",
    "It can be seen that the training accuracy increases throughout. However, the validation accuracy fluctuates and more or less stagnates after 10 epochs. Similar is the case with the loss, where the training loss decreases throughout although the validation loss saturates after about 10 epochs.\n",
    "This is a case of over fitting the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Evaluate the test data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = model.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Modify the test labels and predictions to comparable dimensions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_test2 = y_test.squeeze()\n",
    "predictions = [np.argmax(i) for i in predictions]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_accuracy = sum(y_test2 == predictions)/len(predictions)\n",
    "print('Test Accuracy : ',test_accuracy) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### What increases computing time?\n",
    "In our model, we have used a stride length of 2 for the convolutional layer. A stride lenght of 1 improves the accuracy, although it significantly increases the computing time, to ~20 minutes, from the curent value of\n",
    "~5 minutes.\n",
    "\n",
    "#####  What increases accuracy?\n",
    "For improved accuracy, we could have added more number of convolutional layers, though this would increase computing time. Also, setting strides=1 for the convolutional layer increases accuracy by about 3% (up to 75% from the current 72%). We can also experiment with an improved loss function to increase accuracy.\n",
    "Setting a higher value for epochs also could result in marginally better accuracy, although again at the expense of time."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Create Assignment",
  "kernelspec": {
   "display_name": "Python (tensorflow)",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}