MLP_Tutorial_New.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Untitled1.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/binny-mathew/IITKGP_CS69002_Spring_2019/blob/master/MLP_Tutorial_New.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "GvYHwzMigOg9",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Movie Review Sentiment Analysis"
      ]
    },
    {
      "metadata": {
        "id": "hJRJTkHYgbNm",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Import Header files"
      ]
    },
    {
      "metadata": {
        "id": "FhEievieSXKJ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import pandas as pd\n",
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "BD18oPehUVrj",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "print('Welcome to Computing Lab 2. The current PyTorch version is ', torch.__version__)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "gukcTUtThMZJ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Load the dataset and visualize\n",
        "\n",
        "\n",
        "There are 3 ways to do this:\n",
        "\n",
        "(a)  From Github (Files < 25MB)\n",
        "\n",
        "The easiest way to upload a CSV file is from your GitHub repository. Click on the dataset in your repository, then click on View Raw. Copy the link to the raw dataset and store it as a string variable called url in Colab as shown below (a cleaner method but it’s not necessary). The last step is to load the url into Pandas read_csv to get the dataframe.\n",
        "\n",
        "\n",
        "```\n",
        "url = 'copied_raw_GH_link'\n",
        "df1 = pd.read_csv(url)\n",
        "# Dataset is now stored in a Pandas Dataframe\n",
        "\n",
        "```\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "(b) From a local drive\n",
        "\n",
        "To upload from your local drive, start with the following code:\n",
        "\n",
        "```\n",
        "from google.colab import files\n",
        "uploaded = files.upload()\n",
        "```\n",
        "\n",
        "It will prompt you to select a file. Click on “Choose Files” then select and upload the file. Wait for the file to be 100% uploaded. You should see the name of the file once Colab has uploaded it.\n",
        "\n",
        "Finally, type in the following code to import it into a dataframe (make sure the filename matches the name of the uploaded file).\n",
        "```\n",
        "import io\n",
        "df2 = pd.read_csv(io.BytesIO(uploaded['Filename.csv']))\n",
        "# Dataset is now stored in a Pandas Dataframe\n",
        "```\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "(c) From Google drive\n",
        "\n",
        "This the most complicated of the three methods. Follow this [link](https://medium.freecodecamp.org/how-to-transfer-large-files-to-google-colab-and-remote-jupyter-notebooks-26ca252892fa) to learn more."
      ]
    },
    {
      "metadata": {
        "id": "rYepeso8OiD1",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "url = 'https://github.com/binny-mathew/IITKGP_CS69002_Spring_2019/blob/master/Dataset/check.csv'\n",
        "df1 = pd.read_csv(url, sep='\\t')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "J5ITVAYmiy5H",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from google.colab import files\n",
        "uploaded = files.upload()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "fQLFDGIQjrh-",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "type(uploaded), uploaded.keys(), type(uploaded['check.csv'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "o7sqK9HKoGzv",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import io\n",
        "df = pd.read_csv(io.StringIO(uploaded['check.csv'].decode('utf-8')), sep='\\t')\n",
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "n3E2eUKolVBI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Pandas\n",
        "\n",
        "[Pandas](https://pandas.pydata.org/) is a high-performance, easy-to-use data structure and data analytics tool for Python.\n",
        "\n",
        "Tutorials:\n",
        "\n",
        "*   [Link 1](https://pandas.pydata.org/pandas-docs/stable/getting_started/tutorials.html)\n",
        "*   [Link 2](https://www.tutorialspoint.com/python_pandas)\n",
        "*   [Link 3](https://www.dataquest.io/blog/pandas-python-tutorial/)\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "sgLdaButozDS",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "print('Number of Negative movie reviews', len(df[df['label']==0]))\n",
        "print('Number of Positive movie reviews', len(df[df['label']==1]))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "KHLUDV8ZoZLG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Data pre-processing\n",
        "The first step when building a neural network model is getting your data into the proper form to feed into the network. We'll need to encode each word with an integer. We'll also want to clean it up a bit.\n",
        "\n",
        "You can use any type of pre-processing that you see fit, based on the task. For the tutorial, i will show just one pre-processing step: **lowercasing**."
      ]
    },
    {
      "metadata": {
        "id": "eafpV8pgk5kp",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "text_reviews = df['text'].astype(str).tolist()\n",
        "text_labels = df['label'].astype(int)\n",
        "\n",
        "text_reviews = [x.lower() for x in text_reviews]\n",
        "text_reviews[0], text_labels[0]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "s7IWD6ZXuLFc",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "len(text_reviews)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "HLlNt4uVv_J-",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "zydP-RKS0GgB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Creating Bag Of Word (BOW) representation of sentences."
      ]
    },
    {
      "metadata": {
        "id": "COY4uTBU2OSj",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def generate_word_ids(dataset):\n",
        "  word_to_ix = {}\n",
        "  print(dataset)\n",
        "  for sent,_ in dataset:\n",
        "      for word in sent.split():\n",
        "          if word not in word_to_ix:\n",
        "              word_to_ix[word] = len(word_to_ix)\n",
        "  return word_to_ix"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "O_6P4L4p2Xvx",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "generate_word_ids([['welcome to computing lab 2','1']])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "myexGL002a65",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "data = [(\"me gusta comer en la cafeteria\", \"SPANISH\"),\n",
        "        (\"Give it to me\", \"ENGLISH\"),\n",
        "        (\"No creo que sea una buena idea\", \"SPANISH\"),\n",
        "        (\"No it is not a good idea to get lost at sea\", \"ENGLISH\")]\n",
        "\n",
        "test_data = [(\"Yo creo que si\", \"SPANISH\"),\n",
        "             (\"it is lost on me\", \"ENGLISH\")]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "yLSUWPvg-tvm",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "word_to_ix = generate_word_ids(data+test_data)\n",
        "\n",
        "VOCAB_SIZE = len(word_to_ix)\n",
        "NUM_LABELS = 2\n",
        "print(VOCAB_SIZE, word_to_ix)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "8dE4gEe--uqy",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Intro to PyTorch"
      ]
    },
    {
      "metadata": {
        "id": "ShtFBLsF-ts1",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "from torch.autograd import Variable\n",
        "\n",
        "SEED = 42\n",
        "\n",
        "torch.manual_seed(SEED)\n",
        "torch.cuda.manual_seed(SEED)\n",
        "\n",
        "lin = nn.Linear(5,3)\n",
        "data1 = Variable(torch.randn(10,5)) # create a 10 * 5 matrix of random vars"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "i_Fg0hRz_7oA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "data1, data1.size()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "mq7AeDVN_7lI",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "lin(data1), lin(data1).size()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "NveD4hezA8AN",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "lin.weight.size()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "bI_h0ltKCf6c",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**BE VERY CAREFUL WITH THE DIMENSIONS**"
      ]
    },
    {
      "metadata": {
        "id": "nfDBZ-j1C_BT",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "F.relu(lin(data1))  # Activation Function"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "4KnJSJrwCtXV",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Model Definition for the BOWClassifier"
      ]
    },
    {
      "metadata": {
        "id": "4120zYv8_Uvv",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class BOWClassifier(nn.Module):\n",
        "  def __init__(self, num_labels, vocab_size):\n",
        "    super(BOWClassifier, self).__init__()\n",
        "    self.lin = nn.Linear(vocab_size, num_labels)\n",
        "    \n",
        "  def forward(self, x):\n",
        "    return F.softmax(self.lin(x))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "7F1gFDZOJV3N",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Generate the BOW Vectors"
      ]
    },
    {
      "metadata": {
        "id": "7kVLsLD6JQtj",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def make_bow_vector(sentence, word_to_ix):\n",
        "    # create a vector of zeros of vocab size = len(word_to_idx)\n",
        "    vec = torch.zeros(len(word_to_ix))\n",
        "    for word in sentence.split():\n",
        "        if word not in word_to_ix:\n",
        "            raise ValueError('Word',word,' not present in the dictionary. Sorry!')\n",
        "        else:\n",
        "            vec[word_to_ix[word]]+=1\n",
        "    return vec.view(1, -1)\n",
        "\n",
        "def make_target(label, label_to_ix):\n",
        "    return torch.LongTensor([label_to_ix[label]])\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "wK61RhS_JotP",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "bow = BOWClassifier(NUM_LABELS, VOCAB_SIZE)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "NzPrffWtJpo1",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "for param in bow.parameters():\n",
        "    print(param,param.size())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "muH5n2PGKk5j",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Let's check with a sample case how our model is working"
      ]
    },
    {
      "metadata": {
        "id": "G4_OMpkDJuDM",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "sample_data, sample_label = data[0]\n",
        "bow_vector = torch.autograd.Variable(make_bow_vector(sample_data, word_to_ix))\n",
        "logprobs = bow(bow_vector)\n",
        "print(data[0])\n",
        "print(logprobs)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "DITYJwjIKust",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "label_to_ix = {\"SPANISH\": 0, \"ENGLISH\": 1}\n",
        "ix_to_label = {v: k for k, v in label_to_ix.items()}\n",
        "\n",
        "label_to_ix, ix_to_label"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "9_AdvKxRNgY8",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "for instance, label in test_data:\n",
        "    bow_vec = Variable(make_bow_vector(instance, word_to_ix))\n",
        "    logprobs = bow(bow_vec)\n",
        "    print(logprobs)\n",
        "    pred = np.argmax(logprobs.data.numpy())\n",
        "    print('prediction: {}'.format(ix_to_label[pred]))\n",
        "    print('actual: {}'.format(label))\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "SDxwccp1Ngdm",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# define a loss function and an optimizer\n",
        "loss_function = nn.NLLLoss()\n",
        "opt = torch.optim.SGD(bow.parameters(), lr = 0.1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1VmI9iMkODWr",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# the training loop\n",
        "for epoch in range(100):\n",
        "#     print(epoch)\n",
        "    for instance, label in data:\n",
        "        # get the training data\n",
        "        bow.zero_grad()\n",
        "        bow_vec = Variable(make_bow_vector(instance, word_to_ix))\n",
        "        label = Variable(make_target(label, label_to_ix))\n",
        "        probs = bow(bow_vec) # forward pass\n",
        "        loss = loss_function(probs, label)\n",
        "        loss.backward()\n",
        "#        print('CURRENT LOSS: {}'.format(loss.data))\n",
        "        opt.step()\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "YsfIolPXNgih",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "print('--- AFTER TRAINING ---')\n",
        "for instance, label in test_data:\n",
        "    bow_vec = Variable(make_bow_vector(instance, word_to_ix))\n",
        "    logprobs = bow(bow_vec)\n",
        "    print(logprobs)\n",
        "    pred = np.argmax(logprobs.data.numpy())\n",
        "    print('prediction: {}'.format(ix_to_label[pred]))\n",
        "    print('actual: {}'.format(label))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "LUAx6baDNgld",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "torch.save(bow,'model.json')\n",
        "\n",
        "from google.colab import files\n",
        "files.download(\"model.json\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "kt0wHhXjNgoZ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from google.colab import files\n",
        "temp_test = files.upload()\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "vGYbYaIsy6Iw",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "torch.load(io.BytesIO(temp_test['model.json']))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "m9i8PVtgz0NS",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}