[Part_1]Hate-Speech-Detection_Classical-ML.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "[Part 1] Hate Speech Detection_Classical ML.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/AbeerAbuZayed/QUIUG_Hate-Speech-Detection_OSACT4-Workshop/blob/master/%5BPart_1%5DHate-Speech-Detection_Classical-ML.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mx0Hk8bMzyHk",
        "colab_type": "text"
      },
      "source": [
        "# [Part 1] Hate Speech Detection Using Classical Machine Learning\n",
        "In this notebook, we conduct a preliminary experiment on the detection of hate speech in Arabic tweets as part of our participation in the Hate Speech Detection subtask in [OSACT4 workshop](http://edinburghnlp.inf.ed.ac.uk/workshops/OSACT4/)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evavjP3jjxQt",
        "colab_type": "text"
      },
      "source": [
        "# 1. Data Loading and Basic Data Exploration\n",
        "In this section, we are going to load the train and dev dataset avaliable on the [workshop website](http://edinburghnlp.inf.ed.ac.uk/workshops/OSACT4/) and conduct basic data exploration for a basic understanding of the data.\n",
        "*   [Train dataset](https://edinburghnlp.inf.ed.ac.uk/workshops/OSACT4/datasets/OSACT2020-sharedTask-train.txt)\n",
        "*   [Dev dataset](https://edinburghnlp.inf.ed.ac.uk/workshops/OSACT4/datasets/OSACT2020-sharedTask-dev.txt)\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yk-Eo_LrkHMZ",
        "colab_type": "code",
        "outputId": "4bfc5f25-a0cc-485f-e37e-c6206d26879b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 125
        }
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n",
            "\n",
            "Enter your authorization code:\n",
            "··········\n",
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JuITAb7bJZgU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import re\n",
        "import string\n",
        "import math\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "from sklearn import model_selection, naive_bayes, svm\n",
        "from sklearn import metrics\n",
        "from sklearn.metrics import f1_score\n",
        "from sklearn.metrics import accuracy_score\n",
        "from sklearn.metrics import classification_report as creport\n",
        "\n",
        "\n",
        "from multiprocessing import Pool"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wFDSKk9L6Ga2",
        "colab_type": "text"
      },
      "source": [
        "NOTE: the data files used in this experiment are CSV files (the original txt files are converted to CSV files)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Fa7-DN6vU53M"
      },
      "source": [
        "### **Loading Data**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KmcNjsUNfbFN",
        "colab_type": "code",
        "outputId": "7645cb0b-d017-4053-dd96-966643e9f9ab",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 197
        }
      },
      "source": [
        "train_data = pd.read_csv('train_data.csv')\n",
        "train_data.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Tweet</th>\n",
              "      <th>Offensive</th>\n",
              "      <th>Hate</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>الحمدلله يارب فوز مهم يا زمالك.. كل الدعم ليكم...</td>\n",
              "      <td>NOT_OFF</td>\n",
              "      <td>NOT_HS</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>فدوه يا بخت فدوه يا زمن واحد منكم يجيبه</td>\n",
              "      <td>NOT_OFF</td>\n",
              "      <td>NOT_HS</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>RT @USER: يا رب يا واحد يا أحد بحق يوم الاحد ا...</td>\n",
              "      <td>OFF</td>\n",
              "      <td>HS</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>RT @USER: #هوا_الحرية يا وجع قلبي عليكي يا امي...</td>\n",
              "      <td>NOT_OFF</td>\n",
              "      <td>NOT_HS</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>يا بكون بحياتك الأهم يا إما ما بدي أكون 🎼</td>\n",
              "      <td>NOT_OFF</td>\n",
              "      <td>NOT_HS</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                               Tweet Offensive    Hate\n",
              "0  الحمدلله يارب فوز مهم يا زمالك.. كل الدعم ليكم...   NOT_OFF  NOT_HS\n",
              "1            فدوه يا بخت فدوه يا زمن واحد منكم يجيبه   NOT_OFF  NOT_HS\n",
              "2  RT @USER: يا رب يا واحد يا أحد بحق يوم الاحد ا...       OFF      HS\n",
              "3  RT @USER: #هوا_الحرية يا وجع قلبي عليكي يا امي...   NOT_OFF  NOT_HS\n",
              "4          يا بكون بحياتك الأهم يا إما ما بدي أكون 🎼   NOT_OFF  NOT_HS"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gS7-PdkVfuHT",
        "colab_type": "code",
        "outputId": "aa222121-10b6-4cc3-860b-80909398a172",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 197
        }
      },
      "source": [
        "dev_data = pd.read_csv('dev_data.csv')\n",
        "dev_data.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Tweet</th>\n",
              "      <th>Offensive</th>\n",
              "      <th>Hate</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>فى حاجات مينفعش نلفت نظركوا ليها زى الاصول كده...</td>\n",
              "      <td>NOT_OFF</td>\n",
              "      <td>NOT_HS</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>RT @USER: وعيون تنادينا تحايل فينا و نقول يا ع...</td>\n",
              "      <td>NOT_OFF</td>\n",
              "      <td>NOT_HS</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>يا بلادي يا أم البلاد يا بلادي بحبك يا مصر بحب...</td>\n",
              "      <td>NOT_OFF</td>\n",
              "      <td>NOT_HS</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>RT @USER: يا رب يا قوي يا معين مدّني بالقوة و ...</td>\n",
              "      <td>NOT_OFF</td>\n",
              "      <td>NOT_HS</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>RT @USER: رحمك الله يا صدام يا بطل ومقدام. URL</td>\n",
              "      <td>NOT_OFF</td>\n",
              "      <td>NOT_HS</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                               Tweet Offensive    Hate\n",
              "0  فى حاجات مينفعش نلفت نظركوا ليها زى الاصول كده...   NOT_OFF  NOT_HS\n",
              "1  RT @USER: وعيون تنادينا تحايل فينا و نقول يا ع...   NOT_OFF  NOT_HS\n",
              "2  يا بلادي يا أم البلاد يا بلادي بحبك يا مصر بحب...   NOT_OFF  NOT_HS\n",
              "3  RT @USER: يا رب يا قوي يا معين مدّني بالقوة و ...   NOT_OFF  NOT_HS\n",
              "4     RT @USER: رحمك الله يا صدام يا بطل ومقدام. URL   NOT_OFF  NOT_HS"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d8r2frbKft2I",
        "colab_type": "code",
        "outputId": "190af4c7-cd77-49ce-d47a-5a8dd537c654",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "print(\"Train data shape:{} \\nDev data shape: {}\".format(train_data.shape,dev_data.shape))\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train data shape:(7000, 3) \n",
            "Dev data shape: (1000, 3)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yoKuXMED7dLJ",
        "colab_type": "text"
      },
      "source": [
        "### **Checking Missing Values**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rkLxCSlIfto-",
        "colab_type": "code",
        "outputId": "bed101e4-51bc-48d0-f8b1-2f8a3c57d537",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        }
      },
      "source": [
        "train_data.isnull().sum()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Tweet        0\n",
              "Offensive    0\n",
              "Hate         0\n",
              "dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZMruwp-Zf-1Z",
        "colab_type": "code",
        "outputId": "bc0a9bbc-6dae-4826-f9b1-76333d4e8108",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        }
      },
      "source": [
        "dev_data.isnull().sum()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Tweet        0\n",
              "Offensive    0\n",
              "Hate         0\n",
              "dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BGlLuRWq8Lqm",
        "colab_type": "text"
      },
      "source": [
        "There are no missing values in both train and dev dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pRpVxuuo8cGg",
        "colab_type": "text"
      },
      "source": [
        "### **HS class distribution in train and dev dataset**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SXZNNukLgwAe",
        "colab_type": "code",
        "outputId": "0033e089-90e2-48a0-ce93-76fa327c05a9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        }
      },
      "source": [
        "train_data['Hate'].value_counts()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "NOT_HS    6639\n",
              "HS         361\n",
              "Name: Hate, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tHJkIE6JhHVU",
        "colab_type": "code",
        "outputId": "abd8612a-458d-45df-ce3c-842350d53ac3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 264
        }
      },
      "source": [
        "# Data to plot\n",
        "labels = 'NOT_HS', 'HS'\n",
        "sizes = [train_data['Hate'].value_counts()[0], train_data['Hate'].value_counts()[1]]\n",
        "colors = ['lightskyblue', 'yellow']\n",
        "explode = (0,0.2)  # explode 1st slice\n",
        "\n",
        "# Plot\n",
        "plt.pie(sizes, explode=explode, labels=labels, colors=colors,\n",
        "autopct='%1.1f%%', shadow=True, startangle=140)\n",
        "\n",
        "plt.axis('equal')\n",
        "plt.title(\"Train Data\")\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAD3CAYAAAC+eIeLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3xUVfrH8c+Zkh4SQi+B0CHUUFSw\nRNeCJUYFV7G7rrtrWxGFJa67/nRtUddeVkVUUFCUIjFYQIRQpBNQCEhPCD0hCemZcn5/3AEjKqRM\n5mYmz/v1yiuZ5M69T1j57uHcc8+jtNYIIYTwDYvZBQghRFMioSuEED4koSuEED4koSuEED4koSuE\nED4koSuEED4koSsaDaWUVSlVopTqZHYtQjQUCV1RZ56APP7hVkqVV3t9U23Pp7V2aa0jtNY5dail\nu1JKV7v+QaXUF0qpC2txjjuVUotre20hakNCV9SZJyAjtNYRQA5wZbXvTTv5eKWUzVc1AQnAd0Ca\nUurmhr6uEDUloSsajFLqSaXUDKXUx0qpYuBmpdRwpdRKpVShUuqAUupVpZTdc7zNM1qN87z+yPPz\nr5RSxUqpFUqpLjW5ttb6gNb6JeAJ4DmllPKc819KqV2e821WSiV7vt8feB041zNSzvN8P1kptUEp\ndUwplaOU+reX/5hEEyOhKxraNcB0IAqYATiBsUBL4GzgUuBvp3j/jcC/gRiM0fQTtbz+bKAd0N3z\nepvnulHAU8B0pVQbrfWPwH3AUs9ouaXn+BLgJiAauBIYq5RKqmUNQpwgoSsa2jKt9Rdaa7fWulxr\nvUZrvUpr7dRa7wLeARJP8f6ZWuu1WmsHMA0YVMvr7/d8jgHQWn/qGQW7tdbTgT3A0N97s9b6O631\nZs/xG4FPTlOvEKckoSsa2t7qL5RSvZVS8zw3uo4B/8EY9f6eg9W+LgMiann9Dp7PRz3Xv10ptdEz\nvVEI9D7V9T3TIYuVUkeUUkXAnaepV4hTktAVDe3kbezeBjYB3bXWzYBHAdWA178GI7h3KKW6Av8D\n7gZaaK2jga3Vrv9bW+59AswCYrXWUcC7DVyvCHASusLXIoEioFQp1YdTz+fWmVKqjVLqfuBfwERt\n7GEagRGsR4xD1F8wRrrHHQI6Hr+xV63eo1rrCqXUWcCYhqhXNB0SusLXHgJuA4oxRr0zvHny4+t0\ngR+AkcAorfVUAK31D8BrwGrgANALWFXt7QuA7cAhpdTxaY27gWc8qy/+CXzqzXpF06NkE3MhhPAd\nGekKIYQPSegKIYQPSegKIYQPSegKIYQPSegKIYQPSegKIYQPSegKIYQPSegKIYQPSegKIYQPSegK\nIYQPSegKIYQPSegKIYQPSejWkWcnq+qvb1dKve75updn4+sNSqktSql3zKlSCNHYNHh31ibqVeAl\nrfVcONH0UAghJHQbSDsg9/gLT9NDL1JxQC/Q33j3vI1TaqZDAc2BYMAFuH/vc0qC3WFWnULUhOyn\nW0dKKRdQPUxjgDSt9X1KqT8BLwPfA/OB97XWhV64qtXt5kGLhceACqA36CP1P685UjMdVqArRveG\nDkAroLXnc/WvW1DzAUIZxgbl+zCaUh7/XP3rHAlnYRYJ3TpSSpVorSOqvb4dGKq1vs/zuj1Ge/Gr\nMDoUDNRaV9b1ej/9pIa3acMn0dF0qvbtD0D/qa7n9KXUTEcrYLDnYwDQF+iJMXr1NQewBaO7xEZg\nA7A2JcHuhf9jFOLUJHTr6HShe9Kxm4DbtNbranud5GRlSU3l2Z49GWezYT3pxxo4D/Sy2p63oaVm\nOuIw2uVcDJzFz115GysNbMNo37MKWJySYM8ytyQRiCR06+hUoauUuhRYqLV2KKXaAplAgtb64O+c\n7jelpanOgwaR3qkT/U5x2CYgAbSzDr+G16RmOsKBC4CRWuuRSqkeZtbjJbuBecAXGCFcZXI9IgBI\n6NbRaUL3ReAKjHlXgOe11h/V5vxLl6pbBg3izchIIk5/NBNA/7c25/eG1ExHL4wW5yO11iOUUkG+\nrsGHSjAaV6YD81IS7IdMrkf4KQndRuaVV1TwhRfyYXw811osqBq+rRTjplruaY+sp9RMRzPgeq3d\ndyhlOauhr9dIaYyOwh8BH6Yk2ItMrkf4EQndRuTDD1WvCy/kq/bt6VKHt88GPdrrRXFiydYF2u2+\nAxilLJbQhriOnyrDaCP/dkqCfdXpDhZCQrcRSE5W6tprGZ2czOToaJrV41RXgP7SW3WlZjritNa3\na7f7DovVGuut8wawjcDbwLSUBPsxs4sRjZOErsmSk5X11lt5NCmJh0NCsNfzdLuAfqDL63OS1ExH\nvMvp/D+L1XqtUkoeFa+9UuBj4PWUBPtGs4sRjYuEromSk1Xo3Xcz6ZJLuNFqrfH87ek8CfrfdXlj\naqZjYFVF2VP24NDLlVLeqqcp08Ac4NGUBPtms4sRjYOErkmSk1XMhAnMOvdczvfyqSuBAaC31fQN\nqZmOIVUVZc/Yg0MvkrBtEG7gE+CxlAT7drOLEeaS0DXBtdeqDg8/zLwhQxjYQJf4FvTFpzvo6XUV\nZ7mqqp6xh4Se30B1iF9yAVOBx1MS7NlmFyPMIaHrY6NGqZ4PP8ycYcOIb+BL3QD6k9/6wVOrSzs5\nHVVvB4dFXNrANYjfVgVMBp5MSbDvN7sY4VsSuj6UnKz6PvIIn515Jn18cLkDGGt3T9xFT810hJQc\nPfx0WFTMfRarrb437UT9lQKPAS+nJNhNfaJQ+I7cmfaR5GTV5+GH+dRHgQvG9pJPHH8Rn5gUnb93\n5zcRMa3HSeA2GuHA88C61EzHcLOLEb4hI10fSE5WvSdOZObZZ9PXx5d2VVZazki45LLmwHUxHbtF\nXz4udZTFapN9lBsfDbwDTEhJsBebXYxoODLSbWDJyarXP/7BpyYELoDVatVvK6VvB4qO5u7ckr1x\nxVIT6hCnp4C/AZtSMx2XmF2MaDgSug0oOVn1vPdePjrnHExr12Oz6aHP/XtDJcZG4Kyc8b/lFSVF\n+WbVI06rE/BNaqZjcmqmI8rsYoT3Seg2kORk1W30aN665BKGml3L5X/Yf03HdqVuINxRWe7a+PUM\nrz0qLBrMHRhzvQPMLkR4l4RuA0hOVm3PPZfnb7qJ8xrDowZKETPllZVtMdrf8NOyr3bl5WyXJ6Qa\nv27AytRMxy1mFyK8R0LXy5KTVTPgwZAQrG43jWYZUNvWFVdcl5xdCrQB+P7jN752Oarq3D5I+Ewo\nMDU10/FGaqYjkPcrbjIkdL0oOVlZgHuB6AULyHz6aSYfO0aB2XUBKIV65P7NF9ht7hDAVnggu2Tn\n2sWLzK5L1Ng9wJLUTEdHswsR9SOh2zDsgFq/niMTJvDugQPkmF0QgN2ue/8vdY0C2gOsnjlpdWlh\nfq1aCAlTnQmsT810/MHsQkTdSeh6UVqadgMvAMuBLoD9wAHKxo5lalYWG8ytzjB8aN61vbsXOYFm\nbpdTr0+bki5rtf1KK2B+aqZjnNmFiLqR0PWytDRdBbyHsatURyCsogJXSgpzv/uOb91uTE04pQif\n/MKqrhhLyNTu9Uv3HdqxqdZdioWprMCLqZmOp80uRNSePJHWgJKT1WCMubhSMOZ2R4+m1403Mtpu\nr/eG5fXy6uSeb701tYcFOBAR0zr0yomv3GcPDgkzsyZRJ28Af09JsMtfZD8hI90GlJam12Psf+AG\n2gLMmsVPzz3H5JISTG3ncu/t2y6LCHPYgaCSo4fLt33/zQIz6xF1di8wJTXTYTW7EFEzEroNLC1N\nZ2ME736Mp43UqlUcSklh0uHD7DOrLquVzpNfXBWM56baurQpG4rzDjaKG36i1m4BZqZmOoLNLkSc\nnoSuD6Sl6aPAcxhtu+MAW04OJfffzwfbtrHJrLr69S66dtjA/AqgOVqzeva787Tb7TarHlEvVwPp\nqZmOcLMLEacmoesjaWm6AmMXqdkYI97QsjKc48cza9kyFpsxta4UQW88s2YAEA1Y9mWtO7xvy/qV\nvq9EeMlFwALZs6Fxk9D1obQ07U5L03OB14GWQBTAc8+RMWMGM51O3z/BFhHuGv6vsZsqMPbfZfn0\n1xZXlZdK+3D/NRyYI0+vNV4SuiZIS9OrgacBG579EKZPZ/PLL/NBWRklvq5nzNXZyS2aVyogpLL0\nmGPzd59/7esahFddALyfmuloBDt/iJNJ6JokLU3vBB4H8oFYQC1Zwr5HHmFSfj4+fUrMYqHtlFdW\nNMcz2v1xwcwtBQeypWutf7sReMbsIsSvSeiaKC1N52H8xdiAcYPNunMnx8aO5b1du9jqy1q6dCq9\n6uLzDpRhTHuw6tO3v3K7nI1mwx5RJxNTMx33mF2E+CUJXZOlpeky4E0gHegMBB87hmPcOGasWsXy\n070/Lg7694dBg2Dob+zcO20aDBhgHDNiBGzcaHz/yBE45xzo1w8+/xyUwvrsvzactSdzeXJFybGo\nw7u3FEiXiYDwWmqm4yqzixA/k9BtBNLStGv4/5VXDH1oT0drcFQHoJnW8NRTfDt7Np+7XLhO9f5F\ni2DDBli79tc/69IFMjLgxx/h3/+Gv/7V+P7HH8Ndd8Hq1fDyy8b3Fsx3Dzp7RGVeSESzZgArPnlT\nukz4PwvwcWqm4yyzCxEGCV2TxScmxdz60uxLlcX6UVCz9hcMnZAzOLRVfAzGxiZ88AEbX3+dqRUV\nlNXl/CNGQPPmxtdnnQW5ucbXdjuUlUFlJVit4HQa4TtnZnnc8S4TzqoK6TIRGEKBL1IzHV3NLkRI\n6JoqPjEpNCy6ZUpcwtmfAs0ArPbQvoPuWTcyJv6aMIwNc1i4kJxHH2VSQQFHTj6HUnDJJTBkCLzz\nzqmvN3kyXHaZ8fWNN8LcuXDxxfDPf8Kbb8Itt0BEBM2nvLqyHdW7TGRvN+0BDuE1LYHP5Kk188mG\nNyaJT0yyoNRdVzz4/MMtYrv9amNqrXX5vqXPfpqz8FEFZAPumBiCH3+cazt3pvvx4/btgw4d4PBh\nI0Bfew3OO+/X11u0CO65B5YtgxYtfvmzggK47jqYMwfGjYOjR8EaETt7054Bh4FD0e06R1zx4HP3\nWe1B8hfW/72ZkmC/1+wimjIZ6Zrn8iFX3nr7bwUugFIqtMO5E2+Nv2VeKMbKhqCjR6kcN47p69ez\n6vhxHToYn1u3hmuuMeZoT/bDD3DnncbI9uTABXjiCXjkEWOe95xzYOpUOLh778hfdJlYI10mAsQ9\nqZmO68wuoimT0DVBfGJS31Zxvf7cJzEp4VTHKaVUdPeL/zh43PZ2FntYRyDC6UQ/9hhfp6eTXlyM\nu7jYOLa0FObPN1YjVJeTA6NGwYcfQs+ev77G9u3GPO/55xtzvBaLMWUBhP+iy8Qs6TIRQN6V+V3z\nyPSCj8UnJkVarLanr/7n63+MaNGmTU3f56oq2bjxf8M2VhzdWQzkAfTtS//0dK6xWFBOpzFP+8gj\n8NZbxnvuussY4c6aBZ07G9+z2X65yuG66+Cpp6BHD2OK4uqroagI/vMfGDWK0tF3njN9646oEuBY\nl8HndjjnlnF3qsbQ4ljU1/fAeSkJ9lOujBHeJ6HrQ/GJSQr424gb7vtr9zMvPOUo97dot3PflmlX\nzy/cMd+JsVUkAwbQYsIEboyKIsbb9QIUFNoXnn3VJfuAPYC+5N7/JLXt0X9IQ1xL+NyjKQn2J8wu\noqmR6QXfGtYhfsiV3YZdMLAub1YWW4c+N3/xxw7nTrRhzPNafviB/PHjeXffPvZ4s9Djmkc7Lrzn\ntm1leDZh//7j1xc6KivqtHxNNDqPpmY6hpldRFMjoesj8YlJLewhYXeOGHPfWcpiqfOfu1IqotOF\n/7mt9w0zgzGC137oEOVjx/Lhjz+y3msFV3P3bdt/2WVi+dfSZSIw2DA2xrGZXUhTIqHrA/GJSRbg\n9nNuGjsitFl0vacBlFKWmN7JYxLu39xaWYNjgfCqKtyPPMIX8+fzjbebX1qtdJ780qoQjneZ+GLq\nhuK8A9JlIjD0Bf5udhFNiYSubyR27Dfswo79hvX15klDW/QYOewfuX2ComJbgzGn+/rrrHz/fT6u\nqqLKm9fq16to9BkJeZWc6DIxWbpMBI7HUjMdbc0uoqmQ0G1g8YlJ7YCbhl19x4CGuOtvC4kaOnjs\n1sRmcedF4Nmace5ctqemMrm4mEJvXUcpgt54em1/pMtEIGqG0U5K+ICEbgPyrFa4oe+F13SKbNm2\nQ0Ndx2K1d+57+/xR7c68LwhjpzK1di2HJ0xg0sGD7PXWdcLDpMtEALs5NdNxttlFNAUSug0r3hYU\nPKjfhdc0+B1ipSxRcZe9cHOP0VOCgC6Abf9+ysaOZcqWLfzgreuMuTr7KukyEZAU8HpqpkMyoYHJ\nH3ADiU9MsgE3D736ji7BYZHRvrimUsrWasANNw68Z11zZbF1AkLLy3FNnMicxYtZ6I0l2RYLbaTL\nRMAaBNxldhGBTkK34QwPj2kV13XY+T5fBxnepn/S0Am5PezhbdphzMHy4ossmzaNT51OHPU9f5dO\npVddkvhzl4mVM/73pXSZCBhPpmY6WppdRCCT0G0A8YlJ4cD1w6+7u5fNpJ257GExZw15cOfwiI5n\nRAFtAD79lC3PP897paUU1+fcSmFNfWTDWRaLDgesR/b8VJi9QbpMBIjmwASziwhkEroNY2TrrvHt\n2/UcOMjMIiy2oO7971yS3Hrwn0KAToBasYKDKSm8c+SI8RhxXYUEuwc9/+9MF561uytmSJeJAHJ3\naqbDJ1NiTZGErpfFJya1Ai4fetVtvZTFYvrOMEpZYrolv3Vj16TXjz86bMvOpmTsWN7fsYOs+pz7\n0gsOjKreZWLDV5/M80bNwnSRwH1mFxGoJHS9b1Tz9nFhLWK79zG7kOOUUkFth/31lv5/WR6FsnYC\nQkpKcD70EJ8tX86Sup/3l10mti3/erd0mQgYY1MzHWFmFxGIJHS9yPMgxFkJSTfHNYZR7skiOw67\nauj47C620Jj2QJTW8OyzLPr0U2Y7ndTpRli71hVXjLkquxTPvPH3n7zxjctRVenNuoUpWgJ/MbuI\nQCSh610XBIc3U+169B9sdiG/Jyii9TlDx+8ZFtZ2QDSeEepHH/HjK68wpbyc0rqc8+H7N/8hKMgl\nXSYCz/jUTIfd7CICjYSul8QnJjUDLkhIuql9Y+8lZrGF9B74t1WXt+x/fSie5pcZGeT+619MOnqU\nQ7U9n92me735zFoL0mUi0HQEbjG7iEAjoes9w5XFaus8cLhf7E+qLNbWPUZPHRM38jk7xg026/bt\nFI0dy3t79rCttucbPiRvdJ8eRU6gmdvl1OvTpqRr7ZYd8v3fRHlKzbvkD9ML4hOTgoCkvn+4urmv\nnj7zBqVUSPsRD9za90/fRoDqDAQXFVH1wAN8sno139fuXIS/+8KqbkALQO1ev3TfoR2bG2R/X+FT\nPYHLzS4ikEjoesdAIKLH8Iv9so1NVNx5o4c8uLOjNTiyAxDpdqOffJIFn39OmstFjbdvbB7l+EP1\nLhPLp7/2rXSZCAi3ml1AIJHQrSfPTmJXxfY/IyiyRZvfbKfuD4KjOl4wdHzOoNCWPVvgebz3vffI\nfPNNplZUUF7T81TvMlFacKRCukwEhCvlYQnvkdCtv55Ah55nX9rN7ELqyxoU3n/QvRsvad7ryjCg\nA8CCBWQ/9hiTCguNDsSnPYd0mQhEIcAfzS4iUEjo1t/FymKtbB3Xu5/ZhXiDsljb9b5h5vWxFzxq\nw9gi0pKVRcG4cby7dy87a3KOX3WZmPVuunSZ8HsyxeAlErr1EJ+YFAEk9Dr70gh7SGi42fV4i1Iq\nrGPiI7f1ufmL480vg/LzqRw7lmkbNrDm9O8/qcvElvVHcrPWSZcJ/3Z2aqaji9lFBAIJ3frpB1ji\nBp8TEKPc6pRSqnmPkdcPHru1jcUWGgtEOJ3oRx/lyy+/5Eu3+9Q32MLDXMP/9cCmSjz77n7/8evS\nZcK/KWTNrldI6NbP+VabvSSmY9feZhfSUEJiul4y7B+5fYOj41phLAfjrbdYM2kS06qqOOXjvmOu\nyk5uGVNhAUI9XSa+8kXNosFI6HqBhG4dxScmtQB69hwxsrlZe+b6ijU4MmHw2KwLorpdFIHnBtm8\neex68knePXaMgt97n9FlYmU0niVkPy6YubVgv3SZ8GPdUzMdw80uwt9J6NZdPEDsgDMbzW5iDUlZ\nbLHxt8wb3f7sh040v9ywgbzx45m0fz/Zv/e+uNiTukx8Kl0m/Fyy2QX4OwndujtHKUtxi9juATu1\ncDKlVGTni5++udd1n4RgrGywHzxI+f33M3XzZjJ/+z3SZSLAjDS7AH8noVsH8YlJUUCPzoNGhNmD\nQ5rUnqNKKWuLvqPGDLpvY0tlDY4FwqqqcD/8MGnffst8t5tf7bcgXSYCyqDUTEcbs4vwZxK6ddMb\nUO17D+pkdiFmCWvV57Jh/9jbOyiyfRsgBuDVV1kxdSqfOBxUnXz8pRccGN2pg3SZCAAKGe3Wi4Ru\n3QwGymI6du1sdiFmsoVEDxs8btu5kZ3OjsRzs2z2bLY9+yzvlZRQVP1YpYj+4GXpMhEgLjK7AH8m\noVtLnr0W+qJUUWTLdrFm12M2izWoS787Fl7d9oy7T9xgW72aQxMnMunQIXKrH9tWukwEikSzC/Bn\nErq11xoIbddzYLOmNp/7e5SyRHe5/OVbul8z+fijw7a9eym9/34++OknfjGardZlwi5dJvxWp9RM\nR5zZRfgrCd3a6wSojn2HNOmphZMppWytB91y88C7Vkcpi60zEFpejmvCBGZ9+y0rtef2WrUuE+1A\nukz4MRnt1pGEbu3FA1UtYrs32ZtopxLeblDy0PE5XW3hrdph7L3Aq6/yzeTJLHa7jSfYTu4ysS7t\nA+ky4X/OM7sAfyWhWwue+dz+QFFUmw4Sur/DHt5yxNAHd50Z3n5INJ7527Q0Vj71FPPcbg6c3GVi\nz/pl0mXC/ww0uwB/JaFbO1FATPMOXWz+1JbHDBZbcM8Bf12e1GrQrcFALFC1Zg1HU1J4VWvWN49y\n/OHe27eV4ZlmkC4TfqdPaqZDmV2EP5LQrZ1OgG7fc2A7swvxB0pZWnS/etJNXa545Xjzy0Nbt9Ln\nwQf5CzD7rlu3XxYR5rDh6TLx07Kv5ptasKiNMIz/TUUtSejWTk9AN2vdPsbsQvyFUiqo3Rl339L/\nziWRoDoBZTt3MubOO7nRamWap8tEO4D1X0zdeOzIgd/dx0E0OvFmF+CPJHRrpztQHB7TWkK3liJj\nz7pm6PjsztaQ6Aigx+HDXAT6kR5xxV/071NYDjQHWDP73Xlut0u6TPiHvmYX4I8kdGunHVAeFhUj\noVsHQZFtzxs2PntIaOv4EuD65GTVJiTE9XFhkf0NqnWZ2Je1XrpM+AcZ6daBhG4NxScmhQDNgKqQ\nyCgJ3Tqy2EP7DLp73cUt+o52YgSv+ubjRZuBJUiXCX8jI906kNCtueaAy2KzW4JDI6LMLsafKYu1\nTc8/Th/T+8Y5kdW+PRNwIl0m/ImsYKgDCd2aiwFo1blntLJY5M+tnpRSoTG9rnhixONVjwJkZaQX\nAdORLhP+JBxjOaCoBQmPmosBLDEdu8jUgvco4LHUTMfHqZmOEOB7YCfQCqTLhJ9oa3YB/kZCt+Y6\nAI5mrTtI6HrfGGDxrS/PaQVMwRhBSZcJ/yB/H2pJQrfmOgLlIRHREWYXEqDOBFbf+vKcaOAbqneZ\nKJYuE41YC7ML8DcSujXXHii3BQcHmV1IAIsFlv/xifddQCnHu0x8LV0mGjEZ6daShG4NeDa6iQaq\nbHYJ3QYWHhoZ/fEVD/0XqnWZOJK9TbpMNE4SurUkoVszNoybPtpqD5LQbXiWFrHd/n7ZA6nxVntQ\nO4AVH0uXiUZKQreWJHRrJgiMLrdWu11C10daxfUaesVD/70qsmXbZoUHc0p2rln0ndk1iV+R0K0l\nCd2a+Tl0bTLS9aXotrFtLh37zJ3teye0Wj3r3TWlhXkHzK5J/ILcSKslCd2akZGuiUIjoyPP/3PK\nnT3PHtl1XdqUedJlolFpbnYB/kZCt2ZOBK3VJqFrBps9KGjY1Xfc1LZ7vw7SZaJRkQypJfkDq5kT\nQWuxSuiaRVksqueIkZfZgkKCnVWVFWbXIwBwmF2Av7GZXYCfOBG0ymKxmlmIgJade/TTbpliaCSq\nzC7A38hIt2ZOhK52u2QvgEZAWSyyu1XjIKFbSxK6NWPBWKeL2yWhK0Q1Mr1QSxK6NePEs3rBLSNd\nIaqTkW4tSejWjOv4F7LVoBC/IKFbSxK6NXMiaOVRVCF+QaYXaklCt2Z+Dt2qSgldIX4mI91aktCt\nmRNrQh2VFbI+VIifFZhdgL+R0K2ZSjyrF5yV5RK6QvxM9sKoJQndmjkRtJVlJaVmFiJEI7Pf7AL8\njYRuzVTg+bMqzj8o/5wS4mcSurUkoVsDWRnpLqAcsBXuzzlqdj1CNCISurUkoVtzh4DQI3t+ktAV\nwuAGcswuwt9I6NZcLhBaVpRf6aisKDO7GCEagX0pCXZZMlZLEro1txcIAagsOSajXSFgl9kF+CMJ\n3ZrLw7P/QnlxgYSuEBK6dSKhW3P5eEK3tCBPQlcIyDK7AH8koVtzRzm+bCzvgISuELDa7AL8kYRu\nzZVgbO5hzc/dlWd2MUKYSWvtAtaZXYc/ktCtoayMdA0cBEJzN6895HI6ZHcl0WQppbJSEuzydGYd\nSOjWTg4Q5nY63CX5h3PNLkYIE8nUQh1J6NZOFp5lYwX792SbXIsQZlpldgH+SkK3dnLwrGA4uONH\neRJHNGUy0q0jCd3aOYhxM82Wnfl9rna73WYXJISvaa3LgE1m1+GvJHRrwbPxzRYgqrKs2FFamCd7\niYomRym1PiXB7jr9keK3SOjW3o9AOEDhgRyZYhBN0XdmF+DPJHRrbw+eed3Du7dK6Iqm6HOzC/Bn\nErq1l4sRupY965dmy7yuaEq01jkpCfZMs+vwZxK6tZSVkV6FsdFHs5Kjh8uLDu/bbXZNQviKUmqu\n2TX4OwndulkPRALs37J+s8m1COFLMrVQTxK6dfMDnu7AW5bM2+J2u2SKQQQ8rXUhsMTsOvydhG7d\nHMBo3xNRWnCkouhQ7k6zCzqSNxkAABI9SURBVBKioSml0lMS7E6z6/B3Erp14Nn8ZjEQA7AvS6YY\nRJMgUwteIKFbdz9PMWSkb3W7XLJYXAQsrXUF8I3ZdQQCCd26O4jRfjqy/NjRysKDe3eYXZAQDWhG\nSoK9xOwiAoGEbh1Vm2JoDpC7ea1MMYiApZR6w+waAoWEbv38PMWwOG2rs6qywuR6hPA6l9OxMSXB\nvsbsOgKFhG79HMZ4Qq1ZZVmx4+D2H9ebXZAQ3ma12V82u4ZAIqFbD54phgV4phh+XDBzjdZubW5V\nQniP2+UqBD4xu45AIqFbf2sx9ti1H9nzU2HB/pxtZhckhNcoJqck2GXazIskdOspKyO9DFgItAHY\ntvyrleZWJIR3aK3dFov1TbPrCDQSut6xBLABatv38/eUFBzZb3ZBov6WT3+Nl/84iJeuHciyaa/+\n4mdLP3yJhwcHUVqQ95vv/erlFF66diAvjupP2nPj0FrjrKrkvXuTePmPg1jx6Vsnjp39xN3s29L4\nNu5yu5wLUxLsu8yuI9BI6HpBVkb6AWAj0Apgx4oFy8ytSNTXwR2bWDNnMvdM/Z77P1nH1qVfkpdj\nLMUuPLiX7Su+Jbptp998b/bGFWRvXMHYGet54LMN5G5ey+51S9i2Yj5xCSO4f8Z6MudNA+DAto1o\nt4sOfRJ89rvVlNVmf87sGgKRhK73zAPCAH78dtbWipKifJPrEfVwZPdWYvudQVBoGFabjS5DzmXz\nd8ZTsPNeGM9lDzwNSv3OuxWOygpcjiqcVZW4nQ4iYlpjtdlxlJfhdjpAG/dbF7z5GBff85hvfqla\ncFRWrE5JsH9rdh2BSELXe7Zj7LMbo91uvXP1Ihnt+rE23fqyO3MZpYX5VJWX8dOyryk6lEvW4jSa\nte5Au54Df/e9nQeeRbdh5/P0JZ14emQnegy/mNZd+9D9zIsoOJDNm7edw4gb7iUr4wva906gWav2\nPvzNasZqt08wu4ZAZTO7gECRlZGu4xOT5gLjgKPr0z/c2GXIuWeFRbVoY3ZtovZad+1D4u0TeO+e\nywkKDad9r4E4qypZ9N6z/PmNL0/53rycHRzevZWUr4397SfffRm71y+jy+BzGPP0hwC4HA7eu/cK\nbn1pFukvTKDoYA4JSTcTn3hlg/9up1NVXrry0RHRsoVjA5GRrndtwtj2MUq73fqH+TNlgxA/Nuzq\nP/H36av42+TvCI2Mpk23eAr27eGVMUN59ooeHDucy2s3nUlx3sFfvC9r0Vxi+59BcFgEwWER9Dp7\nJDk//HJRy8rP3mJw0s3k/LiKkIhm3JA6nWUfNo5nEGxBIQ+YXUMgk9D1Ik+L9ul4tnzctvzr3QX7\ns2Xdrp8qOXoYgMIDOWxe9DmDr7yFfy3cx8R525k4bzvNWnfk79NWEdmy7S/eF902lt3rluJyOnE5\nHOxet5TWXXqf+Hn5sQK2Lv2ShKSbcVSUoSwWUApHZblPf7/fUlF67Ot/Dg1ZZXYdgUxC1/s2eT5a\nA6z5/L350rzSP00bfz0vjR7AlAeuIXniq4RGRv/usblZ65j1n78B0O+i0bTo2JVXrkvg1TFDaNtz\nAH0Sk04cu/Cdp7jgzylYLBZ6DL+EPZnLeeW6BBKuuKnBf6dTcbvdLntw6H2mFtEEKK3lqVVvi09M\nigWeAHIA90V3/99l7XsNOsPksoQ4pfLiwg8eP6/Vn8yuI9DJSLcBZGWk7wUWAe0BVn769mKnQ3Yg\nE42X01FZEhoZPd7sOpoCCd2Gkwa4gaCS/IPle9Yvk7vBotEqLzo6LiXBLmvLfUBCt4FkZaQXYPSU\nageweta7q8uPFRwxtyohfq047+Cqpy7u9K7ZdTQVEroN6zugEIhwVlW4Vs2aNEfatYvGxFlVWVF+\nrOB6s+toSiR0G1BWRnoFxhKy1oDK2bjiQHbm9xkmlyXECUWH9v7fi6MHZJtdR1Miodvw1gIr8dxU\nWz791aUl+YdzzS1JCCjJP7zx+eQ+sqmNj0noNjBPd4mPgHIg0u1y6uUfvzbH5XQ6TC5NNGFOR1VV\naVH+H82uoymS0PWBrIz0Y8AkjK0fLYd2bDq6fcX8+SaXJZqwokO5T780esB2s+toiiR0fWcTRj+1\njgCrZ01aW3jQs0GrED5UdCh39fNX9nrc7DqaKgldH/FMM8wE8vA0slwy5cW5zqoK8x+4F01GaWF+\n/q51S8zfyqwJk9D1oayM9HLgHSAKsBUeyC5ZO3fKZ7I3g/AFR2VF1fYVC26c8chth82upSmT0PWx\nrIz0HcBcIBaMnch+Wv71V+ZWJQKddrv1rjWLH//kn7fIvQSTSeiaIx3YjGcZ2epZk9bu25op2+mJ\nBpO7ee3nq2a+nWp2HUJC1xRZGekO4C2Mp9VaAiya9PQ3cmNNNIS8nB1bFk1+5sasjHSZxmoEJHRN\n4llG9jJgByLcLqde8ObjM2V/BuFNpYX5R7cuSb/c83SkaAQkdE2UlZG+H3gFY7QbVH7saOXi956d\n7qisKDO5NBEAKkuLy7YunXd9+gsT9phdi/iZhK7JsjLSs4ApGOt3LUf2/FS4etakGW6Xy2VyacKP\nVZWVlK+d+8F9nz36Z2mj3shI6DYOi4Gvgc4AO1d/l5P55bTPZEcyURdVFWWVKz97++mdq7+bYnYt\n4tckdBsBz4MTnwIb8Cwl27xwzk8bv/pkpqzhFbXhqKyoWj3znf/uyVyWKjfOGicJ3UYiKyPdifHg\nRDbQAeDHBTO3/DD/s9na7ZZGduK0nFWVjjWz331t19qMxz3/PYlGSEK3EcnKSC8FXgL24VnDu/Hr\nTzb/MP+zWTLVIE7F5XQ418794O0dqxb+07MkUTRS0g24EYpPTGoGjAfaAAcA+v7h6p4JV9x8ncVq\ntZpanGh0XE6Hc/0XU9/fkpF+vywNa/wkdBupasHbFtgP0Ovcy7sOvepPY6w2m93U4kSj4agor1g1\n8+33d63NGJ+VkS5LDf2AhG4jFp+YFAmMAzoBuQDdhl0Qe8a1fx1jDw4JM7U4Ybry4sLiJVNeeP/Q\njk3/9ExNCT8godvIxScmhQP3Az2AHICWnXtEnX/HxBvColq0MbU4YZpjRw7kLZr8zJtFB/c+n5WR\nXmJ2PaLmJHT9QHxiUghwNzAII3hdweHN7Bfd9eg1LWK79TG3OuFrh3Zuzl783nP/rSw99k5WRnqV\n2fWI2pHQ9RPxiUk24Frgcow53gqU4rxbH0rsPGjE+UopcwsUPrF95bc/rJzxv6e0ds/KykiXpxb9\nkISuH4lPTFLACOBOjB3KigD6X3xtnwEjr7vGarPLDbYA5XI6HJnzpn2ftWjuf4BFngdqhB+S0PVD\n8YlJ3YGxgA04BNCx77A2Z9/09zHBYZHRphYnvK447+DhpR++9F1e9rYnPHt1nJZSSgMvaq0f8rwe\nD0RorR/zvP4r8KDn8GPAg1rrZUqpOUAXIAKjkepuzzH3aK2//43rLAbGa63Xel7HAela635KqTCM\nhqwDAIUxULhUa92k56Dl4Qg/5Ok+8RhG4HYCVO7mNYfmvfCPd/Jydmw2tTjhNVq79c41izfOTb1/\nSl72tgdrGrgelcAopVTLk3+glEoC/gaco7XuDdwFTFdKtdVaX6O1HoTxr6mlWutBno9fBW4NjAUO\naa37a637AX8GmvyDGxK6fiorIz0fSAVWYoxM7CX5B8u/fHHCzI1fz5gp20P6t8rS4mNLPvjv/OXT\nXnnD7XQ8npWRfqCWpzj+WPm43/jZRGCC1joPQGu9HmOnu3vrVfSvtcN4uhLPdX7SWld6+Rp+R6YX\n/Fx8YpIFuBi4HigHjgA0a90h/LxbH0yK6di1t5n1ido7tHPzjowP/rusorjwf8CauszfKqVKMB4l\n/wEYCPwFz/SCUuoo0EVrXVTt+KuA27TWozyvz8eYNkg6zXUWY4Tr8a7WQYDbM70wCJgP7AQWAlO0\n1ttr+7sEGpvZBYj68ewk9U18YlIWxj8J44B9xw7vK03/70MzEq64qX+fxCsvtwUFh5haqDgtl6Oq\ncuPXM9ZsWjj7S+DdrIz0enUR0VofU0pNxVjnXX664+vhppPndD3X36CU6gpcAlwErFFKDddab2nA\nWho9mV4IEFkZ6XuBJ4GZGCOcFgCZ86b9+OVLE98o2J/d5EcYjZXWmoM7Nm394vlxn29aODsVeL6+\ngVvNyxhzqeHVvpcFDDnpuCEYzVK9SmtdorWerbW+B/gIY8ljkyYj3QDi2V0qPT4x6UeMf052BvYV\nHsgu+eK5B6YPuvzGfr3OufRCWeHQeJTkH963es67mbmb1mwB3srKSN/lzfNrrY8qpT7FCN73PN9+\nDnhWKXWp1jrfMw1wO3CmN6+tlDobyNJaFyilgoB4jA37mzQJ3QCUlZGeHZ+Y9B+MUcXVGEuCjm74\ncvqmzQvnbBk26s9nxCWcc64tKDjU3EqbrqqKspKsRXNX//DNZ9mg5wNpDbhhzQvAfcdfaK3TlFId\ngO89S8uKgZu11rW9WXc63YD/KePJHQswD5jl5Wv4HbmRFuDiE5O6AHdgdKTIA0oAwqNbhpxx7V/P\n7dAn4QyL1Sb/5+sjbpfLlfPDynUrP3trR1VZyTpgRlZG+r7TvlEEDAndJiA+MckKDAXGANHAQYx1\nnLTo1D1q2DV/vqBVXM8BSlnkWeIGot1unZezfcvKz97eUrBv9y7gQ2CTPFnW9EjoNiHxiUnBwHnA\naMCOsUG6EyC23xltBl1+44XR7Tr1kH0cvMfldDoO7dy0Yf0XH+46mrurCONGZ4a/bVRT7Um16iZq\nrb8xox5/JqHbBHk2SB8JXAq4MEa+boD2vRNa9b949PBWcb0HSJeKunNUVpTty1q7dl3a1OzSgiMu\nIAP4PCsjvdDs2oS5JHSbsPjEpNYYN9qGYzyeeQgjhIlq0zF84GU3DOvQO2GIPSQ0wsQy/UplaXHh\n7vVLV2fOm3bAUVHmBJYAC+rwRJkIUBK6gvjEpA4Yi9fPw9iY5BCeOV+rPcjS78JRfboMOXdos1bt\n48yrsvHS2q2LDu3btXvdks2bFs45qt2uSuArYElWRnqB2fWJxkVCV5wQn5jUHDgHuAwIxdg68sQ/\nh9v1GtSy5/CL+7fu2qdPaLPmrUwqs9EoP1ZwZN/WzA2bv/t8T9HBvXaMpXlzgZXSr0z8Hgld8Sue\nG24DMcK3C1CFsafDiR2i2vUc2LL7mX/o07pb3/jw6BZtzanU9yqKi/IP7cravGPlt9v3bVkPxvrT\n/Rhhu0Han4vTkdAVv8uzaXonIBFj83Q7xpxvHkYQA9Aqrld0j+GXxLft3q9PeEyrjoG0+sHtcjpL\n8g/n5ufu3L1rbcb2fVnrNMamLmUYT1etBvbK0i9RUxK6okbiE5PsQHeM9b5nYUw/HA/gE9v1Rbft\nFNFpwJmxLeN6xUa16RgbHt2ynT+tgnC7nM6So4dzj+7dtWf/to179mQuP+isLG+OEbROYA2wAvhJ\nRrWiLiR0Ra15+rV1AQYDZ2N0GXBjzGmWeL4GwB4cao3tf0a7Nt37xcZ06BIb2bJdx6DQsEgz6j6Z\n1m5dVVZaWH6sIL/w4N7cAz9t3LM7c9lBZ2V5BMYGMRojaNdi7Fu8LSsjvcnvByvqR0JX1ItnP984\noB9GW5Y4jBUQCmM7wWNUGwkDNG8fF9kitlvzZq07NI+IaR0dFhXTPCQyOjokollze0hopLefjHM6\nKisqiovyy4qO5pXkH8ovOpSbdzR3V97hXVuOOirLFRDFL0M2C9iA0apmf1ZGutOb9YimTUJXeFV8\nYlIQxtaSsUBfjJ2lIjACTWMEcYXn41fdbK32IEuL2O5R0e1io4PDIoPtwaFB9pDQYFtwaJDVbrdb\nLFaLslitFovFoixWi8tR5XBUlFVUVZRVVJWXVVaVl1RUlhZXVpYeqygvLqwsLyqoKCvKr8TY3CkU\nCANCMEbj2lODhKzwGQld0aA8N+NigI4YUxIdgLZAa4wbc26MUbEFY3VEBcZo0+X5cPJzQJ78H6sN\nsHo+V/84/j0871EYN/72ATmejyNAPnBEQlb4koSuMIUnjMOB5hib8ERjjJDbYIyMQ6t9BGEE6fFp\nh+P/0VYCpRgrCUo8n4s9XxcDBRjBmgcUywoD0RhI6Aq/4QlqK8ao2OlpVSSEX5HQFUIIH5IeaUII\n4UMSukII4UMSukII4UMSukII4UMSukII4UMSukII4UMSukII4UMSukII4UMSukII4UMSukII4UMS\nukII4UMSukII4UMSukII4UP/D+VhaKokAaYCAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZIG8M1tI9hB2",
        "colab_type": "text"
      },
      "source": [
        "*   There are 7,000 training tweets, only 361 of them (5.2%) are labelled as hate speech. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7HNQQinugvnQ",
        "colab_type": "code",
        "outputId": "9f57dc0e-dba0-44ae-e04b-e1bc55482030",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        }
      },
      "source": [
        "dev_data['Hate'].value_counts()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "NOT_HS    956\n",
              "HS         44\n",
              "Name: Hate, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1-xUPCvGjk2-",
        "colab_type": "code",
        "outputId": "1c00c448-6441-4db1-9af0-904cd62be1c8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 264
        }
      },
      "source": [
        "# Data to plot\n",
        "labels = 'NOT_HS', 'HS'\n",
        "sizes = [dev_data['Hate'].value_counts()[0], dev_data['Hate'].value_counts()[1]]\n",
        "colors = ['lightskyblue', 'yellow']\n",
        "explode = (0,0.2)  # explode 1st slice\n",
        "\n",
        "# Plot\n",
        "plt.pie(sizes, explode=explode, labels=labels, colors=colors,\n",
        "autopct='%1.1f%%', shadow=True, startangle=140)\n",
        "\n",
        "plt.axis('equal')\n",
        "plt.title(\"Dev Data\")\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAD3CAYAAAC+eIeLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3hUVf7H8feZkp4QOoQWmsHQEkBE\nBYON4oZYWXF1dVnbLuqirq7x566r7q5mRcW2qy42RBQLxRgVBIFIB0mooUNCS0hISG9Tzu+PGTQW\nJJPMzE35vp5nHkgyc+9nAvlwOffcc5XWGiGEEP5hMjqAEEK0JlK6QgjhR1K6QgjhR1K6QgjhR1K6\nQgjhR1K6QgjhR1K6QgjhR1K6wmuUUtlKqSqlVJlSqlgptVYp9QellNf/nimlxiqlnEqpcvfjqFLq\nI6XUeR5s43Gl1HvezibEL5HSFd42SWsdDvQCUoCHgTd9tK/jWuswIBwYBewGVimlLvPR/oRoNCld\n4RNa6xKtdSpwA3CrUmoQgFIqUCn1rFLqsFLqhFLqNaVUsPtru5RSiae3oZSyKKUKlFLDzrIvrbU+\nqrV+DHgD+HedbbyolDqilCpVSm1WSo1xf34C8H/ADe4j5a3uz0915yhTSh1USt3l3e+MaO2kdIVP\naa03AkeBMe5PpQDnAHFAP6Ab8Jj7ax8AN9Z5+XjgpNY6w4NdLgCGKaVC3R9vcu+rHfA+8LFSKkhr\nvRh4CvhQax2mtR7qfn4+kAhEAFOBmWcrfSE8IaUr/OE40E4ppYA7gfu11kVa6zJcxTfF/bz3gSSl\nVIj749/gKmJP96WASACt9Xta60KttV1r/RwQCMSc6cVa68+11gfcR8/pwFd8/w+GEI1mMTqAaBW6\nAUVARyAE2OzqX8BVkGYArfV+pdQuYJJS6jMgCYhvwL40UAyglHoQuA2Icn8+AuhwphcrpSYCf8d1\nNG5y593uYQYhzkhKV/iUezZBN2A1cBKoAgZqrY+d4SWnhxhMQJbWer+Hu7wGyNBaV7jHb/8CXAbs\n1Fo7lVKncBU9uEq4btZAYD5wC/Cp1tqmlFpU5/lCNJoMLwifUEpFuE+KzQPe01pv11o7gVm4xkk7\nuZ/XTSk1vs5L5wHjgD/iGm6oz76Uezt/B27HdYIMXLMa7EABYFFKPYbrSPe0E0B0nSltAbiGHwoA\nu/uod5yn712IXyKlK7ztM6VUGXAEeBR4HtcJqdMeBvYD65VSpcAy6oyxaq1zgXXAhcCHZ9lXlFKq\nHCjHdcJsMDBWa/2V++tLgMXAXiAHqHbnOu1j96+FSqkM9xjzn4CPgFO4xpRT6//WhTg7JYuYCyGE\n/8iRrhBC+JGUrhBC+JGUrhBC+JGUrhBC+JGUrhBC+JGUrhBC+JGUrhBC+JGUrhBC+JGUrhBC+JGU\nrhBC+JGUrhBC+JGUrhBC+JGUrhBC+JGUrhBC+JHcOcJDSqly922/T3/8O2CE1voepVQM8Dqu+3MF\nAqu01ncak7RlS8m0mYAgoDY53mo3Oo8Q9SWl610vATO11p8CKKUGG5ynWUjJtHUBegPR7l97Az1x\n3eUhGFe5/vhXa53XV+K6J1rJGX7NBQ64HweT4601fnhbQvwsKV3v6orrduMAaK3lhoZu7iPTc4Hz\ngGG4br/eG+iFq0gbI8T9iKrHc50pmbYjQBawo85jp5Sx8Ae5c4SHlFIOfnh32HZAqnt4YSrwArAW\n162739ZaF3thr5cBp0BnNH5b/pGSaYsARuO6ffmFWuthSqmws7zMSNXAemCF+7EhOd5aa2wk0RJJ\n6Xrol8Z03R9HAROAq3Dd+2uo1rpBR1BbtqiQIUN4ymTiT8AuYDjo6sa+B19wH8mOBq7GdZ+yoXVu\n+NgcVeL6x/N0CW+SsWPhDVK6Hjpb6f7ouTuAW7XWmz3dz4oVaszQoXzSrh2d6nz6OdAPNiS3L6Rk\n2izAWOA6rfW1p+/w20KVA0uBuUCaDEWIhpLS9dBZZi9MAL7WWtuUUl2ATCBea51X3+0nJSnTAw/w\n6AUX8LfAwO9PFrk5gQTQq73wVhokJdMWAFwOXKe182qlTO2MymKgEmA+8B6QnhxvdRqcRzQjUroe\nOkvpPg/8Ctf4IMAMrfV79d32gw+q8Jtu4pO4OMYpdcanHQCGgq5o0BtooJRM21DgXq31ZKVUhD/3\n3cQdAz4A3kuOt241Ooxo+qR0m4g331SDrriCz3r2JLoeT/8v6Lt9nSkl02YGkhy22gfN1oALfb2/\nFmAH8Arwjgw/iDOR0m0CPv5Y3TB+PG9ERFDfs/saGAd6mS/ypGTaIrXTeYfT6Zhutli7+WIfLVwe\n8CLwanK8tcToMKJpkdI1UFKSstx8M49ddRXJPzN+ezZHgEGgS72VJyXTNsBht92vTObfmkymxs6d\nFVAKvAa8kBxvzTU6jGgapHQNkpSkQm6/nf/96lfcaDY3eA2Mt0H/vrFZUjJtPW011c9aAgKvV+oX\nRpNFQ9UAc4AZyfHWvUaHEcaS0jVAUpJqe++9zLv88l88YVZfk0CnNeSFKZm2yKqy4qcDQ8JvM5nN\nnh5pC885cZ10eyQ53nrE6DDCGFK6fpaUpLo+/DCLLrqIkV7aZB4wEHRRfV+QkmkLqCorfiggKCTZ\nbA1oyleJtVRVwPNASnK8tdzoMMK/pHT9KClJ9XrkERZdcAFxXt70PNA3nu1JKZk2VV1ReovZYk2x\nBgZ38XIG4blc4BHg3eR4q/wgthJSun6SlKTOeeAB5o0dS7yPdvFr0B+f6YuPry48TynT7MCQsHN9\ntH/RcKuBacnxVlkgqRWQ0vWDpCTV7+67+WD8eEb4cDcncc1mOFH3k39euCMoICTslfAOXaeaTKbm\nvBZCS2cHXgYekyGHlk1K18eSklT0nXfyfmIiF/hhd5+Cvvr0B/fP3zquTadu7waFtensh30L7zgA\n/CY53rrR6CDCN+TIx4eSklSPqVOZ7afCBbgK1C2xCYnm2ITEsceyMpKlcJudvsCalEzbI+6V20QL\nI3+oPpKUpKKuuYbXr76ai/25X615cdCA4ieBqTuWzd9YePTgLn/uX3iFBXgKWJaSaZMrAlsYKV0f\nSEpSnS64gOduvpkr/H2pgVJE/it56xSgDKheNfu5NFtNdaV/UwgvuQTYmpJpu/qszxTNhpSulyUl\nqTDg/ltv5RKr1ZjbIfXvXd5n+u27hwHW0oLjlTuWzf/MiBzCK9oDC1Myba+lZNrk0uwWQErXi5KS\nlBWYBnT45z+Zk5eHYVcd3XbjwbgB/UpiALYv/WR3QfYemY7UvN0FbEzJtPUyOohoHCld74rBddPF\nE0ePUjF9OrN372abEUEsFh0884mMEUrpjgDfvPPsF7VVlWVGZBFeMwhYn5JpG250ENFwUrrelYXr\ndi49geCqKhx/+QsLV65kuREz83p1r+z1l2m7hgMBFcUnq7cunpfq/xTCy7oA6SmZtl8ZHUQ0jMzT\n9YGkJDUC+COuk1nFADfcwLk33MA1FovHSzg2isNBzc33XPjl1qy2WwHG3fOPSV36DRrmzwzCJxzA\nPcnx1teMDiI8I6XrI0lJqjdwP2AFTgBcdBFd772XG0NCCPdnluMngo5MuPGSL+wOU15weGRA0iMv\nTwsMCWvjzwzCZ2YAD8vaDc2HDC/4SGqqPgQ8AeTjGm5Qa9aQm5zMrIIC/LqgdVTn6h6PTt85HAis\nKiuuzUybs0j+sW0xHgI+SMm0BRodRNSPlK4PpabqQiAF+BaIBszZ2ZTddx9vHziAXy9amDzp8NCR\ncYWxAHvXfpWdu2erXGbactwALE7JtIUYHUScnZSuj6Wm6irgdeBToBcQVFaG7YEH+GjdOlb5K4fJ\nhHXGY5kjAgIcUQDfzH52aXV5SaG/9i98biyu+bwBRgcRv0xK1w9SU7UDWAi8CnQC2mgNTz/N8o8/\nZoHDgcMfOTq2r4l68sHtw4Hg2qoK+7eL3l6ktVPGGVqOccC8lEybIRfliPqR0vWT1FStU1P1OuBp\nXCfXOgHMmcP2F1/knaoqKvyRY9K4Y0PHnJ8/CFAHv00/enTn5rX+2K/wm2uAt1MybXKvuyZKStfP\nUlP1PuBJ4BTQA2DlSo7+9a/MKioi39f7Vwrz0/+3dVhIsL07wOo5M1dUlp7y+X6FX90M/NfoEOLn\nSekaIDVV5+NaRWo70Bsw79tHyfTpvJmdjc/vFtsusrbLv5K3DgdCbDVVjo3zZy3STqfT1/sVfvWH\nlEzbDKNDiJ+S0jVIaqquAF4BvsR1gi2wpITa++5j3qZNrPP1/scl5A2+4uLcIYA6vHVdbs62dX47\nqSf85sGUTNtjRocQPyQXRxgsKUkp4GJgKlCI6yo2bruN+MREEs1m3/3DWFpmKRh34yWfl5YF5Jgs\nVtM1f/3v7aGRHbr6an/CMFOT463vGB1CuMiRrsHcJ9jS+171+mcxUz65QpkDOgK8+SaZr73GnJoa\nqny174hwe8d/P7p1OBDmtNuc6z98daHT4fDLTArhV6+lZNrONzqEcJHSNVhsQmLIjU/PGdx52NQX\n2p+bNHjoHzZeaQ3r0htgyRKyH3+cN4qLOeN8WocD4uMhMfHM+5g/H5SCb791fbxmDQwZAiNGQNf2\n+YMmjTs6xFZdFfzNu89POLBp5QrvvkPRBAQCC1IybfK/mCZAStdAsQmJFktg0D294i76GugOENIp\nNjru7oyJYVHDBwOmnTspeuAB3jhyhIM/t40XX4Rzf+Gm6mVlruecX+c457nn4Isv4IUX4PXXUY/d\nv2N48dGdie2691m1/qP/ri0rPHHUm+9TNAlRuIpXLp4wmJSuQWITEhVw/cW3/Pm2yC49Otb9mjWk\nQ8dBt60Y12HwDecDASdPUn3ffczdsoVNdZ939Ch8/jncfvuZ9/O3v8HDD0NQUJ3tW6Gy0vWwWiEv\n19Eupm9R97ZdexZpp1Ov/eCVhQ673ebN9yuahFHAs0aHaO2kdI1zwYAxV/6h+8AR5/zcF02WoLD+\n182+tMclfx8LhNlsOB97jC++/JIvnU40wH33wTPPgOkMf4oZGXDkCPzqRyuvPvII3HILPP003HMP\nPPoovP2mLWLypJyhgOnE/h1FBzZ+vcyL71U0HfemZNomGx2iNZPSNUBsQmKv8A5d74lP/O3IX3qe\nUiZLj7GPXhDz63njUOb2AK++ysY332TuggXYOnWC4We4h4DTCQ884BpK+LG4OFi/HlasgIMHoWtX\n0Bq2rdkxJnfX2ptrKspC13/8+sbS/OPZjX6zoil6IyXT1t/oEK2VTBnzs9iExEDg77/687M3tu/R\nN7q+r6vI3Xpw57sTv7FXnswBOHiQxMJChgUGoqqrobQUrr0W3nvP9fySEujbF8LCXB/n5UG7dpCa\n6jqBBq6iHT8e5s2De++Fp56CtMVhhx/5W3he93OHfdmhV/824+/51zSz1SrjgC1PJjAyOd5qNzpI\nayNHuv6XNHTilIs9KVyA0K5D+8TdnTkxpMuQgYDq04e0SZOYsW4dOfPmwaWXfl+4AG3awMmTkJ3t\neowa9cPCBXj3XbjySlcZV1a6himGxJb37N+7Mgown8zZV7JnzeIlXnjPoumJBx40OkRrJKXrR7EJ\niTHtuveZMuiya89ryOsDwjp3Hnz7qvHtBlw1ErDm5VE1fTpzdu1i/+nnPPaYq1zPprIS3nkH7r7b\n9fEDD7gK+L774K1ZpW17RFX0Bfh20VsZp3Jz9jUkr2jy/i7DDP4nwwt+EpuQGKpMpievSn5pSkSn\nblGN2ZZ2OmyHv35sw7HVM9YBlQDTp3PhJZdwucmEV1aXytobse/6O8YsBoraRkWHT7wvZZolIDDo\nrC8UzU06cInc7sd/zI8//rjRGVo89/Swm8+/7o7ru507LKax21PKZI7se1nPoPb9Qot2fZoPVG/Y\nwBGbjbyBAznHbMbc2H10bF/T3m5Xts3b2h+pLiuutgQElnbuG/sLM4JFMxUNHB/d1bzZ6CCthQwv\n+MfQDr36T+p/wbh4b26045Ab4wbfsfpKc1Bkd4D589nzzDO8VV5OiTe2P+13++L79irrD5D5+dzt\nhUcO+PUWQ8JvnknJtDXqf1+i/qR0fSw2ITESuP386+8aZDKbG30E+mPh3c/rFzct48rgDjHnAmrD\nBk48/DCz8vM51thtW606dOYTGSOADgCr3n0+zVZTXdnY7Yompw3wH6NDtBZSur53XfSwMb3a9+jb\nz1c7CGzTveuQu9aNi+w/YThgOXKEij/9iXf27mVHY7fdr3d5n+m37x4GWEsLjlfuWDb/s8YnFk3Q\n1SmZtuuNDtEaSOn6UGxCYjSoMcMn3XKGSxi8xxwQFnnubxZc0XXUPaOB4MpK7A8+yPxVq1jZ2HOl\nt914MG5Av5IYgO1LP9ldkL1nuxcii6ZnptzK3fekdH3EffJsypDxv+4e2tY/a9QqkyUgesJzF/dN\nevUyIBJgxgzS583jE7udBk+Ct1h08MwnMkYopTsCfPPOs1/UVlWWeSm2aDq6A38wOkRLJ6XrO0Os\nQSGDYsdOatCc3IZSSqnOw28bPnDq1xNM1tCuAB98wM6ZM3m7spLyhm63V/fKXg/fnTUcCKgoPlm9\ndfG8eswGFs3QIymZtlCjQ7RkUro+EJuQaAVuGnnd7b0DgkMjjMjQJnpMTNzdmVcGte3TH1CrVnH8\nkUf438mT5DV0mzddlx0/NPbUuQC70j/bn7d/R4bXAoumojNwr9EhWjIpXd8YE96hS6/o+NEjzv5U\n3wlqG919yB82TozoPTYeMB86RNn06bx18CC7G7I9s4mA5x7PGG4xO7sArJr93JKayrJir4YWTcFf\nUjJtbYwO0VLJxRFeFpuQGA5Mv/iWB4a06dStu9F5TJbA4A6Dp/SyVRSoiuMZJ2prqV2yhJ29e2Pp\n3p2enm4vPMzepn3bWlau63zYXltdW1tZcaL7wBFxSnnlQjjRNAQDtaO7mlcaHaQlkiNd75sQ0qZ9\neOd+gwcbHeQ0k9ka1CfxlYTeE5+/FIjQGp56iq/nz2eRw4HH90S7PvHw0PPjTw4E2Lfuq+zcPVs3\nej20MNr9KZm29kaHaImkdL0oNiGxLTA+PvHmbmaLxWp0nrqUUqauo+45L/aWLyYqS1AXgNmz2fry\ny8yursajCx5MJqzP/G3L8IAARxTAN7OfXVpdXnLG+7iJZikceNjoEC2RlK53jTZZrOYeg0YaOpb7\nSyL7Xj4g7o/fTgyI6NEXYPlyjjz2GLNOnaLAk+10bF8T9eSD24cDwbVVFfZvF729SGunLJrSstyd\nkmlra3SIlkZK10tiExKDgIlDxk1uHxAcEm50nl8S3OGcnnHTvp0Q3mPUUMC0ezfF99/Pmzk53y8R\nWR+Txh0bevGo/EGAOvht+tGjOzev9U1iYZAQ4BajQ7Q0UrreMwwI7jvykiZ7lFuXJbhth4G/W3pF\nx6E3XwAEFBVRc//9vL95M+vruw2lMD/9yJbhIcH27gCr58xcUVl6Kt9noYUR7jI6QEsjsxe8IDYh\n0QTc3WfE2A79Rl46yug89aVMloB2Ayb1MluCA0sOLj/hdFKTns6BsDDK+/WjX33W5g0OcoZFd69Q\ni1dGHXE67LXlRfnHeg4dFa+USaYztAwdV+c5l4/uaj5sdJCWQo50veNcoFPsJUlxRgfxlFImc7cx\nD50/4DcLxiuTpRPArFlsfv115tTUUF2fbVyRkDfkiotzhwDq8NZ1uYe3rv/Gp6GFv8mlwV4kpesd\nEzv07G9uG9XrZ2+n3hy0i0kcOPQPmyZawzpHAyxeTPYTTzCrpISzzkpQCtM//rJtWER4bU+A1XNf\nXFVRfDLXx5GF/1yXkmnrYHSIlkJKt5FiExK7AwMHXnp1r+b+X+qQzgOj46ZlTAyNGjYYMO3YQdGf\n/8wbR49y6GyvjQi3d/z3o1uHA2FOu825/sNXFzodDo/nAIsmKRD4ndEhWgop3ca7FLB17jdokNFB\nvMEa2rHT4NtWjms/8LqRgDU/n+rp03lv2zbOejuXi0flD5o07ugQwHRsV0bBoc3frPB9YuEnd6Zk\n2pr1QUVTIaXbCLEJiYHA6Oj4iyxBYRHtjM7jLSZLUNg5k+de1n3sX8cCoTYbzr/+lbQlS1jsdHLG\nubhKoR67f8fwdpE10QBr572ytqzwxFE/xRa+1R+4xOgQLYGUbuMMAKx9R14Wa3QQb1PKZOl5yWMX\nnjN57jiUuT3Af/7Dhrfe4v3aWmrO9LrQEEe7Z/+eORyI0E6nXvvBKwsddrvNb8GFL00xOkBLIKXb\nOGNAVXWMjmlxpXtah0GThwy5c+1ES3C7ngCpqex/+mneLCvjjKuLjRpWGDt5Us5QwHRi/46iAxu/\nXua3wMKXrkrJtElnNJJ8AxsoNiExDIjvPXxMYFO/Aq2xwqLi+8bdnTkxpNOgWEBt3kzBQw8xKzeX\nM87dTL4na3jnjlW9AdZ//PrGkvxjZz0ZJ5q8TsCFRodo7qR0G24AoKLjR8cYHcQfAsK7dhl8x+rx\nbWMSzwMsx49TOX0672ZlsfXnnh8c5Gwz8/GM84BItGbN3Bc/ddhstf5NLXzgaqMDNHdSug03Cqjq\nGH3OuUYH8RdzQEjEgCkfXxZ10Z8TgJDqahzJySxavpxlP3fzy7hBxTG3TD44FDCfzNlXsmfN4iX+\nziy87iqjAzR3UroNEJuQGAwM7Roz1BQU1qbFzFqoD2UyB0SPe3p0v2vevBxUW4AXXmDNnDl8aLPx\nkxNm99+5Z3iPqIq+AN8ueivjVG7OPn9nFl7VLyXT1tfoEM2ZlG7D9AfMPQaN7GF0EKN0ivtt/ODb\n0yeaAyO6AXzyCbtnzOCtigpK6z4vMMAZPvOJjBFAO4DVc174zF5bU6/Li0WTNcHoAM2ZlG7DDAdq\n2/fo6/HtblqS8B6j+sdNy5gY1L5/DKDWrycvOZlZBQUcr/u82HNK+9918754wHLqeHbZrvTPvjAm\nsfASKd1GkNL1UGxCogIGAcURnaJa7ZHuaYGRPbsNvWvDhDZ9Lx8GWHJyKJ8+nbf37mVn3edN+92+\n+L69ys4ByPx87vbCIwd2GRJYeMMlKZm2JnVnlOZEStdzkUDbtt16WwNDwiONDtMUmAPDImNvTr2i\ny/nTRgNB5eXYH3qIT1auZN3p51itOnTmkxnDgQ4Aq959Ps1WU+3RbYJEkxEKNJl7ADY3Urqe6wno\nnkPOb9VDCz+mTJbA3hNnXtwn8ZXLgTZaw/PP89U777DC4XCdYOsXXd7nvjt2DwOspQXHK3csm/+Z\nsalFIwwzOkBzJaXruf6A7tgrRkr3R5RSqst5dw4f+LulE0zW0K4ACxaw6qmnWGa3cxLg91MOxg3o\nVxIDsH3pJ7sLsvdsNzKzaLDhRgdorqR0PTcEKGnTpYeU7hm06Z0wIG7atxMDI3v1B/SmTex84AG+\ncDjYYbHo4BeezBihlO4I8M07z35RW1VZZnBk4Tkp3QaS0vWAe35ut+DwyNqQiLadjc7TlAW169tj\n6B83TYiIvjgeqM7Opub3v+dtrfm0Z7fKXg/fnTUcCKgoPlm9dfG8VKPzCo8NkZNpDSOl65keAD2G\nnB+lTM17wXJ/sARFtou95csrOg2beiFQdOoUcZMnkwI8c9O12XFDY0+dC7Ar/bP9eft3ZBibVngo\nEBhodIjmSErXM70BIrv0bG90kObCZLYG9016LSF6woxLgYraWm5NSuIZp1Z/fOZvmXEWs7MLwKrZ\nzy2pqSw748plokmSIYYGkNL1zECgIqxdp1Z16W9jKaXMURdMP+/cmz+7WJkDuwK3Xnetnl1QGHTf\ntKn7YoHAqrLi2ozP3vtU/9wiDqKpkhkMDSCl65luQGVIm/ZtjQ7SHLXtPz526B+/jbeGdr4CuGjY\n4KJPs/ZEvBQSbO8OsG/dV9m5e7ZuNDimqD8p3QaQ0q2n2IREK9AWqAkKb12L3HhTSMeYnnH3bBkR\n3mPUtKQk1WnZqq4LKqss24AuAN/MfnZpdXnJWe9ALJqE3kYHaI6kdOsvEtCADgyVK9EawxrSvsPA\n3y2dFD3h2alZ6WlO4G1AAcG1VRX2bxe9vUhrp4wzNH2dUjJtFqNDNDdSuvXXFtBtOncPNVusAUaH\nae5MlsDQrqPu/WdKpu2urPS0k8C7QFdAHfw2/ejRnZvXGhxRnJ3C9WcmPCClW3/tAFP7nv1kaMFL\nlFIW4LWUTNuL4+75x3pgM+4f4tVzZq6oLD2Vb2hAUR9RRgdobqR0668T4Izs3ENOonnfn7r0G5Q2\n8LJrFgJ2IMRWU+XYOH/WIqfT4TQ6nPhF3YwO0NxI6dZfd6AqrF0nKV3fmDB80i1fxoyesATXSTV1\neOu63MNb139jdDDxi+RI10NSuvUXBVRbg0OCjA7SgsWOvO7ONwaMuTIf9w/z6rkvrqooPplrcC5x\nZlK6HpLSrQf3wuWdgCqzxSrXm/uQUqrDedfefu/QCVN6A2FOu825/sNXFzodDofR2cTPktL1kJRu\n/ZgBK+A0WwNk5oKPKaUChk644bLRN983HqVMx3ZlFBza/M0Ko3OJnyVjuh6S0q0fK645usiRrv/0\nGZEwaOL0p28JDI2wrp33ytqywhNHjM4kfiLM6ADNjZRu/XxXuiYpXb/qGB3TK/HBZ+9oGxUdvvaD\nVxY57Paf3OZdGEo6xEPyDaufAORI1zChbTt2HHfPP+4IDo8MOrDx62VG5xE/IB3iIfmG1c93RWuS\nq9EMERAUEjb65ulTq8tKKkvyjx0yOo/4jnSIh+QbVj/fla7ZbJEjXYOYzBbL4HGTr6spLy1x2G21\nRucRgHSIx+QbVj/fHd2aLFK6RlJK0anPuXFKmcxGZxGAa2aP8ICUbv1I0TYxJrNZftibBukQD8k3\nrH6suFZUwilnz4WoSzrEQ/INq5/v1nZ1OqR0hahDOsRD8g2rHxvu4nVI6QpRl/w8eEhKt37sp3/j\ntNvkL5kQ3ysyOkBzI6VbP98Vrb2mutrIIEI0MVK6HpLSrZ/vStdWU1VlZBAhmhgpXQ9J6dZPNe7Z\nC7XVlVK6QnxPStdDUrr1U4W7dG1VlTK8IMT38owO0NxI6dbP6SNdVVlaVGZ0GCGakONGB2hupHTr\nISs9TQMVgKX4eE6h0XmEaOKVJjEAABQXSURBVEKkdD0kpVt/p4CA/EO7pXSF+J6UroekdOvvKBBS\ncaqg2l5bLSfThAAnrp8L4QEp3frLAYIAqspK5GhXCNifHG+VAxAPSenWXz7uGQxVJUVSukLANqMD\nNEdSuvVXiHv9hfKifCldIaR0G0RKt/4KcR/plhbkSukKAVuNDtAcSenWXwVQA1hOHTskpSuEHOk2\niJRuPbnn6h4HgvP27yh0Oh0OozMJYaCS5HhrttEhmiMpXc8cAUJs1ZX2ilMnZX6iaM22Gx2guZLS\n9cwhIBDg1LHsHIOzCGEkGVpoICldz+TgnsGQt397trFRhDDUZqMDNFdSup45CjgAc3bG6iPa6XQa\nHUgIgyw1OkBzJaXrgaz0NDuwF2hTXV5SW1FSKMvaiVZHa52VHG89YnSO5kpK13OZQBhA8fGcbGOj\nCOF/SqklRmdozqR0PXcI97juif075WSaaI0WGx2gOZPS9dxRXKVrOpSxKkdrpzY6kBD+orWuBNKN\nztGcSel6KCs9rRbYB0RUlhTWlBcVHDM6kxD+opRKT4631hidozmT0m2YLUAEQO7erTsMziKEP8l4\nbiNJ6TbMvtO/2f3NFztliEG0IjKe20hSug2TDZQAwcW5OeWlBbnZhqYRwg+01nuT4617jM7R3Enp\nNkBWepoT18mEDgDHsjJkiEG0eEqpt43O0BJI6TbcZtzr6+5K/yxLVh0TLZnW2gHMNjpHSyCl23BH\ngZNAWMWpguqSvCMHjA4khK9o7VySHG/NNTpHSyCl20Du9XVXAO0AjuzYJEvdiRbLZDLPMjpDSyGl\n2ziZfDfEkLbHYbfbDM4jhNc5nY6TQJrROVoKKd3GOYFrYfOImopSW0H2bjnaFS2OUqZ3kuOtdqNz\ntBRSuo1QZ4ghEmDn1wvXG5tICO9TSr1ldIaWREq38TJxrcVgPrYro6DkxNGDRgcSwlucDsfG5Hjr\nLqNztCRSuo2UlZ5WDKwGOgPsW79MjnZbkDXvv8wLk+OYef1QVs99CYBlrz3J0+OjeWnKCF6aMoLd\nq7/82ddWlRUz96EbeP7aQTx/7WBytrr+anz54iO8+OthfPS3qd89N/Pzud9tvykxmc3PG52hpbEY\nHaCF+BpIAMhambpv4CVXFQRHtO1ocCbRSHn7d7Bp4ZtMe3ctZmsAb9+TyIAxVwJw0U1/4uJbHvjF\n13824wHOuXA8N834ELutFlt1JdVlJRzfvYXpH2Uw/8m7yNu3nfY9+rE59V2mvtK0zlU5bLYcs9X6\nsdE5Who50vWOI8AeoD1ac2DTyjVGBxKNV3BoNz0GjSQgOASzxULv4WPYuXxRvV5bXVZCdsZqRlzt\nOpq1WAMIDo9EmUw47Da01tiqKzFZrHwz53kumDINs9Xqy7fjOcW/kuOtcksqL5PS9QL3CbU0IBxg\ny5cfbK+pLC8xNpVorM59B3IoczUVxYXUVlWyZ/ViSk4cBWDdh6/y4q+H8cnjd1BVeuonry06fojQ\nth345PHbeenG85j/5F3UVlUQGBpOzEUTePnG8wjv0JWgsDYc2b6JgZdc5e+394scttoTZov1HaNz\ntERKa1kgyxtiExJNwD+BYKDk/OvvOi9m9IQrDY4lGmnTordZ/9FrBASH0rlvLGZrIGN//xdCIzuA\nUiz9798pO5nH9Y//8NqBo1mbefXW0dz1Vjo9B4/ksxkPEBgazrhpT/zgefOfvItRk//A8d2Z7Fu/\nlC79B3Pp7f/nz7f4s+y22j/9dWToy0bnaInkSNdL3IvgLMA9fSwjbU5mbVVFqbGpRGOdd/VU7n1/\nA3e9uZzg8Eg69OpPePvOmMxmTCYTI6+9jaM7N/3kdW06dSOiU3d6Dh4JwKDLruX47i0/eM7x3Zmg\nNR2jz2H70vn85t8fUHTkICcP7/vJ9vzJXluTZ7EGvG5oiBZMSte7tgBFQJitutK+Z/WXy4wOJBqn\nvCgfgOLcw+xcsYi4iVMoLfh+CYKdyz+lc9+BP3ldeIcuRHbuTkG2ayXEAxuX06n3uT94zlf/fYIr\npj2Ow27D6XStl6RMJmzVlb56O/WitfPvyfHWWkNDtGAye8GLstLT7LEJifOBO4HyzM/nbo8eNmZk\nePvO3Y3OJhpm7oM3UFlSiMliJenhlwgOjyT13/eRu3crCkXbqF5c/eh/ASgtOM78J//A1JdTAZj0\n8Ew+fPRWHLZa2nXvzfWPv/Hddneu+JTuscOI6BgFQFTMUF74dTxd+g+m6zlD/f9G3Ww11UesgcFv\nGhagFZAxXS+LTUi0AI/jup1PUXT86Kgxtzxwh1LK2GBC1IOtpurXfxsVIdPEfEiGF7wsKz3NDrwH\ntAFUdubq4wXZe7YaHEuIs6oqLV4thet7Urq+sQfYCHQBWP/Ra8scdpuMkYkmy2G32Rz22t8ZnaM1\nkNL1Afe83Y9xjZlbi3NzynO2rF1tcCwhzqi8MP+Ff17WTRbi9wMpXR/JSk/Lx3XBRFeADZ/8b11N\nRVmxsamE+Knq8tKjbTp3M35ycCshpetbS4AKINRWXWnfsWzBz6+MIoRBtNZUlxVPlfVy/UdK14ey\n0tMqgblAJ4CdKxbtzdu/I8PYVEJ8r7zwxKcpV/aV+eR+JKXre5uA/biLN/3tGYury0sKjY0kBNhq\nqsvN1oDfG52jtZHS9bGs9DQHMAuwAkE1FaW2DZ/MWuB0OmT1JmEYrTWl+cfueXJs5yKjs7Q2Urp+\nkJWelgfMBqIAlbNlzfHsjNUrjU0lWrOTh/ctmpE0YLbROVojKV3/WYNr7m4UwJr3X1pddjL3sLGR\nRGtUWpB7JGvFpzcbnaO1ktL1E/fc3XeBKiBcO5161ZyZC+y22hqDo4lWpLa6sjpn69prvnzx/yqM\nztJaSen6UVZ6WinwOtABMJ/M2Veya2Xq5wbHEq2Edjr1ke0b/jL3oSmbjc7Smknp+llWetpO4Eug\nG0Dm53O3H9uVITezFD6Xt39H6pq5L71idI7WTkrXGAuBXKAjwPJZ//qq6NihPcZGEi1ZSf6x7C1f\nvD/FPcwlDCSla4Cs9LQa4D+AGff47tL//H1+xamTuWd5qRAeqy4vKcvJXDMpffZz1UZnEVK6hslK\nTzsOvAC0A4JqKstsX//vn+/LDS2FN9VWVVTvXL7olveTb9phdBbhIqVroKz0tN3AG7imkVmKc3PK\n18x98X2HzGgQXmCvrbFlpL33xM7liz41Oov4npSu8dbiGuPtCaijO7/N35z67sfa6ZQr1kSDOR12\nx5YvP/jP3jWLZ8g4btMipWsw9w/Ep8BqXMXL7lWfH9i9+osvDA0mmi3tdOodyxa8l7Xi04fdl6GL\nJkRKtwlw3759NrAP9xVrmxa8uXn/huVLDQ0mmqXdqz5P3fLlB9Oy0tPkbiVNkJRuE+Ge0fAKUIx7\nKtnaD15eK8UrPHFg4/L0TQvfutW9rKhogqR0mxD3FWvPAbW4rlpzF+/XXxkaTDQLBzd/s2HN+y9f\nn5WeJjNgmjC5BXsTFJuQ2BlIBgKAAoDzJ9818pwLx0+UW7mLH9Nas3vVF2s2LXjjhqz0tGNG5xG/\nTEq3ifq54h2W+NshAy+9+iplMsn/UATgOmm27auPv9m6eN7vs9LTDhqdR5ydlG4T5i7eh4BQ4ATA\noMuujYm78jeTTWaz2dBwwnAOu82e8dmcr3alfzYtKz0tx+g8on6kdJu42ITE9sCfgfa41mug/6gr\neo245ve/tgYGhRgaThjGVl1VvXbeKwtztqz9c1Z6mlw+3oxI6TYDsQmJEcD9uObxHgHo0Kt/m4Sp\nD08JjWzfxdBwwu+qSk+Vpr8z4/38g7v+mpWeJvfba2akdJuJ2ITEUOCPwCBcxesICA61XHbX3yZ1\njI4ZYmw64S+njmfnpr8zY1Zp/vFns9LTyozOIzwnJ2Saiaz0tArgRWAJ0AsIqq2qsH/5QvLCveu+\nWiyXDbdsWjv1vnVLt6Y9++ATpfnH/1XfwlVKaaXUc3U+flAp9Xidj+9USu12PzYqpUa7P79QKbVF\nKbVfKVXi/v0WpdSFZ9jPSqXUiDofRyuldrh/H6KUmquU2q6U2qGUWq2UCmvgt6LZsxgdQNRfVnqa\nLTYhcR5wCLgDqACK1n/46obCnP15Ms7bMtVWVVSs//j1NdkZq/4HLHRfwVhfNcC1SqmntdYn635B\nKZUI3AWM1lqfVEoNAxYppUZqra9xP2cs8KDWOrERb2E6cEJrPdi9zRjA1ojtNWtypNvMZKWn6az0\ntPXAk7j+4kYB7Fu/NOerV/76esWpguOGBhReVXTs0JG0Zx/8JDtj1V+y0tPme1i4AHbgf7jOCfzY\nw8BDp8tYa52B63L0uxsV+qe6At/NH9Za79Fat9qV9GRMtxlzn2C7ExgMHAYc1sBg80U3TR/bY9B5\nF8p83uZLO51679qvMjfM/99CtH49Kz2toCHbUUqV4/qHeRswFNf/kMK01o8rpYqA3lrrkjrPvwq4\nVWt9rfvjsdTjSFcptRJXuVa5PxUAOLXWg5RSccBXwAHga2C21npfQ95PSyA/lM2Y+7LhF4DPcY3z\nhthqqhwr30r5On32s29UlhSeMDahaIiayvKy9NnPLtnwyetPofW/G1q4p2mtS3HdifpP3kl4Rjdp\nreO01nHAlXX2vwXoA8zAtWj/JqXUuT7O0mTJmG4zl5WeZo9NSPwYOAj8HmgLHD+8dV3usazN/7vw\nxntG94q78GKTSS6maOq006mP7Ny0Y/2Hr26oLi95KSs9bbsXN/8CkAG8XedzWcBwYHmdzw0Hdnpx\nvwBorcuBBcACpZQTVynv8vZ+mgMp3RbAvSbvt7EJifuBG4ELgHyHrbZ81bvPf3Nw08pd50++66qw\ndp26GZtUnElZ4Ylj6z/8b0bu3m3rgP95e/6t1rpIKfURcBvwlvvTzwD/VkpN0FoXuocBfgec7819\nK6UuArK01qeUUgFALLDSm/toTmR4oQXJSk8rBl4DZgKBQHfAdGxXRsGif9395v4Ny5c6HXa7oSHF\nD9hra6q2L/3km4X/+OPi3L3bXgNm+PCCh+dwr14HoLVOxVXAa5VSu4FZwM1aa29f4dYXSFdKbQcy\ngW+B+V7eR7MhJ9JaqNiExHDgBuBi4CRQCtAxOiZyxNVTL+nQq/9gpUyyZJlBtNbk7du+Y+0HL++s\nOHVyLfBBY8duRfMgpduCxSYkKmAgrv9SRgDHAQdA94EjOsVdedOl7bpFxxgYsVWqOFWQu3HBmxlH\ntm/YC7wDbJf7mLUeUrqtgPsS4knAFbhKNw9wAvQZkdB9yPgbLo/o2LWXgRFbhfLC/KM7li/ctnfN\n4hNAKrA4Kz2t2uhcnlBKLQR6/+jTD2utlxiRpzmS0m1FYhMSO+Eq3zFANa7lIjXAgIt/1XfgpVdf\nFhrZoauBEVuk0oLcnB3L5mfu3/B1Ca6ZAXOz0tPkIpZWSkq3FYpNSOwOXINrelAF7kXSUYoh4ybH\n9jv/sgtlpkPjlZw4enDbko+2HMpYVYbr0u0FwM4GXFUmWhAp3VbKPd7bB5gMDMB1oq3o9Nd7DxvT\nbcDFiee379l3oMlkllku9aS1U586nrN325IPtx/etqEC1x2eFwK7ZNxWgJRuq+cu33NxlW80rgVS\n8nGP+UZ27RU2+IrrhkUNiB8WGBLWxrCgTVxNZXlJ7p4tW7Z99cnh4twcJ64LDxYB+6RsRV1SugL4\nrnz74TrZNgLXWG8+rhJGmUwqZvTEPn1HXjq8bVSvGDn6BafDbi86dmjPwU3p2/es/rJca6cV2Irr\nJNlBKVvxc6R0xU/EJiR2wHWy7XIgBCjHNfSgAULbdgw658Jx/bvGxMW07dqrv9lqDTAurX9p7dSl\n+ccPHdmxaVvWik9zq8tLgnH9w7QSWJOVnnbE2ISiqZPSFWcUm5AYgGue7+W4Lt3UwCnguwW0LQFB\n5n7nXxbdY9B5Me179osJCA6NMCat79RUlpcU5+YczNu/48CBDcvzyovyQwEF7Me1etb25jb1SxhH\nSlfUi3u6WTwwGjg9s6EcKMY9/gvQc+gFXaPjR8e0796nT0hkhy5mi8Xq/7SN47DZaksLjmUXZO85\nkL1l7YG8vdtqgUhcRXsS1/KEmVnpafmGBhXNkpSu8Jj7DsUxuBbWicVVRrVAIXXuCGAyW1TUgPhO\nXfoPimrXrXe38I5do4Ij2nZuSuPBTqfDWVNeWlhx6uSJ0oLj+Xn7tuccylhV4LDVRgKn/8E4imu9\ngK3AYRmrFY0hpSsaxX21W39cJ99G4CoqhWucswyorPt8a2CwudvA4V069xkYFdEpqlNQWJuIwNCw\nNgHBoRGWgKBgX+XUWlNbWV5cUVx4orwwL78470h+Qfae/Lx92086bLUmoA0QimsIpRzYjGvh74Pu\ndYuF8AopXeE1sQmJVlx3KYjCdSQcA3TGVWQK110FynBdDfcTAcGhlrbdeke06dytTVi7zhEhbdpF\nBIW1CTNZLBazxWoxmS0Wk9liUSb1gyNlrbW211RX11ZVVtZWllfWVJZVVpeXVFaXFVdWlhRVlhcV\nVJadzK102GrBdWIwFNcqbKdz1eJa23Uzrrsb5MnRrPAVKV3hU7EJicG4Srg7roswzsF19wCH+ykK\n1xKjtbiOjk8/PF2C0ozrKPvHD0udfWlci/4cdD/y3Y9TUrLCX6R0hd/FJiQG4Vr17PQjEujkfnTE\nteZrEK6SPP04k7qlXep+lLgfp9y/FuFaZ6IwKz3NcYbtCOEXUrqiyXFfqGH50eP0WPGPVQOVWelp\nrfaW3qJ5kdIVQgg/ajJTd4QQojWQ0hVCCD+S0hVCCD+S0hVCCD+S0hVCCD+S0hVCCD+S0hVCCD+S\n0hVCCD+S0hVCCD+S0hVCCD+S0hVCCD+S0hVCCD+S0hVCCD/6f7cbGiai8A3dAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0NyIvVnD8vGZ",
        "colab_type": "text"
      },
      "source": [
        "*   There are 1,000 dev tweets, only 44 of them (4.4%) are labelled as hate speech.\n",
        "\n",
        "**This shows how the two classes in the dataset are clearly imbalanced.** "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GOjj6XvckTSB",
        "colab_type": "text"
      },
      "source": [
        "# 2. Data Cleaning\n",
        "To prepare our dataset for the feature extraction process, basic text preprocessing is done as follows:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cTEWf1FbyXxu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def clean(df):\n",
        "    df = remove_diacritics(df)\n",
        "    df = normalize_arabic(df)\n",
        "    df = remove_punctuations(df)\n",
        "    df = remove_repeating_char(df)\n",
        "    df= remove_english_word_and_numbers(df)\n",
        "    df=clean_space(df)\n",
        "    return df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Y2ur_EkyeQy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "arabic_punctuations = '''`÷« »×؛<>٩٨'٧٦٥٤٣٢١٠_()↗*•&^%][ـ،/:\"؟.,'{}⋮≈~¦+|٪!”…“–ـ/[]%=#*+\\\\•~@£·_{}©^®`→°€™›♥←×§″′Â█à…“★”–●â►−¢¬░¶↑±▾\t═¦║―¥▓—‹─▒：⊕▼▪†■’▀¨▄♫☆é¯♦¤▲è¸Ã⋅‘∞∙）↓、│（»，♪╩╚³・╦╣╔╗▬❤ïØ¹≤‡₹´'''\n",
        "english_punctuations = string.punctuation\n",
        "punctuations_list = arabic_punctuations + english_punctuations\n",
        "\n",
        "arabic_diacritics = re.compile(\"\"\"\n",
        "                             ّ    | # Tashdid\n",
        "                             َ    | # Fatha\n",
        "                             ً    | # Tanwin Fath\n",
        "                             ُ    | # Damma\n",
        "                             ٌ    | # Tanwin Damm\n",
        "                             ِ    | # Kasra\n",
        "                             ٍ    | # Tanwin Kasr\n",
        "                             ْ    | # Sukun\n",
        "                             ـ     # Tatwil/Kashida\n",
        "                         \"\"\", re.VERBOSE)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ODlY489hyuNN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def remove_diacritics(df):\n",
        "    df['Tweet'] = df['Tweet'].apply(lambda x: _remove_diacritics(x))\n",
        "    return df\n",
        "def _remove_diacritics(x):\n",
        "    x = str(x)\n",
        "    x = re.sub(arabic_diacritics, '', x)\n",
        "    return x\n",
        "\n",
        "def normalize_arabic(df):\n",
        "    df['Tweet'] = df['Tweet'].apply(lambda x: _normalize_arabic(x))\n",
        "    return df\n",
        "def _normalize_arabic(x):\n",
        "    x = str(x)\n",
        "    # added space around puncts after replace\n",
        "    x = re.sub(\"[إأآا]\", \"ا\", x)\n",
        "    x = re.sub(\"ى\", \"ي\", x)\n",
        "    x = re.sub(\"ؤ\", \"ء\", x)\n",
        "    x = re.sub(\"ئ\", \"ء\", x)\n",
        "    x = re.sub(\"ة\", \"ه\", x)\n",
        "    x = re.sub(\"گ\", \"ك\", x)\n",
        "    return x\n",
        "\n",
        "def remove_punctuations(df):\n",
        "    df['Tweet'] = df['Tweet'].apply(lambda x: _remove_punctuations(x))\n",
        "    return df\n",
        "def _remove_punctuations(x):\n",
        "    x = str(x)\n",
        "    #translator = str.maketrans(' ', ' ', punctuations_list)\n",
        "    translator = str.maketrans(punctuations_list, ' '*len(punctuations_list))\n",
        "    return x.translate(translator)\n",
        "\n",
        "def remove_repeating_char(df):\n",
        "    df['Tweet'] = df['Tweet'].apply(lambda x: _remove_repeating_char(x))\n",
        "    return df\n",
        "def _remove_repeating_char(x):\n",
        "    x = str(x)\n",
        "    return re.sub(r'(.)\\1+', r'\\1', x)\n",
        "\n",
        "def remove_english_word_and_numbers(df):\n",
        "    df['Tweet'] = df['Tweet'].apply(lambda x: _remove_english_word_and_numbers(x))\n",
        "    return df\n",
        "def _remove_english_word_and_numbers(x):\n",
        "    x = str(x)\n",
        "    return re.sub(r'[a-zA-Z0-9]+', '', x)\n",
        "\n",
        "def clean_space(df):\n",
        "    compiled_re = re.compile(r\"\\s+\")\n",
        "    df['Tweet'] = df[\"Tweet\"].apply(lambda x: _clean_space(x, compiled_re))\n",
        "    return df\n",
        "def _clean_space(x, compiled_re):\n",
        "    return compiled_re.sub(\" \", x)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CqmgO_XDzILE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "num_cores = 2\n",
        "def df_parallelize_run(df, func, num_cores=2):\n",
        "    df_split = np.array_split(df, num_cores)\n",
        "    pool = Pool(num_cores)\n",
        "    df = pd.concat(pool.map(func, df_split))\n",
        "    pool.close()\n",
        "    pool.join()\n",
        "    return df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m870svOszLGn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_data_cleaned = df_parallelize_run(train_data, clean)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oVYFpHN5zSNN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#train_data_cleaned.to_csv(\"train_data_cleaned.csv\", index=False)    #save the train_data_cleaned dataframe to csv"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eFYPAqUvzUYF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#!cp -R train_data_cleaned.csv \"/content/drive/My Drive/OSACT4\"            #save the csv file to the drive\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iIKDeUdtzWxt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dev_data_cleaned = df_parallelize_run(dev_data, clean)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "haaij8GizY5J",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#dev_data_cleaned.to_csv(\"dev_data_cleaned.csv\", index=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cJLwCfsrzapF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#!cp -R dev_data_cleaned.csv \"/content/drive/My Drive/OSACT4\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eBYS-8uTznar",
        "colab_type": "text"
      },
      "source": [
        "# 3. Classical Machine Learning Algorithms with tf- idf\n",
        "We experimented with various classical machine learning models:\n",
        "\n",
        "1.  SVM\n",
        "2.  Random Forest\n",
        "3.  XGBoost\n",
        "4.  Extra Trees\n",
        "5.  Decision Trees\n",
        "6.  Gradient Boosting\n",
        "7.  Logistic Regression\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SoS1I5G1I6bB",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wTn4UNFMKQUb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dataset = pd.read_csv('/content/drive/My Drive/OSACT4/train_data_cleaned.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lcxWn8RAJpTI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x=dataset.iloc[:,0]\n",
        "y=dataset.iloc[:,2]\n",
        "X=x.to_dict()\n",
        "\n",
        "X=[]\n",
        "for d in range(len(x)):\n",
        "    b=x[d]\n",
        "    X.append(b) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4zxndc1XKlsJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "count_vect=CountVectorizer()\n",
        "X_train_counts=count_vect.fit_transform(X)\n",
        "tfidf_transformer = TfidfTransformer()\n",
        "X_train_tfidf = tfidf_transformer.fit_transform(X_train_counts)\n",
        "X_train_tfidf= X_train_tfidf.toarray()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BmDn9ggoP848",
        "colab_type": "code",
        "outputId": "56e63699-acc1-41ee-b30a-308e54444341",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "print(X_train_tfidf.shape)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(7000, 28701)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bDZe9r5lmTLi",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "## SVM Classifier"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GHjD4YI0K7I3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.svm import SVC\n",
        "clf= SVC(random_state = 0)\n",
        "clf.fit(X_train_tfidf, y)\n",
        "clf.score(X_train_tfidf, y)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MUevBk3jLSsh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dev_data = pd.read_csv('/content/drive/My Drive/OSACT4/dev_data_cleaned.csv')\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pY-Vle45PZnD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x_dev=dev_data.iloc[:,0]\n",
        "y_dev=dev_data.iloc[:,2]\n",
        "#X=x_dev.to_dict()\n",
        "\n",
        "X=[]\n",
        "for d in range(len(x)):\n",
        "    b=x[d].lower()\n",
        "    X.append(b)  "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uotO18mPPotP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X_dev_tfidf=count_vect.transform(x_dev)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uVUXXgPQQTt0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "y_pred=clf.predict(X_dev_tfidf)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DohnKD7m3Fvs",
        "colab_type": "code",
        "outputId": "131ab59e-3aec-486b-f120-739bbf196909",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 232
        }
      },
      "source": [
        "print(creport(y_dev, y_pred,target_names=['HS', 'NOT_HS'],digits=4))\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "          HS     0.0000    0.0000    0.0000        44\n",
            "      NOT_HS     0.9560    1.0000    0.9775       956\n",
            "\n",
            "    accuracy                         0.9560      1000\n",
            "   macro avg     0.4780    0.5000    0.4888      1000\n",
            "weighted avg     0.9139    0.9560    0.9345      1000\n",
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1272: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
            "  _warn_prf(average, modifier, msg_start, len(result))\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bYJqH--qhCF1",
        "colab_type": "text"
      },
      "source": [
        "## Random Forest Classifier\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-XFPHAeCjhJA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn import model_selection, naive_bayes, svm, ensemble, tree\n",
        "from xgboost import XGBClassifier\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Txx3mRTFuN6P",
        "colab_type": "code",
        "outputId": "764719f3-2109-4509-bb05-86af69fab1d8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "RF= ensemble.RandomForestClassifier()\n",
        "RF.fit(X_train_tfidf, y)\n",
        "RF.score(X_train_tfidf, y)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uOaZqwLBjqJH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x_dev=dev_data.iloc[:,0]\n",
        "y_dev=dev_data.iloc[:,2]\n",
        "X=x_dev.to_dict()\n",
        "\n",
        "X=[]\n",
        "for d in range(len(x)):\n",
        "    b=x[d].lower()\n",
        "    X.append(b)\n",
        "\n",
        "X_dev_tfidff=count_vect.transform(x_dev)\n",
        "y_pred=RF.predict(X_dev_tfidf)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZGDgqxbP3LJl",
        "colab_type": "code",
        "outputId": "1254d452-5b78-4eda-ec44-1a234cc32e13",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        }
      },
      "source": [
        "print(creport(y_dev, y_pred,target_names=['HS', 'NOT_HS'] ,digits=4))\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "          HS     0.9000    0.2045    0.3333        44\n",
            "      NOT_HS     0.9646    0.9990    0.9815       956\n",
            "\n",
            "    accuracy                         0.9640      1000\n",
            "   macro avg     0.9323    0.6017    0.6574      1000\n",
            "weighted avg     0.9618    0.9640    0.9530      1000\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g4VBK6MVj4GX",
        "colab_type": "text"
      },
      "source": [
        "## XGB Classifier"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LJGcctCZjw9s",
        "colab_type": "code",
        "outputId": "c7324ca2-8ac4-4175-89bc-7ef5ce511fe2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "XGB= XGBClassifier()\n",
        "XGB.fit(X_train_tfidf, y)\n",
        "XGB.score(X_train_tfidf, y)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9554285714285714"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cfnPh1m-j8wP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x_dev=dev_data.iloc[:,0]\n",
        "y_dev=dev_data.iloc[:,2]\n",
        "X=x_dev.to_dict()\n",
        "\n",
        "X=[]\n",
        "for d in range(len(x)):\n",
        "    b=x[d].lower()\n",
        "    X.append(b)\n",
        "\n",
        "X_dev_tfidf=count_vect.transform(x_dev)\n",
        "y_pred=XGB.predict(X_dev_tfidf)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FkWs38Wz3Msv",
        "colab_type": "code",
        "outputId": "bfd5f059-05dc-4dc4-cd98-40f10a32a9f7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 232
        }
      },
      "source": [
        "print(creport(y_dev, y_pred,target_names=['HS', 'NOT_HS'],digits=4))\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "          HS     0.0000    0.0000    0.0000        44\n",
            "      NOT_HS     0.9560    1.0000    0.9775       956\n",
            "\n",
            "    accuracy                         0.9560      1000\n",
            "   macro avg     0.4780    0.5000    0.4888      1000\n",
            "weighted avg     0.9139    0.9560    0.9345      1000\n",
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1272: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
            "  _warn_prf(average, modifier, msg_start, len(result))\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aRTKKteNkVgq",
        "colab_type": "text"
      },
      "source": [
        "##Extra Trees Classifier"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dnfip-aPkPLt",
        "colab_type": "code",
        "outputId": "b981ab38-7e3e-439e-9c3f-19288dabc0f8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "extrees= ensemble.ExtraTreesClassifier()\n",
        "extrees.fit(X_train_tfidf, y)\n",
        "extrees.score(X_train_tfidf, y)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rqGUDRq0kYQj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x_dev=dev_data.iloc[:,0]\n",
        "y_dev=dev_data.iloc[:,2]\n",
        "X=x_dev.to_dict()\n",
        "\n",
        "X=[]\n",
        "for d in range(len(x)):\n",
        "    b=x[d].lower()\n",
        "    X.append(b)\n",
        "\n",
        "X_dev_tfidf=count_vect.transform(x_dev)\n",
        "y_pred=extrees.predict(X_dev_tfidf)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rodaGaRo3Oel",
        "colab_type": "code",
        "outputId": "363fb7d8-983e-4805-c60f-512675d90f2b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        }
      },
      "source": [
        "print(creport(y_dev, y_pred,target_names=['HS', 'NOT_HS'],digits=4))\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "          HS     0.6667    0.2727    0.3871        44\n",
            "      NOT_HS     0.9674    0.9937    0.9804       956\n",
            "\n",
            "    accuracy                         0.9620      1000\n",
            "   macro avg     0.8170    0.6332    0.6837      1000\n",
            "weighted avg     0.9542    0.9620    0.9543      1000\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pjmL4pjmk2nN",
        "colab_type": "text"
      },
      "source": [
        "## Decision Tree Classifier"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8p440U3zkt72",
        "colab_type": "code",
        "outputId": "a4f8a27f-a603-4303-95d5-6b5b20862981",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "DTC= tree.DecisionTreeClassifier()\n",
        "DTC.fit(X_train_tfidf, y)\n",
        "DTC.score(X_train_tfidf, y)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OBKWjj3KlUaE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x_dev=dev_data.iloc[:,0]\n",
        "y_dev=dev_data.iloc[:,2]\n",
        "X=x_dev.to_dict()\n",
        "\n",
        "X=[]\n",
        "for d in range(len(x)):\n",
        "    b=x[d].lower()\n",
        "    X.append(b)\n",
        "\n",
        "X_dev_tfidf=count_vect.transform(x_dev)\n",
        "y_pred=DTC.predict(X_dev_tfidf)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XKVzPHvyleXf",
        "colab_type": "code",
        "outputId": "66649b05-03b1-46ad-aa42-6bb8862b6957",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        }
      },
      "source": [
        "print(creport(y_dev, y_pred,target_names=['HS', 'NOT_HS'],digits=4))\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "          HS     0.1565    0.4091    0.2264        44\n",
            "      NOT_HS     0.9706    0.8985    0.9332       956\n",
            "\n",
            "    accuracy                         0.8770      1000\n",
            "   macro avg     0.5636    0.6538    0.5798      1000\n",
            "weighted avg     0.9348    0.8770    0.9021      1000\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7oeQU0BQmDlR",
        "colab_type": "text"
      },
      "source": [
        "## Gradient Boosting Classifier"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zPRVertul-z4",
        "colab_type": "code",
        "outputId": "4aeda6c3-eab9-4bb2-a5a0-7e6d7ca45064",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "GBC= ensemble.GradientBoostingClassifier()\n",
        "GBC.fit(X_train_tfidf, y)\n",
        "GBC.score(X_train_tfidf, y)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9628571428571429"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZexBEzkkmAjk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x_dev=dev_data.iloc[:,0]\n",
        "y_dev=dev_data.iloc[:,2]\n",
        "X=x_dev.to_dict()\n",
        "\n",
        "X=[]\n",
        "for d in range(len(x)):\n",
        "    b=x[d].lower()\n",
        "    X.append(b)\n",
        "\n",
        "X_dev_tfidf=count_vect.transform(x_dev)\n",
        "y_pred=GBC.predict(X_dev_tfidf)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pnKvNSXKmAwl",
        "colab_type": "code",
        "outputId": "7bb6e40d-5b37-4fc6-f672-1c034d53a4c1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        }
      },
      "source": [
        "print(creport(y_dev, y_pred,target_names=['HS', 'NOT_HS'],digits=4))\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "          HS     0.8182    0.2045    0.3273        44\n",
            "      NOT_HS     0.9646    0.9979    0.9810       956\n",
            "\n",
            "    accuracy                         0.9630      1000\n",
            "   macro avg     0.8914    0.6012    0.6541      1000\n",
            "weighted avg     0.9582    0.9630    0.9522      1000\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jfkZiMViTRk4",
        "colab_type": "text"
      },
      "source": [
        "## Logistic Regression Classifier"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1dm__bipRfAQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn import linear_model"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oS49ikeRsVcM",
        "colab_type": "code",
        "outputId": "80b4ff92-7b59-4a0b-84f7-1281fa0dbccf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "LR= linear_model.LogisticRegression()\n",
        "LR.fit(X_train_tfidf, y)\n",
        "LR.score(X_train_tfidf, y)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9484285714285714"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "exECcudzsi03",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x_dev=dev_data.iloc[:,0]\n",
        "y_dev=dev_data.iloc[:,2]\n",
        "X=x_dev.to_dict()\n",
        "\n",
        "X=[]\n",
        "for d in range(len(x)):\n",
        "    b=x[d].lower()\n",
        "    X.append(b)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MOkqzXv5slhb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X_dev_tfidf=count_vect.transform(x_dev)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vcXyRIf6snrv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "y_pred=LR.predict(X_dev_tfidf)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "de02a805-386f-4947-e835-1cd30ab24100",
        "id": "V6ad1wJrZenL",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        }
      },
      "source": [
        "print(creport(y_dev, y_pred,target_names=['HS', 'NOT_HS'],digits=4))\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "          HS     0.3750    0.0682    0.1154        44\n",
            "      NOT_HS     0.9587    0.9948    0.9764       956\n",
            "\n",
            "    accuracy                         0.9540      1000\n",
            "   macro avg     0.6668    0.5315    0.5459      1000\n",
            "weighted avg     0.9330    0.9540    0.9385      1000\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wENmretwby8V",
        "colab_type": "text"
      },
      "source": [
        "#4. Classical Machine Learning Algorithms with Word Embeddings\n",
        "\n",
        "Go to: [Colab Notebook](https://colab.research.google.com/drive/1Yk12Ng68RJ5Mn24vHgNg-nEdznIEDArZ)"
      ]
    }
  ]
}