NLP Project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this Project, my objective is to analyze text data and to predict topics of the customers' requests.\n",
    "<br>\n",
    "The data contain about 7000 requests of customers for fixing bugs in a product.\n",
    "<br>\n",
    "Using labeled data and requests' details, data analysis and prediction are possible. \n",
    "<br><br>\n",
    "Made by: Alon Cohen\n",
    "<br>\n",
    "bigalon1990@gmail.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Analysis "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Case Number</th>\n",
       "      <th>Title</th>\n",
       "      <th>Description</th>\n",
       "      <th>Case Comments</th>\n",
       "      <th>Topic - for PC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10087</td>\n",
       "      <td>Rodopi (IngentaConnect) - special character 'ß'</td>\n",
       "      <td>Migration date: 7/14/2013 4:56:11 PM. Dear Pri...</td>\n",
       "      <td>Status: 1st. line W4I\\nDear Christian,\\n\\nThis...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10087</td>\n",
       "      <td>Rodopi (IngentaConnect) - special character 'ß'</td>\n",
       "      <td>Migration date: 7/14/2013 4:56:11 PM. Dear Pri...</td>\n",
       "      <td>need to reload pubtech collection and test tha...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10087</td>\n",
       "      <td>Rodopi (IngentaConnect) - special character 'ß'</td>\n",
       "      <td>Migration date: 7/14/2013 4:56:11 PM. Dear Pri...</td>\n",
       "      <td>Hi Elad,\\n\\nCould you check if this case is re...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10087</td>\n",
       "      <td>Rodopi (IngentaConnect) - special character 'ß'</td>\n",
       "      <td>Migration date: 7/14/2013 4:56:11 PM. Dear Pri...</td>\n",
       "      <td>Please, reload Rodopi collection subset from t...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10087</td>\n",
       "      <td>Rodopi (IngentaConnect) - special character 'ß'</td>\n",
       "      <td>Migration date: 7/14/2013 4:56:11 PM. Dear Pri...</td>\n",
       "      <td>Dear Shahar,\\n\\nthanks for correcting!\\n\\nBest...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Case Number                                            Title  \\\n",
       "0        10087  Rodopi (IngentaConnect) - special character 'ß'   \n",
       "1        10087  Rodopi (IngentaConnect) - special character 'ß'   \n",
       "2        10087  Rodopi (IngentaConnect) - special character 'ß'   \n",
       "3        10087  Rodopi (IngentaConnect) - special character 'ß'   \n",
       "4        10087  Rodopi (IngentaConnect) - special character 'ß'   \n",
       "\n",
       "                                         Description  \\\n",
       "0  Migration date: 7/14/2013 4:56:11 PM. Dear Pri...   \n",
       "1  Migration date: 7/14/2013 4:56:11 PM. Dear Pri...   \n",
       "2  Migration date: 7/14/2013 4:56:11 PM. Dear Pri...   \n",
       "3  Migration date: 7/14/2013 4:56:11 PM. Dear Pri...   \n",
       "4  Migration date: 7/14/2013 4:56:11 PM. Dear Pri...   \n",
       "\n",
       "                                       Case Comments Topic - for PC  \n",
       "0  Status: 1st. line W4I\\nDear Christian,\\n\\nThis...   Data display  \n",
       "1  need to reload pubtech collection and test tha...   Data display  \n",
       "2  Hi Elad,\\n\\nCould you check if this case is re...   Data display  \n",
       "3  Please, reload Rodopi collection subset from t...   Data display  \n",
       "4  Dear Shahar,\\n\\nthanks for correcting!\\n\\nBest...   Data display  "
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Loading the data\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv('updated 12082018.csv', encoding=\"latin-1\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             Case Number  Title  Description  Case Comments  Topic - for PC\n",
      "Case Number                                                                \n",
      "10087                  1      1            1              7               1\n",
      "10099                  1      1            1             22               1\n",
      "11750                  1      1            1              3               1\n",
      "12439                  1      1            1             13               1\n",
      "12644                  1      1            1              4               1\n"
     ]
    }
   ],
   "source": [
    "unique_cases=df.groupby('Case Number').nunique()\n",
    "print(unique_cases.head())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Case Number</th>\n",
       "      <th>Title</th>\n",
       "      <th>Description</th>\n",
       "      <th>Case Comments</th>\n",
       "      <th>Topic - for PC</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Topic - for PC</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Availability</th>\n",
       "      <td>184</td>\n",
       "      <td>184</td>\n",
       "      <td>184</td>\n",
       "      <td>1812</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Citation tools</th>\n",
       "      <td>12</td>\n",
       "      <td>12</td>\n",
       "      <td>12</td>\n",
       "      <td>169</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coverage</th>\n",
       "      <td>39</td>\n",
       "      <td>39</td>\n",
       "      <td>39</td>\n",
       "      <td>442</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Data</th>\n",
       "      <td>232</td>\n",
       "      <td>229</td>\n",
       "      <td>232</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Data availability</th>\n",
       "      <td>650</td>\n",
       "      <td>638</td>\n",
       "      <td>650</td>\n",
       "      <td>5543</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   Case Number  Title  Description  Case Comments  \\\n",
       "Topic - for PC                                                      \n",
       "Availability               184    184          184           1812   \n",
       "Citation tools              12     12           12            169   \n",
       "Coverage                    39     39           39            442   \n",
       "Data                       232    229          232           1973   \n",
       "Data availability          650    638          650           5543   \n",
       "\n",
       "                   Topic - for PC  \n",
       "Topic - for PC                     \n",
       "Availability                    1  \n",
       "Citation tools                  1  \n",
       "Coverage                        1  \n",
       "Data                            1  \n",
       "Data availability               1  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "groupby_topic=df.groupby('Topic - for PC').nunique()\n",
    "groupby_topic.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAHfCAYAAACiSPpCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3WmYZVV59vH/TSuIIpO0E6igdlQc\nQEVEwQGMCE7ggEhEEQloRAPEqGgGiEMcEpwIYlBBcAISRFFRQECRCDLLoPjSigqCCIKIIqP3+2Gt\nQ5+uPl3dXbX3qarV9++66qo665yzn13dVfXsvYZnyTYRERHRhlVm+gQiIiKiO0nsERERDUlij4iI\naEgSe0REREOS2CMiIhqSxB4REdGQJPaIiIiGJLFHREQ0JIk9IiKiIfea6ROYqvXWW88bbrjhTJ9G\nRETEWJx//vk32J6/rNfN2cS+4YYbct555830aURERIyFpF8uz+vSFR8REdGQJPaIiIiGJLFHREQ0\nJIk9IiKiIUnsERERDUlij4iIaEgSe0REREOS2CMiIhqyzMQu6WGSTpf0E0mXSdqnth8o6deSLqof\nLxx6z7skLZT0U0kvGGrfrrYtlLT/UPtGkn4o6QpJx0hatetvNCIiYmWwPHfsdwFvs/04YAtgb0kb\n1+c+anvT+nEiQH3u1cDjge2AT0qaJ2kecAiwPbAxsMvQcT5Uj7UAuAnYo6PvLyIiYqWyzMRu+1rb\nF9SvbwF+Aqw/yVt2AI62fbvtK4GFwOb1Y6Htn9u+Azga2EGSgG2A/63vPxLYcarfUERExMpshcbY\nJW0IPBn4YW16i6SLJR0uaZ3atj5w1dDbrq5tS2t/APB723dNaI+IiIgVtNyJXdIawHHAvrb/ABwK\nPArYFLgWOGjw0hFv9xTaR53DXpLOk3Te9ddfv7ynHhERsdJYrsQu6d6UpP5F218BsH2d7btt/wX4\nNKWrHcod98OG3r4BcM0k7TcAa0u614T2Jdg+zPZmtjebP3+ZO9dFRESsdJa5bWsdA/8s8BPbHxlq\nf4jta+vDlwGX1q9PAL4k6SPAQ4EFwDmUO/MFkjYCfk2ZYPc3ti3pdOCVlHH33YCvdfHNRYzL1ft/\nf4Xfs8EHn9XDmUTEym559mPfEngtcImki2rbuymz2jeldJv/AngjgO3LJB0L/Jgyo35v23cDSHoL\ncBIwDzjc9mX1eO8Ejpb0PuBCyoVERERErKBlJnbbZzJ6HPzESd7zfuD9I9pPHPU+2z9nUVd+RERE\nTFEqz0VERDQkiT0iIqIhSewRERENSWKPiIhoSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdE\nRDQkiT0iIqIhSewRERENSWKPiIhoSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0i\nIqIhSewRERENSWKPiIhoSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIhSewR\nERENSWKPiIhoSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIhSewRERENSWKP\niIhoSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIhSewRERENSWKPiIhoSBJ7\nREREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIhSewRERENSWKPiIhoSBJ7REREQ5LY\nIyIiGrLMxC7pYZJOl/QTSZdJ2qe2ryvpFElX1M/r1HZJ+oSkhZIulvSUoWPtVl9/haTdhtqfKumS\n+p5PSFIf32xERETrlueO/S7gbbYfB2wB7C1pY2B/4FTbC4BT62OA7YEF9WMv4FAoFwLAAcDTgc2B\nAwYXA/U1ew29b7vpf2sRERErn2UmdtvX2r6gfn0L8BNgfWAH4Mj6siOBHevXOwBHuTgbWFvSQ4AX\nAKfYvtH2TcApwHb1uTVtn2XbwFFDx4qIiIgVsEJj7JI2BJ4M/BB4kO1roSR/4IH1ZesDVw297era\nNln71SPaIyIiYgUtd2KXtAZwHLCv7T9M9tIRbZ5C+6hz2EvSeZLOu/7665d1yhERESud5Ursku5N\nSepftP2V2nxd7Uanfv5tbb8aeNjQ2zcArllG+wYj2pdg+zDbm9nebP78+ctz6hERESuV5ZkVL+Cz\nwE9sf2ToqROAwcz23YCvDbW/rs6O3wK4uXbVnwRsK2mdOmluW+Ck+twtkraosV43dKyIiIhYAfda\njtdsCbwWuETSRbXt3cAHgWMl7QH8CtipPnci8EJgIXArsDuA7RslvRc4t77uPbZvrF//HfA5YHXg\nW/UjIiIiVtAyE7vtMxk9Dg7wvBGvN7D3Uo51OHD4iPbzgCcs61wiIiJicqk8FxER0ZAk9oiIiIYk\nsUdERDQkiT0iIqIhSewRERENSWKPiIhoSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQk\niT0iIqIhSewRERENSWKPiIhoSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIh\nSewRERENSWKPiIhoSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIhSewREREN\nSWKPiIhoSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIhSewRERENSWKPiIho\nSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIhSewRERENSWKPiIhoSBJ7RERE\nQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIhSewRERENWWZil3S4pN9KunSo7UBJv5Z0\nUf144dBz75K0UNJPJb1gqH272rZQ0v5D7RtJ+qGkKyQdI2nVLr/BiIiIlcny3LF/DthuRPtHbW9a\nP04EkLQx8Grg8fU9n5Q0T9I84BBge2BjYJf6WoAP1WMtAG4C9pjONxQREbEyW2Zit30GcONyHm8H\n4Gjbt9u+ElgIbF4/Ftr+ue07gKOBHSQJ2Ab43/r+I4EdV/B7iIiIiGo6Y+xvkXRx7apfp7atD1w1\n9Jqra9vS2h8A/N72XRPaIyIiYgqmmtgPBR4FbApcCxxU2zXitZ5C+0iS9pJ0nqTzrr/++hU744iI\niJXAlBK77ets3237L8CnKV3tUO64Hzb00g2AayZpvwFYW9K9JrQvLe5htjezvdn8+fOncuoRERFN\nm1Jil/SQoYcvAwYz5k8AXi1pNUkbAQuAc4BzgQV1BvyqlAl2J9g2cDrwyvr+3YCvTeWcIiIiAu61\nrBdI+jLwXGA9SVcDBwDPlbQppdv8F8AbAWxfJulY4MfAXcDetu+ux3kLcBIwDzjc9mU1xDuBoyW9\nD7gQ+Gxn311ERMRKZpmJ3fYuI5qXmnxtvx94/4j2E4ETR7T/nEVd+RERETENqTwXERHRkCT2iIiI\nhiSxR0RENCSJPSIioiFJ7BEREQ1JYo+IiGhIEntERERDktgjIiIaksQeERHRkCT2iIiIhiSxR0RE\nNCSJPSIioiFJ7BEREQ1JYo+IiGhIEntERERDktgjIiIaksQeERHRkCT2iIiIhiSxR0RENCSJPSIi\noiFJ7BEREQ1JYo+IiGhIEntERERDktgjIiIaksQeERHRkCT2iIiIhiSxR0RENCSJPSIioiFJ7BER\nEQ1JYo+IiGhIEntERERDktgjIiIaksQeERHRkCT2iIiIhiSxR0RENCSJPSIioiFJ7BEREQ1JYo+I\niGhIEntERERDktgjIiIaksQeERHRkCT2iIiIhiSxR0RENCSJPSIioiFJ7BEREQ1JYo+IiGhIEntE\nRERDktgjIiIaksQeERHRkCT2iIiIhiSxR0RENCSJPSIioiFJ7BEREQ1JYo+IiGjIMhO7pMMl/VbS\npUNt60o6RdIV9fM6tV2SPiFpoaSLJT1l6D271ddfIWm3ofanSrqkvucTktT1NxkREbGyWJ479s8B\n201o2x841fYC4NT6GGB7YEH92As4FMqFAHAA8HRgc+CAwcVAfc1eQ++bGCsiIiKW0zITu+0zgBsn\nNO8AHFm/PhLYcaj9KBdnA2tLegjwAuAU2zfavgk4BdiuPrem7bNsGzhq6FgRERGxgqY6xv4g29cC\n1M8PrO3rA1cNve7q2jZZ+9Uj2iMiImIKup48N2p83FNoH31waS9J50k67/rrr5/iKUZERLRrqon9\nutqNTv3829p+NfCwoddtAFyzjPYNRrSPZPsw25vZ3mz+/PlTPPWIiIh2TTWxnwAMZrbvBnxtqP11\ndXb8FsDNtav+JGBbSevUSXPbAifV526RtEWdDf+6oWNFRETECrrXsl4g6cvAc4H1JF1Nmd3+QeBY\nSXsAvwJ2qi8/EXghsBC4FdgdwPaNkt4LnFtf9x7bgwl5f0eZeb868K36EREREVOwzMRue5elPPW8\nEa81sPdSjnM4cPiI9vOAJyzrPCIiImLZUnkuIiKiIUnsERERDUlij4iIaEgSe0REREOS2CMiIhqS\nxB4REdGQJPaIiIiGJLFHREQ0JIk9IiKiIUnsERERDUlij4iIaEgSe0REREOS2CMiIhqSxB4REdGQ\nJPaIiIiGJLFHREQ0JIk9IiKiIUnsERERDUlij4iIaEgSe0REREOS2CMiIhqSxB4REdGQJPaIiIiG\nJLFHREQ0JIk9IiKiIUnsERERDUlij4iIaEgSe0REREOS2CMiIhqSxB4REdGQJPaIiIiGJLFHREQ0\nJIk9IiKiIfea6ROI6NNBO794hd/ztmO+0cOZRESMR+7YIyIiGpLEHhER0ZAk9oiIiIYksUdERDQk\niT0iIqIhSewRERENSWKPiIhoSBJ7REREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIh\nSewRERENybatMXaHvOm0FX7P3p/apocziYhoT+7YIyIiGpLEHhER0ZAk9oiIiIYksUdERDRkWold\n0i8kXSLpIknn1bZ1JZ0i6Yr6eZ3aLkmfkLRQ0sWSnjJ0nN3q66+QtNv0vqWIiIiVVxd37Fvb3tT2\nZvXx/sCpthcAp9bHANsDC+rHXsChUC4EgAOApwObAwcMLgYiIiJixfTRFb8DcGT9+khgx6H2o1yc\nDawt6SHAC4BTbN9o+ybgFGC7Hs4rIiKiedNN7AZOlnS+pL1q24NsXwtQPz+wtq8PXDX03qtr29La\nIyIiYgVNt0DNlravkfRA4BRJl0/yWo1o8yTtSx6gXDzsBfDwhz98Rc81IiKiedO6Y7d9Tf38W+B4\nyhj5dbWLnfr5t/XlVwMPG3r7BsA1k7SPineY7c1sbzZ//vzpnHpERESTppzYJd1P0v0HXwPbApcC\nJwCDme27AV+rX58AvK7Ojt8CuLl21Z8EbCtpnTppbtvaFhEREStoOl3xDwKOlzQ4zpdsf1vSucCx\nkvYAfgXsVF9/IvBCYCFwK7A7gO0bJb0XOLe+7j22b5zGeUVERKy0ppzYbf8c2GRE+++A541oN7D3\nUo51OHD4VM8lIiIiilSei4iIaEgSe0REREOS2CMiIhqSxB4REdGQJPaIiIiGJLFHREQ0JIk9IiKi\nIUnsERERDUlij4iIaMh0d3eLiIiIEQ7a+cVTet/bjvnGtOLmjj0iIqIhSewRERENSWKPiIhoSBJ7\nREREQ5LYIyIiGpLEHhER0ZAk9oiIiIYksUdERDQkiT0iIqIhqTwXERGTeuKRT1zh91yy2yU9nEks\nj9yxR0RENCSJPSIioiFJ7BEREQ1JYo+IiGhIEntERERDktgjIiIaksQeERHRkCT2iIiIhiSxR0RE\nNCSJPSIioiFJ7BEREQ1JrfiImFEPPv2iFX7Pb7betIcziWhDEnvc4yePfdwKv+dxl/+khzOJiIip\nSld8REREQ5q7Y99w/2+u8Ht+8cEX9XAmERER45c79oiIiIY0d8ceETFKevNiZZHEHhExFx241hTe\nc3P35xGzTrriIyIiGpLEHhER0ZAk9oiIiIZkjD1iDjnwwAPH8p6ImLtyxx4REdGQJPaIiIiGpCs+\nIpZw6mmPWuH3PG+bn/VwJhGxonLHHhER0ZAk9oiIiIakKz4iImaFbB3djdyxR0RENCSJPSIioiFJ\n7BEREQ1JYo+IiGhIJs/NAU888okr/J5LdrukhzOJiIjZLok9IqJDG+7/zRV+zy8++KIeziRWVrOm\nK17SdpJ+KmmhpP1n+nwiIiLmollxxy5pHnAI8HzgauBcSSfY/vHMntkyHLjWFN5zc/fnERERUc2K\nxA5sDiy0/XMASUcDOwCzO7FHRMScc8ibTlvh9+z9qW16OJN+zJau+PWBq4YeX13bIiIiYgXI9kyf\nA5J2Al5g+2/r49cCm9t+64TX7QXsVR8+BvjpCoZaD7hhmqe7MsYZZ6zEmf2xEmf2x0qc2R9rKnEe\nYXv+sl40W7rirwYeNvR4A+CaiS+yfRhw2FSDSDrP9mZTff/KGmecsRJn9sdKnNkfK3Fmf6w+48yW\nrvhzgQWSNpK0KvBq4IQZPqeIiIg5Z1bcsdu+S9JbgJOAecDhti+b4dOKiIiYc2ZFYgewfSJwYs9h\nptyNv5LHGWesxJn9sRJn9sdKnNkfq7c4s2LyXERERHRjtoyxR0RERAeS2GNGSXpCS3EihklabXna\nIrrUdFe8pHm27x5DnEcBV9u+XdJzgScBR9n+fY8x72f7T30df1wknQmsCnwO+FJf/2ZjjPOJEc03\nA+fZ/lofMYdirwKsYfsPfcaJ5SfpAttPWVZbB3G2BC6y/SdJuwJPAT5u+5ddxom5ofXEfiXwv8AR\nfdadl3QRsBmwIWVm/wnAY2y/sIdYzwQ+Q/kD/nBJmwBvtP3mrmONi6QFwBuAnYBzKP9fp8zFOJIO\nAx4L/E9tegVwGaVOw89t79txvC8BbwLuBs4H1gI+Yvs/Ojj2wcBS/0DY/vvpxhgRc1TCuxn4pe27\n5kocSQ+mVM/8AvA3gOpTawKfsv3Y6caYEO9iYBPKTcXngc8CL7f9nC7jjJOk+cCelL+r90z0tv2G\nDmO8fLLnbX+lq1g13j7AEcAtlL/jTwb2t31yp3EaT+z3p6yJ350y7HA4cHTXdzSDK3BJbwdus32w\npAttP7nLODXWD4FXAicMji/pUtuddzVL2gI4GHgc5W53HvAn22v2EGsesCPwCeAPlD+E7+7hF6vX\nOJJOA7YdJAdJ9wJOpmxwdIntjbuIMxTvItubSnoN8FTgncD5tp/UwbF3m+x520dON8aImGdT7jYv\npvzfPKF+/QDgTV39Aew7Tv23ez3lgv9cFiX2W4DP9fBzPfgb9K/Ar21/to+egRprS+BA4BGUhCvA\nth/ZcZwfAN+nXLDe0/Nq+7gOYxzBkhevg/8rd3kRUeP9yPYmkl4A7A38C+UGo9P/p1mz3K0Ptm8B\nPg18WtKzgS8DH5X0v8B7bS/sKNSdknYBdgNeUtvu3dGxl2D7KknDTX0NN/wX5cLofyh/oF4HPLrL\nAJKeRLnwehFwCvAS2xdIeihwFtBVwh1LHMpd2v0od3/Urx9q+25Jt3cUY9i9Jd2bcrHyX7bvnPCz\nMWV9JO7l8Atgj0EdC0kbA28H3kv5P+rqzqbXOPXf7khJr+gyEU3iFknvAl4LPKtewPb1N+izwH5M\nSLg9uK/td/Z4fIBLJzw2cD1wpu0re4g3+OV8ISWh/0hd/cIOaXrynKR5kl4q6Xjg48BBwCOBr9Pt\nmvndgWcA77d9paSNKF1wfbiqdsdb0qqS/hH4SU+xqBc/82zfbfsIYOuOQ/wXcAGwie29bV9Q414D\n/PMcjPNh4CJJR0j6HHAh8J+S7gd8p8M4A/9NSVL3A86Q9AgWXVR0QtJ8Sf8p6URJpw0+uowx5LHD\nxanqENqTBzs/zsE4G0haU8VnJF0gaduOYwDsDNwOvMH2bygXmNMejlmKm21/y/Zvbf9u8NFDnG9I\n6nw4c4I1Jnzcn3IT8y1Jr+4h3vmSTqYk9pNqr/Jfug7Self8z4HTgc/a/sGE5z7Rxxhh3yStR7lI\n+WvK1d/JwD59/GJJOqPG+QzwG+Ba4PW2N+kwxr62PzahbR/bH+8qxrhJeghlK2IB59SLh75ibTR8\nZ1Gv/h9t+4oOY5wMHAP8I2U8fzfg+j7upiQdA9wIHF2bdqZslvFayl3U0+ZYnLF0vdZYjwAW2P6O\npPtSLshv6SHOBynDcl+hXEwAMLhY7uD4t1DunEW5YL0duJNFXf6dDwWOOId1ge/0MMlxFWBTynyb\n30t6ALC+7Ys7jdN4Yt/K9pkT2ra0/X8dHf8SRk8uGvwATnuccybVPxTXUcbX96NMzDrE9s86jDFq\n1nDn8xPqxLkPABsD9xm0dz0uWGOtz6Lxx0GcM7qOU2ON+vc73/ZTO4xxvu2nSrp48DMt6Xt9TMyS\ntDrwZmAryu/RmcAngdsoXbN/nGNxLrb9JEkfB75r+/iefr73pOx8ua7tR9Wf90/Zfl6XcWqs00c0\n2/bc2bB8OfT0/yTgNcAjbb9H0sOBB9s+p8s4TY+xUyZITbziOnhE21S9uKPjLDeNdznVjvXO+Tbg\n32r8fSg9BtNS5yT8DbCRpOENf+4P9NGtdwRwAPBRynDC7iwa7+qMpA9R7v4uY1EXm4FOE7ukxwKP\nB9bS4jN712TowqUjd9bP10p6EWXnxQ06jgGA7T9ThswOGvF0J8l2nHFY1PW6EfCuvrpeKb0BmwM/\nBLB9haQH9hAH210Px40k6WXAabZvro/XBp5r+6tjiL0NcFMPh/4k5f9/G+A9lMmUxwGd9BANNJnY\nJT0DeCYwX9I/DD21JqULqRMeWiMq6UEs+s85x/Zvu4ozwX0YvZxqD0lbu9vlVLuxZBJ//Yi2qfgB\npWt/PRb/43oLZXZy11a3faok1f+3AyV9n5Lsu7QjZaljHxPlhj2GcmG5NosmbEL599uz41jvk7QW\n8DbKhfGalB6czo2YcQ1037MyrjjAHizqer21dr3u3nEMgNtt3zGYh6WyGqOX7tj6s3AA8Oza9D3g\nPYME3KEDbB8/eFC7rg8AOkvsS+l1XZdy8fq6ruIMeXpdvXAhgO2bVHY07VSTiZ3SdbwG5fu7/1D7\nHyhLxTol6VWUiSrfpdwFHizp7bb/t+tYlFnp23jRcqpDGVpO1UWAcdxN1+T6S8qkw3G4rY5vXaGy\nk+CvgT7uaH5OmY3ca2KvvTNfk/QM22f1FafOrl5g+xuUnqG+79bGNeN6XHFMGf55MeUO7X5036MC\n8D1J7wZWl/R8yjDD13uIA2XZ8KXAq+rj11J6xCZdEz4FoyZ3d52zJva6Gvid+yv+dWf9nTLcs1Y/\nk+dWhKRHeAyVlyT9CHj+4C69/md9p8tJZkOxfgpsPtQ9tRbwQ9uP7WpMqI6tb0QZk95/6KlbgIvd\nTQGPM21vNTRR5p6n6GGCjKSnUVYPrE1Z0rQW8GHbZ3cc5zhKoZBTWXxiUS8TNSX9FXAo8CDbT1BZ\n1vdS2+/rMMbpY+x+/aHtpzcU51Bq16vtx0laBzi5q8l5Q3FWofQObEv5HToJ+Ix7+AOvWjthWW0d\nxDkc+D1wCOVvxFuBdWy/vss446RSb2JnynDwkZQbzX+xfWyncVpM7JI+ZntfSV9nRHeU7Zd2HO8S\n208cerwK8KPhtg5j7UFZnvVdyi/ws4F/p6zRP9D227uOGctPSynq4p7WhEv6HmX99X+7p4JFkt5P\nuRA6BrjnTqarWdATYvU643oG4gwKx1w49P/zoz4u+sdF0lnA2wcTk+uwxn/a7rT3TWWJ6L9QVuZA\n6Zl8f49302NR58c8j/L3+1TbnS9XbrUr/vP183+OKd63JZ1ESa5Qrsh62VvepaLUiSxaTvXuoeVU\nnSZ19Vh5ri4nWSrbN043Ro0z8uJuKE6nF3l9JfBJ3Nf2OVq8xkVnpVerZ9bP7xlqM2UCUNcGd9Gb\n9RxrXHF67XqdZGUOAD2tzPk7SvGdtSh/g26kzL3pTP03a+5GRdLnbb8WuHxEW2eaTOy2z6+fvzem\neG+vM5MHS2cOG5700YPbKBPP7gM8WtKje1pO1WflufNZtFZ1IlMKCXVhLBd3ko61/aql/aHtcenj\nDSqbEA0SxyspPxudGVc3/DhjjfF7+gRwPPDA2vPxSrotiDT2lTm2LwI2kbRmfdz5pkMulRo7W7I5\nizx++EG9gOn8+2y1K37sV7F1VvzmNW5vs+Il/S2wD2W50UXAFsBZfawhlXSe7c20+PrlH9h+5rLe\nOxvV2aePpfwf/dT2HR0e+yG2r63zE5bQ11wPSY8EDqPcVd8EXAnsavsXHcZ4EGW456G2t1cpv/oM\n25/tMMautr8wYRXLPWx/ZC7FmRCz967XcRj3v52kg4AFlBuL4SGgTuvsj4NKud93A6sDt7LohuYO\nyo3gu7qM1+QdO2O+ih3zrPh9KMvqzra9df2j8W89xAG4tSbDiyR9mHIneL8uDizpsbYv1+hdtvoY\n63wR8CngZ5T/o40kvdH2t7o4vu3BXfLGE48p6U01dudcSqD+dR2PXMU9VBqjbHV7BPBP9fH/o4y3\nd5bYWfRzdf9JXzV34gCgUoDkVoZmqEt6uO1fdRxneBLqqpSVGV1v2DTZv10fd4jrUlbhDN+0mO72\ndRgb2x8APiDpA10n8VGavGMftzHPij/X9tNUtop9usse8J3PSK2xRlWe+6Q72DxH0mG299KYqlhJ\nuhx48eDca/f1N9399pk/AP7Z9mn18TspRTW27zLOULx/p8zu/319vA7wNtuddfcO/cwNTwDr5Wdu\nXCStO3EehyaU5+0ozqD3UJShs40ovUWPn/SN04+7I2X1zLt7OPYS1TtHtcVo9Xd0AYtXwOx0KLXJ\nO/YRS6k0/LnrpVSUO6Xhrvff0d8GO1erVGD6KnCKpJsoxRQ6Z/uX9SIF2532Ctjeq34e11jnbydc\nkPwc6GO45KWUzSveDmxH6frvdILeBNsP//F2KXjxQrodx/2TSmGVwTj+FnS80cxAHVr4OGWIyZSd\n9/Zz95uzfF3S9oPxYUmPo3T5drr98cSVMbWH6o1dxlhK3K9K2n/Zr5ySUdU7u6zoCYCk+1CW8D2e\nxZNgp1upjtPShlLpeNJmk4kvVzZpAAAeYklEQVTd9lb181i62xg9K76TLt6JbL+sfnlgvdtdC/h2\nlzFUplgfALyFcjG0iqS7gINtv2fSN694rPuwqGa3Kfsvf8r2bV3GAS6rqwmOrXF2As6tkx47G7ez\nfYOkl1J2cjsfeKX77RabJ2k110p3KjXQV+s4xj8AJwCPkvR/wHx6KPRUfYmybnnwc/5qyu9V12vO\n/52S3F9EqeJ3FKWGd69ctgrudA07gBYvK7wKZbJrpz93GlNFzyGfp8wefwFlRcZr6HEnyzEZy1Bq\nk4l9WL1CHiSNM21f2HWMcc2KV1kff/FgjXKPs/73BbYEnjbomqx3UodK2s/2RzuMdRSl8M3B9fEu\nlF/onTqMAeWK/zpgsHHJ9ZQxvJfQwbidliy0syplZv8rJfXRSzTwBeBUSUfU+G+gFL7oTE1Gz6Ek\nQFG6ku9cxtumSrY/P/T4CyqVAjtl+5sq+9ifTBkz3tEd7og3MCEBrkK5q72+6zgsXlb4LspWvjt0\nHGOsFT0puxTuJGkH20dK+hKl8M5cdpvt2yRRL8gvl/SYroM0PcYu6V8pCWLwR3tH4H/cYVWuGudD\nnrCF5ai2jmJ9EXhX15NvJsS4kDJn4IYJ7fMpVbM62/FII4p1jGqLpZO0PYtmXZ9su/M/fpKeCWzI\n4nXVj+ohzgcp1caOplyo7EzpgTikxpxWfQNJB7P4Bdg2lGGZX9Tjd1ohUKW2+cAg4R7XQ4/UqNj3\ncw/FXDS+ip7n2N5cZfvoN1O2jj7HPezIOC6SjqfsFbAv5WfvJuDetjvdd771xP4T4MmDX6LaTXmB\n7cd1HGfU1pn3LBHrONZplK6cc1h8CUhn47iapHLZZM9NMdbnKF3vZ9fHTwd2s/3mrmLU434YeB/w\nZ8rQxSbAvra/0GWcGmts27aOg6TPA4+ijAkO6qq76yRYY002ec3T/aOupVQGHAow7gJD01Z/3h5C\n6c27Q2VXt32B19t+aA/x5gPvYMmx764nvP4tZeezJ1FWZawB/KvtXlaYjFvtBVsL+LY7XHoL7XfF\n/4Lygze4Ol6NstypE5L+jnIl+UhJwzuS3R/oa4ZoX0vbhk32Q9bJD+DQbOF7A6+T9Kv6+BHAj7uI\nMcG2tt+hshXk1ZSenNMpXdmd0aJtW3/MUBKk421bh+L1Vh1wyGaUZXy93wXY3qjn4481cWvxTZSW\nMN0Lckn7UpYhLgRWU9n3/SOUIa6+Crx8kbLc8cXAmyi7QHY+vGD7M/XL79FdwaoZodGVNgebdq1B\nqd7XmSYT+1B32+2USVOn1MfPB87sMNSXKJPkltgsZbpdhktj+3t1GdoC29+RdF+6n7iyiaRR1aQG\nS3a6MO6KWfeun18IfNn2jVq8DGtXxrVt60Cf1QEHLgUeTMcV7UaRNHKrzK67/bXktq2DFTNdJ5Ar\nKf92gwvIXSg3HF0Nl+xF+Xm7UWXN/ELg2e54c6MJHuBS2nqfOs/neyp7FnRKYyiMNEbDlTYfTumC\nF2VTql9RlkF2psnEDpxXP59PKec48N0ug7jssHYz5Zd1LCTtSfllXpfSPbo+pfjJ87qKYbuPGa4T\nYyw2Rle7D/vYznLg6ypr2f8MvLl2J/YxzjmWbVuH2V4oaZ7tu4EjVNbSd2k94MeSzmHxDVP6WMY3\nPGP8PpSf6wsod6BdGte2rU+2/eyhx1+XdEaH68tvG9xE2P6VpP/Xc1IHGEycvLauKriGsnyra5+j\n/8JIYzHoiZL0KeAE2yfWx9uzaJObzjSZ2OfiONkK2JtSuvaHALavqElxTqpLww4CHkpZV/4IypKW\nTgt42N6/dpP/waUO9Z/oftYwlCpjF0kay7at9FgdcMiBHR9vqWy/dfixykYjn1/Ky6fjZndUdXAZ\n5kt65GAdvqSNKMsFu7KBpE8MPX7g8OOefu7eV/9f3kYZBlqTcpHUtfVsH6tSjhXbd0nq8yJsHJ5m\n+02DB7a/Jem9XQdpMrEPSFpA6SbfmMUneczl8Zrb6wQZACTdi37KOY7LeylFGr5j+8mStqbDHhBJ\n29g+bXid74Qu+K7LU55QP8bltZRlVG+h/HF9GPCKLgOMafhnaW6lVOnq2umS/oOet22l/J98V9Kg\nwM6GdFugZuLuZ+d3eOwlqGxassD2Nyi9lX0WmBpbYaQxukHSP1OGZgzsSilo1qmmEzulG+cA4KOU\nH8DdYeRuYnPJ9yS9G1hd0vMpk/e+voz3zGZ32v6dpFUkrWL79Hpn3ZXnAKex+Drfgc7rTo+rt0i1\n3vjQkMZt9DSxchzDP0OxhrfZnUeZFHhs13EY07attr9dbzAGpYsv73L+xbh7J2tv10spf1P7Ns7C\nSOOyCyUnDYaIz6CHodzWl7udb/upki5xLe0o6fu2n9VxnJcDHwIeSLlw6Kt07aBIzR7AtjXOScBn\nxjFjuQ+SvkOZcPYByljubyndVXNqBzmNedvW4SWWko6z3eld+oRYF1GHf7yoVvw9v1Mdx3rO0MO7\ngF/avrrrOONSezf+AXiE7T1rkn9MveOdk1S2n12LMt49vOS2k94Olcp8V9n+Te2RfCOlF+rHlOVu\nvUxMHqc6lPEX97NpU/N37LfVRHiFSvWqX1OSb9c+DLzE49mOcQfgKNufHkOscdiBcre5H6Vk5FqU\n8pGd0FK2mBxwd1tN7lM/j2u2/3DPU99DS2Mb/qnd/g9i0SS6TqvBafzbth5B6R5/Rn18NWUFw5xN\n7JSysrD472mXvR3/zaIJZc+kTJ57K7ApZYviOXvXXi9aDqdW7pN0M/AG250OobSe2PcF7gv8PWUs\ndxvKmsuuXTempA5lQ5GPqVRjOho4yfZdY4rduQmVsfroVhyUvnwMJVkMxr9fQodry123bR1HRa5B\nyKV83YexDf+o/y2Qx7316KNs7yxpFwDbf1ZP6yzHaA9P2JRHpeR0V+YN3ZXvTCnRfRxwXO09mss+\nC7zZ9vcBJG1Fufjrtkdvjvbgziq1KMSDKTuuDU/E6WXfYJUa19tTfui3Ak6x/bd9xOqLlqytfs9T\n9DCMIelk4BWDri9J96eUF96uo+OP+/u5m9INKmB1yiSzXuJNGP6BcjH5mUneMp1YY9sCeUTsfW1/\nrONj/oAyF+H/bD9FZbvgL9vevOM484E9WbLsb+c7oWl0pc3zbXdSEEfSpcCmdRb85cBerpUb1XHl\ny3GT9H+2t1xW23Q1ecc+YQLOEnpYf7sm5Q/rtkNtnU/MuufA9p2SvlVjrE7pzp5Tid3j23lv4OEs\nXjXvDsofwU6M+/sZR60BSTsAG9g+BPh0nUQ3H3iqpN93eBc9bJxbIE/0D0CniZ0yUerbwMNU9nnY\nEnh9xzEAvkbZGfE79LQuX2UnsscDa2nx3eTWpNsaFF+m9BLdQKk7Mbi7fTRzf1b8OZL+m/I9DvZC\n+K7KZmXdzVNo8Y59wgScJbi/XdF6J2k7SqWxrSndlcdQNv6YU93xkta0/QeNLrU47c0+RsT7J+BV\nlNmopmwLeoztD3QZpyV1JvKrbV9VH19EGc5aAzjCdh+z4v+D0i05vAXyxe5hQ6URsa+y/bAOjydK\n4ZZbKUs6Rdmu84ZJ3zi1WBfZ3rTr406IsQNloutLWXxJ5y3A0bY7K4xUl7Y9hPK37U+17a+ANXpY\nkjg2Klttw6Ibz+FhGbujevtNJvZxk7QBpVDDltTtYYF9+pjNK+loytj6t7pcNjNukr5h+8Uqm34M\nSi0OuI9aA/WqeLAi4gz3sIVvSySda/tpQ4//y/Zb6tdn296ip7jDWyCf4R62QF5K3F/ZfnjHx+ys\ni3oZcd4H/MC1olnPsZ5h+6y+47RkaLLm4O+cKfX1z3TdGrvTeC0m9hlYenQKpW78oELWrsBrbD+/\nyzhD8YZnDZ8zoesyohOSFtoeWXde0s9sP6rjePMo4/edl9gcijHZXIjVbXc6PCnpEOBzts/t8rgj\n4txCmRh4B4tKvva15HZsOyW2Qotv3zuwLvAC4EDbR3car9HE/hDb16pUy1pC1zOXR3WD9dU1Jmkn\n4D9ZNGv4WUCXs4bHTtI6lOpiw9UB5+w2p62oY8Lfnbi0UtIbgefa7r6wRtkN7bUu+zDMeZJ+DPwV\n8EsWTXZ01zcX4zT426ayU+KOlKWqp49jgmNr6lDkdyZORpyuJifPzcDSoxsk7cqiccFd6KFMYPXP\nlAIui80aBuZkYlfZc3kfyljkRZSxyLPouALYuGg826iOy37AVyX9DWUjFihbga5G+YPeh9uAS2ov\n2HDxk75q7fdt+3EFUqkIN9hw5rs9FsEZ106Jzav/dp3/4zWZ2AfG+Ef2DZTtMz9K6eb7QW3rw0zO\nGu7DPpRhhbNtb11n3o5jz/m+jGMb1bGoP2fPlLQNizbl+abt03oM+836AaMnGM0p47q5kPRByu/R\nF2vTPpK2sr3/JG+bqnHtlNi8+rt1U+fHbbErfkDSeYz4I2v7nyZ94yw2k7OG+zCYoFVnXD/d9u19\nDGOM6yJP0nm2N5N08aC7VdIPPMdK5I7bhKV1qGwRO5+S3N9p+39m8vxmO0kXU9Z+/6U+ngdc2FeX\nfx0+G+yUeF9gTdu/6SNWC5Yy32tdypa3r7N9eZfxmr5jh373qpb0DtsflnQwoyfpddZ9WNdwPsj2\n2yfMGj6LRVfpc9HVktamFPc5RdJNlB/2ro3rTnoc26i26B2U/5+BVSnd/mtQKnMlsS/b2sBgmeha\nXR9c498psSUTS00b+J0Xr7zZmdYTe99/ZAdlZM/r8JhL8zHg3XBPRbuvAEjarD43aveyWc/2y+qX\nB9Y1nmtRZtr2Eau3i7who7ZRffmk7wiAVQfr5aszay2DGyXlwmjZPgBcWH+HRBlrf1fHMca6U2JL\nxjjfC2i/K/4RwHWUq//9KEnjk7YXdhxnp4ldhaPaphljqaUU1dNOW+OgUo73mC6LWywlzhmUjSU+\nA/yGcpH3+q5n8krax/bHl9UWixv30rpx0Xh3fnwIZZxdlJ340jW+kmo9sb8MOLHvQi4aXTt5ibZp\nxpjsD99Sn5vtJO1GmSfwV5SqcMfY7rwHZCkXeYfY/lnHcUb9LFzout1pjDYTS+vGQdJCetz5UdJj\nbV9eiy8toY8qbRq9M97NwPm25/omLU1oPbEfQVk21ctOaJK2pyz5eBWltOvAmsDG7nCjB0lfBk4b\n8YdvD2Bb2zt3FWsm1PWcr6CMsz7c9oKOj9/rnbTK7l1/Q5n78P2hp+4P3N1n0ZUWSHogizZRWmJp\nne3rZurcpkM9bPAx4fiH2d5Li0qVDrM7KlE6IeaXKPNUBjv8vQg4F3gsZWOlD3cdM1ZM04kdQD3u\nhCZpE8oewe8B/nXoqVsoBRs6W8ZQq80dT6ksNdi7dzPKHejL5nq3m6TNKf9HOwI/tt3pnIG+76Rr\nj8BGlLHO4SVGt1BWLcypWv4zZcLSust6XlrXO41p50dJ97F927LaOop1EmWnxD/Wx2tQ6mi8jHLX\nvnHXMWPFNJ/Y4Z7kvh1lbfmzbK/X9fFt37nsV3YSa2tgMNbewh++D1Eml/0MOBb4iu3fd3j83EnH\njKm9hhPZHW+nOo7hwKHj/gTYxPYd9fFqwEW2H5dhp9mh6VnxWrQT2jbA6cBhwE49hNpQ0geAjVm8\nLGrnG5nYPp3yvbTiSuAZ7mHHq+oHlIly6wEHDbXfAlzcdbDGKs/FNNnevc/jS3owsD6wuqQns6iY\nz5rAfXsK+yXgbElfq49fAny5rl74cU8xYwU0fceushPal4Fv18InWwG72N674zhnUvZd/ijlh3x3\nyr/tqML/MYEaqhXfYlGkmDr1vPNjnXz6esrP2vCk01som8/0sgRN0lNZVEvjzD4mvMbUNZ3YASRt\nSqndvjPl7vArtg/uOMb5tp86vOxM0vdtP2tZ713ZaSm14rue9JPKczETNKadHyW9wvZxXR5zGfG2\nAhbYPkKlpOwa7mH70ZiaJrviJf0V5a5psBnLMZSLmK17CnmbpFWAKyS9Bfg1Zd1qLNu4asWn8lzM\nhPm2h8fZPydp366D2D5O0osoEw+He77e03UslS1INwMeQ6kKeG/gC5ReiZgF5vLmIZO5HHgeZf3o\nVvUO/e4e4+1LGc/6e8oSnV0piSOW7bbBzF1Jq9WayY/pI1AtTDTP9t31j20fF3rDlef+RKk894oe\n4sTccIOkXSXNqx+70sPOj5I+RemVfCule3wnYOS21R14GfBS6u57tq+hTEaNWaLJO3YWrYc+XdK3\nKWvY+9whakPb5wJ/pIyvD/ZN/2GPMVsxrlrxY7mTtv3L2jWJ7bm8S110Y1w7Pz7T9pPqENC/STqI\n/kq83mHbkgyQkr+zT9Nj7PUHbkdKl/w2wJHA8bZP7jjO2JaatEzSc6i14gdLaTo8dq/lhSWJMoHy\nLZSLyFWAu4CD++gOjRgm6Ye2ny7pbMry0d8Bl3Zd6KnG+kfKZNfnU+o2vIGyL/snuo4VU9N0Yh9W\nK5vtBOzc1cSscVaei+kbupO+vodj70f5WdhrMIlI0iOBQykXKh/tOmbMXhrjzo813r9QJoc+Dzik\nxvyM7X/pMs5QvOcD21IuYk+yfUofcWJqVprE3odxVp6LqRnXnbSkC4HnT1yPXy8mTk7RjpWLpJfY\n/npdjrYE20f2GHs14D62b+4rxoR484BX257L20c3pdUx9rGw/SPgR5K+mJKhs9a+lNm6T5t4Jy1p\nvw7vpO89qsiO7etr5cNYidge1FG/1SN2fuw6nqSfAf9h+1Mum17dLukbtifuAz6dGGsCe1MK4pwA\nnFIfv52yVDWJfZbIHfs0SDrW9qskXcLi3W2DrRmfNEOnFtW47qQnm1OR+RYrr3HNv5F0OfAj4Fbg\njbbv6Lq8a600dxNwFqXLfx3KnJV9nF3dZpXcsU/PPvVzZ1fFK5sxFI4Z1530JpL+MKJdDK0rjpXD\n0Pyb9SUNTypbkzIU1LVbbe8s6R3A9yW9ihFj+9P0yKECXJ8BbqDsxHhLx3FimpLYp+d+kra0/X/D\njZKeRT9LtlrUd+GYyWbXdzbz3va8ro4VTbiGUuL1pSzajRHK/Jv9eogngDph73zgJGDdjmPcs9GV\n7bslXZmkPjslsU/Px4B3j2j/c32u061HW2V7oaR5tu8GjpD0gw4PnzvpGLuh+TfHU3qg7oZ7Jpqt\n1kPIeybv2j5V0guAkRP3pmH4d0mUjWf+wKKhx2x0NEsksU/PhraX2CHM9nmSNhz/6cxJvRaOyZ10\nzLCTgb+mFK8CWL22dbJ3gKTH1mqNv5Y0cdz+G13EGMjv0tyRxD49k93xrT62s5jbhkuw7kcpwfry\nGT2jiO7cx/YgqWP7j5K63E71bcCeLL4l8T3hKIW5YiWTxD4950ra0/anhxsl7cHi42qxdDva/jhw\nG3XzF0n7AB+f0bOK6MafJD3F9gVwz3anf+7q4Lb3rJ/72uAq5qAsd5sGSQ8CjqdMwhok8s0os7tf\nZvs3M3Vuc8VSlgN1ukwnYqZIehplr4rBZNqHUKpfdnLhL2nS3q2+9mOP2S2JvQOStgaeUB9eZvu0\nmTyfuUDSLsDfAFsB3x966v7A3bb/ekZOLKJjdVnlYyiTzC63fecy3rIixz5ikqdtu48NZ2KWS2KP\nGVE3ZdmIsonE/kNP3QJcnEp+0YI6nv4PwCNs7ylpAfAY251ObIsYlsQeEdETScdQhuleZ/sJklYH\nzrK9aQ+xXgQ8nqFJvdlZcOW0ykyfQKzcJG0h6VxJf5R0h6S7l7LuPGIuepTtD1OLu9j+M7WYTJck\nfQrYGXhrPf5OwCO6jhNzQxJ7zLT/AnYBrqAsEfxbSonZiBbcUe/SDSDpUcDtPcR5pu3XATfZ/jfg\nGZSlo7ESynK3mHE9V56LmEkHAN8GHibpi5SdBl/fQ5zBErpbJT0U+B1lDkushJLYY6b1WnkuYibZ\nPkXSBcAWlC7yfUZtStSBb0haG/gP4AJKD8FneogTc0Amz8WMqrPjr6Os/d8PWAv4pO2FM3piEdMw\norzrYgYFa3qKvRql4t3NfcWI2S2JPWZc3Rsd29fP9LlEdEHS6ZM8bdudlnqtm8u8CNiQoZ5Y2x/p\nMk7MDemKjxkhSZTxx7dQuihXkXQXcHCW6MRcNwMlXr9OKct8CfCXMceOWSaJPWbKvpSJRE+zfSWA\npEcCh0raz/ZHZ/TsIqZhBkq9bmD7SR0fM+aodMXHjJB0IfD8iROJarf8yakVH3PZuEu9SvoQcKrt\nk7s8bsxNuWOPmXLvUbODbV9fa2tHzFm2dx9zyLOB4yWtQimGo3IaXnPM5xGzQArUxEy5Y4rPRcwZ\nktaS9BFJ59WPgySt1UOogyhFae5re03b909SX3mlKz5mhKS7gT+NeoqyVCd37THnSToOuBQ4sja9\nFtjE9qRj8FOIcxKwve1MnIsk9oiIvki6aOKGL6PaOojzOeCRwLcYKlmb5W4rp4yxR0T058+StrJ9\nJoCkLVlU/rVLV9aPVetHrMSS2CMi+vN3wJFD4+o30XGt+FqcZg3bb+/yuDF3pSs+IqJnktYEsN3L\nlsSSTrX9vD6OHXNPZsVHRPRE0r9LWtv2H2z/QdI6kt7XQ6iLJJ0g6bWSXj746CFOzAG5Y4+I6Imk\nCycWW5J0ge1JN4mZQpxRBXE6L4QTc0PG2CMi+jNP0mq2bweQtDqwWtdBZqAgTsxi6YqPiOjPF4BT\nJe0h6Q3AKSxa094ZSRtIOl7SbyVdJ+k4SRt0HSfmhnTFR0T0SNJ2wF9Tii+dbPukHmKcAnwJ+Hxt\n2hV4je3ndx0rZr8k9oiIOW5chXBibkhXfETE3HeDpF0lzasfuwK/m+mTipmRO/aIiDlO0sOB/6Js\nBGPgB8A+tn85oycWMyKJPSKiJ5K2Ac62fetMn0usPJLYIyJ6IukoYAtKt/j368eZtm/q6Pj/OsnT\ntv3eLuLE3JLEHhHRM0kPBV4J/CPwUNud1BCR9LYRzfcD9gAeYHuNLuLE3JLEHhHRkzqJ7VnAE4Eb\ngDOB79s+q4dY9wf2oST1Y4GDbP+26zgx+yWxR0T0RNINwM+ATwGn2/5FDzHWBf4BeA2l+M3Hu+rq\nj7kpiT0iokeSHg88G9gKWAD81PZrOzr2fwAvBw4DDrH9xy6OG3NbEntERE/qdq1bAs+hdMmvR5kl\nv1tHx/8LcDtwF2WZ2z1PUSbPrdlFnJhbktgjInoi6WLKuPqZwBm2r57hU4qVQBJ7RETPJN3P9p9m\n+jxi5ZCSshERPZH0DEk/Bn5SH28i6ZMzfFrRuCT2iIj+fAx4AbVuu+0fUSbSRfQmiT0ioke2r5rQ\ndPeMnEisNDqpfhQRESNdJemZgCWtCvw9tVs+oi+ZPBcR0RNJ6wEfB/6asgTtZMqua9lSNXqTxB4R\nEdGQdMVHRHQsu67FTMode0REx7LrWsykJPaIiB5l17UYt3TFR0T0YMSua0/JrmsxDknsEREdm7Dr\n2hOz61qMU7riIyI6ll3XYiYlsUdERDQkJWUjIiIaksQeERHRkCT2iIiIhiSxR8whkh4g6aL68RtJ\nvx56vOoKHusISY/p4JweJOkcSRfWDU+mcowvSLqyfh/nS3p6bZekd0j6qaRL6/Ovme45R7Qsy90i\n5pC6ecimAJIOBP5o+z+neKzdOzqt5wOX2N5jed8gaZ7tiduX7mf7q5JeCBwKPAXYG9ga2Mz2LZLW\nBl7a0XlHNCl37BGNqHe2l9aPt9a2R0u6TNLnJV0i6VhJq9fnzpQ0uEh4kaQLJP1I0skrEHMz4N+B\nlw56DSTtWmNdKunf6+vuJen3kt4n6Rxg80kOewbw6Pr1u4E32b4FwPbvbR+1gv80ESuVJPaIBkja\nnFLhbHPgGcCbJT2pPr0xcIjtJwK3AW+c8N4HU+6QX2Z7E+DVyxvX9nnAe4Av2t4UeCDwPspd9pOB\nLSW9uL58LeAC25vbPmuSw74EuETSOsC9bf9yec8nIpLYI1rxLOA427fWu9uvAlvV5660fXb9+gtD\n7QPPAE4fJFDbN07jPJ4OnGb7Btt3Al8Cnl2fuwM4fpL3flTSRcDuwJ6UYi4RsYIyxh7RhsmS4MQq\nVBMfa0Tb4i+QjgKeBPzK9mRj3JOdx589eUWs/Wx/dULcOyU93PavJju/iFgkd+wRbTgDeJmk1SWt\nAewAfL8+t5Gkp9WvdwHOnPDe/wO2kfQIuGfzksXYfp3tTZeR1AHOBraus/fvRenW/97UviUAPgh8\nsu6QhqS1Je05jeNFNC937BENsH2OpC8D59amQ21fIunRwGXAnpI+C1xO2Zhk+L3XSfo74GuSBFwD\nbD/F87ha0r8C36XcvX/d9jdrkp+Kgyn7mJ8v6Q7gTuDDUzxWxEohteIjGlYT+//WiW0RsRJIV3xE\nRERDcsceERHRkNyxR0RENCSJPSIioiFJ7BEREQ1JYo+IiGhIEntERERDktgjIiIa8v8BWB5Q+OGU\nnkYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plotting the requests and their quantities\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "fig = plt.figure(figsize=(8,6))\n",
    "df.groupby('Topic - for PC').Title.count().plot.bar(ylim=0)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6917"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating dictionary of the cases the their topics\n",
    "cases_dict=dict()\n",
    "count=0\n",
    "for i in df['Case Number']:\n",
    "    if i not in cases_dict.keys():\n",
    "        cases_dict[(df['Case Number'][count])]=df['Topic - for PC'][count]\n",
    "    count+=1\n",
    "len(cases_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(10087, 'Data display'),\n",
       " (10099, 'Data display'),\n",
       " (11750, 'Data display'),\n",
       " (12439, 'Data display'),\n",
       " (12644, 'Data availability'),\n",
       " (12665, 'Updates'),\n",
       " (13399, 'Updates'),\n",
       " (13415, 'General'),\n",
       " (13418, 'Data linking'),\n",
       " (13420, 'Data display'),\n",
       " (13489, 'Facets'),\n",
       " (14435, 'Data display'),\n",
       " (15053, 'Data'),\n",
       " (15247, 'Availability'),\n",
       " (15758, 'Normalization Rules'),\n",
       " (15824, 'Facets'),\n",
       " (16117, 'Linking'),\n",
       " (16118, 'Data display'),\n",
       " (17753, 'Data availability'),\n",
       " (18208, 'Data linking')]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(iter(cases_dict.items()))[0:20]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Topic</th>\n",
       "      <th>Number of cases</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Data display</td>\n",
       "      <td>850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Data availability</td>\n",
       "      <td>650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Updates</td>\n",
       "      <td>357</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>General</td>\n",
       "      <td>158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Data linking</td>\n",
       "      <td>3120</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Facets</td>\n",
       "      <td>162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Data</td>\n",
       "      <td>232</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Availability</td>\n",
       "      <td>184</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>191</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Linking</td>\n",
       "      <td>182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Grouping</td>\n",
       "      <td>266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Search</td>\n",
       "      <td>97</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>UI</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Registration</td>\n",
       "      <td>352</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>New collection request</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Citation tools</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Coverage</td>\n",
       "      <td>39</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     Topic  Number of cases\n",
       "0             Data display              850\n",
       "1        Data availability              650\n",
       "2                  Updates              357\n",
       "3                  General              158\n",
       "4             Data linking             3120\n",
       "5                   Facets              162\n",
       "6                     Data              232\n",
       "7             Availability              184\n",
       "8      Normalization Rules              191\n",
       "9                  Linking              182\n",
       "10                Grouping              266\n",
       "11                  Search               97\n",
       "12                      UI               13\n",
       "13            Registration              352\n",
       "14  New collection request               52\n",
       "15          Citation tools               12\n",
       "16                Coverage               39"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "casess_counter=dict(Counter(cases_dict.values()))\n",
    "df_1=pd.DataFrame(list(casess_counter.items()),\n",
    "                      columns=['Topic','Number of cases'])\n",
    "\n",
    "\n",
    "df_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\scipy\\stats\\stats.py:1633: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.\n",
      "  return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWkAAAEFCAYAAAAhTRZvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xt4FPW9P/D37MxecyEJCSCXIAgI\n3uCgtVVBvAHqUUG8FC9wfNB6qR5EakUoKhbkYtWjYrV4aZ/zU59jqVhrbRVBq4gFilYgICACAgkB\nciW3vc3M9/fH7szukt1kE0mY7Lxfz+OTZDO7fHfM884nn/l+vyMJIQSIiMiSHCd6AERElBpDmojI\nwhjSREQWxpAmIrIwhjQRkYUxpImILIwhbSOlpaUYNmwYJkyYgAkTJuDqq6/G5MmT8fe//9085rnn\nnsO7777b4uu88MILWL16ddLvxT//1FNPRXV1dZvGuGXLFjz66KMAgJKSEkyfPr1Nz28PTdNwzz33\nYPz48XjjjTdSjmfDhg246qqrOnw87fXhhx9iypQpJ3oYdJwpJ3oA1Lk8Hg/+8pe/mF+XlZXhtttu\ngyzLGD9+PO6///5WX2PDhg0YNGhQ0u+l8/yWfPfddzh8+DAA4Mwzz8Tzzz//g14vHYcPH8batWux\nadMmyLKccjxEJwJD2ub69OmD6dOn47XXXsP48ePx8MMPY/Dgwbj99tvx/PPPY9WqVXA6ncjPz8ei\nRYuwatUqbN26FU8++SRkWcbHH3+M2tpaHDhwABdddBGqqqrM5wPAs88+i5KSEui6jhkzZuDiiy/G\nO++8g5UrV2LZsmUAYH49b948PP/886ivr8fs2bMxceJEzJ8/H++//z7q6+vx+OOPY8eOHZAkCaNH\nj8bMmTOhKArOPPNM3Hnnnfjiiy9w5MgR3HHHHbj55pubvdcvv/wSTz75JPx+P5xOJ2bMmIGRI0fi\njjvugKqqmDRpEpYuXYri4mIAQHl5ebPxNDU14YEHHsCePXsQDAaxYMECnHPOOQiFQnjqqaewceNG\naJqG0047DXPnzkV2drb579fU1OCSSy7BF198AZ/Ph0cffRR79uwxq/dx48bhpZdegq7r+PWvf43a\n2lpIkoRp06Zh4sSJ2LBhA5544gn4fD40NjZixYoVeOmll/DXv/4VeXl56N+/f8J7Xbx4MXRdBwDc\nddddGD9+fMf8EFHHEmQbBw4cECNGjGj2+LfffiuGDx8uhBBi1qxZ4tVXXxUHDx4UI0eOFMFgUAgh\nxGuvvSZWrVolhBDi1ltvFR988IF5/H/913+Zr2U8XwghhgwZIpYtWyaEEGLnzp3i3HPPFVVVVWLF\nihXizjvvNJ8T/3X85+vXrxf/+Z//KYQQ4qGHHhLz588Xuq6LYDAopk2bZr72kCFDxOuvvy6EEKKk\npEScccYZIhAIJLzH6upqcd5554lNmzaZ7/ncc88V+/fvT3leko1n2LBh5mv84Q9/EFOnThVCCLF0\n6VKxePFioeu6EEKIp59+Wjz22GPNXm/KlCnik08+EUIIMW7cOHH++eeLhoYGsWvXLnHFFVeIcDgs\nLr30UrFy5UohhBCHDh0So0ePFv/+97/F+vXrxdChQ0VpaakQQohVq1aJK6+8UtTX14twOCzuvPNO\nceuttwohhJg6dap4//33hRBCbN++XcybNy/p+yPr67Ce9ObNm9vVH3vnnXcwZcoUTJkyBTfeeCPO\nPPNM1NXVdcAIySBJEjweT8JjPXv2xNChQ3HttddiyZIlGDZsGC677LKkzz/77LNTvvZNN90EABgy\nZAhOOeUUfP311+0a45o1a3DrrbdCkiS4XC5MnjwZa9asMb9/6aWXAgBOP/10hEIhNDU1JTx/y5Yt\nKC4uxvDhwwEAgwcPxsiRI/Gvf/2rTePo16+f+RpDhw41e+6ffvopPvnkE0ycOBETJkzA6tWrsXv3\n7mbPHzt2LNasWYPdu3ejZ8+eGDlyJDZu3IiPP/4Y48aNw/fff49gMIhx48YBiPx/GDduHD7//HMA\nwEknnYQ+ffoAANatW4exY8ciOzsbiqLguuuuM/+dK664Ar/+9a/xi1/8Atu2bcPMmTPb9D7JOjqk\n3fHKK6/gvffeg9frbfNzJ02ahEmTJgEAHn/8cVx33XXIzc093kOkOCUlJRgyZEjCYw6HA2+88QZK\nSkqwbt06LFy4EKNHj8ZDDz3U7Pk+ny/lazscsTpA13UoigJJkiDitowJh8OtjlHXdUiSlPC1qqrm\n1263GwDMY8QxW9JompbwfOOY+NdIh9PpND+Pfx+6rmPOnDkYM2YMAKCxsRHBYLDZ88eOHYtbbrkF\nJ598Mi644ALk5uZi7dq1KCkpweOPP97qOI891/HvM76fPnnyZFx88cX44osv8Pnnn+OFF17Ahx9+\naJ4n6jo6pJIuLi7G0qVLza937txpVsf//d//jfr6+lZfo6SkBN999x1++tOfdsQQKWrv3r148cUX\nMW3atITHd+zYgauuugqnnHIK7rrrLtx2220oKSkBEAmDdMPtz3/+MwBg27Zt2L9/P4YPH46CggLs\n2rULwWAQ4XAYK1euNI9P9dqjRo3CG2+8ASEEQqEQli9fjvPPPz/t9zlixAjs2bMHW7ZsAQDs2rUL\nGzduxLnnntvi89J9r6NGjcKbb76JUCgEXdfxyCOP4Jlnnml2XK9evZCfn4+33noLF1xwAUaNGoWP\nPvoItbW1GDp0KAYOHAhFUfDRRx8BiFzUXLlyZdL3euGFF+LDDz9EXV0ddF1PuCA8efJkbN++HZMm\nTcL8+fNRV1eHioqKVt8HWU+HVNLjx49HaWmp+fUjjzyChQsXYtCgQfjTn/6EV199FWeffTaefPLJ\nhOfNmDHD/JN62bJluPfeeztieLYWCAQwYcIEAJEq1+12Y+bMmbjooosSjhs6dCiuuOIKXHfddfD5\nfPB4PJg7dy4A4JJLLsEzzzyTVgV84MABTJw4EZIk4ZlnnkFeXh4uuOAC/OhHP8IVV1yBoqIi/PjH\nP8bOnTsBRML0t7/9Le67776EdtncuXOxYMECXH311QiHwxg9ejTuvvvutN93QUEBnnvuOcyfPx+B\nQACSJGHRokUYMGBAws/qsVKN51g///nPsWTJElx77bXQNA3Dhg3Dww8/nPTYsWPH4ve//z1OO+00\nOBwOeDwe8+fe6XTixRdfxIIFC7B06VJomoZ7770XP/nJT7Bhw4aE1xkzZgx27txp/rU5dOhQ1NTU\nAAAefPBBLFy4EM8++ywkScJ9992Hvn37pn2+yDokcezfhcdJaWkpZs6cieXLl+Pss8/GaaedBiDy\np+2AAQOwaNGilM+tq6trNn+XiMiOOmUK3oABA7BkyRL07t0bX331Vat/dm3cuLFNf8oSEWWqTgnp\nefPmYdasWdA0DQDwxBNPtHj83r17+acZERE6sN1BREQ/HPfuICKysOPe7qioaH16XSr5+T7U1DS1\nfqAN8FxE8DzE8FzEZOK5KCrKSfq4pSppRZFbP8gmeC4ieB5ieC5i7HQuLBXSRESUiCFNRGRhDGki\nIgtL68LhxIkTkZMTaWr37du3xdWCRER0/LQa0sZOXq+//nqHD4aIiBK1uphl8+bNeOihh9CnTx+o\nqoqZM2dixIgRKY9XVc1WV16JiDpSqyG9c+dObN68GTfccAO+//57/OxnP8OHH34IRUlehP+QedJF\nRTk/6PmZhOciguchhuciJhPPRap50q22OwYMGID+/ftDkiQMGDAAeXl5qKiowEknnXTcB5mKLgQC\nQQ0+D2/JSET20ursjrfffhuLFy8GENmAvKGhAUVFRR0+sHgrPtuNX7z4BfzBtt1Fg4ioq2s1pK+/\n/nrU19fjpptuwgMPPICFCxembHV0lIoaP4IhDY3+1jeZJyLKJK2mrcvlwtNPP90ZY0lJ1SJtc1Xn\nhn1EZC9dYjFLWNMBAGr0IxGRXXSJkFbVSDhrGitpIrKXrhHSRiWts5ImInvpEiFttDtYSROR3XSJ\nkDYvHLInTUQ20zVC2uhJc3YHEdlMlwhpzu4gIrvqEiGtsidNRDbVpUKalTQR2U2XCOmwGqmg2ZMm\nIruxfEgLIVhJE5FtWT6k46tnlT1pIrIZy4d0WI1Vz2x3EJHdWD6k41scGtsdRGQzXSCk49sdDGki\nshfLh3Q4LpjZkyYiu7F8SKvsSRORjVk/pBMqabY7iMheLB/S4YQLh6ykicheLB/S8e0ObvpPRHZj\n/ZCOq55ZSROR3Vg+pBNmd7CSJiKbsXxIJ7Q7WEkTkc1YP6S54pCIbMzyIZ0wu4PzpInIZiwf0lwW\nTkR2Zv2QZk+aiGzM+iGtx7c7WEkTkb1YP6RZSRORjVk+pMMJi1lYSRORvVg+pFVuVUpENmb9kFbZ\nkyYi+7J+SLOSJiIbs3xIG4tZFFliT5qIbCetkK6qqsKYMWOwe/fujh5PM0b17HEpXHFIRLbTakiH\nw2E8+uij8Hg8nTGeZoyetNsps91BRLbTakgvWbIEkydPRo8ePTpjPM0Y7Q6PW+aycCKyHaWlb77z\nzjsoKCjA6NGj8fLLL6f1gvn5PiiK3O4BFRXlJHztcER+j+T4XCiraERhYTYkSWr363clx54Lu+J5\niOG5iLHLuWgxpFesWAFJkrBu3Tps374ds2bNwksvvYSioqKUz6mpaWr3YIqKclBRUZ/wWJM/BABw\nRHP50OE6KLLlr3f+YMnOhR3xPMTwXMRk4rlI9UunxZB+8803zc+nTJmCefPmtRjQHSGsCSiywwxm\nTRP4AYU6EVGXYvmSVNV0OBUJcrSU5i20iMhOWqyk473++usdOY6UVE2HIjsgRytpzvAgIjuxfCUd\nVvVouyNSSXNBCxHZieVDWtV0OGUHlOgsD5ULWojIRrpASAsoCitpIrIny4d0WNOhyBJkR2x2BxGR\nXVg+pFU10u6QZc7uICL7sXRI60JA0xPnSXN2BxHZiaVD2ug/K4rDnCfNnjQR2YmlQzqsRqpmZ9wU\nPM7uICI7sXRIq3Eb/seWhbOSJiL76BohrXDFIRHZk6VDOnbrrLieNNsdRGQjlg5p464sCT1ptjuI\nyEasHdLR1kbiFDyGNBHZh6VD2mx3KJK5mIUrDonITiwd0gntDmNZOHvSRGQj1g7p+AuH7EkTkQ1Z\nOqTjZ3ewJ01EdmTpkDYuHDoTloWz3UFE9mHtkFabrzjksnAishNLh3Q4SU+ay8KJyE4sHdJG/9mp\nxN0+i+0OIrIRa4e0mqSS5qb/RGQjlg7p5LM7WEkTkX1YOqTN2R2yBIWb/hORDVk8pJNsVcrZHURk\nI5YO6XCynjQraSKyEUuHtDm7Q+bsDiKypy4R0ooSf49DVtJEZB+WDmnjRrSKzK1KicieLB3S8e0O\nhyRBAnvSRGQvXSKkFcUBSZIgyw7O7iAiW7F0SMcvZol8lLhVKRHZiqVDOv7OLAAgOyTemYWIbMXa\nIa0JOCQJjuhqQ0V2cAoeEdmK0toBmqZh7ty52Lt3L2RZxqJFi1BcXNwZY0NY06Eokvm1Iku8cEhE\nttJqJf2Pf/wDAPDWW29h+vTpWLRoUYcPyqBqutnqABC5cMiQJiIbabWSvuyyy3DRRRcBAA4ePIjC\nwsKOHpNJVXXzoiHAnjQR2U+rIQ0AiqJg1qxZWLVqFZ5//vkWj83P90FR5HYPqKgox/xcB+B2yeZj\nHreCusZQwjGZzC7vszU8DzE8FzF2ORdphTQALFmyBA8++CBuvPFG/O1vf4PP50t6XE1NU7sHU1SU\ng4qKevPrYEiD163EHhMCYVVPOCZTHXsu7IrnIYbnIiYTz0WqXzqt9qTfffddLFu2DADg9Xqji0ra\nXym3harqcMqxC4eyg7M7iMheWq2kx40bh9mzZ+OWW26BqqqYM2cO3G53Z4wNqpbYk1ZkCboQ0EVk\nah4RUaZrNaR9Ph+ee+65zhhLM5EpeImzO4DIJksOhSFNRJnPsotZNF2HEEiYgmfcQovT8IjILiwb\n0mp0m1I5vidtVNKchkdENmHZkNaim/sbd2QBYG78z1WHRGQXlg1pY0tS+ZjZHQBvoUVE9mHZkDbu\nwCI7EvfuAHgLLSKyDwuHdOJe0kDi7A4iIjuwbkjrSSppzu4gIpuxbEjHetLxFw45u4OI7MWyIW20\nO+IraeMiIitpIrIL64Z0knaH8Tl70kRkF9YPaTl+dkd0Ch5ndxCRTVg3pM12R/OeNOdJE5FdWDak\njQuHisx2BxHZl2VDuqXFLBrbHURkE9YNab15u0M22x0MaSKyB+uGtJZs7w5jCh7bHURkD9YNaaMn\n7Wg+u4OLWYjILiwc0tF2h9x8q1K2O4jILqwb0kkuHHKDJSKyG8uGtMoNloiIrBvSybYqNT4PhLQT\nMiYios5m3ZBOUkn365kN2SFh+77qEzUsIqJOZf2QjpuCl+VxYki/POwtr0dNffBEDY2IqNNYN6ST\n7N0BACMGFwIANu+u7PQxERF1NuuGdJJ2BwCMGBQJ6U27GNJElPmsH9JyYkgX5XnRtygL33xfg0BI\nPRFDIyLqNJYNaTVFuwOItDxUTce2vbyASESZzbIhbSxYUY6ppAHgPwYXAWDLg4gyn3VDOkVPGgD6\n98pBrs+J7ftrOntYRESdysIh3XzvDoNDkpCf40Gjnz1pIsps1g3pJHt3xPO6ZQTDGnTuiEdEGcy6\nId1CuwMAPC4FAJeIE1Fms2xImxssJWl3AIDHLQMAp+ERUUazbEjHVhymaHdEK2l/kCFNRJlLaemb\n4XAYc+bMQVlZGUKhEO655x5ceumlnTIwLcndwuMZlbSf7Q4iymAthvR7772HvLw8/OY3v0FNTQ2u\nvfbazgvpFhazALFKOsBKmogyWIshffnll2P8+PHm17Isd/iADKmWhRu87mi7g5U0EWWwFkM6KysL\nANDQ0IDp06djxowZrb5gfr4PitL+MC8qygEASA4HHBLQs0du0uN6FEbGprgU8zmZJlPfV1vxPMTw\nXMTY5Vy0GNIAUF5ejnvvvRc333wzrr766lZfsKamqd2DKSrKQUVFPYBIG8PhcJhfHyscbXNUVDak\nPKYriz8XdsbzEMNzEZOJ5yLVL50WQ7qyshLTpk3Do48+ivPOO69DBpaKpuspWx0A4HXxwiERZb4W\np+D97ne/Q11dHV588UVMmTIFU6ZMQSAQ6JSBaZowbzybjMdtLGbhhUMiylwtVtJz587F3LlzO2ss\nCVRdpFzIAsRdOAyykiaizGXpxSypFrIAgMfFFYdElPmsG9K6aDGkYysOWUkTUeaydki30O5wOR2Q\nJMDPSpqIMph1Q1rTW7xwKEkSvC6FKw6JKKNZN6RbaXcAkT2l2e4gokxm2ZBWNdHiPGkgMg2PFw6J\nKJNZNqQ1XU+5uZLB61LgD2oQgndnIaLMZMmQ1oWAEKm3KTV43DJ0IRBW9U4aGRFR57JkSLd2f0OD\nOQ2PS8OJKENZM6RbuFN4PHNBC2d4EFGGsmhIp1lJm3tKM6SJKDNZM6TTbHcYlTSn4RFRprJkSKta\neu0Oo5Jmu4OIMpUlQ5rtDiKiCEuHdKtT8NjuIKIMZ82QbuVO4QYvN/4nogxnzZBOt93hMkKalTQR\nZSZrh3QaKw4BwM8Lh0SUoawZ0uYUvPQWs7AnTUSZypohbaw4THN2B3vSRJSpLBnSqpZmu8PFdgcR\nZTZLhrRRSSutLGaRHQ64nA5usEREGcuaIZ3msnAAvIUWEWU0a4Z0mlPwgMjdWVhJE1GmsmRIq2lu\nVQoAXpfMC4dElLEsGdJtane4FYTCutnHJiLKJNYM6ba0O4yN/9nyIKIMZM2QNrcqTa+SBjgNj4gy\nkyVDWjV2wWtlxSEQfwutlivp3WVH8V3p0R8+OCKiTmTJkNbSXMwCpL+n9Kvvf4NX3t/2wwdHRNSJ\nlBM9gGRiy8LTr6Rba3fUNoTgSKPHTURkJRYN6fQraZ/HCaDlTZbCqo5gWIMkAUIISBLDmoi6Bku3\nO5Q0Kl9ftN3R1EIl3RgIAwCEiAQ2EVFXYc2Q1tPbqhSI9aSbokGcTIM/9j1O1SOiriStkN68eTOm\nTJnS0WMxqW2YgufzpFFJx4d0mCFNRF1Hqz3pV155Be+99x68Xm9njAdA2xazGO0OfyB1SDf4Y98L\nspImoi6k1Uq6uLgYS5cu7YyxmGIXDlsv9NOqpAPx7Q4ueiGirqPVSnr8+PEoLS1N+wXz831QFLnd\nAyoqyoHTGXl+j8JsFBVlt3h8bp4PAKDqkecmFdfb9njdqY+zmK4yzo7G8xDDcxFjl3Nx3Kfg1dQ0\ntfu5RUU5qKioR2NTCABwtLYJTogWnyOEgCJLqK0PoKKiPukxh6saYp9X1KOie+e1btrLOBd2x/MQ\nw3MRk4nnItUvHWvP7kij3SFJEnxuBU0t9KQbObuDiLooa4a0lt6NaA1ej7OV2R1xFw45u4OIupC0\nQrpv375Yvnx5R4/FlO6NaA2tVdKJ86RTHxcMafh/K3fiSK0/zZESEXUsa1bSbZiCB0RmeKiajrCa\nvEpOnN2RupLesqcKn35dhvVbD7VhtEREHceiIZ3e3cIN5tLwFNV0uisOq44GAACNLVTlRESdyZoh\n3YbbZwGtz5VuDKjIih7T0mKWyqORNkdLS8yJiDqTNUNaF3BIUtq71bVUSQfDGsKqju7dPABa7kmz\nkiYiq7FoSOtpXzQEWq6kjel33XOjId3C7I7KukDK1yEiOhGsGdKaSLvVAbRcSRv96PwcNyQpdbtD\nCGFW0mx3EJFVWDKkVb1tIe1No5LO9jrhcckpLxw2BVXze2x3EJFVWDKkNU1Pa7WhweeO3J0lWQVs\nBG6W1wm3U05ZSVfWBszPW5pzTUTUmawZ0npkP450GT3pZLfQakiopJWUFw6r6mIhHQxr5p7WREQn\nkmVDul096WTtjkAspN0uOeWFw8poP9r4d3nxkIiswJohrelp3TrLYM7uSNLuMCrpLI8THqeMUFiH\nrjffWc+4aNi7MCv6WgxpIjrxrBnSumjbFLyWKuno5krZXgUeV2Sf6mSbLBkLWYp7RPavZkgTkRVY\nMqTbOrvDqTigyFLSW2iZlXS03QEkXxpeVReAS3GgR0HkJgKchkdEVmDJkG5ru8PcUzpFT1qSIncV\n97giFXeyi4dVRwPo3s1jLh/nNDwisgKLhnTb2h1AdE/pFJV0lscJhySlbHf4gyoaAyq6d/M0629r\nuo5D1e2/2wwR0Q9huZDWdQEBQGlDuwNAC5W0iixvZB61O3rvxMAxU/WM6XeF3bzI8kTnXEdfa82m\ng5jz8nrsO5RZt+ohoq7BciFtbFPalsUsAOBzywiriXtKCyHQ6A8jO1ode9zRkD6mkjam33XPdZsX\nIY12x/4jkfsjllY0oCXVdQHUNgTbNGYiotZYLqTVNm5TavCaFXAsgAMhDZouzEraE62kj111aEy/\nK+zmbdbuqK4LRj8G0JKn3tqEpSu2tGnMREStOe53C/+h2npXFkNsk6UwumW5AMT27TBaGKkuHBoh\nHblwGDnWqKSr6wPRj6mr5FBYw6HqJiiyA7qIbLNKRHQ8WK6SbsudwuMl2660IW61IQBzCt6xlfTh\nmsiFwe658RcOoyFtVtKpQ9roaauajrrGUJvGTUTUEuuFdHTPjPZcOASQMFd614GjAIBu2ZHKOtk8\n6bCq45t9NeiR50VetguK7IDL6UBTQIU/GPkPiFXUyRiV+LGfW8Hhmib8bd330EXzVZZEZH2WC2m1\nve2OYyrpuqYQ/rJ2L7xuGReceRIAmFPw4i8c7thfg2BIw4jBheadYLI8TjQGwgktjvhKWtX0hGq8\nMq5fXRkX0uu2Hjrh0/c+WL8fKz7bg+9Kj57QcRBR+1gupI1Kuu2zOxLbFO98tgdNQRUTRw00e9TG\nhcP4SvrrXZUAgP8YXBh7LY8Cf1BFTVz4xlfVb328C7OWrUMoGvbx1bOxvPxwTRNeef8brPhsd5ve\nx/FWXtWY8JGIuhbrhbTZk25/Jb23vA6fbz6IPoVZuHhkH/MY48JhMHrhUAiBTbsqkOVRMKhvN/O4\nLLeCpoBq9ppdSuQ0GZX19n01qGsM4WA0+CqTtDtKj0S+V1bReeEohMDe8rqE1oZRyZdXcUEOUVdk\nvZBu5xQ8Y+P/Q1VNWPbeNggAN48dAiWuIj+2J/39oXrUNoQwfFBhwjJ0n8cJAaCsMhKwA07KBQDU\n1AUQVnUcro5Uy0YAVx71w5jQYQS2EeBHavwIq52zN/WmXZWY/79fYv22QwAiqy3rmyIXT09024WI\n2sd6IR2tpJU27N0BxG6htbakHEdq/Ljq/P4Y1j8/4RjPMSGdrNUBxKry0uhCFqPKrq4P4lB1k1mp\nxkI6gMJuHmR7nWZIl0cDXhcChzspIHceqAUAfBu9YBofzAxpoq7JgiFt9KTbN7sDAC4c3hvXjh7Y\n7BhFjuyWZ+zdsWlXJRTZgdMHFCS+lhHS0RA+pXc0pOsCKItbeVhW2YiwquFoQwiF3bzo3s2DqroA\nhBBmFW4cd7w1+MP45Yv/xKovD5iP7T8cWbq+L/oxvg9dUdt5FT0RHT+WC+n2rjjM8TnRs8CHn5zW\nE1PHn2rO1DiW2xm5GW11XQClFQ0Y1j/f7FUbjMBv8IeR7XWiZ4EXQKSSTgzfBlRFZ3107+ZBYTcP\nwqqO2oYQyquazEUtBzsgpHfsq0FVXcBsbehCmOFcVtEAVYttDNWzwAchgCO1/uM+DiLqWJYLabOS\nbmNIK7IDC3/2Y9x5zelwtPBcj0tBMKRi+74aAGhWRQOxFYoAUJDjRkGuB0CkJ220OIp7ZKO6Lmi2\nRApzIyENRAJU1XQM7Z8HINafPp6+jbY29h1qQCCkorLWb97jUdUEDlY24lD0YqHRzjnEGR5EXY5l\nQrqsogHfldbGLhy2cQoegJTVczyPK1JJ74iG9NDivGbHGO0OACjI9cDtlJHlUaKVdANyfU4Mjfa7\nt+yuAmBU0pGKe8ueyGPD+ufD51Y6pJI2QloXArvL6rDvcOSXRc/oTQv2HarHoeom+NwKhvSNvMf2\n9qUbA2HUtLAsnog6jmVC+o2PvsWcF78wF6O0dcVhuoyQ3r6/BtleJ/pGb5cVL76Szs91A4iEdeXR\nACprA+hdmGXeC3HL7sjFx8Ik93q7AAAMcElEQVRuHnSPVtJboyHdpzAbvQuzcLjaf1zvPt4UCOPA\nkQZzauDOA7XmVqoXDo8s3NlTXocjNX706u7DSd0jwd3eaXjP/mkzHvv9v5otpyeijmeZkD61OA/+\noIoN3xwG0L5KOh1ulwxNF6iuC2JocV7SzZASKukct/kxrOoQiIRvn6JISNdFp7gVdvOa7Q5jc6be\nhT70LsyCLsRxnV3xXdlRCACjh/eGhEhVve9QHQDg/NN7QXZI+PrbCmi6wEkFPhTmeSA7pHaNYd+h\neuwuq0ODP4yNO44ct/dgFSs+242Hl60zb7NGZDWWCWlj6XZJtApta086XcbG/wCaTdEzHNvuiP8I\nAH2KstC7e5b5teyQkJfjQve4Y1yKA4XdvGbFfbCyEeVVjfif5ZvxXVnbl2gfbQyZqymNqXYjBhWi\nX49s7DlYh+8P1aMoz4Nu2W70Lswyf3n06u6D7HCgR74X5VVNEG3cw+PzLQeTfp4Jyioa8Pf1+3Ck\nxo+Pvyo90cOhLkwIgaMdtLmaZUK6KM+LM08phJEhHRXS8TM5hqYI6WMvHAJAQbTtAQC9C7PgdStm\nKOfnuCE7HPC6FXPHvV7dfXA4JPSJhnRpRSN+//ftKNlThVf+us2cBrjt+2ose29bQs93x74a/O2L\nvWagHq5pwpyX1+HR329Agz+MXQeOwiFJOKVPLob0y4Oq6WgMqOjfMwcAzI8A0Ksg8u+f1D0L/qCK\nuqYwRLSy/2xTGV7/aCdWfLYb//h3abPZH6GwhvXbDqNbtgtDi/Owq/RoRi0v/+Mn30GIyEXn1V8e\nMJf9E7WFput45f1v8MDStR0y3bbV/aR1Xce8efOwc+dOuFwuLFiwAP379z/uAwGAy87th5Joj7et\n86TTZSxo6ZbtQq/oRbZjJa2kcxIraeNjVV3AbHMAkQuIDf6wGc5GJf3xV6XwB1VkeRRU1Abw3hd7\nMXJIEZa+vQUhVUdZRSMevmUkviurxdIVJdB0gd3n9MP1Fw3E797dBn9Qgz+o4eX3tmFveR3698qG\nx6VgSL88rI5Wgf175Zgf15aUA4j8sgBgvtdvD9Tik69KzWo8ntct44EbRpiLd/79bQWagiqu/I/+\nKO6ZjR37a/H5lnLcePGg9E+4RZXsqcLWvdU47eR8nFqcjz+v2YN/fF2GK3/SMT/blJlUTccrf/0G\nG3ccwaA+3dAjz3vc/41WQ3r16tUIhUL44x//iE2bNmHx4sV46aWXjvtAAOD8M3vjxRVbEAxpbbpb\neFsYIT2sf37K2SAuxQHZIUHTBfKPqaTzsl1mpd2nMAtbdleZszqAyAXEfYfqzXDOy3bB65bhD6rw\nuRU8ctuP8NT/fY2VGw5gzaaDCGs6zhhQgK17q/Gb//saZZWNkB0SehT4sOrLA/jm+2qUVTbigjN7\nobY+iK17qwEAQ/pFZmwM7hebnWKGdLSSdkiS+UNjXDxc9pdt0IXA6SfnY+SQIgzonYtgSMPe8nq8\n/eluPP3HTfj5tWdgwEm5WLM50t4YfdZJKMiN3En9nyXlGDm4CJV1fijRNkpethsOhwRdCARDGpoC\nKnQh4PMocCkyGv1h1DWFIDsk5Ga54HbKONoYwtHGEHxuBXnZLkiShJr6IBr9YeT4XMjLcSGs6jhY\nG8DhI/XIy3GjW5YLDf7I7oRCCBTkeJDjc6KmIYiqowG4FBmF3TxwuWRU1PhRVRdAjs+JHvk+SIj8\nRXK42o8jNU3YvLsKkgRMvmQwCnI9+HDDfqz8134oDglf7qyAEALnDO2B0wcUoLI2gAMVDfC5FfTv\nmYPcbBeqav2org8ix+dCUbTnX1MfRFNARW6WC3nZbqiajqONIYQ1Hbk+F7K9TgRCKuqbInewz/Y6\n4XFF7s3Z6A/DqTiQ7XVCkR1oDIThD6pwO2Vke50QAmgI6zh0pB5etwKfW4Gq6WgKqND0yLn2uOTI\n+Q+qkKTIfH+XM/Kz1xRU4ZQd8HkUyA4HmgJhBEIa3C4ZWR4ndF2gMRBGWNXhdSvwuhWENR1NgXDk\n7kYep3nRvdEfhsMhIcujwKnIaAqqaAqE4VJkZHkVOCQJDf7I63tckfFrukCDP4xQWEOW14ksjxPB\nsIb6phB0XSDH54LXraAxEEZ9Ywiy7EBulgsuxYG6phDqGsPwumV0y4r8rBypD6Gs/GjkZyXbhZCq\no7ougEBIQ36OG3nZbtT7QzhS44cuBIryvMj1uVBZ68eh6ia4XTJO6p4Fj0vGgSMNKK9qQn6OG/17\n5kDVdOw8UItDVU0o7pmNwX3zcKi6CV99W4Ha+iDOHFiAMwZ2x75D9fhiazl2l9VhSN9uuP+G4XAq\nxz+3Wg3pr776CqNHjwYAjBgxAlu3bj3ugzB43Ap+NLQH1m4pT9hz43j/GwAwtDh5qwOITOUz2hbG\nOPKjFbVRIQOxijq+kjY+N3rWkiShd2EWdpfV4cZLBqFHnhdTLz8Vz/xxMxoDKqZefiouPKs3Xnx3\nK/79bQUUWcJ9k87C8GG98Mvn1qCsshF9CrNw67hTEQxpeOwP/8LRhpA5ra5bVuQvgkPVTSiOhnO/\nHtmQJKAwz2P+0BgVtRAC140ZiCt/0j/hl9Spxfnome/FS3/Ziv9Zvtl8fEi/PHNa33ln9MLqL0ux\n8I2v2nDGreuaC042Z/dcdnZf/PWf3+OtT76DJAESJOw+WHeCR0gnXEnilxIiF+7//Ple87Hhp3TH\n3RPOMPcGOt4k0cqVpF/96lcYN24cxowZAwC46KKLsHr1aiiK5e68RUSUcVotV7Ozs9HYGGuG67rO\ngCYi6iSthvTIkSOxZs0aAMCmTZswZMiQDh8UERFFtNruMGZ3fPvttxBCYOHChTjllFM6a3xERLbW\nakgTEdGJY5nFLERE1BxDmojIwhjSREQWZom5dJ259PxEmDhxInJyIgtN+vbti5/+9Kd44oknIMsy\nRo0ahfvuuy/lOdi0aVPax1rZ5s2b8dRTT+H111/Hvn378PDDD0OSJAwePBiPPfYYHA4HXnjhBXz6\n6adQFAVz5szBWWeddVyOtZr4c7Ft2zbcfffdOPnkkwEAN910E6688sqMPxfhcBhz5sxBWVkZQqEQ\n7rnnHgwaNMjWPxcpCQtYuXKlmDVrlhBCiK+//lrcfffdJ3hEx08gEBATJkxIeOyaa64R+/btE7qu\nizvuuENs3bo15Tloy7FW9fLLL4urrrpK3HDDDUIIIe666y6xfv16IYQQjzzyiPjoo4/E1q1bxZQp\nU4Su66KsrExMmjTpuBxrNceei+XLl4vXXnst4Rg7nIu3335bLFiwQAghRHV1tRgzZoytfy5aYolf\nJ5259Lyz7dixA36/H9OmTcPUqVOxceNGhEIhFBcXQ5IkjBo1CuvWrUt6DhoaGtI+1sqKi4uxdOlS\n8+tt27bh3HPPBQBceOGF+Oc//4mvvvoKo0aNiiyj790bmqahurr6Bx9rNceei61bt+LTTz/FLbfc\ngjlz5qChocEW5+Lyyy/H/fffb34ty7Ktfy5aYomQbmhoQHZ27A4psixDVTNj20iPx4Pbb78dr732\nGh5//HHMnj0bXm9sQ6asrCzU19cnPQfHPtbSsVY+X+PHj09YpSqEMPcNSfWejMd/6LFWc+y5OOus\ns/DQQw/hzTffRL9+/fDb3/7WFuciKysL2dnZaGhowPTp0zFjxgxb/1y0xBIhnclLzwcMGIBrrrkG\nkiRhwIAByMnJQW1tbJvQxsZG5ObmJj0Hxz7W0rFd6XzF9wNTvafGxkbk5OT84GOtbuzYsTjjjDPM\nz7/55hvbnIvy8nJMnToVEyZMwNVXX82fixQsEdKZvPT87bffxuLFiwEAhw8fht/vh8/nw/79+yGE\nwNq1a3HOOeckPQfZ2dlwOp1pHduVnHbaadiwYQMAYM2aNeZ7Wrt2LXRdx8GDB6HrOgoKCn7wsVZ3\n++23Y8uWLQCAdevW4fTTT7fFuaisrMS0adPwy1/+Etdffz0A/lykYokVh5m89DwUCmH27Nk4ePAg\nJEnCgw8+CIfDgYULF0LTNIwaNQoPPPBAynOwadOmtI+1stLSUsycORPLly/H3r178cgjjyAcDmPg\nwIFYsGABZFnG0qVLsWbNGui6jtmzZ+Occ845LsdaTfy52LZtG+bPnw+n04nCwkLMnz8f2dnZGX8u\nFixYgA8++AADBw40H/vVr36FBQsW2PbnIhVLhDQRESVniXYHERElx5AmIrIwhjQRkYUxpImILIwh\nTURkYQxpIiILY0gTEVnY/weoi1VMvYgoRgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Distribution of the words\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "dist_list=[]\n",
    "for i in clean_text_counter.values():\n",
    "    # print(i)\n",
    "    dist_list.append(i)\n",
    "dist_list = np.asarray(dist_list)\n",
    "sns.set(color_codes=True)\n",
    "sns.distplot(dist_list,hist=False)\n",
    "plt.title('Distribution of the words')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAETCAYAAAAs4pGmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XtYE3eiPvB3kkAVElY4Ylu8FRAv\n6FoXkaon4KW12i5IRS1aF+vqVm3VilYLUgERFK3KcYXV2rtLpRWVanu2PWqxFkEEjytrwVsvigrU\nqugKEQWS7+8Pf+RIGTBRclHfz/P4PGTyneTNoHn9ziQzkhBCgIiI6DcUtg5ARET2iQVBRESyWBBE\nRCSLBUFERLJYEEREJIsFQUREslgQZBU9evRAZWVlo2VZWVmYMWMGAOCvf/0rduzY0eJjpKWl4Ztv\nvrFYRks6fvw4nnnmGYSFheH8+fON7rv9dUVHR+ODDz64p+caPnw4vv/++3t6jOZMnTrV+Hu05POQ\nfVDZOgARAMydO/eOYwoKCtCtWzcrpGl92dnZeOqpp7Bs2bIm991PrysvL8/WEciKWBBkF6Kjo+Hj\n44Np06Zh3bp12LNnDxwcHODq6ork5GTs2bMHxcXFePvtt6FUKjFw4EAkJCTgxIkTkCQJgYGBmD9/\nPlQqFb777jusXr0aCoUCvXr1woEDB5CRkYHCwkJs27YNNTU1UKvV2LhxI5YsWYLS0lJcvXoVzs7O\nWL16Nby8vBAREYHevXujqKgIlZWVePHFF3Hp0iUUFhaipqYGa9euRY8ePZq8jr/97W/4xz/+AaVS\nCU9PT8TGxiI/Px+ffvop9Ho9bty4gTVr1hjHb968udHrAoAjR45gwoQJuHTpEnx8fLBmzRo4OTnh\np59+wrJly3D16lXo9XpERERg3LhxLW7XvXv3YsOGDairq0ObNm0QFRWFP/zhD0hNTUVZWRkuXryI\nsrIyPProo1i1ahU6dOiAo0ePYsmSJairq0OXLl1QXl6O6Oho4wzv5ZdfxrvvvgsA2LJlC+Lj41FZ\nWYnQ0FDMmzcPOp0OixYtQmlpKRQKBXr37o2lS5dCoeAOi/uOILKC7t27i+DgYDF69GjjnyFDhojp\n06cLIYSIiooS77//vigvLxd+fn7i5s2bQgghPvjgA7Fnzx4hhBB/+tOfxNdffy2EEOLNN98UiYmJ\nwmAwiJs3b4qpU6eKjRs3isrKShEQECCOHz8uhBAiKytLdO/eXZw7d05s375dDBgwQFRVVQkhhPj6\n669FYmKiMWNsbKxYunSp8blmz54thBCiqKhIdO/eXWRnZwshhFi2bJlYvHhxk9e4bds2ER4eLnQ6\nnRBCiHXr1ompU6caf05ISJDdNre/rqioKDFu3Dhx/fp1UV9fL8aMGSM+//xzUVdXJ55//nlRXFws\nhBDi2rVr4rnnnhNHjhxp8njDhg0TR48eFadPnxbBwcGisrJSCCHEqVOnxH/+538KnU4n1q1bJ55+\n+mnjtpgxY4b461//Kurq6kRQUJDYt2+fEEKI/Px80aNHD3Hw4EHj7/Hy5cvG52nYXr/++qvo06eP\nKC8vF59//rnxddfX14u33npLnDlzRva1k33jDIKsZtOmTXBzczPezsrKwq5duxqNefTRR9GzZ0+M\nGTMGQUFBCAoKwqBBg5o8Vk5ODj799FNIkgRHR0dMmDABmzZtgqenJ7y9vdGzZ08AwJgxY5CUlGRc\nr0ePHlCr1QCAUaNGoXPnzkhPT0dpaSkKCwvxhz/8wTh2xIgRAIDOnTsDAAIDAwEAXbp0QWFhoWym\nsLAwODk5AQAmT56Md955B7W1tWZtp2eeeQZt27YFAPj4+KCyshJnzpzB2bNnERMTYxx348YNHDt2\nDP369ZN9nLy8PPz666+YMmWKcZkkSTh79iwAICAgwLgtfH198e9//xunTp0CAAwZMgQAMHDgQPj4\n+DSbNTg4GADg7u6O9u3b4/Lly+jfvz/+67/+CxERERg8eDBefvlldO3a1axtQPaBBUF2RaFQ4JNP\nPsH333+P/Px8LF++HIGBgXjzzTcbjTMYDJAkqdHt+vp6KJVKiN+cXuz2XRsNb94AkJGRgczMTEya\nNAkhISFo165dowPIjo6OjR7HwcGhxezNZTKXSvV//ywlSYIQAnq9HhqNBjt37jTed+nSJWg0mhbz\nDBo0CGvXrjUuq6ioQIcOHbBnzx60adOmyfPIbb+GXV+mZu3cuTP27NmDgoICHDx4EH/+85+xdOlS\nDB8+3LQNQHaDOwXJrpw4cQLBwcHw9vbGjBkzMGXKFOMnZZRKpfENV6vV4pNPPoEQArW1tcjMzMTg\nwYPh5+eHM2fO4MSJEwCAXbt24dq1a43euBvk5uZizJgxGD9+PDw9PbF3717o9fq7zh4YGIjt27fj\n+vXrAID09HQMGDCgSdH81u2vqzmenp5o06aNsSAqKioQHByM4uLiZtcZNGgQ8vLy8NNPPwEAvvvu\nO4wePRo3btxodh1vb284OjoiJycHAHD06FGcOnXKuP1MyZqRkYFFixZBq9Vi4cKF0Gq1OHbsWIvr\nkH3iDILsSs+ePfHcc89h7NixcHJyQps2bbB48WIAtz5WmZKSgrq6OixevBhJSUkICQlBXV0dAgMD\nMXPmTDg6OiIlJQVRUVFQKBTo06cPVCqVcZfN7aZOnYq4uDhs27YNANCvXz/jLpa7MW7cOFRUVGD8\n+PEwGAzo2rUrVq9efcf1bn9dzXF0dMT69euxbNkyvP/++6ivr8fcuXPRv3//Ztfp1q0bli5divnz\n50MIAZVKhQ0bNsDZ2bnZdVQqFVJTUxEfH4+UlBQ88cQTaN++vXG2MWrUKERERCA1NbXZx3jhhRdQ\nWFiI559/Hm3btsXjjz+OiIiIO24Hsj+S+O18kug+Vl1djfXr12POnDlo27YtSkpKMGPGDOzfv192\nFkFNrVy5EtOmTUP79u1RUVGB0NBQfPPNN3BxcbF1NLIyziDogaJWq+Hg4IBx48ZBpVJBpVJh7dq1\nLAczdOzYEVOmTIFKpYIQAklJSSyHhxRnEEREJIsHqYmISBYLgoiIZD0wxyAuXqy663VdXZ1w5cr1\nVkzTeuw1G3OZh7nMw1zmu9ts7u7Nf5eGMwgAKlXzXwSyNXvNxlzmYS7zMJf5LJGNBUFERLJYEERE\nJMtixyDq6uoQHR2NsrIyKBQKJCYmQqVSITo6GpIkwcfHB/Hx8VAoFEhLS8O+ffugUqkQExODvn37\norS0VHYsERFZh8Xecb/77jvU19fjs88+w6xZs7B27VokJycjMjISGRkZEEIgOzsbJSUlKCwsxNat\nW5GSkoKEhAQAkB1LRETWY7GC8PT0hF6vh8FgQHV1NVQqFUpKShAQEAAACAoKwoEDB3D48GFotVpI\nkgQPDw/o9XpUVlbKjiUiIuux2C4mJycnlJWV4bnnnsOVK1fwzjvv4NChQ8ZTHjg7O6OqqgrV1dVo\n166dcb2G5UKIJmNb4urqdE9H8Vv6qJet2Ws25jIPc5mHuczX2tksVhAff/wxtFot3njjDVRUVODl\nl19udLZKnU4HFxcXqNVq6HS6Rss1Gk2j4w0NY1tyL59NdnfX3NP3KCzJXrMxl3mYyzzMZb67zWaT\n70G4uLgYL2byu9/9DvX19fD19UVBQQGAW1ff8vf3h5+fH3Jzc2EwGFBeXg6DwQA3NzfZsUREZD0W\nm0FMmTIFMTExeOmll1BXV4d58+ahT58+iI2NRUpKCry8vDBy5EgolUr4+/sjPDwcBoMBcXFxAICo\nqKgmY4mIyHoemLO53su0zxrTxqkr9sou/zC65csw2uuUlrnMw1zmYS7z3Ve7mIiI6P7GgiAiIlks\nCCIiksWCICIiWSwIIiKSxYIgIiJZD8wV5exFcx9nJSK633AGQUREslgQREQkiwVBRESyWBBERCSL\nBUFERLJYEEREJIsFQUREslgQREQkiwVBRESyWBBERCSLBUFERLJYEEREJMtiJ+vLysrC559/DgC4\nefMmjh8/jvT0dCxbtgxKpRJarRazZ8+GwWDAkiVLcPLkSTg6OiIpKQldu3ZFUVFRk7FERGQ9FiuI\nsLAwhIWFAQASEhIwduxYxMfHIzU1FZ07d8b06dNRUlKCsrIy1NbWYsuWLSgqKsKKFSuwYcMG2bG9\ne/e2VFwiIvoNi5/u+/vvv8ePP/6IN954Ax9//DG6dOkCANBqtcjPz8fFixcRGBgIAOjXrx+Ki4tR\nXV2N2traJmNbKghXVyeoVMq7zunurrnrde+FKc9rq2x3wlzmYS7zMJf5WjubxQti48aNmDVrFqqr\nq6FWq43LnZ2dce7cuSbLlUpls2NbcuXK9bvO6O6uwcWLVXe9/r240/PaMltLmMs8zGUe5jLf3WZr\nqVQsWhDXrl3Dzz//jIEDB6K6uho6nc54n06ng4uLC27cuNFoucFggFqtlh1LRETWY9FPMR06dAiD\nBw8GAKjVajg4OODs2bMQQiA3Nxf+/v7w8/NDTk4OAKCoqAjdu3dvdiwREVmPRWcQp0+fRqdOnYy3\nExISsGDBAuj1emi1Wjz55JP4/e9/j7y8PEyYMAFCCCxfvrzZsUREZD0WLYi//OUvjW7369cPmZmZ\njZYpFAosXbq0ybpyY4mIyHr4RTkiIpJl8U8xPaimrthr6whERBbFGQQREcliQRARkSwWBBERyWJB\nEBGRLBYEERHJYkEQEZEsfsz1/2vuY6sfRg+3chIiIvvAGQQREcliQRARkSwWBBERyWJBEBGRLBYE\nERHJYkEQEZEsFgQREcliQRARkSwWBBERybLoN6k3btyIvXv3oq6uDhMnTkRAQACio6MhSRJ8fHwQ\nHx8PhUKBtLQ07Nu3DyqVCjExMejbty9KS0tlxxIRkXVY7B23oKAAR44cwaeffor09HT88ssvSE5O\nRmRkJDIyMiCEQHZ2NkpKSlBYWIitW7ciJSUFCQkJACA7loiIrMdiBZGbm4vu3btj1qxZmDlzJoYO\nHYqSkhIEBAQAAIKCgnDgwAEcPnwYWq0WkiTBw8MDer0elZWVsmOJiMh6LLaL6cqVKygvL8c777yD\n8+fP49VXX4UQApIkAQCcnZ1RVVWF6upqtGvXzrhew3K5sS1xdXWCSqVs9dfh7q5p9cc09/EtneFu\nMZd5mMs8zGW+1s5msYJo164dvLy84OjoCC8vLzzyyCP45ZdfjPfrdDq4uLhArVZDp9M1Wq7RaBod\nb2gY25IrV67fddaWNmrIGzvv+nFNcfFiy8Xn7q654xhbYC7zMJd5mMt8d5utpfc/i+1i6t+/P/bv\n3w8hBC5cuICamhoMGjQIBQUFAICcnBz4+/vDz88Pubm5MBgMKC8vh8FggJubG3x9fZuMJSIi67HY\nDGLYsGE4dOgQxo0bByEE4uLi0KlTJ8TGxiIlJQVeXl4YOXIklEol/P39ER4eDoPBgLi4OABAVFRU\nk7FERGQ9khBC2DpEa7iXaZ+7u8biu5Kac6cLEtnrlJa5zMNc5mEu891Xu5iIiOj+xoIgIiJZLAgi\nIpLFgiAiIlksCCIiksWCICIiWRY9myvd2dQVe2WX3+njr0RElsYZBBERyWJBEBGRLBYEERHJYkEQ\nEZEsFgQREcliQRARkSwWBBERyWJBEBGRLBYEERHJYkEQEZEsFgQREcmy6LmYXnjhBWg0ty5n16lT\nJ4SHh2PZsmVQKpXQarWYPXs2DAYDlixZgpMnT8LR0RFJSUno2rUrioqKmowlIiLrMbkgamtr4ejo\niNLSUpw+fRpBQUFQKJqfgNy8eRMAkJ6eblwWGhqK1NRUdO7cGdOnT0dJSQnKyspQW1uLLVu2oKio\nCCtWrMCGDRsQHx/fZGzv3r3v4aUSEZE5TCqItLQ0/Pzzz1iwYAEmTZqEbt26ITc3F4sXL252nRMn\nTqCmpgZTp05FfX095syZg9raWnTp0gUAoNVqkZ+fj4sXLyIwMBAA0K9fPxQXF6O6ulp2LAuCiMh6\nTCqIvXv3IiMjA3//+98xevRovPnmmwgLC2txnTZt2mDatGkYP348zpw5g1deeQUuLi7G+52dnXHu\n3DlUV1dDrVYblyuVyibLGsa2xNXVCSqV0pSXc19wd9fI/mxPmMs8zGUe5jJfa2czqSAMBgPatGmD\nb7/9FpGRkTAYDKipqWlxHU9PT3Tt2hWSJMHT0xMajQZXr1413q/T6eDi4oIbN25Ap9M1ei61Wt1o\nWcPYlly5ct2UlyLLHn/hFy9WAbiVreFne8Jc5mEu8zCX+e42W0vvfyZ9imnQoEEIDg5GXV0dBgwY\ngD/96U8YPrzlC9ps27YNK1asAABcuHABNTU1cHJywtmzZyGEQG5uLvz9/eHn54ecnBwAQFFREbp3\n7w61Wg0HB4cmY4mIyHpMmkHMnDkTERERePTRR6FQKBAbG3vH/9GPGzcOixYtwsSJEyFJEpYvXw6F\nQoEFCxZAr9dDq9XiySefxO9//3vk5eVhwoQJEEJg+fLlAICEhIQmY4mIyHpaLIiKigoIITB9+nS8\n9957uHDhAgBArVZj2rRp+J//+Z9m13V0dMSaNWuaLM/MzGx0W6FQYOnSpU3G9evXr8lYIiKynhYL\nYt26dSgoKMCvv/6KSZMm/d9KKhWGDh1q6WxERGRDLRZEcnIyAODdd9/F9OnTrRKIiIjsg0nHIMLD\nw7F582ZcvXoVQgjjcn67mYjowWVSQURGRkKj0cDHxweSJFk6ExER2QGTCuLSpUv46KOPLJ2FiIjs\niEnfg+jVqxdOnDhh6SxERGRHTJpB/PDDDxgzZgz+4z/+A4888giEEJAkCdnZ2ZbOR0RENmLyyfqI\niOjhYtIupo4dO+Kf//wnMjMz4ebmhkOHDqFjx46WzkZERDZkUkGsXr0a3333HXbv3g29Xo/t27cb\nz7NEREQPJpMKIjc3F6tWrcIjjzwCtVqNjz76yHiCPSIiejCZVBANV45r+A5EbW1ti1eTIyKi+59J\nB6lHjRqFyMhI/Pvf/8bHH3+ML774AsHBwZbORkRENmRSQUyfPh379++Hh4cHKioqMGfOHAwbNszS\n2YiIyIZMKggA6NChQ6OLBB06dAgDBgywSCgiIrI9kwpi3rx5OHbsGDp06GBcJkkS/v73v1ssGBER\n2ZZJBXHixAl89dVXUCqVls5DRER2wqSPIj355JMoLS21dBYiIrIjJs0gBg4ciODgYHTo0AFKpZLn\nYiIiegiYVBAbN27Epk2b4OHhYdaDX758GWFhYfjwww+hUqkQHR0NSZLg4+OD+Ph4KBQKpKWlYd++\nfVCpVIiJiUHfvn1RWloqO5aIiKzHpHddV1dX+Pv7o2PHjo3+tKSurg5xcXFo06YNgFuXL42MjERG\nRgaEEMjOzkZJSQkKCwuxdetWpKSkICEhodmxRERkXSbNIJ544gm8+OKLGDx4MBwcHIzLW7rk6MqV\nKzFhwgS8++67AICSkhIEBAQAAIKCgpCXlwdPT09otVpIkgQPDw/o9XpUVlbKjh0xYkSLGV1dnaBS\nPTgH0d3dNbI/2xPmMg9zmYe5zNfa2UwqCA8PD7N2L2VlZcHNzQ2BgYHGgmg4bgEAzs7OqKqqQnV1\nNdq1a2dcr2G53Ng7uXLlusn5fssef+EXL956ze7uGuPP9oS5zMNc5mEu891ttpbe/0wqiJZmCnK2\nb98OSZKQn5+P48ePIyoqCpWVlcb7dTodXFxcoFarodPpGi3XaDSNjjc0jCUiIusy6RhEz5490atX\nr0Z/hgwZ0uz4zZs345NPPkF6ejp69eqFlStXIigoCAUFBQCAnJwc+Pv7w8/PD7m5uTAYDCgvL4fB\nYICbmxt8fX2bjCUiIusy+YtyDerq6vDNN9+gqKjIrCeKiopCbGwsUlJS4OXlhZEjR0KpVMLf3x/h\n4eEwGAyIi4trdiwREVmXJIQQd7NiaGgodu7c2dp57tq97Bd0d9cg5A37eS0A8GH0rfNe2es+T+Yy\nD3OZh7nMZ7NjEDt27DD+LITADz/8AJXK5PP8ERHRfcikd/mG4wENXF1dsXbtWosEIiIi+2BSQSQn\nJ+PYsWPw9fVFVVUViouL0blzZ0tnIyIiGzKpINasWYOSkhJ8+OGHqKmpwfr16/G///u/mDNnjqXz\nPbSmrtgru7zh2AQRkaWZ9DHXb7/9Fu+99x6AWxcO+uijj7B7926LBiMiItsyqSDq6+tx48YN4+26\nujqLBSIiIvtg0i6mCRMmICwszHjJ0ZycHEyaNMmiwYiIyLZMKogpU6agf//+OHToEBwcHLBq1Sr4\n+vpaOhsREdmQSQUhhMD333+PI0eOQK/Xw2AwoGfPnrxGAxHRA8ykgnj77bdRWlqKsWPHQgiBrKws\nnDt3DosXL7Z0PiIishGTCiIvLw87duwwzhiGDh2KkJAQiwYjIiLbMmkfkV6vR319faPbSuWDc3Ee\nIiJqyqQZREhICCZPnow//vGPAIB//OMfCA4OtmgwIiKyLZMKYvjw4XjssceQnp4ODw8PvPrqqy1e\nD4KIiO5/LRbE5cuX8frrr+PHH39E165doVQqcfDgQdy4cQN+fn7QaOzvUp1ERNQ6WjwGsWbNGvTv\n3x+5ubnIzMxEZmYmDhw4gJ49e2LZsmXWykhERDbQ4gziyJEj+Prrrxstc3BwwPz58xEaGmrRYERE\nZFstziAeeeQR2eWSJPFLckRED7gWZxCSJN3VfcCtj8IuXrwYp0+fhlKpRHJyMoQQiI6OhiRJ8PHx\nQXx8PBQKBdLS0rBv3z6oVCrExMSgb9++KC0tlR1LRETW0WJB/PDDD3j66aebLBdC4OLFiy0+8Lff\nfgsA+Oyzz1BQUGAsiMjISDz11FOIi4tDdnY2PDw8UFhYiK1bt6KiogJz5szB9u3bkZyc3GTsiBEj\n7uGlEhGROVosiF27dt31Az/zzDMYOnQoAKC8vBzt27fHvn37EBAQAAAICgpCXl4ePD09odVqIUkS\nPDw8oNfrUVlZiZKSkiZjWRBERNbTYkF07Njx3h5cpUJUVBT27NmDdevW4dtvvzXumnJ2dkZVVRWq\nq6vRrl074zoNy4UQTca2xNXVCSrVg//tbnd3+/losT1luR1zmYe5zGOvuYDWz2bSF+XuxcqVK7Fg\nwQK8+OKLuHnzpnG5TqeDi4sL1Go1dDpdo+UajabR8YaGsS25cuX6XWe051/4b1282HJRWou7u8Zu\nstyOuczDXOax11zA3Wdr6f3PYkd9d+zYgY0bNwIA2rZtC0mS0KdPHxQUFAC4ddEhf39/+Pn5ITc3\nFwaDAeXl5TAYDHBzc4Ovr2+TsUREZD0Wm0E8++yzWLRoESZNmoT6+nrExMTA29sbsbGxSElJgZeX\nF0aOHAmlUgl/f3+Eh4fDYDAgLi4OABAVFdVkLBERWY8khBC2DtEa7mXa5+6uQcgbO1sxjeV8GD3c\n1hEA2O9Um7nMw1zmsddcwH22i4mIiO5vLAgiIpLFgiAiIlksCCIiksWCICIiWSwIIiKSxYIgIiJZ\nLAgiIpLFgiAiIlksCCIiksWCICIiWSwIIiKSxYIgIiJZLAgiIpJl8SvKUeuaumKv7HJ7OQ04ET04\nOIMgIiJZLAgiIpLFgiAiIlkWOQZRV1eHmJgYlJWVoba2Fq+++iq6deuG6OhoSJIEHx8fxMfHQ6FQ\nIC0tDfv27YNKpUJMTAz69u2L0tJS2bFERGQ9FnnX/eKLL9CuXTtkZGTgvffeQ2JiIpKTkxEZGYmM\njAwIIZCdnY2SkhIUFhZi69atSElJQUJCAgDIjiUiIuuySEGMGjUKc+fONd5WKpUoKSlBQEAAACAo\nKAgHDhzA4cOHodVqIUkSPDw8oNfrUVlZKTuWiIisyyK7mJydnQEA1dXVeP311xEZGYmVK1dCkiTj\n/VVVVaiurka7du0arVdVVQUhRJOxd+Lq6gSVSmmBV3N/cHfXPBTPaQrmMg9zmcdecwGtn81i34Oo\nqKjArFmz8NJLLyEkJASrVq0y3qfT6eDi4gK1Wg2dTtdouUajaXS8oWHsnVy5cv2us9rzL9xUFy/e\nuURbk7u7xurPaQrmMg9zmcdecwF3n62l9z+L7GK6dOkSpk6dioULF2LcuHEAAF9fXxQUFAAAcnJy\n4O/vDz8/P+Tm5sJgMKC8vBwGgwFubm6yY4mIyLosMoN45513cO3aNaxfvx7r168HALz11ltISkpC\nSkoKvLy8MHLkSCiVSvj7+yM8PBwGgwFxcXEAgKioKMTGxjYaS0RE1iUJIYStQ7SGe5n2ubtrEPLG\nzlZMY33WPtWGvU61mcs8zGUee80F3Ee7mIiI6P7HgiAiIlksCCIiksWCICIiWSwIIiKSxYIgIiJZ\nvKLcA4JXmiOi1sYZBBERyWJBEBGRLBYEERHJYkEQEZEsFgQREcliQRARkSx+zPUBx4+/EtHd4gyC\niIhksSCIiEgWC4KIiGSxIIiISBYLgoiIZFm0IP71r38hIiICAFBaWoqJEyfipZdeQnx8PAwGAwAg\nLS0N48aNw4QJE3D06NEWxxIRkfVYrCDee+89LF68GDdv3gQAJCcnIzIyEhkZGRBCIDs7GyUlJSgs\nLMTWrVuRkpKChISEZscSEZF1Wex7EF26dEFqairefPNNAEBJSQkCAgIAAEFBQcjLy4Onpye0Wi0k\nSYKHhwf0ej0qKytlx44YMaLF53N1dYJKpbTUy3ngNPf9CAD4ck2oSY/h7q5prTitirnMw1zmsddc\nQOtns1hBjBw5EufPnzfeFkJAkiQAgLOzM6qqqlBdXY127doZxzQslxt7J1euXL/rrPb8C7eFixfv\nvL3d3TUmjbM25jIPc5nHXnMBd5+tpfc/qx2kVij+76l0Oh1cXFygVquh0+kaLddoNLJjiYjIuqx2\nqg1fX18UFBTgqaeeQk5ODgYOHIguXbpg1apVmDZtGn755RcYDAa4ubnJjqX7D0/zQXR/s1pBREVF\nITY2FikpKfDy8sLIkSOhVCrh7++P8PBwGAwGxMXFNTuWiIisSxJCCFuHaA33sl/Q3V2DkDd2tmKa\nB9Pt//M3ZX+nLWYQ9rqPmLnMw1zmu6+PQRAR0f2FBUFERLJYEEREJIsFQUREsnhFOTIZP7ZK9HDh\nDIKIiGRxBkH3rKXzOhHR/Yt4VtspAAALbUlEQVQzCCIiksWCICIiWdzFRFbHg91E9wcWBNkNFgeR\nfeEuJiIiksUZBNm91vqUFGciROZhQdBDozU/jsuyoYcBdzEREZEsziCI7gJ3e9HDgAVBZEPmFg0L\nhayJBUF0H2GhkDXZbUEYDAYsWbIEJ0+ehKOjI5KSktC1a1dbxyK6r3BXGN0Luy2Ib775BrW1tdiy\nZQuKioqwYsUKbNiwwdaxiB5K1jgho6VL6H76Iqa9ZLXbgjh8+DACAwMBAP369UNxcbGNExGRJdnq\nrMD309mIrV0cdlsQ1dXVUKvVxttKpRL19fVQqeQju7tr7un5vlwTek/rExHZ2r2+D/6W3X4PQq1W\nQ6fTGW8bDIZmy4GIiFqf3RaEn58fcnJyAABFRUXo3r27jRMRET1cJCGEsHUIOQ2fYjp16hSEEFi+\nfDm8vb1tHYuI6KFhtwVBRES2Zbe7mIiIyLZYEEREJIsFQUREsh7qz43ay+k8XnjhBWg0tz6/3KlT\nJ4SHh2PZsmVQKpXQarWYPXu2VbP+61//wurVq5Geno7S0lJER0dDkiT4+PggPj4eCoUCaWlp2Ldv\nH1QqFWJiYtC3b99mx1oiV0lJCWbOnIknnngCADBx4kQ8//zzVs1VV1eHmJgYlJWVoba2Fq+++iq6\ndetm8+0ll+uxxx6z+fYCAL1ej8WLF+P06dNQKpVITk6GEMLm20wuV1VVlV1sMwC4fPkywsLC8OGH\nH0KlUllve4mH2K5du0RUVJQQQogjR46ImTNnWj3DjRs3RGhoaKNlo0ePFqWlpcJgMIi//OUvori4\n2GpZ3333XREcHCzGjx8vhBBixowZ4uDBg0IIIWJjY8Xu3btFcXGxiIiIEAaDQZSVlYmwsLBmx1oq\nV2Zmpvjggw8ajbF2rm3btomkpCQhhBCVlZViyJAhdrG95HLZw/YSQog9e/aI6OhoIYQQBw8eFDNn\nzrSLbSaXy162WW1trXjttdfEs88+K3788Uerbq+HeheTPZzO48SJE6ipqcHUqVMxefJkHDp0CLW1\ntejSpQskSYJWq0V+fr7Vsnbp0gWpqanG2yUlJQgICAAABAUF4cCBAzh8+DC0Wi0kSYKHhwf0ej0q\nKytlx1oqV3FxMfbt24dJkyYhJiYG1dXVVs81atQozJ0713hbqVTaxfaSy2UP2wsAnnnmGSQmJgIA\nysvL0b59e7vYZnK57GWbrVy5EhMmTECHDh0AWPff5ENdEM2dzsOa2rRpg2nTpuGDDz5AQkICFi1a\nhLZt2xrvd3Z2RlVVldWyjhw5stE31oUQkCSpxSwNy+XGWipX37598eabb2Lz5s3o3Lkz/va3v1k9\nl7OzM9RqNaqrq/H6668jMjLSLraXXC572F4NVCoVoqKikJiYiJEjR9rFNpPLZQ/bLCsrC25ubsb/\nHALW/Tf5UBeEPZzOw9PTE6NHj4YkSfD09IRGo8HVq1eN9+t0Ori4uNgs6+37K5vLotPpoNFoZMda\nyogRI9CnTx/jz8eOHbNJroqKCkyePBmhoaEICQmxm+3121z2sr0arFy5Ert27UJsbCxu3rzZ5Pls\nle32XFqt1ubbbPv27Thw4AAiIiJw/PhxREVFobKysslzWSrXQ10Q9nA6j23btmHFihUAgAsXLqCm\npgZOTk44e/YshBDIzc2Fv7+/zbL6+vqioKAAAJCTk2PMkpubC4PBgPLychgMBri5ucmOtZRp06bh\n6NGjAID8/Hz07t3b6rkuXbqEqVOnYuHChRg3bhwA+9hecrnsYXsBwI4dO7Bx40YAQNu2bSFJEvr0\n6WPzbSaXa/bs2TbfZps3b8Ynn3yC9PR09OrVCytXrkRQUJDVttdD/U1qezidR21tLRYtWoTy8nJI\nkoQFCxZAoVBg+fLl0Ov10Gq1mDdvnlWznj9/HvPnz0dmZiZOnz6N2NhY1NXVwcvLC0lJSVAqlUhN\nTUVOTg4MBgMWLVoEf3//ZsdaIldJSQkSExPh4OCA9u3bIzExEWq12qq5kpKS8PXXX8PLy8u47K23\n3kJSUpJNt5dcrsjISKxatcqm2wsArl+/jkWLFuHSpUuor6/HK6+8Am9vb5v/HZPL9fjjj9v879jt\nIiIisGTJEigUCqttr4e6IIiIqHkP9S4mIiJqHguCiIhksSCIiEgWC4KIiGSxIIiISBYLgu5r58+f\nR48ePZCXl9do+fDhw3H+/Pl7fvzWepyWlJeXY+TIkQgNDUV1dbVFnwsAevToYfHnoAcDC4Luew4O\nDoiNjbXKm6slFBYWok+fPti5c2ej0yUQ2RoLgu57HTp0wODBg7Fy5com9xUUFCAiIsJ4Ozo6GllZ\nWTh//jxCQ0Mxb948hISEICoqCp999hnCw8MxatQo/PTTT8Z10tLS8MILLyA8PBwnTpwAcOvbyq+9\n9hrCwsIwduxY40nQUlNTMW3aNDz//PPIyMholOX06dOIiIhASEgIwsPDcfToURw/fhxr167F/v37\nERcXZxxbWVnZ6Pw7gYGB+OqrrwAAGzduxPvvv4+amhq88cYbCA4ORkhICHbs2AHg1vl7Gp4nJSUF\n58+fx8SJExEaGtroOfLz8xEWFoawsDD8+c9/bnQKByKABUEPiOjoaOTm5jbZ1dSSkydP4pVXXsHO\nnTvxz3/+E2VlZdiyZQuCg4OxZcsW47iuXbtix44deO211xAdHQ0AWLZsGcaOHYusrCxs2LABcXFx\nxhlMbW0tvvrqK7z00kuNnm/hwoWIiIjAl19+iUWLFmHu3Lnw9vbG66+/juHDh2Pp0qXGsW5ubnj8\n8cdx6tQp/PTTT9Dr9SgsLAQA7N+/H8OGDUNqaipcXV3x3//939i0aRNSU1ONBXbhwgV8/vnnmD9/\nPhITExEWFoadO3fCz8/P+Bzr16/HkiVLkJWVhcGDB+PYsWNmbnV60LEg6IGgVquRmJho1q6m9u3b\nw9fXFwqFAo899hgGDRoEAPDw8MC1a9eM48aPHw8AGDJkCMrLy3Ht2jUcOHAA69atQ2hoKF555RXU\n19fj3LlzAG6dafa3dDodzp49i2effRbArVO2/+53v8PPP//cbL6goCDk5+fj4MGDmDx5Mg4fPoyq\nqipcunQJ3t7eOHjwoPFcS25ubnj66aeNJeLr62s8mWNhYSGee+45AMDo0aPh4OAAAHj66acxe/Zs\nLF26FL6+vtBqtSZtN3p4sCDogaHVapvsapIkCbefTaaurs74s6OjY6P1mztHze3LhRBQqVQwGAzY\ntGkTdu7ciZ07dyIzM9N4AsU2bdo0eQy5M9oIIaDX65t9PUOHDkV+fj4KCgowYsQIKBQKfPnll8Y3\n8t8+5u2P99sMDWMlSTKe4XPKlClIT09Hly5dsGrVKmzYsKHZLPRwYkHQA6VhV9Ovv/4KAHB1dcW5\nc+dw8+ZNXL16FYcPHzb7Mb/88ksAwJ49e+Dt7Q0nJycMHDjQeIzhxx9/REhICGpqapp9DLVajU6d\nOmH37t0Abp2R99KlS/Dx8Wl2nd69e+P06dM4c+YMvL298dRTT2HDhg0YNmwYAGDgwIHYtm0bgFvH\nLLKzs40Xh7nd4MGD8cUXXwAAdu/ebTy99vjx46HT6TBlyhRMmTKFu5ioiYf6mtT04GnY1TRt2jQA\ngI+PD4YMGYI//vGP6NixI/r372/2Y545cwahoaFwdnY2npp98eLFiIuLQ0hICADg7bffvuMnkFat\nWoUlS5YgNTUVDg4OSE1NbTKLuZ0kSejfv7+xeAYOHIitW7diwIABAIBZs2ZhyZIlCAkJgV6vx8yZ\nM9G7d2+cPHmy0ePExcVh4cKF2LJlC/r06QNnZ2cAwPz58xEdHQ2VSgUnJyckJSWZvW3owcazuRIR\nkSzuYiIiIlksCCIiksWCICIiWSwIIiKSxYIgIiJZLAgiIpLFgiAiIln/Dwyr7SBLW2FpAAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Histogram of the lengths\n",
    "\n",
    "df['Case Comments']=df['Case Comments'].astype(str)\n",
    "df['Case Comments length'] = df['Case Comments'].apply(len)\n",
    "\n",
    "df['Case Comments length'].plot(bins=50, kind='hist')\n",
    "plt.title('Histogram of the lengths')\n",
    "plt.xlabel('Number of words')\n",
    "plt.ylabel('Documents')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Cleaning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Function that cleaning the data\n",
    "\n",
    "import re\n",
    "def my_function(raw):\n",
    "    raw=raw.lower() \n",
    "    raw= raw.replace(']', '')\n",
    "    raw= raw.replace('[', '')\n",
    "    raw= raw.replace(')', '')\n",
    "    raw= raw.replace('(', '')\n",
    "    raw= raw.replace(':', '')\n",
    "    raw= raw.replace('.', '')\n",
    "    raw= raw.replace(',', '')\n",
    "    raw= raw.replace('  ', ' ')\n",
    "    raw= raw.replace('\"', '')\n",
    "    raw= raw.replace('\\n', ' ')\n",
    "    raw= raw.replace('\\t', ' ')\n",
    "    raw= raw.replace('?', '')\n",
    "    raw= re.sub(r\"http\\S+\", \"\", raw)\n",
    "    raw= re.sub('https?://(?:[-\\w.]|(?:%[\\da-fA-F]{2}))+', \"\", raw)\n",
    "    raw= re.sub(\" \\d{13} \", \" isbn \", raw)\n",
    "    raw= re.sub(\" \\d{10} \", \" isbn \", raw)\n",
    "    raw= re.sub(\" \\d{7}\\d{1}[\\dx] \", \" issn \", raw)\n",
    "    raw= re.sub(\" \\d{4}[-]\\d{3}[\\dx] \", \" issn \", raw)\n",
    "    raw= re.sub(\" 10\\.\\S+ \", \" doi \", raw)\n",
    "    raw= re.sub('<.*>', \"\", raw)\n",
    "    raw= re.sub('\\S+@\\S+', \"email\", raw)\n",
    "    raw= re.sub('[0-9]+', \"\", raw)\n",
    "    raw= re.sub(r'(\\d+/\\d+/\\d+)', \"date\", raw)\n",
    "    for i in raw.split():\n",
    "        if len(i) > 22:\n",
    "            raw = raw.replace(i,'')\n",
    "    raw= raw.replace('$', '')\n",
    "    raw = raw.replace('!', '')\n",
    "    raw = raw.replace(\"'\", '')\n",
    "    raw = raw.replace(\"->\", '')\n",
    "    raw= raw.replace('&', '')\n",
    "    raw= raw.replace('/', '')\n",
    "    raw= raw.replace('%', ' ')\n",
    "    raw= raw.replace(' - ', ' ')\n",
    "    raw= raw.replace('+', '')\n",
    "    raw= raw.replace('_', '')\n",
    "    raw= raw.replace('@', '')\n",
    "    raw = raw.replace('--', '')\n",
    "    raw = raw.replace('#', '')\n",
    "    raw = raw.replace('=', '')\n",
    "    raw = raw.replace('â', '')\n",
    "    raw = raw.replace('*', '')\n",
    "    raw = raw.replace('-', '')\n",
    "    raw = raw.replace(';', '')\n",
    "    raw = raw.replace('<', '')\n",
    "    raw = raw.replace('>', '')\n",
    "    raw = raw.replace('ß', '')\n",
    "    ' '.join(raw.split())\n",
    "    raw= raw.replace('  ', ' ')\n",
    "    raw= raw.replace('  ', ' ')\n",
    "\n",
    "\n",
    "    return raw\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Case Number</th>\n",
       "      <th>Mixed Comments</th>\n",
       "      <th>Categorie</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10087</td>\n",
       "      <td>status st line wi dear christian this is a de...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10087</td>\n",
       "      <td>rodopi ingentaconnect special character  migra...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10099</td>\n",
       "      <td>hi rachel olga performed an in house analysis...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10099</td>\n",
       "      <td>directory of open access journals doaj wrong r...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11750</td>\n",
       "      <td>status st line wi dear kasper thank you for r...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Case Number                                     Mixed Comments     Categorie\n",
       "0       10087   status st line wi dear christian this is a de...  Data display\n",
       "1       10087  rodopi ingentaconnect special character  migra...  Data display\n",
       "2       10099   hi rachel olga performed an in house analysis...  Data display\n",
       "3       10099  directory of open access journals doaj wrong r...  Data display\n",
       "4       11750   status st line wi dear kasper thank you for r...  Data display"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a new data frame that separating the comments' text and the title & description text\n",
    "\n",
    "merged_cases = pd.DataFrame( columns=['Case Number','Mixed Comments','Categorie'],)\n",
    "cases_by_id=df.groupby('Case Number')\n",
    "\n",
    "counter=0\n",
    "for i in cases_dict.keys():\n",
    "    case=cases_by_id.get_group(i)\n",
    "\n",
    "    mix_comment=''\n",
    "    \n",
    "    for x in case['Title']:\n",
    "        title=x\n",
    "    for y in case['Description']:\n",
    "        description=y\n",
    "\n",
    "    mix_title=my_function(str(title)) + ' ' + my_function(str(description))\n",
    "    for comment in case['Case Comments']:\n",
    "        if len(str(comment).split())>4:\n",
    "            mix_comment+=(' '+my_function(str(comment)))\n",
    "        \n",
    "    for t in case['Topic - for PC']:\n",
    "        topic=t\n",
    "    merged_cases.loc[counter]=[i,mix_comment,topic]\n",
    "    counter+=1\n",
    "    merged_cases.loc[counter]=[i,mix_title,topic]\n",
    "    counter+=1\n",
    " \n",
    " \n",
    "merged_cases.head()\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:5: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  \"\"\"\n",
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:6: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  \n",
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:7: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  import sys\n",
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:8: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  \n",
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:9: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  if __name__ == '__main__':\n"
     ]
    }
   ],
   "source": [
    "# Removing topics with low frequencies\n",
    "\n",
    "merged_cases.columns = ['Case Number','Mixed_Comments','Categorie']\n",
    "merged_cases_A = merged_cases[merged_cases['Categorie'] != 'Citation tools']\n",
    "merged_cases_A = merged_cases_A[merged_cases['Categorie'] != 'UI']\n",
    "merged_cases_A = merged_cases_A[merged_cases['Categorie'] != 'Coverage']\n",
    "merged_cases_A = merged_cases_A[merged_cases['Categorie'] != 'New collection request']\n",
    "merged_cases_A = merged_cases_A[merged_cases['Categorie'] != 'Search']\n",
    "merged_cases_A = merged_cases_A[merged_cases['Categorie'] != 'General']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Merging categories that technically are the same  \n",
    "\n",
    "merged_cases_A['Categorie'].replace('Availability', 'Data availability', inplace=True)\n",
    "merged_cases_A['Categorie'].replace('Linking', 'Data linking', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAFwCAYAAACxVaymAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XncVWW5//HPl0mcUFEsEhMHHEpE\niUGzNPUcULNwSLNMwVTyZB2tDoonf2pYWWlZzlloWmapmZpWigMOpxxAccQCHMkJxQE1FPH6/bHu\nDZuH/Uy4n70We33fr9d+PXvda+21r/U8sK617nUPigjMzKx8uuUdgJmZ5cMJwMyspJwAzMxKygnA\nzKyknADMzErKCcDMrKScAKw0JIWkzfKOIy+SDpJ0Y95xWHE4AVhdSfqipGmS3pD0nKS/SPpE3nEV\ngaRxku7M6/sj4tKIGJXX91vxOAFY3Uj6JvBT4PvAB4APA+cCY/KMy0BSj7xjsOJxArC6kLQWMAk4\nKiKuiog3I2JRRPwpIiakbUZI+rukV9PdwdmSeqV1knSGpBclvSbpQUlbp3WrSDpd0tOSXpB0vqRV\n07r1JF2X9jlf0h2S2vp3vaekxyW9JOk0Sd3S/udLGlx1POtL+rekfq0c7xGSZkpaIOlRSUNT+URJ\nc6rK90nlWwHnAzuku6NX2zu2tP7Y9Lt6VtLh1dVYktaSdImkeZKeknRC5djT3cb/pd/pfODklncg\nkraUNCUd+z8kHVC1bs8U/wJJ/5L0Px39t2ArkYjwy6/3/QJ2B94FerSxzceA7YEewEBgJnBMWjca\nmA6sDQjYCuif1v0UuBboC6wJ/Ak4Na07lezE2jO9Pgmole8P4Na0nw8D/wQOT+vOBX5Yte3RwJ9a\n2c/+wL+A4SnWzYCNqtZ9iOzi6vPAm1XHMQ64s8W+2jq23YHngY8CqwG/TsewWVp/CXBN+tzAdDyH\nVX3Xu8DX0+971ervB1YHngEOTeuHAi8BH03rnwM+md6vAwzN+9+YX/V/5R6AX83xAg4Cnu/kZ44B\n/pje75pOYNsD3aq2UTqJblpVtgPwRHo/KZ0EN+vA9wWwe9XyV4Gb0/uR6YTYLS1PAw5oZT83AEd3\n8BhnAGPS+2USQAeO7cJKMkjLm1USANAdeBv4SNX6rwBTq77r6RaxVCeAzwN3tFj/c+Ck9P7ptL8+\nef/b8qvrXq4Csnp5GVivrbpmSZun6prnJb1O9qxgPYCIuAU4GzgHeEHSBZL6AP3Irn6np2qeV4G/\npnKA04DZwI2pamdiO3E+U/X+KbKrdSLibrKT8c6StiQ7yV7byj42BOa0coyHSJpRFevWlWOsob1j\n+1CLeKvfrwf0SsdQfTwbtLJ9SxsBIyvfm777IOCDaf1+wJ7AU5Juk7RDG/uylZQTgNXL34GFwN5t\nbHMe8BgwKCL6AP9LdhUMQEScGREfI6vy2ByYQFYt8W+yqom102utiFgjfWZBRHwrIjYBPgN8U9Ju\nbcSwYdX7DwPPVi1fDHwJOBi4MiIWtrKPZ4BNWxZK2gj4BfA1YN2IWBt4uOoYWw692+axkVXDDGgl\n9peARWQn8urj+VfVcltD/T4D3Fb1vWtHxBoR8V8AEXFvRIwB1geuBi5vY1+2knICsLqIiNeAE4Fz\nJO0taTVJPSXtIelHabM1gdeBN9JV9n9VPi9puKSRknqSXYkvBBZHxHtkJ9UzJK2ftt1A0uj0fi9J\nm0lS2vfi9GrNBEnrSNqQrJ7/91Xrfg3sQ5YELmljH78E/kfSx9LD683SyX91spPuvBTboWR3ABUv\nAAOUHny3d2xkJ91DJW0labX0+638vhen9d+TtGb6/m8Cv2kj7mrXAZtLOjj9nXqmv8FWknop6zOw\nVkQsYunv1ZqME4DVTUT8hOwkdALZSfAZsqvhq9Mm/wN8EVhAduKrPvn2SWWvkFVlvAycntYdR1bN\nc1eqOroJ2CKtG5SW3yC7Czk3Iqa2EeY1ZA+bZwDXA5Or4p8L3Ed2Er+jjeO8Avge8Nt0LFcDfSPi\nUeDHKY4XgMHA/1V99BbgEeB5SS+1d2wR8RfgTLIH17PTfiGr+4fsAe+bwOPAnSmeC9s49upjWACM\nAg4kuwt6HvghsEra5GDgyRTTkWRJ0ZqMIjwhjFmFpAuBZyPihLxjaSk1JX0YWCUi3s07Hlv5OQGY\nJZIGkt0ZbBcRT+QbTSb1I7ierHrpYuC9iGjrOYtZh7kKyAyQdArZ1fVpRTn5J18hq06bQ1YP/19t\nb27Wcb4DMDMrKd8BmJmVVKEHiFpvvfVi4MCBeYdhZrZSmT59+ksRUXMcq2qFTgADBw5k2rRpeYdh\nZrZSkfRU+1u5CsjMrLScAMzMSsoJwMyspAr9DMDM3r9FixYxd+5cFi5sbWw7W1n17t2bAQMG0LNn\nzxX6vBOAWZObO3cua665JgMHDiQbM8+aQUTw8ssvM3fuXDbeeOMV2oergMya3MKFC1l33XV98m8y\nklh33XXf152dE4BZCfjk35ze79/VCcDMrKT8DMCsZAZOvL6u+3vyB59udxtJfPOb3+THP/4xAKef\nfjpvvPEGJ5988vv+/nHjxrHXXnvxuc997n3vqy1XXHEFJ554Ih/84Ae59dZbu/S7GsUJwHI1c8ut\n6ravrR6bWbd9WX2tssoqXHXVVRx//PGst15rUyQ33uLFi+nevXuHtp08eTLnnnsuu+yySxdH1Tiu\nAjKzLtejRw/Gjx/PGWecsdy6cePGceWVVy5ZXmONbErkqVOnsvPOO3PAAQew+eabM3HiRC699FJG\njBjB4MGDmTNnzpLP3HTTTXzyk59k880357rrrgOyk/uECRMYPnw422yzDT//+c+X7HeXXXbhi1/8\nIoMHD14unssuu4zBgwez9dZbc9xxxwEwadIk7rzzTo488kgmTJiw3Gd+9KMfMXjwYIYMGcLEiRMB\n+MUvfsHw4cMZMmQI++23H2+99RaQ3UlsvfXWDBkyhJ122qnNWJ977jl22mkntt12W7beemvuuKPV\niepWiO8AzKwhjjrqKLbZZhuOPfbYDn/mgQceYObMmfTt25dNNtmEww8/nHvuuYef/exnnHXWWfz0\npz8F4Mknn+S2225jzpw57LLLLsyePZtLLrmEtdZai3vvvZe3336bHXfckVGjRgFwzz338PDDDy/X\nfPLZZ5/luOOOY/r06ayzzjqMGjWKq6++mhNPPJFbbrmF008/nWHDhi3zmb/85S9cffXV3H333ay2\n2mrMnz8fgH333ZcjjjgCgBNOOIHJkyfz9a9/nUmTJnHDDTewwQYb8OqrrwLZ3UWtWK+66ipGjx7N\nt7/9bRYvXrwkidSLE4CZNUSfPn045JBDOPPMM1l11VU79Jnhw4fTv39/ADbddNMlJ/DBgwcvUw9/\nwAEH0K1bNwYNGsQmm2zCY489xo033siDDz645O7itddeY9asWfTq1YsRI0bUbDt/77338qlPfYp+\n/bKBNA866CBuv/129t679UnYbrrpJg499FBWW201APr27QvAww8/zAknnMCrr77KG2+8wejRowHY\ncccdGTduHAcccAD77rsvQKuxDh8+nC9/+cssWrSIvffem2233bZDv7eOcgIws4Y55phjGDp0KIce\neuiSsh49evDee+8BWeemd955Z8m6VVZZZcn7bt26LVnu1q0b7767dFrkls0hJRERnHXWWUtOvBVT\np05l9dVXrxnfikyQFRE1m2OOGzeOq6++miFDhvCrX/2KqVOnAnD++edz9913c/3117PtttsyY8aM\nVmMFuP3227n++us5+OCDmTBhAoccckinY2yNnwGYWcP07duXAw44gMmTJy8pGzhwINOnTwfgmmuu\nYdGiRZ3e7xVXXMF7773HnDlzePzxx9liiy0YPXo055133pL9/fOf/+TNN99scz8jR47ktttu46WX\nXmLx4sVcdtll7Lzzzm1+ZtSoUVx44YVLqmcqVUALFiygf//+LFq0iEsvvXTJ9nPmzGHkyJFMmjSJ\n9dZbj2eeeabVWJ966inWX399jjjiCA477DDuu+++Tv9u2uI7ALOS6Uizza70rW99i7PPPnvJ8hFH\nHMGYMWMYMWIEu+22W6tX523ZYost2HnnnXnhhRc4//zz6d27N4cffjhPPvkkQ4cOJSLo168fV199\ndZv76d+/P6eeeiq77LILEcGee+7JmDFj2vzM7rvvzowZMxg2bBi9evVizz335Pvf/z6nnHIKI0eO\nZKONNmLw4MEsWLAAgAkTJjBr1iwigt12240hQ4awzTbb1Ix16tSpnHbaafTs2ZM11liDSy65pNO/\nm7YUek7gYcOGhSeEaW5uBtr1Zs6cyVZb1e/3bMVS6+8raXpEDGvlI0u4CsjMrKScAMzMSsoJwKwE\nilzVayvu/f5dnQDMmlzv3r15+eWXnQSaTGU+gN69e6/wPtwKyKzJDRgwgLlz5zJv3ry8Q7E6q8wI\ntqKcAMyaXM+ePVd4xihrbq4CMjMrqQ4lAElrS7pS0mOSZkraQVJfSVMkzUo/10nbStKZkmZLelDS\n0Kr9jE3bz5I0tqsOyszM2tfRO4CfAX+NiC2BIcBMYCJwc0QMAm5OywB7AIPSazxwHoCkvsBJwEhg\nBHBSJWmYmVnjtZsAJPUBdgImA0TEOxHxKjAGuDhtdjFQGS5vDHBJZO4C1pbUHxgNTImI+RHxCjAF\n2L2uR2NmZh3WkTuATYB5wEWS7pf0S0mrAx+IiOcA0s/10/YbAM9UfX5uKmut3MzMctCRBNADGAqc\nFxHbAW+ytLqnllrT1Ecb5ct+WBovaZqkaW62ZmbWdTqSAOYCcyPi7rR8JVlCeCFV7ZB+vli1/YZV\nnx8APNtG+TIi4oKIGBYRwyqTMpiZWf21mwAi4nngGUlbpKLdgEeBa4FKS56xwDXp/bXAIak10PbA\na6mK6AZglKR10sPfUanMzMxy0NGOYF8HLpXUC3gcOJQseVwu6TDgaWD/tO2fgT2B2cBbaVsiYr6k\nU4B703aTImJ+XY7CzMw6rUMJICJmALXGlt6txrYBHNXKfi4ELuxMgGZm1jXcE9jMrKScAMzMSsoJ\nwMyspJwAzMxKygnAzKyknADMzErKCcDMrKScAMzMSsoJwMyspJwAzMxKygnAzKyknADMzErKCcDM\nrKScAMzMSsoJwMyspJwAzMxKygnAzKyknADMzErKCcDMrKScAMzMSsoJwMyspJwAzMxKygnAzKyk\nOpQAJD0p6SFJMyRNS2V9JU2RNCv9XCeVS9KZkmZLelDS0Kr9jE3bz5I0tmsOyczMOqIzdwC7RMS2\nETEsLU8Ebo6IQcDNaRlgD2BQeo0HzoMsYQAnASOBEcBJlaRhZmaN936qgMYAF6f3FwN7V5VfEpm7\ngLUl9QdGA1MiYn5EvAJMAXZ/H99vZmbvQ0cTQAA3SpouaXwq+0BEPAeQfq6fyjcAnqn67NxU1lr5\nMiSNlzRN0rR58+Z1/EjMzKxTenRwux0j4llJ6wNTJD3WxraqURZtlC9bEHEBcAHAsGHDlltvZmb1\n0aE7gIh4Nv18EfgjWR3+C6lqh/TzxbT5XGDDqo8PAJ5to9zMzHLQbgKQtLqkNSvvgVHAw8C1QKUl\nz1jgmvT+WuCQ1Bpoe+C1VEV0AzBK0jrp4e+oVGZmZjnoSBXQB4A/Sqps/9uI+Kuke4HLJR0GPA3s\nn7b/M7AnMBt4CzgUICLmSzoFuDdtNyki5tftSMzMrFPaTQAR8TgwpEb5y8BuNcoDOKqVfV0IXNj5\nMM3MrN7cE9jMrKScAMzMSsoJwMyspJwAzMxKygnAzKyknADMzErKCcDMrKScAMzMSsoJwMyspJwA\nzMxKygnAzKyknADMzErKCcDMrKScAMzMSsoJwMyspJwAzMxKygnAzKyknADMzErKCcDMrKScAMzM\nSsoJwMyspJwAzMxKqsMJQFJ3SfdLui4tbyzpbkmzJP1eUq9Uvkpanp3WD6zax/Gp/B+SRtf7YMzM\nrOM6cwdwNDCzavmHwBkRMQh4BTgslR8GvBIRmwFnpO2Q9BHgQOCjwO7AuZK6v7/wzcxsRXUoAUga\nAHwa+GVaFrArcGXa5GJg7/R+TFomrd8tbT8G+F1EvB0RTwCzgRH1OAgzM+u8jt4B/BQ4FngvLa8L\nvBoR76blucAG6f0GwDMAaf1rafsl5TU+s4Sk8ZKmSZo2b968ThyKmZl1RrsJQNJewIsRMb26uMam\n0c66tj6ztCDigogYFhHD+vXr1154Zma2gnp0YJsdgc9K2hPoDfQhuyNYW1KPdJU/AHg2bT8X2BCY\nK6kHsBYwv6q8ovozZmbWYO3eAUTE8RExICIGkj3EvSUiDgJuBT6XNhsLXJPeX5uWSetviYhI5Qem\nVkIbA4OAe+p2JGZm1ikduQNozXHA7yR9F7gfmJzKJwO/ljSb7Mr/QICIeETS5cCjwLvAURGx+H18\nv5mZvQ+dSgARMRWYmt4/To1WPBGxENi/lc9/D/heZ4M0M7P6c09gM7OScgIwMyspJwAzs5JyAjAz\nKyknADOzknICMDMrKScAM7OScgIwMyspJwAzs5JyAjAzKyknADOzknICMDMrKScAM7OScgIwMysp\nJwAzs5JyAjAzKyknADOzknICMDMrKScAM7OScgIwMyspJwAzs5JyAjAzKyknADOzkmo3AUjqLeke\nSQ9IekTSd1L5xpLuljRL0u8l9Urlq6Tl2Wn9wKp9HZ/K/yFpdFcdlJmZta8jdwBvA7tGxBBgW2B3\nSdsDPwTOiIhBwCvAYWn7w4BXImIz4Iy0HZI+AhwIfBTYHThXUvd6HoyZmXVcuwkgMm+kxZ7pFcCu\nwJWp/GJg7/R+TFomrd9NklL57yLi7Yh4ApgNjKjLUZiZWad16BmApO6SZgAvAlOAOcCrEfFu2mQu\nsEF6vwHwDEBa/xqwbnV5jc9Uf9d4SdMkTZs3b17nj8jMzDqkQwkgIhZHxLbAALKr9q1qbZZ+qpV1\nrZW3/K4LImJYRAzr169fR8IzM7MV0KlWQBHxKjAV2B5YW1KPtGoA8Gx6PxfYECCtXwuYX11e4zNm\nZtZgHWkF1E/S2un9qsB/ADOBW4HPpc3GAtek99emZdL6WyIiUvmBqZXQxsAg4J56HYiZmXVOj/Y3\noT9wcWqx0w24PCKuk/Qo8DtJ3wXuByan7ScDv5Y0m+zK/0CAiHhE0uXAo8C7wFERsbi+h2NmZh3V\nbgKIiAeB7WqUP06NVjwRsRDYv5V9fQ/4XufDNDOzenNPYDOzknICMDMrKScAM7OScgIwMyspJwAz\ns5JyAjAzKyknADOzknICMDMrKScAM7OScgIwMyspJwAzs5JyAjAzKyknADOzknICMDMrKScAM7OS\ncgIwMyspJwAzs5JyAjAzKyknADOzknICMDMrKScAM7OScgIwMyupdhOApA0l3SpppqRHJB2dyvtK\nmiJpVvq5TiqXpDMlzZb0oKShVfsam7afJWls1x2WmZm1pyN3AO8C34qIrYDtgaMkfQSYCNwcEYOA\nm9MywB7AoPQaD5wHWcIATgJGAiOAkypJw8zMGq/dBBARz0XEfen9AmAmsAEwBrg4bXYxsHd6Pwa4\nJDJ3AWtL6g+MBqZExPyIeAWYAuxe16MxM7MO69QzAEkDge2Au4EPRMRzkCUJYP202QbAM1Ufm5vK\nWitv+R3jJU2TNG3evHmdCc/MzDqhwwlA0hrAH4BjIuL1tjatURZtlC9bEHFBRAyLiGH9+vXraHhm\nZtZJHUoAknqSnfwvjYirUvELqWqH9PPFVD4X2LDq4wOAZ9soNzOzHHSkFZCAycDMiPhJ1aprgUpL\nnrHANVXlh6TWQNsDr6UqohuAUZLWSQ9/R6UyMzPLQY8ObLMjcDDwkKQZqex/gR8Al0s6DHga2D+t\n+zOwJzAbeAs4FCAi5ks6Bbg3bTcpIubX5SjMzKzT2k0AEXEntevvAXarsX0AR7WyrwuBCzsToJmZ\ndQ33BDYzKyknADOzknICMDMrqY48BC68gROvr8t+nvzBp+uyHzOzlUFTJIBCOnmtOu3ntfrsx8ys\nBVcBmZmVlBOAmVlJOQGYmZWUE4CZWUk5AZiZlZQTgJlZSTkBmJmVlBOAmVlJOQGYmZWUE4CZWUk5\nAZiZlZQTgJlZSTkBmJmVlBOAmVlJOQGYmZWUE4CZWUk5AZiZlVS7CUDShZJelPRwVVlfSVMkzUo/\n10nlknSmpNmSHpQ0tOozY9P2sySN7ZrDMTOzjurIHcCvgN1blE0Ebo6IQcDNaRlgD2BQeo0HzoMs\nYQAnASOBEcBJlaRhZmb5aDcBRMTtwPwWxWOAi9P7i4G9q8ovicxdwNqS+gOjgSkRMT8iXgGmsHxS\nMTOzBlrRZwAfiIjnANLP9VP5BsAzVdvNTWWtlS9H0nhJ0yRNmzdv3gqGZ2Zm7an3Q2DVKIs2ypcv\njLggIoZFxLB+/frVNTgzM1tqRRPAC6lqh/TzxVQ+F9iwarsBwLNtlJuZWU5WNAFcC1Ra8owFrqkq\nPyS1BtoeeC1VEd0AjJK0Tnr4OyqVmZlZTnq0t4Gky4BPAetJmkvWmucHwOWSDgOeBvZPm/8Z2BOY\nDbwFHAoQEfMlnQLcm7abFBEtHyybmVkDtZsAIuILrazarca2ARzVyn4uBC7sVHRmZtZl3BPYzKyk\nnADMzErKCcDMrKTafQZgzWPwxYPrtq+Hxj5Ut32ZWT6cAMzMutjNt2xat33ttuucuu3LVUBmZiXl\nBGBmVlJOAGZmJeUEYGZWUn4IbGZN5eSTTy7kvorIdwBmZiXlOwCzlcSPP79XXfbzrd9fV5f92MrP\ndwBmZiXlBGBmVlKuAjKzFTZ34h112c+AH3yyLvuxzvEdgJlZSfkOwKyFc468pW77Our8Xeu2L7N6\n8x2AmVlJOQGYmZWUE4CZWUk5AZiZlZQTgJlZSTkBmJmVVMMTgKTdJf1D0mxJExv9/WZmlmloApDU\nHTgH2AP4CPAFSR9pZAxmZpZp9B3ACGB2RDweEe8AvwPGNDgGMzMDFBGN+zLpc8DuEXF4Wj4YGBkR\nX6vaZjwwPi1uAfyjTl+/HvBSnfZVL46p44oYl2PqGMfUcfWKa6OI6NfeRo0eCkI1ypbJQBFxAXBB\n3b9YmhYRw+q93/fDMXVcEeNyTB3jmDqu0XE1ugpoLrBh1fIA4NkGx2BmZjQ+AdwLDJK0saRewIHA\ntQ2OwczMaHAVUES8K+lrwA1Ad+DCiHikQV9f92qlOnBMHVfEuBxTxzimjmtoXA19CGxmZsXhnsBm\nZiXlBGBmVlJNmwAkbZ13DC0VMSYzK6+mfQYg6U6gF/Ar4LcR8Wq+ERUzpqKSdGaN4teAaRFxTaPj\nqUVSN2CNiHg971iKRNKOwIyIeFPSl4ChwM8i4qmcQ7MWmvYOICI+ARxE1u9gmqTfSvpPx7Q8SUdL\n6qPMZEn3SRqVc1i9gW2BWem1DdAXOEzST/MKKv3N+khaHXgU+IekCXnFU1DnAW9JGgIcCzwFXJJv\nSCBpaI3XppJynRtd0o6Spkj6p6THJT0h6fGGfHlENPWLrLnpfsC/gJnAY8C+jmmZeB5IP0eT9csY\nAtyX8+/oFqBH1XKPVNYdeDTHuGaknwcBPwF6Ag/m/LvanqyPzRvAO8Bi4PUc47kv/TwROKy6LOff\n013p9zMNmA68nX5vjwOjcozrMbIBMtcH1q28GvHdTXsHIGkbSWeQnWB3BT4TEVul92c4pmVDSz/3\nBC6KiAeoPWxHI20ArF61vDrwoYhYTPYfNy89JfUE9gauiYhFOcZScTbwBbI7pVWBw4GzcoxngaTj\ngYOB69MowD1zjKfiSWC7iBgWER8DtgMeBv4D+FGOcb0WEX+JiBcj4uXKqxFfnOutTxc7G/gF8L8R\n8e9KYUQ8K+kEx7SM6ZJuBDYGjpe0JvBejvFA9h9yhqSpZMloJ+D7qerlphzj+jnZieQB4HZJG5E9\nm8hVRMyW1D0lyIsk/S3HcD4PfBH4ckQ8L+nDwGk5xlOxZVR1PI2IRyVtFxGPS7le79wq6TTgKqou\nbiLivq7+4qZ9CFxEko6JiJ+2KDs6In6WV0wphm5k9e2PR8SrktYFNoiIB3OOqz/ZEOIC7omI3MeN\nkrRxRDxRtSxgs4iYlWNMt5Ndxf4SeB54DhgXEUNyjGkjYFBE3CRpNaB7RCzIK54U0++B+WTD0EOW\nqNYju1O5MyKG5xTXrTWKIyJ27fLvbtYEIGkQcCrZxDO9K+URsUmOMd0XEUNblN0fEdvlFVOKQWR1\n2ptExKR0xfbBiLgn57g2ADai6k41Im7PL6JW/4bTU5VCXjFtBLxA1sLsG8BawDkRMSeneI4gG9K9\nb0Rsmv4vnh8Ru+URT1VcqwJfBT5BdlFxJ3AusBBYLSLeyDG8XDRzFdBFwElkdeu7AIeSU722pC+Q\n3RJvLKl68Ls1gYbU9bXjXLIqn12BScAC4A9ALldEAJJ+SHaF9ghLq6MCyCUBSNoS+CiwlqR9q1b1\noeoCIyd7p7vIhcB3ILuzBPK6szyK7M7tboCImCVp/ZxiWSJVu/44vVrK7eQvaS2yc9VOqeg2YFJE\ndHnVYjMngFUj4mZJiqz98cmS7iD7RTfa38huy9dj2X98C4Bcq1mSkRExVNL9ABHxShqtNU97A1tE\nRJ4PfKttAewFrA18pqp8AXBELhEtNZblT/bjapQ1ytsR8U6lXj01s8y9qiH1TziZ5e8qc6sVSC4k\nexh9QFo+mOwCdt9WP1EnzZwAFqa67VlpBNJ/kTWzariUgJ4Cdsjj+ztgUWqpEQCS+pH/Q+DHyVqO\nFCIBRNb57BpJO0TE3/OOBwp9Z3mbpP8FVk39XL4K/CnHeComk1WRTSdrKlsUm0bEflXL35E0oxFf\n3MwJ4BhgNeC/gVPIqjfG5hGIpDsj4hOSFrDslZDIHvb0ySOuKmcCfwTWl/Q94HPA/8s3JN4iawV0\nM8u2jPjv/EIC4OUU0wciYmtJ2wCfjYjv5hBLUe8sJwKHAQ8BXwH+TPaAOm+vRcRf8g6ihn9L+kRE\n3AlL7lT+3c5n6qJpHwJb56Q67t3IktLNETEz53hqJuuIuLjRsVSTdBswAfh55eG9pIcjwuM8FZyk\nH5B1JGx4c8u2SNoWuJjs4b0Sk6icAAAT00lEQVTIWiqNS/1xuva7my0BSPoTbdQ3RsRnGxgOAJL6\ntrU+IuY3KpZaJP06Ig5ur8xA0r0RMby69ZakGRGxbY4xbU/W8WsrspZA3YE3G31nKekh2v6/t00D\nw1lOns0tO0JSH4Bo4NhSzVgFdHreAdQwnew/Rq1WSAHk/RDqo9UL6XlALs0aJV0eEQe0djLJ+yQC\nvCRpU5Y+L/kcWTVMns4mm171CmAYcAiwWQ5x7JXDd3ZYROySdwzVJH0pIn4j6ZstygGIiJ90dQxN\nlwAi4rbK+9SSZUuy/6z/iIh3copp4zy+tz2pu37lYd3rLE1Q75DflHlHp59FPZkcRfa72VLSv4An\ngC/lG1IxegJHQUf7bO1EW9GIE20rKkOdrFljXUOqZpouAVRI+jRwPjCH7MS2saSv5PEQSNKWEfGY\npKG11udVBxkRpwKnSjo1Io7PI4aWIqJyNf2Rln8rSUeS/U1zExGPA/+RhqTolnfv1uStdLEzQ9KP\nyO5IVm/nM12mRWOHXmStuRpeJVWlrRNtbiLi5+ntTRHxf9Xr0oPgLtd0zwAqJD0G7BURs9PypsD1\nEbFlDrFcEBHji1wHKWkdYBDL9prOrddtuoI9ISJuScvHAZ+KiD3yiinF8X3gR5Hmcki/t29FRG5j\nObXSE/jcyr/9vEnaGxgREf+bdyxF1Erv8uXKuuS7mzgB3B4RO1UtC7ituswykg4nq3oZAMwgG174\n73kmJknrAdeRtbjZnawq78DIefTNWkN3NOo/a1tS3w0iYl6ecbRG0l0RsX3OMWxC1jlue7I7lL8D\n30h3dXnEswPwcbIm69WjAfcB9mnEWE5NWwUEPCLpz8DlZH/s/YF7K934I+KqRgckqTdLxyIJ4A6y\nMVIWNjqWFo4mG/bhrojYJTUJ/U6eAUXES5I+Szby53Tgc1GMq5Xuklap9FBO48uskkcg6aLmJOBr\nZNWc3SS9C5wVEZPyiCnFVd2DtRvZg+ki/O1+C5wD7JOWDwQuA0bmFE8vYA2y83B19dTrZH1xulwz\n3wFc1MbqiIgvNyyYRNLlZJ10fpOKvgCsExH7NzqWalVNG2eQDQvxdl5NG2t0lusFvJvKcu80J+lY\n4LNkXfUD+DJwbUQ0fDx5Sd8gm8NhfKQRStNV7nnAXyMir3kvqv/vvUs2fPYvIuLFPOKpkHR3RIxs\nUVaEO5ON8nqA3rQJoIgkPdDytq5WWaNJ+iPZYHnHkPWYfgXoGRF75hlXUUnag6Wd5m6MiBtyiuN+\n4D8j4qUW5f1SXLmOMltN0uoR8WbOMfwAeJVsOOggG2xwFbK7gtz646S/17FkzbGrn8F5OOgVlVpD\nfJesS/VfyaY5PCYiftPmB7s2pl+RVfnclZZHAmMj4qt5xdSSpJ3JHiL+Na9ms1WxFG446CJpqwdy\nXr2T09+sP9k0me8oGwX0GLKerR9qdDwtYnuijdUROQ0Kp2wypt8D/wMcSTZkzbyIOK6rv7uZnwGM\niohjJe0DzCV7BnArS6tfGqaqU1NP4BBJT6fljcgmFs9FKz2UH0o/1yDrkp4LLR0O+lGWDtyV23DQ\nFUXpdZu0laAbnrwlHQN8G5gNrCLpZ2TzJl9CTh0LqxW1Pw7Z/L+TlU0OdRvZYHq3tfupOmjmBFCZ\ng3RP4LKImK/8pn0raqem6h7KHyar+hHZkMdPk00RmZeiDQddUZRetwBDUge+lkQ+cxSMJ/ubzVc2\nqdBsYKfKHW/eJB1SqzwiLml0LC1UWrY9l/ovPUvWIq/LNXMC+FPqC/Bv4Kupni2X1jYtH/Ck2+K8\nJxFZckUk6XyyB5l/Tst7kE0xmKdCDQddrQi9blMc3fP43jYsrNSjR8TTkv5ZlJN/Uj3BUW+y5zj3\nkd2h5Om7yiaF+RbZ3WUfsv4cXa5pnwHAkk46r0fEYmXzkvaJiOdzjOezZMP2fgh4kawKaGZEfLTN\nD3Z9XMtNaShpWkQMyzGmP5A9tynUcNAq4Py7RSHpRZbOtwvZndKS5bz/di2lk+6vI4cBIqti6A78\nd14ttpruDkDSrhFxS3Vb5BZVPw1v/1/lFLJOKDdFxHaSdiFrCpq3lySdQPZ8JMjGtsl7qspr06to\nDiZr2/41squ0DYH92vxEeUxosTw9lyg67i2y3u+5SRenn2XZjmAN03R3AJK+ExEntdIPIJf2/xWV\nq2pJDwDbRcR7ku6JiBF5xZTi6suyc5LeDnwnr2ZxRSTpwxHxdN5x2IrTskPFdyd7kH95REzMLypQ\nNgnTWmQtgZY0lW3EGGFNlwCKTNJNZA83TyWbxelFYHhEfDzXwJJ0S/xengOcqaDDQVcP9yDpD7Hs\nFH62EkhNnCveBZ6KiLl5xVOR5xhhTZcA1MqQrxWR39CvKBtBciFZK42DyLL+pRGRa3WLpOFkE1NX\nuqO/Bnw5Ihp+Cy+pf0Q8lwY4W05uPSaXnQBmufGAbOUg6QMsfRh8T969kyHrvd1yPKJaZV2h6Z4B\nsPQktgXZH7pSj/wZcm5D3qInZK5TG7YwGfhqRNwBIOkTZEMdNPxqO9Jw0Hmd6NsQrby3lYSkA4DT\ngKlkF2FnSZoQEVfmGhhcCbQcTPAKGtB3oukSQER8B5b0rhtaqc6QdDLZL7Xhaoxvs2QVBRjfBlhQ\nOfkDRMSdKeaGK/DvqtLmXiydQKcIcRVOanJ9BDCQZXtx5/b8Lfk2WZXri7AkzpvITsANlwZd/Ciw\nVosB9PrQoGbiTZcAqnyYZXtDvkP2D7LhIqJQE1HUcI+kn5ONjFgZI2Wq0gQ2jXgYVVHU31UB29wX\n2TVkI93exNJe3EXQrUWVz8tkLbrysgVZJ9G1yWooKhaQJdAu13TPACokfRs4APgj2UltH+D3kc2C\n1ehY+kTE660MvVCESeErD6Eq/xiq28025GGUNY+8RpJtj6TTyKo1L0tFnycbs6jLx9xpi6QdIuLv\nuXx3syYAgHQF+8m0eHtE3J9THNdFxF5pMKqWk8PnOQhV5YF5JZ4A5gF3Rhpe2KyzJH0X+FulZ3mR\npKqWT5D9m789Iv6Yc0i5DlzZ1AnA2ibppBrFfYHRwMkR8bsa683alJ7jrE5W7VoZ5ybX5ySpx+0N\nEZH3ECfLqdwxpYEr9ybrYHhrI3qXN/MzgEJSgeberTwwbylVVd3Est36zTqkiM9xUo/btyStFRGv\n5R1PC7kNXOkE0EBqZe5dsklYCiP9A8xt6FQo3LDL1klpeINKz/KpEXFdnvEkC4GHJE1h2R63eY9R\nlNvAlU4AjVW4uXdrkVSZFSxPRRp22TpB2cxbw4FLU9HRkj6R95ALwPXpBbUbPOQiIiam+S8qA1e+\nCYxpxHc3bQIo6BXkwohYKAllE4s/JmmLvIJpZbiFvmTjkdccO72RijLssnXansC2EfEegKSLgfuB\nXBKApDHAgIg4Jy3fA/Qj+7efWwugIgxc2bQJgGJeQc6VtDZwNTBF0itkJ9u8tJyoJoCXI+e5W5O3\nJPUCZqRWEs+RPVi0lcPaLJ1Rbq08AyGbb/fAquVeZL1s1yDr8Z5LB1FgZ+AWlu0DUBE0IAE0bSug\nqpE3H6wMICbpbwUaeK0wc+8WURoL6AWy/6zfIPtdnRMRc3INzNol6QvAD8imYBXZs4Dj82pVJune\niBhetXx2RHwtvb8rIrbPI64iaOYEULiJO5TNkfr7iHBVRjuUzY/6s/bKrJgk9Sd7DiDg7sh3IqbZ\nEVHz7l/SnIjYtNExtYih1gCWrwHTI2JGV353nt2gu1r1xB1vkk3csW+bn+h69wEnSJot6TRJuc24\ntRIYW6NsXKODsI5LjRoqHTD7A3OBZ4APVYYVycndkpYbWkHSV4B7coinpWHAkcAG6TUe+BTwC0nH\nduUXN/MdQGGvIFM7+/3I6iU/HBG5zkpUJKn64ItkvTXvqFq1JrC4iB15LCPpgogYn+f49rUom4P7\narKpRSvjWn0MWAXYOyJeyCOuCkk3APtFxBtpeQ2yAer2IbsL+EhXfXczPwQeC7Q82Y+rUZaHzYAt\nyQanezTfUArnb2TVdeuRzZ9csQB4MJeIrEMiYnx6u0dELNOOXVJDRresJQ0A9/HUvLky//b1EXFL\nXjG10HLgykXARhHxb0lvt/KZumi6BFB1BbmxpOo5Zdck53luU1vffYE5wOXAKRHxap4xFU2aB+Ap\nYIe8Y7EV9jeWH9++VllDpRN+UU761X4L3CXpmrT8GeCyNIFUl14gNl0CoNhXkE8AO0TESznHUXgF\n7cdhbZD0QbI67FUlbcfSTlZ9gNVyC6zgIuIUSX9m6SB1R0bEtLT6oK787qZ9BlBURRoLqMgkTaNG\nP46I+HaugVmrJI0lq2YdBkyrWrUA+FVEdHm79pVVmoVvUERclIaCWKMRI/I2bQIo4hVka2MBebz9\n5RW9H4e1TtJ+EfGHvONYWaRReYcBW0TE5pI+BFwRETt29Xc3YxVQRRF7Aq8UYwEVhHsCr6Qi4g+S\nPk32wLX6TndSflEV2j7AdqQWShHxrKSGjKjazP0AiIjZQPeIWBwRFwG75BzSwkrriMpYQGTTwtny\navXj2C/XiKxDJJ1PNtvW18nqtPcHNso1qGJ7J7KqmABID38bopnvAIp4BVm0sYAKKyKeSnWhrc5b\nYIX18YjYJlXffUfSj2nAuDYrscuVzcm9duqw9mWyEQy6XDM/A6g1lsy56a4gdx4LqLY0D8FJZFf+\nIrsLeBc4y1UIKwdJd0fESEl3kTV7fhl42B0eWyfpP4FRZP/mb4iIKQ353mZNAABVV5Dz8o7FOkbS\nN8iGEx5faQUhaRPgPLJkeUae8Vn7JP0/sgYYuwHnkFVt/DIi/l+uga0klE1feWBEXNruxu/3u5ot\nAfgKcuUm6X7gP1v2lUjJ/MaI2C6fyGxFSFoF6F3AaRhzJ6kPcBRZ34lrgSlpeQIwIyK6fFKYZnwI\nfAywIzA8ItaNiHWAkcCO6erSiq1nrY5y6S6uZ43trWAkzZF0JEBEvB0Rr0kqwpSQRfNrskYgDwGH\nAzeSPTAf04iTPzTnHYCvIFdiku6LiJpDBrS1zopD2fy2DwBvAV+JiHck3e//e8uS9FBEDE7vuwMv\nkQ0OuaBRMTTjHUBhryAlbS/pXklvSHpH0mJJr+cZUwENkfR6jdcCYHDewVmHvBURnwdmAnekBhnN\ndaVZH4sqb9K0p0808uQPzdkMtK0WNXm3tili57RCiYjuecdg75sAIuJHkqYDN5DNNW3LGlJ1ASiy\nMZReT++jEaMWNGMCGNLKVbWo6pWYF090biVwYuVNRNwsaTS1J/gptSJc7DRdAijCL7UNReycZlYX\nkrZMvdv/VWMGMD8ELqCmewhcZK10TvNE59YUJP0iIo4o2oxg1jongAYq8jSVZlY+TgANVKsZo5vH\nWbOQtG9b6z0fQPE03TOAIiryNJVmdfSZNtYFHhCucHwH0ACp7n9j4FRgYtWqBcCDEfFuLoGZWak5\nAZhZ3XlCmJVDM/YELiz3BLYy8IQwKw8ngMY6G/gCMAtYlWwAqLNyjcis/j4eEYcAr6TJfHYgm9HN\nCsYJoMEKOE2lWb39O/18K01wvojsGZgVjFsBNZZ7AlsZXJemPj2NbKLzoEFTHFrn+CFwAxV9mkqz\nevOEMMXmBNBgnqbSml0a2/7TwECqahki4id5xWS1uQqoAWpNUynJ01Ras/oTsJBspqv3co7F2uAE\n0BjV01QuM9G5pG94onNrMgMiYpu8g7D2uQqoATxNpZWJpB8CN0fEjXnHYm3zHUBjtDpNpSRPdG7N\n5i7gj5K6kTUBbdgMV9Y5TgCNUeRpKs3q7cdknb8eClcxFJoTQGMUeppKszqbBTzsk3/xOQE0QMGn\nqTSrt+eAqZL+ArxdKXQz0OJxAjCzensivXqllxWUE4CZ1U3qBLZGREzIOxZrnweDM7O6iYjFwNB2\nN7RC8B2AmdXbjDT16RXAm5VCzwlcPE4AZlZvfcnmut61qsxzAheQewKbmZWUnwGYWV1JGiDpj5Je\nlPSCpD9IGpB3XLY8JwAzq7eLgGuBDwEbkI0OelGuEVlNrgIys7qSNCMitm2vzPLnOwAzq7eXJH1J\nUvf0+hLZQ2ErGN8BmFldSfowcDbZgHAB/A04OiKeyjUwW44TgJlZSbkfgJnVhaQT21gdEXFKw4Kx\nDvEdgJnVhaRv1SheHTgMWDci1mhwSNYOJwAzqztJawJHk538Lwd+HBEv5huVteQqIDOrG0l9gW8C\nBwEXA0Mj4pV8o7LWOAGYWV1IOg3YF7gAGBwRb+QckrXDVUBmVheS3iObAexdsuafS1bhSeELyQnA\nzKyk3BPYzKyknADMzErKCcDMrKScAKz0JK0raUZ6PS/pX1XLvTq5r4skbdFVsZrVkx8Cm1WRdDLw\nRkScnncsZl3NdwBmbZB0rKSH0+vrqWwzSY9I+rWkhyRdLmnVtO5OSdum95+WdJ+kByTdmOdxmNXi\nBGDWCkkjyHq0jiAb2virkrZJqz8CnBMRg4GFwFdafPaDwHnAPhExBDiwYYGbdZATgFnrPgn8ISLe\niogFwNXAJ9K6JyLirvT+N1XlFTsAt1bGwI+I+Y0I2KwznADMWqc21rV8eNZyWTXKzArFCcCsdbcD\n+0haVdIawBjgjrRuY0nD0/svAHe2+Oz/AbtK2giWDJJmVihOAGatiIh7gMuAe4G7gPMi4qG0+hHg\nCEkPko15f0GLz74A/BdwjaQHgEsbFrhZB7kZqFknSdoMuDIits07FrP3w3cAZmYl5TsAM7OS8h2A\nmVlJOQGYmZWUE4CZWUk5AZiZlZQTgJlZSf1/FizybYt5n1YAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plotting the requests and their quantities after the topics were removed and merged\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "from collections import Counter\n",
    "\n",
    "casess_counter=dict(Counter(merged_cases_A['Categorie']))\n",
    "df_2=pd.DataFrame(list(casess_counter.items()),\n",
    "                      columns=['Topic','Number of cases'])\n",
    "\n",
    "\n",
    "ax = df_2.plot.bar(x='Topic', y='Number of cases', rot=0)\n",
    "plt.title('Cases by categories')\n",
    "plt.xticks(rotation=90)\n",
    "plt.show()\n",
    "fname=\"label.png\"\n",
    "plt.savefig(fname, bbox_inches=\"tight\")\n",
    "plt.close(\"all\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(13092, 109)\n"
     ]
    }
   ],
   "source": [
    "# Create a Terms Matrix\n",
    "\n",
    "import pickle\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "tfidf = TfidfVectorizer(sublinear_tf=True, min_df=0.2, norm='l2', encoding='latin-1', ngram_range=(1, 2), stop_words='english')\n",
    "features = tfidf.fit_transform(merged_cases_A.Mixed_Comments).toarray()\n",
    "labels = merged_cases_A.Categorie\n",
    "print(features.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "158779"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Data before cleaning\n",
    "all_text=''\n",
    "for i in df['Case Comments']:\n",
    "    all_text+=str(' '+ str(i))\n",
    "\n",
    "\n",
    "from collections import Counter\n",
    "split_clean_text=all_text.split()\n",
    "clean_text_counter=Counter(split_clean_text)\n",
    "len(clean_text_counter)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "47974"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Data after cleaning\n",
    "all_text_clean=''\n",
    "\n",
    "for i in merged_cases_A['Mixed_Comments']:\n",
    "    all_text_clean+=str(' '+ str(i))\n",
    "\n",
    "split_clean_text=all_text_clean.split()\n",
    "clean_text_counter=Counter(split_clean_text)\n",
    "len(clean_text_counter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(13092, 48547)\n",
      "# 'Data':\n",
      "  . Most correlated unigrams:\n",
      ". shor\n",
      ". liron\n",
      ". shai\n",
      ". ihp\n",
      ". shir\n",
      "  . Most correlated bigrams:\n",
      ". central analyst\n",
      ". tier analyst\n",
      ". lexisnexis records\n",
      ". invalid characters\n",
      ". ronit primo\n",
      "# 'Data availability':\n",
      "  . Most correlated unigrams:\n",
      ". indication\n",
      ". holdings\n",
      ". available\n",
      ". false\n",
      ". availability\n",
      "  . Most correlated bigrams:\n",
      ". ft ft\n",
      ". availability indication\n",
      ". false positivenegative\n",
      ". file url\n",
      ". text available\n",
      "# 'Data display':\n",
      "  . Most correlated unigrams:\n",
      ". author\n",
      ". authors\n",
      ". incomplete\n",
      ". characters\n",
      ". display\n",
      "  . Most correlated bigrams:\n",
      ". does data\n",
      ". data exist\n",
      ". expected display\n",
      ". display proof\n",
      ". exist source\n",
      "# 'Data linking':\n",
      "  . Most correlated unigrams:\n",
      ". link\n",
      ". openurl\n",
      ". broken\n",
      ". linking\n",
      ". target\n",
      "  . Most correlated bigrams:\n",
      ". right information\n",
      ". giving right\n",
      ". target url\n",
      ". linking issue\n",
      ". broken link\n",
      "# 'Facets':\n",
      "  . Most correlated unigrams:\n",
      ". peerreviewed\n",
      ". refine\n",
      ". vals\n",
      ". facets\n",
      ". facet\n",
      "  . Most correlated bigrams:\n",
      ". collection facet\n",
      ". resource type\n",
      ". type facet\n",
      ". facet value\n",
      ". language facet\n",
      "# 'Grouping':\n",
      "  . Most correlated unigrams:\n",
      ". duplicate\n",
      ". deduping\n",
      ". frbr\n",
      ". grouping\n",
      ". grouped\n",
      "  . Most correlated bigrams:\n",
      ". behavior grouped\n",
      ". grouping customers\n",
      ". data add\n",
      ". add fix\n",
      ". records grouped\n",
      "# 'Normalization Rules':\n",
      "  . Most correlated unigrams:\n",
      ". mapping\n",
      ". bepress\n",
      ". lds\n",
      ". nr\n",
      ". normalization\n",
      "  . Most correlated bigrams:\n",
      ". data resolved\n",
      ". nr need\n",
      ". wiso wiso\n",
      ". digital commons\n",
      ". author names\n",
      "# 'Registration':\n",
      "  . Most correlated unigrams:\n",
      ". collections\n",
      ". profile\n",
      ". activations\n",
      ". deactivated\n",
      ". registration\n",
      "  . Most correlated bigrams:\n",
      ". export active\n",
      ". new collections\n",
      ". collections update\n",
      ". active resources\n",
      ". central office\n",
      "# 'Updates':\n",
      "  . Most correlated unigrams:\n",
      ". downloader\n",
      ". ronit\n",
      ". frequency\n",
      ". ftp\n",
      ". updates\n",
      "  . Most correlated bigrams:\n",
      ". update frequency\n",
      ". hi ronit\n",
      ". running properly\n",
      ". thanks ronit\n",
      ". updates hi\n"
     ]
    }
   ],
   "source": [
    "# Split the data to test and train\n",
    "# Finding the most correlated words for each topic using TF-IDF algorithm\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import pydotplus\n",
    "import graphviz\n",
    "import matplotlib.pyplot as plt\n",
    "import re\n",
    "from io import StringIO\n",
    "from sklearn.feature_selection import chi2\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "merged_cases_A['problem_id'] = merged_cases_A['Categorie'].factorize()[0]\n",
    "\n",
    "problem_id_df = merged_cases_A[['Categorie', 'problem_id']].drop_duplicates().sort_values('problem_id')\n",
    "problem_to_id = dict(problem_id_df.values)\n",
    "id_to_topic = dict(problem_id_df[['problem_id', 'Categorie']].values)\n",
    "\n",
    "tfidf = TfidfVectorizer(sublinear_tf=True, min_df=5, norm='l2', encoding='latin-1', ngram_range=(1, 2), stop_words='english')\n",
    "features = tfidf.fit_transform(merged_cases_A.Mixed_Comments).toarray()\n",
    "labels = merged_cases_A.problem_id\n",
    "print(features.shape)\n",
    "\n",
    "N = 5\n",
    "for topic, problem_id in sorted(problem_to_id.items()):\n",
    "  features_chi2 = chi2(features, labels == problem_id)\n",
    "  indices = np.argsort(features_chi2[0])\n",
    "  feature_names = np.array(tfidf.get_feature_names())[indices]\n",
    "  unigrams = [v for v in feature_names if len(v.split(' ')) == 1]\n",
    "  bigrams = [v for v in feature_names if len(v.split(' ')) == 2]\n",
    "  print(\"# '{}':\".format(topic))\n",
    "  print(\"  . Most correlated unigrams:\\n. {}\".format('\\n. '.join(unigrams[-N:])))\n",
    "  print(\"  . Most correlated bigrams:\\n. {}\".format('\\n. '.join(bigrams[-N:])))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\gensim\\utils.py:1197: UserWarning: detected Windows; aliasing chunkize to chunkize_serial\n",
      "  warnings.warn(\"detected Windows; aliasing chunkize to chunkize_serial\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, '0.028*\"case\" + 0.027*\"record\" + 0.024*\"link\" + 0.023*\"thank\"'), (1, '0.030*\"articl\" + 0.024*\"link\" + 0.022*\"record\" + 0.017*\"primo\"'), (2, '0.020*\"thank\" + 0.020*\"record\" + 0.018*\"case\" + 0.017*\"updat\"'), (3, '0.028*\"email\" + 0.018*\"isbn\" + 0.011*\"de\" + 0.008*\"book\"')]\n"
     ]
    }
   ],
   "source": [
    "#Training LDA model\n",
    "from nltk.tokenize import RegexpTokenizer\n",
    "from stop_words import get_stop_words\n",
    "from nltk.stem.porter import PorterStemmer\n",
    "from gensim import corpora, models\n",
    "import gensim\n",
    "\n",
    "tokenizer = RegexpTokenizer(r'\\w+')\n",
    "en_stop = get_stop_words('en')\n",
    "p_stemmer = PorterStemmer()\n",
    "\n",
    "\n",
    "data_list=[]\n",
    "for i in merged_cases_A['Mixed_Comments']:\n",
    "    data_list.append(i)\n",
    "\n",
    "texts = []\n",
    "for i in data_list:\n",
    "    raw = i.lower()\n",
    "    tokens = tokenizer.tokenize(raw)\n",
    "    stopped_tokens = [i for i in tokens if not i in en_stop]\n",
    "    stemmed_tokens = [p_stemmer.stem(i) for i in stopped_tokens]\n",
    "    texts.append(stemmed_tokens)\n",
    "\n",
    "\n",
    "dictionary = corpora.Dictionary(texts)\n",
    "corpus = [dictionary.doc2bow(text) for text in texts]\n",
    "ldamodel = gensim.models.ldamodel.LdaModel(corpus, num_topics=4, id2word = dictionary, passes=20)\n",
    "\n",
    "print(ldamodel.print_topics(num_topics=9, num_words=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Training a Word2Vev Model\n",
    "# Create a cleaning function for the Word2Vev model\n",
    "\n",
    "import re\n",
    "def my_function_for_w2v(raw):\n",
    "    raw=raw.lower() \n",
    "    raw= raw.replace(']', '')\n",
    "    raw= raw.replace('[', '')\n",
    "    raw= raw.replace(')', '')\n",
    "    raw= raw.replace('(', '')\n",
    "    raw= raw.replace(':', '')\n",
    "    raw= raw.replace('.', '')\n",
    "    raw= raw.replace(',', '')\n",
    "    raw= raw.replace('  ', ' ')\n",
    "    raw= raw.replace('\"', '')\n",
    "    raw= raw.replace('\\t', ' ')\n",
    "    raw= raw.replace('?', '')\n",
    "    raw= re.sub(r\"http\\S+\", \"\", raw)\n",
    "    raw= re.sub('https?://(?:[-\\w.]|(?:%[\\da-fA-F]{2}))+', \"\", raw)\n",
    "    raw= re.sub(\" \\d{13} \", \" isbn \", raw)\n",
    "    raw= re.sub(\" \\d{10} \", \" isbn \", raw)\n",
    "    raw= re.sub(\" \\d{7}\\d{1}[\\dx] \", \" issn \", raw)\n",
    "    raw= re.sub(\" \\d{4}[-]\\d{3}[\\dx] \", \" issn \", raw)\n",
    "    raw= re.sub(\" 10\\.\\S+ \", \" doi \", raw)\n",
    "    raw= re.sub('<.*>', \"\", raw)\n",
    "    raw= re.sub('\\S+@\\S+', \"email\", raw)\n",
    "    raw= re.sub('[0-9]+', \"\", raw)\n",
    "    raw= re.sub(r'(\\d+/\\d+/\\d+)', \"date\", raw)\n",
    "    for i in raw.split():\n",
    "        if len(i) > 22:\n",
    "            raw = raw.replace(i,'')\n",
    "    raw= raw.replace('$', '')\n",
    "    raw = raw.replace('!', '')\n",
    "    raw = raw.replace(\"'\", '')\n",
    "    raw = raw.replace(\"->\", '')\n",
    "    raw= raw.replace('&', '')\n",
    "    raw= raw.replace('/', '')\n",
    "    raw= raw.replace('%', ' ')\n",
    "    raw= raw.replace(' - ', ' ')\n",
    "    raw= raw.replace('+', '')\n",
    "    raw= raw.replace('_', '')\n",
    "    raw= raw.replace('@', '')\n",
    "    raw = raw.replace('--', '')\n",
    "    raw = raw.replace('#', '')\n",
    "    raw = raw.replace('=', '')\n",
    "    raw = raw.replace('â', '')\n",
    "    raw = raw.replace('*', '')\n",
    "    raw = raw.replace('-', '')\n",
    "    raw = raw.replace(';', '')\n",
    "    raw = raw.replace('<', '')\n",
    "    raw = raw.replace('>', '')\n",
    "    raw = raw.replace('ß', '')\n",
    "    ' '.join(raw.split())\n",
    "    raw= raw.replace('  ', ' ')\n",
    "    raw= raw.replace('  ', ' ')\n",
    "\n",
    "\n",
    "    return raw\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Case Number</th>\n",
       "      <th>Mixed Comments</th>\n",
       "      <th>Categorie</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10087</td>\n",
       "      <td>status st line wi\\ndear christian\\n\\nthis is ...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10087</td>\n",
       "      <td>rodopi ingentaconnect special character  migra...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10099</td>\n",
       "      <td>hi rachel\\n\\nolga performed an in house analy...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10099</td>\n",
       "      <td>directory of open access journals doaj wrong r...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11750</td>\n",
       "      <td>status st line wi\\ndear kasper\\n\\nthank you f...</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Case Number                                     Mixed Comments     Categorie\n",
       "0       10087   status st line wi\\ndear christian\\n\\nthis is ...  Data display\n",
       "1       10087  rodopi ingentaconnect special character  migra...  Data display\n",
       "2       10099   hi rachel\\n\\nolga performed an in house analy...  Data display\n",
       "3       10099  directory of open access journals doaj wrong r...  Data display\n",
       "4       11750   status st line wi\\ndear kasper\\n\\nthank you f...  Data display"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Create a new data frame for the Word2Vev model \n",
    "\n",
    "merged_cases_for_w2v = pd.DataFrame( columns=['Case Number','Mixed Comments','Categorie'],)\n",
    "cases_by_id_for_w2v=df.groupby('Case Number')\n",
    "\n",
    "counter=0\n",
    "for i in cases_dict.keys():\n",
    "    case_for_w2v=cases_by_id_for_w2v.get_group(i)\n",
    "\n",
    "    mix_comment_for_w2v=''\n",
    "    \n",
    "    for x in case_for_w2v['Title']:\n",
    "        title_for_w2v=x\n",
    "    for y in case_for_w2v['Description']:\n",
    "        description_for_w2v=y\n",
    "    mix_title_for_w2v=my_function_for_w2v(str(title_for_w2v)) + ' ' + my_function_for_w2v(str(description_for_w2v))\n",
    "    for comment_for_w2v in case_for_w2v['Case Comments']:\n",
    "        if len(str(comment_for_w2v).split())>4:\n",
    "            mix_comment_for_w2v+=(' '+my_function_for_w2v(str(comment_for_w2v)))\n",
    "        \n",
    "    for t in case_for_w2v['Topic - for PC']:\n",
    "        topic_for_w2v=t\n",
    "    merged_cases_for_w2v.loc[counter]=[i,mix_comment_for_w2v,topic_for_w2v]\n",
    "    counter+=1\n",
    "    merged_cases_for_w2v.loc[counter]=[i,mix_title_for_w2v,topic_for_w2v]\n",
    "    counter+=1\n",
    " \n",
    "\n",
    "merged_cases_for_w2v.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Preparing the data to fit the model\n",
    "clean_all_text=''\n",
    "for i in merged_cases_for_w2v['Mixed Comments']:\n",
    "    clean_all_text+=my_function_for_w2v(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Removing stop words and splitting the text to splitted sentences\n",
    "import re\n",
    "from nltk.corpus import stopwords\n",
    "enter_model=[]\n",
    "pattern = re.compile(r'\\b(' + r'|'.join(stopwords.words('english')) + r')\\b\\s')\n",
    "text_without_stopwords = pattern.sub('', clean_all_text)\n",
    "text_without_stopwords=text_without_stopwords.split('\\n')\n",
    "for i in text_without_stopwords:\n",
    "    if len(i.split())>4:\n",
    "        enter_model.append(i.split())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\gensim\\utils.py:1197: UserWarning: detected Windows; aliasing chunkize to chunkize_serial\n",
      "  warnings.warn(\"detected Windows; aliasing chunkize to chunkize_serial\")\n",
      "2019-01-28 00:37:45,515:WARNING:consider setting layer size to a multiple of 4 for greater performance\n",
      "2019-01-28 00:37:45,524:INFO:collecting all words and their counts\n",
      "2019-01-28 00:37:45,529:INFO:PROGRESS: at sentence #0, processed 0 words, keeping 0 word types\n",
      "2019-01-28 00:37:45,667:INFO:PROGRESS: at sentence #10000, processed 112739 words, keeping 8504 word types\n",
      "2019-01-28 00:37:45,815:INFO:PROGRESS: at sentence #20000, processed 228260 words, keeping 12711 word types\n",
      "2019-01-28 00:37:45,967:INFO:PROGRESS: at sentence #30000, processed 343594 words, keeping 15949 word types\n",
      "2019-01-28 00:37:46,113:INFO:PROGRESS: at sentence #40000, processed 456267 words, keeping 18670 word types\n",
      "2019-01-28 00:37:46,248:INFO:PROGRESS: at sentence #50000, processed 566473 words, keeping 21368 word types\n",
      "2019-01-28 00:37:46,380:INFO:PROGRESS: at sentence #60000, processed 672937 words, keeping 23824 word types\n",
      "2019-01-28 00:37:46,510:INFO:PROGRESS: at sentence #70000, processed 777152 words, keeping 26368 word types\n",
      "2019-01-28 00:37:46,629:INFO:PROGRESS: at sentence #80000, processed 882660 words, keeping 28448 word types\n",
      "2019-01-28 00:37:46,740:INFO:PROGRESS: at sentence #90000, processed 987074 words, keeping 30564 word types\n",
      "2019-01-28 00:37:46,847:INFO:PROGRESS: at sentence #100000, processed 1094007 words, keeping 32795 word types\n",
      "2019-01-28 00:37:46,936:INFO:PROGRESS: at sentence #110000, processed 1197026 words, keeping 34680 word types\n",
      "2019-01-28 00:37:47,042:INFO:PROGRESS: at sentence #120000, processed 1302837 words, keeping 36747 word types\n",
      "2019-01-28 00:37:47,159:INFO:PROGRESS: at sentence #130000, processed 1411465 words, keeping 39019 word types\n",
      "2019-01-28 00:37:47,265:INFO:PROGRESS: at sentence #140000, processed 1519522 words, keeping 41179 word types\n",
      "2019-01-28 00:37:47,348:INFO:PROGRESS: at sentence #150000, processed 1628847 words, keeping 43086 word types\n",
      "2019-01-28 00:37:47,418:INFO:PROGRESS: at sentence #160000, processed 1736191 words, keeping 45160 word types\n",
      "2019-01-28 00:37:47,539:INFO:PROGRESS: at sentence #170000, processed 1842372 words, keeping 47464 word types\n",
      "2019-01-28 00:37:47,553:INFO:collected 47851 word types from a corpus of 1862560 raw words and 171901 sentences\n",
      "2019-01-28 00:37:47,557:INFO:Loading a fresh vocabulary\n",
      "2019-01-28 00:37:47,738:INFO:min_count=2 retains 24438 unique words (51% of original 47851, drops 23413)\n",
      "2019-01-28 00:37:47,745:INFO:min_count=2 leaves 1839147 word corpus (98% of original 1862560, drops 23413)\n",
      "2019-01-28 00:37:48,027:INFO:deleting the raw counts dictionary of 47851 items\n",
      "2019-01-28 00:37:48,032:INFO:sample=0.001 downsamples 62 most-common words\n",
      "2019-01-28 00:37:48,038:INFO:downsampling leaves estimated 1586697 word corpus (86.3% of prior 1839147)\n",
      "2019-01-28 00:37:48,241:INFO:estimated required memory for 24438 words and 150 dimensions: 41544600 bytes\n",
      "2019-01-28 00:37:48,244:INFO:resetting layer weights\n",
      "2019-01-28 00:37:49,113:INFO:training model with 3 workers on 24438 vocabulary and 150 features, using sg=0 hs=0 sample=0.001 negative=5 window=10\n",
      "2019-01-28 00:37:50,181:INFO:EPOCH 1 - PROGRESS: at 26.07% examples, 435969 words/s, in_qsize 6, out_qsize 0\n",
      "2019-01-28 00:37:51,181:INFO:EPOCH 1 - PROGRESS: at 56.29% examples, 452319 words/s, in_qsize 6, out_qsize 0\n",
      "2019-01-28 00:37:52,200:INFO:EPOCH 1 - PROGRESS: at 88.83% examples, 467057 words/s, in_qsize 6, out_qsize 0\n",
      "2019-01-28 00:37:52,524:INFO:worker thread finished; awaiting finish of 2 more threads\n",
      "2019-01-28 00:37:52,527:INFO:worker thread finished; awaiting finish of 1 more threads\n",
      "2019-01-28 00:37:52,542:INFO:worker thread finished; awaiting finish of 0 more threads\n",
      "2019-01-28 00:37:52,544:INFO:EPOCH - 1 : training on 1862560 raw words (1586810 effective words) took 3.4s, 470135 effective words/s\n",
      "2019-01-28 00:37:53,585:INFO:EPOCH 2 - PROGRESS: at 28.20% examples, 466093 words/s, in_qsize 6, out_qsize 0\n",
      "2019-01-28 00:37:54,601:INFO:EPOCH 2 - PROGRESS: at 57.37% examples, 455720 words/s, in_qsize 5, out_qsize 0\n",
      "2019-01-28 00:37:55,606:INFO:EPOCH 2 - PROGRESS: at 87.23% examples, 457371 words/s, in_qsize 6, out_qsize 1\n",
      "2019-01-28 00:37:55,998:INFO:worker thread finished; awaiting finish of 2 more threads\n",
      "2019-01-28 00:37:56,007:INFO:worker thread finished; awaiting finish of 1 more threads\n",
      "2019-01-28 00:37:56,016:INFO:worker thread finished; awaiting finish of 0 more threads\n",
      "2019-01-28 00:37:56,018:INFO:EPOCH - 2 : training on 1862560 raw words (1587049 effective words) took 3.5s, 459696 effective words/s\n",
      "2019-01-28 00:37:57,047:INFO:EPOCH 3 - PROGRESS: at 26.54% examples, 445409 words/s, in_qsize 5, out_qsize 0\n",
      "2019-01-28 00:37:58,051:INFO:EPOCH 3 - PROGRESS: at 54.65% examples, 439345 words/s, in_qsize 5, out_qsize 0\n",
      "2019-01-28 00:37:59,053:INFO:EPOCH 3 - PROGRESS: at 81.96% examples, 433188 words/s, in_qsize 5, out_qsize 0\n",
      "2019-01-28 00:37:59,735:INFO:worker thread finished; awaiting finish of 2 more threads\n",
      "2019-01-28 00:37:59,753:INFO:worker thread finished; awaiting finish of 1 more threads\n",
      "2019-01-28 00:37:59,762:INFO:worker thread finished; awaiting finish of 0 more threads\n",
      "2019-01-28 00:37:59,765:INFO:EPOCH - 3 : training on 1862560 raw words (1586280 effective words) took 3.7s, 425667 effective words/s\n",
      "2019-01-28 00:38:00,790:INFO:EPOCH 4 - PROGRESS: at 26.07% examples, 439877 words/s, in_qsize 5, out_qsize 0\n",
      "2019-01-28 00:38:01,799:INFO:EPOCH 4 - PROGRESS: at 58.47% examples, 469611 words/s, in_qsize 6, out_qsize 0\n",
      "2019-01-28 00:38:02,802:INFO:EPOCH 4 - PROGRESS: at 90.98% examples, 480917 words/s, in_qsize 6, out_qsize 0\n",
      "2019-01-28 00:38:03,054:INFO:worker thread finished; awaiting finish of 2 more threads\n",
      "2019-01-28 00:38:03,069:INFO:worker thread finished; awaiting finish of 1 more threads\n",
      "2019-01-28 00:38:03,071:INFO:worker thread finished; awaiting finish of 0 more threads\n",
      "2019-01-28 00:38:03,072:INFO:EPOCH - 4 : training on 1862560 raw words (1586404 effective words) took 3.3s, 482862 effective words/s\n",
      "2019-01-28 00:38:04,097:INFO:EPOCH 5 - PROGRESS: at 27.67% examples, 464084 words/s, in_qsize 5, out_qsize 0\n",
      "2019-01-28 00:38:05,100:INFO:EPOCH 5 - PROGRESS: at 57.88% examples, 466147 words/s, in_qsize 6, out_qsize 0\n",
      "2019-01-28 00:38:06,112:INFO:EPOCH 5 - PROGRESS: at 87.23% examples, 460422 words/s, in_qsize 5, out_qsize 0\n",
      "2019-01-28 00:38:06,562:INFO:worker thread finished; awaiting finish of 2 more threads\n",
      "2019-01-28 00:38:06,569:INFO:worker thread finished; awaiting finish of 1 more threads\n",
      "2019-01-28 00:38:06,579:INFO:worker thread finished; awaiting finish of 0 more threads\n",
      "2019-01-28 00:38:06,582:INFO:EPOCH - 5 : training on 1862560 raw words (1586897 effective words) took 3.5s, 454669 effective words/s\n",
      "2019-01-28 00:38:06,584:INFO:training on a 9312800 raw words (7933440 effective words) took 17.5s, 454152 effective words/s\n"
     ]
    }
   ],
   "source": [
    "# Train the Word2Vec model\n",
    "import gensim, logging\n",
    "logging.basicConfig(format='%(asctime)s:%(levelname)s:%(message)s',level=logging.INFO)\n",
    "model=gensim.models.Word2Vec(enter_model,min_count=2,window=10,size=150)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The names of the different analysts in the company has similar vectors\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[('vicky', 0.9005892276763916),\n",
       " ('sharon', 0.8919146060943604),\n",
       " ('ksenia', 0.8675036430358887),\n",
       " ('shahar', 0.8522517681121826),\n",
       " ('bob', 0.8393317461013794),\n",
       " ('daria', 0.8332645893096924),\n",
       " ('alison', 0.8294651508331299),\n",
       " ('veerle', 0.8286025524139404),\n",
       " ('olga', 0.8285531997680664),\n",
       " ('rachel', 0.8250610828399658)]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Illustrating the embeddings semantics\n",
    "print('The names of the different analysts in the company have similar vectors')\n",
    "model.wv.most_similar('tomer',topn=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The names of the different vendors in the has similar vectors\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[('sage', 0.747258186340332),\n",
       " ('elsevier', 0.7268825769424438),\n",
       " ('sciencedirect', 0.6844416260719299),\n",
       " ('springer', 0.671920120716095),\n",
       " ('journals', 0.6593728065490723),\n",
       " ('ingenta', 0.65279221534729),\n",
       " ('ovid', 0.6363487839698792),\n",
       " ('wiley', 0.6345236897468567),\n",
       " ('ebsco', 0.6253975629806519),\n",
       " ('doaj', 0.6209652423858643)]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Illustrating the embeddings semantics\n",
    "print('The names of the different vendors in the have similar vectors')\n",
    "model.wv.most_similar('jstor',topn=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create a function that removes stop words\n",
    "\n",
    "from sklearn.pipeline import Pipeline\n",
    "import string\n",
    "def text_clean(mess):\n",
    "    nopunc = [char for char in mess if char not in string.punctuation]\n",
    "    \n",
    "    nopunc = ''.join(nopunc)\n",
    "    try:\n",
    "        return [word for word in nopunc.split() if word.lower() not in stopwords.words(english)]\n",
    "    except:\n",
    "        return [word for word in nopunc.split()]\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classifying the requests' topics\n",
    "<br><br>\n",
    "In this section I am trying different classifications models to find the one with the highest accuracy.\n",
    "<br>\n",
    "Two general methods will be tested:\n",
    "<br>\n",
    "1) One stage model: classifying all the different topics in one stage.\n",
    "<br>\n",
    "2) Two stages model: classifying to the dominant topic (Data Linking) and others. If a request belongs to \"Others\" then a second classifier will be applied. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## One stage model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 252,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.00      0.00      0.00       139\n",
      "  Data availability       0.00      0.00      0.00       469\n",
      "       Data display       0.33      0.00      0.00       528\n",
      "       Data linking       0.51      1.00      0.67      1999\n",
      "             Facets       0.00      0.00      0.00        93\n",
      "           Grouping       0.00      0.00      0.00       167\n",
      "Normalization Rules       0.00      0.00      0.00       119\n",
      "       Registration       0.00      0.00      0.00       196\n",
      "            Updates       0.00      0.00      0.00       218\n",
      "\n",
      "        avg / total       0.30      0.51      0.34      3928\n",
      "\n",
      "\n",
      "\n",
      "[[   0    0    0  139    0    0    0    0    0]\n",
      " [   0    0    0  469    0    0    0    0    0]\n",
      " [   0    0    1  527    0    0    0    0    0]\n",
      " [   0    0    0 1999    0    0    0    0    0]\n",
      " [   0    0    1   92    0    0    0    0    0]\n",
      " [   0    0    0  167    0    0    0    0    0]\n",
      " [   0    0    0  119    0    0    0    0    0]\n",
      " [   0    0    0  196    0    0    0    0    0]\n",
      " [   0    0    1  217    0    0    0    0    0]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "#Multinomial NB\n",
    "\n",
    "import numpy as np\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "pipeline_MultinomialNB = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean,stop_words='english', ngram_range=(1, 2))),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', MultinomialNB())\n",
    "])\n",
    "pipeline_MultinomialNB.fit(X_train, y_train)\n",
    "predictions_MultinomialNB = pipeline_MultinomialNB.predict(X_test)     \n",
    "\n",
    "print(classification_report(y_test, predictions_MultinomialNB))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test, predictions_MultinomialNB))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       1.00      0.01      0.03       139\n",
      "  Data availability       0.81      0.38      0.52       469\n",
      "       Data display       0.69      0.10      0.18       528\n",
      "       Data linking       0.59      0.98      0.74      1999\n",
      "             Facets       0.80      0.30      0.44        93\n",
      "           Grouping       0.98      0.26      0.42       167\n",
      "Normalization Rules       0.00      0.00      0.00       119\n",
      "       Registration       0.87      0.59      0.71       196\n",
      "            Updates       0.87      0.32      0.46       218\n",
      "\n",
      "        avg / total       0.68      0.63      0.55      3928\n",
      "\n",
      "\n",
      "\n",
      "[[   2    2    1  130    0    0    0    1    3]\n",
      " [   0  180    0  284    1    0    0    4    0]\n",
      " [   0    5   55  458    2    0    0    4    4]\n",
      " [   0   20    7 1968    1    1    0    2    0]\n",
      " [   0    3    1   60   28    0    0    1    0]\n",
      " [   0    1    1  118    2   44    0    0    1]\n",
      " [   0    0   10  105    1    0    0    2    1]\n",
      " [   0    9    2   68    0    0    0  116    1]\n",
      " [   0    3    3  140    0    0    0    3   69]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "#Random Forest\n",
    "\n",
    "pipeline_rf = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean, stop_words='english', ngram_range=(1, 7))),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', RandomForestClassifier(n_estimators=300))\n",
    "])\n",
    "pipeline_rf.fit(X_train, y_train)\n",
    "predictions_rf = pipeline_rf.predict(X_test)\n",
    "print(classification_report(y_test, predictions_rf))    \n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test, predictions_rf))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.00      0.00      0.00       139\n",
      "  Data availability       0.00      0.00      0.00       469\n",
      "       Data display       0.00      0.00      0.00       528\n",
      "       Data linking       0.51      1.00      0.67      1999\n",
      "             Facets       0.00      0.00      0.00        93\n",
      "           Grouping       0.00      0.00      0.00       167\n",
      "Normalization Rules       0.00      0.00      0.00       119\n",
      "       Registration       0.00      0.00      0.00       196\n",
      "            Updates       0.00      0.00      0.00       218\n",
      "\n",
      "        avg / total       0.26      0.51      0.34      3928\n",
      "\n",
      "\n",
      "\n",
      "[[   0    0    0  139    0    0    0    0    0]\n",
      " [   0    0    0  469    0    0    0    0    0]\n",
      " [   0    0    0  528    0    0    0    0    0]\n",
      " [   0    0    0 1999    0    0    0    0    0]\n",
      " [   0    0    0   93    0    0    0    0    0]\n",
      " [   0    0    0  167    0    0    0    0    0]\n",
      " [   0    0    0  119    0    0    0    0    0]\n",
      " [   0    0    0  196    0    0    0    0    0]\n",
      " [   0    0    0  218    0    0    0    0    0]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "# SVC\n",
    "\n",
    "from sklearn.svm import SVC\n",
    "pipeline_svm = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean)),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', SVC())\n",
    "])\n",
    "pipeline_svm.fit(X_train, y_train)\n",
    "predictions_svm = pipeline_svm.predict(X_test)\n",
    "print(classification_report(y_test, predictions_svm))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test, predictions_svm))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.70      0.05      0.09       139\n",
      "  Data availability       0.73      0.59      0.65       469\n",
      "       Data display       0.63      0.48      0.54       528\n",
      "       Data linking       0.73      0.95      0.83      1999\n",
      "             Facets       0.78      0.66      0.71        93\n",
      "           Grouping       0.92      0.51      0.66       167\n",
      "Normalization Rules       1.00      0.05      0.10       119\n",
      "       Registration       0.81      0.79      0.80       196\n",
      "            Updates       0.78      0.56      0.65       218\n",
      "\n",
      "        avg / total       0.74      0.73      0.70      3928\n",
      "\n",
      "\n",
      "\n",
      "[[   7    7   21   97    0    0    0    2    5]\n",
      " [   0  278    7  171    1    1    0    9    2]\n",
      " [   2   15  252  229    6    2    0    8   14]\n",
      " [   0   54   30 1907    1    2    0    2    3]\n",
      " [   0    1   15   12   61    0    0    2    2]\n",
      " [   0    2   12   61    5   85    0    0    2]\n",
      " [   1    5   45   53    3    1    6    2    3]\n",
      " [   0   11    8   17    1    0    0  155    4]\n",
      " [   0   10    9   64    0    1    0   11  123]]\n"
     ]
    }
   ],
   "source": [
    "# Logistic Regression\n",
    "\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "pipeline_lr = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer='word', stop_words='english')),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', LogisticRegression())\n",
    "])\n",
    "pipeline_lr.fit(X_train, y_train)\n",
    "predictions_lr = pipeline_lr.predict(X_test)\n",
    "print(classification_report(y_test, predictions_lr))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test, predictions_lr))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 431,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.70      0.05      0.09       139\n",
      "  Data availability       0.73      0.59      0.65       469\n",
      "       Data display       0.63      0.48      0.54       528\n",
      "       Data linking       0.73      0.95      0.83      1999\n",
      "             Facets       0.78      0.66      0.71        93\n",
      "           Grouping       0.92      0.51      0.66       167\n",
      "Normalization Rules       1.00      0.05      0.10       119\n",
      "       Registration       0.81      0.79      0.80       196\n",
      "            Updates       0.78      0.56      0.65       218\n",
      "\n",
      "        avg / total       0.74      0.73      0.70      3928\n",
      "\n",
      "\n",
      "\n",
      "[[   7    7   21   97    0    0    0    2    5]\n",
      " [   0  278    7  171    1    1    0    9    2]\n",
      " [   2   14  252  230    6    2    0    8   14]\n",
      " [   0   54   30 1907    1    2    0    2    3]\n",
      " [   0    1   15   12   61    0    0    2    2]\n",
      " [   0    2   12   61    5   85    0    0    2]\n",
      " [   1    5   45   53    3    1    6    2    3]\n",
      " [   0   11    8   17    1    0    0  155    4]\n",
      " [   0   10    9   64    0    1    0   11  123]]\n"
     ]
    }
   ],
   "source": [
    "# SGDClassifier\n",
    "\n",
    "from sklearn.linear_model import SGDClassifier\n",
    "pipeline_SGD = Pipeline([('vect', CountVectorizer()),\n",
    "                    ('tfidf', TfidfTransformer()),\n",
    "                    ('clf', SGDClassifier(loss='hinge', penalty='l2',\n",
    "                    alpha=1e-3, random_state=42,max_iter=5, tol=None)),\n",
    "                    ])\n",
    "pipeline_SGD.fit(X_train, y_train)\n",
    "predictions_SGD = pipeline_lr.predict(X_test)\n",
    "print(classification_report(y_test, predictions_SGD))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test, predictions_SGD))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.12      0.32      0.18       139\n",
      "  Data availability       0.20      0.45      0.28       469\n",
      "       Data display       0.42      0.16      0.23       528\n",
      "       Data linking       0.65      0.67      0.66      1999\n",
      "             Facets       0.85      0.12      0.21        93\n",
      "           Grouping       0.23      0.10      0.14       167\n",
      "Normalization Rules       0.11      0.05      0.07       119\n",
      "       Registration       0.42      0.17      0.24       196\n",
      "            Updates       0.33      0.11      0.17       218\n",
      "\n",
      "        avg / total       0.49      0.45      0.44      3928\n",
      "\n",
      "\n",
      "\n",
      "[[  44   32    5   51    0    0    4    1    2]\n",
      " [  48  210   12  179    0    5    9    4    2]\n",
      " [  41  149   82  213    1    9    8   12   13]\n",
      " [ 135  394   53 1333    0   34   14   16   20]\n",
      " [  10   34    1   28   11    1    2    3    3]\n",
      " [  15   55   14   55    1   16    4    2    5]\n",
      " [  15   41   10   42    0    1    6    2    2]\n",
      " [  22   66    5   61    0    2    3   33    4]\n",
      " [  30   57   12   83    0    1    4    6   25]]\n"
     ]
    }
   ],
   "source": [
    "#KNN\n",
    "\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "pipeline_knn = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean)),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', KNeighborsClassifier())\n",
    "])\n",
    "pipeline_knn.fit(X_train, y_train)\n",
    "predictions_knn = pipeline_knn.predict(X_test)\n",
    "print(classification_report(y_test, predictions_knn))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test, predictions_knn))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Two stage model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Case Number</th>\n",
       "      <th>Mixed_Comments</th>\n",
       "      <th>Categorie</th>\n",
       "      <th>problem_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10087</td>\n",
       "      <td>status st line wi dear christian this is a de...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10087</td>\n",
       "      <td>rodopi ingentaconnect special character  migra...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10099</td>\n",
       "      <td>hi rachel olga performed an in house analysis...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10099</td>\n",
       "      <td>directory of open access journals doaj wrong r...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11750</td>\n",
       "      <td>status st line wi dear kasper thank you for r...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Case Number                                     Mixed_Comments  \\\n",
       "0       10087   status st line wi dear christian this is a de...   \n",
       "1       10087  rodopi ingentaconnect special character  migra...   \n",
       "2       10099   hi rachel olga performed an in house analysis...   \n",
       "3       10099  directory of open access journals doaj wrong r...   \n",
       "4       11750   status st line wi dear kasper thank you for r...   \n",
       "\n",
       "      Categorie  problem_id  \n",
       "0  Data display           0  \n",
       "1  Data display           0  \n",
       "2  Data display           0  \n",
       "3  Data display           0  \n",
       "4  Data display           0  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Testing the models on all the topics (Excepct for Data Linking)\n",
    "# Create a new data frame of all the topics except for \"Data linking\"\n",
    "\n",
    "not_linking=merged_cases_A[merged_cases_A['Categorie'] != 'Data linking']\n",
    "not_linking.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAFwCAYAAACxVaymAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XmYHGW5/vHvnZCFLUBIUEyAsIRF\nCcGcLCAKAkc2OYZFEUQ2gYiiB0TDcuQHCEdRAVF20YRFOSgoAgIe9vUoS4JhDUrCOqyBsAQwkITn\n90e9nXQm3TOTmUlX9dT9ua6+puut6uqnJ5N6ut5VEYGZmZVPr7wDMDOzfDgBmJmVlBOAmVlJOQGY\nmZWUE4CZWUk5AZiZlZQTgJWGpJC0Qd5x5EXSvpJuyjsOKw4nAOtWkr4iaYqkdyS9JOkvkj6dd1xF\nIOlASffk9f4RcVlE7JDX+1vxOAFYt5F0FPBz4EfAR4C1gfOA8XnGZSBpubxjsOJxArBuIWkV4GTg\n8Ii4KiLejYh5EfHniJiYjhkr6W+S3kx3B+dI6pv2SdKZkl6V9JakhyVtmvb1k3S6pOckvSLpAknL\np32DJF2Xzjlb0t2S2vq73kXSU5Jek3SapF7p/LMljaj6PGtI+pekwXU+76GSpkuaI+lxSaNS+bGS\nZlaV757KNwEuALZMd0dvtvfZ0v6j0+/qRUmHVFdjSVpF0qWSZkl6VtLxlc+e7jb+L/1OZwMntb4D\nkbSxpJvTZ/+HpL2q9u2S4p8j6QVJ3+vo34I1kYjww48uP4CdgPnAcm0c82/AFsBywDBgOnBk2rcj\nMBVYFRCwCbBm2vdz4FpgILAy8Gfg1LTvVLILa5/0+AygOu8fwO3pPGsD/wQOSfvOA35SdewRwJ/r\nnOdLwAvAmBTrBsA6Vfs+Rvbl6svAu1Wf40Dgnlbnauuz7QS8DHwCWAH4TfoMG6T9lwLXpNcNS5/n\n4Kr3mg98O/2+l69+f2BF4HngoLR/FPAa8Im0/yXgM+n5asCovP/G/Oj+R+4B+NEzHsC+wMtL+Zoj\ngT+l59ulC9gWQK+qY5QuoutXlW0JPJ2en5wught04P0C2Klq+5vAren5uHRB7JW2pwB71TnPjcAR\nHfyM04Dx6fliCaADn21yJRmk7Q0qCQDoDbwPfLxq/9eBO6re67lWsVQngC8Dd7fa/0vgxPT8uXS+\nAXn/bfmx7B6uArLu8jowqK26ZkkbpuqalyW9TdZWMAggIm4DzgHOBV6RdKGkAcBgsm+/U1M1z5vA\n/6ZygNOAGcBNqWrn2HbifL7q+bNk39aJiPvILsbbSNqY7CJ7bZ1zrAXMrPMZ95c0rSrWTSufsYb2\nPtvHWsVb/XwQ0Dd9hurPM6TO8a2tA4yrvG96732Bj6b9ewK7AM9KulPSlm2cy5qUE4B1l78Bc4Hd\n2jjmfOAJYHhEDAD+i+xbMAARcVZE/BtZlceGwESyaol/kVVNrJoeq0TESuk1cyLiuxGxHvAfwFGS\ntm8jhrWqnq8NvFi1fQnwVWA/4A8RMbfOOZ4H1m9dKGkd4FfAt4DVI2JV4NGqz9h66t02PxtZNczQ\nOrG/Bswju5BXf54Xqrbbmur3eeDOqvddNSJWiohvAETEAxExHlgDuBq4oo1zWZNyArBuERFvAScA\n50raTdIKkvpI2lnST9NhKwNvA++kb9nfqLxe0hhJ4yT1IfsmPhdYEBEfkl1Uz5S0Rjp2iKQd0/Nd\nJW0gSencC9KjnomSVpO0Flk9/++r9v0G2J0sCVzaxjl+DXxP0r+lxusN0sV/RbKL7qwU20FkdwAV\nrwBDlRq+2/tsZBfdgyRtImmF9Put/L4XpP0/lLRyev+jgN+2EXe164ANJe2X/p36pH+DTST1VTZm\nYJWImMei36v1ME4A1m0i4mdkF6HjyS6Cz5N9G746HfI94CvAHLILX/XFd0Aqe4OsKuN14PS07xiy\nap57U9XRLcBGad/wtP0O2V3IeRFxRxthXkPW2DwNuB6YVBV/C/Ag2UX87jY+55XAD4H/SZ/lamBg\nRDwOnJHieAUYAfxf1UtvAx4DXpb0WnufLSL+ApxF1nA9I50Xsrp/yBp43wWeAu5J8Uxu47NXf4Y5\nwA7A3mR3QS8DPwH6pUP2A55JMR1GlhSth1GEF4Qxq5A0GXgxIo7PO5bWUlfSR4F+ETE/73is+TkB\nmCWShpHdGXwyIp7ON5pMGkdwPVn10iXAhxHRVjuLWYe5CsgMkHQK2bfr04py8U++TladNpOsHv4b\nbR9u1nG+AzAzKynfAZiZlVShJ4gaNGhQDBs2LO8wzMyaytSpU1+LiJrzWFUrdAIYNmwYU6ZMyTsM\nM7OmIunZ9o9yFZCZWWk5AZiZlZQTgJlZSRW6DcDMum7evHm0tLQwd269ue2sWfXv35+hQ4fSp0+f\nTr3eCcCsh2tpaWHllVdm2LBhZHPmWU8QEbz++uu0tLSw7rrrduocrgIy6+Hmzp3L6quv7ot/DyOJ\n1VdfvUt3dk4AZiXgi3/P1NV/VycAM7OSchuAWckMO/b6bj3fMz/+fLvHSOKoo47ijDPOAOD000/n\nnXfe4aSTTury+x944IHsuuuufPGLX+zyudpy5ZVXcsIJJ/DRj36U22+/fZm+V6P0mATQ3X/U0LE/\nbDNrX79+/bjqqqs47rjjGDSo3hLJjbdgwQJ69+7doWMnTZrEeeedx7bbbruMo2ocVwGZ2TK33HLL\nMWHCBM4888wl9h144IH84Q9/WLi90krZksh33HEH22yzDXvttRcbbrghxx57LJdddhljx45lxIgR\nzJw5c+FrbrnlFj7zmc+w4YYbct111wHZxX3ixImMGTOGzTbbjF/+8pcLz7vtttvyla98hREjRiwR\nz+WXX86IESPYdNNNOeaYYwA4+eSTueeeezjssMOYOHHiEq/56U9/yogRIxg5ciTHHnssAL/61a8Y\nM2YMI0eOZM899+S9994DsjuJTTfdlJEjR7L11lu3GetLL73E1ltvzeabb86mm27K3XfXXaiuU3rM\nHYCZFdvhhx/OZpttxtFHH93h1zz00ENMnz6dgQMHst5663HIIYdw//3384tf/IKzzz6bn//85wA8\n88wz3HnnncycOZNtt92WGTNmcOmll7LKKqvwwAMP8P7777PVVluxww47AHD//ffz6KOPLtF98sUX\nX+SYY45h6tSprLbaauywww5cffXVnHDCCdx2222cfvrpjB49erHX/OUvf+Hqq6/mvvvuY4UVVmD2\n7NkA7LHHHhx66KEAHH/88UyaNIlvf/vbnHzyydx4440MGTKEN998E8juLmrFetVVV7Hjjjvy/e9/\nnwULFixMIt3FCaDRTlplGZzzre4/p1k3GzBgAPvvvz9nnXUWyy+/fIdeM2bMGNZcc00A1l9//YUX\n8BEjRixWD7/XXnvRq1cvhg8fznrrrccTTzzBTTfdxMMPP7zw7uKtt97iySefpG/fvowdO7Zm3/kH\nHniAz372swwenE2kue+++3LXXXex2271F2G75ZZbOOigg1hhhRUAGDhwIACPPvooxx9/PG+++Sbv\nvPMOO+64IwBbbbUVBx54IHvttRd77LEHQN1Yx4wZw9e+9jXmzZvHbrvtxuabb96h31tHOQGYWcMc\neeSRjBo1ioMOOmhh2XLLLceHH34IZIObPvjgg4X7+vXrt/B5r169Fm736tWL+fMXLYvcujukJCKC\ns88+e+GFt+KOO+5gxRVXrBlfZxbIioia3TEPPPBArr76akaOHMnFF1/MHXfcAcAFF1zAfffdx/XX\nX8/mm2/OtGnT6sYKcNddd3H99dez3377MXHiRPbff/+ljrEetwGYWcMMHDiQvfbai0mTJi0sGzZs\nGFOnTgXgmmuuYd68eUt93iuvvJIPP/yQmTNn8tRTT7HRRhux4447cv755y883z//+U/efffdNs8z\nbtw47rzzTl577TUWLFjA5ZdfzjbbbNPma3bYYQcmT568sHqmUgU0Z84c1lxzTebNm8dll1228PiZ\nM2cybtw4Tj75ZAYNGsTzzz9fN9Znn32WNdZYg0MPPZSDDz6YBx98cKl/N23xHYBZyeTdu+273/0u\n55xzzsLtQw89lPHjxzN27Fi23377ut/O27LRRhuxzTbb8Morr3DBBRfQv39/DjnkEJ555hlGjRpF\nRDB48GCuvvrqNs+z5pprcuqpp7LtttsSEeyyyy6MHz++zdfstNNOTJs2jdGjR9O3b1922WUXfvSj\nH3HKKacwbtw41llnHUaMGMGcOXMAmDhxIk8++SQRwfbbb8/IkSPZbLPNasZ6xx13cNppp9GnTx9W\nWmklLr300qX+3bSl0GsCjx49Ojq6IEzTdAN1G4A12PTp09lkk03yDsOWkVr/vpKmRsToOi9ZyFVA\nZmYl1W4CkDRZ0quSHm1V/m1J/5D0mKSfVpUfJ2lG2rdjVflOqWyGpGO792OYmdnS6kgbwMXAOcDC\nyidJ2wLjgc0i4n1Ja6TyjwN7A58APgbcImnD9LJzgc8BLcADkq6NiMe764OYWX31eqpYc+tqFX67\ndwARcRcwu1XxN4AfR8T76ZhXU/l44HcR8X5EPA3MAMamx4yIeCoiPgB+l441s2Wsf//+vP76612+\nWFixVNYD6N+/f6fP0dleQBsCn5H0Q2Au8L2IeAAYAtxbdVxLKgN4vlX5uFonljQBmACw9tprdzI8\nM6sYOnQoLS0tzJo1K+9QrJtVVgTrrM4mgOWA1YAtgDHAFZLWA2rdYwa17zRqfh2JiAuBCyHrBdTJ\n+Mws6dOnT6dXjLKerbMJoAW4KrJ7yvslfQgMSuVrVR03FHgxPa9XbmZmOehsN9Crge0AUiNvX+A1\n4Fpgb0n9JK0LDAfuBx4AhktaV1Jfsobia7savJmZdV67dwCSLgc+CwyS1AKcCEwGJqeuoR8AB6S7\ngcckXQE8DswHDo+IBek83wJuBHoDkyPisWXweczMrIPaTQARsU+dXV+tc/wPgR/WKL8BuGGpojMz\ns2XGI4HNzErKCcDMrKScAMzMSsoJwMyspJwAzMxKygnAzKyknADMzErKCcDMrKScAMzMSsoJwMys\npJwAzMxKygnAzKyknADMzErKCcDMrKScAMzMSqrdBCBpsqRX0+Ivrfd9T1JIGpS2JeksSTMkPSxp\nVNWxB0h6Mj0O6N6PYWZmS6sjdwAXAzu1LpS0FvA54Lmq4p3JloEcDkwAzk/HDiRbSWwcMBY4UdJq\nXQnczMy6pt0EEBF3AbNr7DoTOBqIqrLxwKWRuRdYVdKawI7AzRExOyLeAG6mRlIxM7PG6VQbgKQv\nAC9ExEOtdg0Bnq/abkll9cprnXuCpCmSpsyaNasz4ZmZWQcsdQKQtALwfeCEWrtrlEUb5UsWRlwY\nEaMjYvTgwYOXNjwzM+ugztwBrA+sCzwk6RlgKPCgpI+SfbNfq+rYocCLbZSbmVlOljoBRMQjEbFG\nRAyLiGFkF/dREfEycC2wf+oNtAXwVkS8BNwI7CBptdT4u0MqMzOznHSkG+jlwN+AjSS1SDq4jcNv\nAJ4CZgC/Ar4JEBGzgVOAB9Lj5FRmZmY5Wa69AyJin3b2D6t6HsDhdY6bDExeyvjMzGwZ8UhgM7OS\ncgIwMyspJwAzs5JyAjAzKyknADOzknICMDMrKScAM7OScgIwMyspJwAzs5JyAjAzKyknADOzknIC\nMDMrKScAM7OScgIwMyspJwAzs5LqyIIwkyW9KunRqrLTJD0h6WFJf5K0atW+4yTNkPQPSTtWle+U\nymZIOrb7P4qZmS2NjtwBXAzs1KrsZmDTiNgM+CdwHICkjwN7A59IrzlPUm9JvYFzgZ2BjwP7pGPN\nzCwn7SaAiLgLmN2q7KaImJ827yVb5B1gPPC7iHg/Ip4mWxpybHrMiIinIuID4HfpWDMzy0l3tAF8\nDfhLej4EeL5qX0sqq1e+BEkTJE2RNGXWrFndEJ6ZmdXSpQQg6fvAfOCySlGNw6KN8iULIy6MiNER\nMXrw4MFdCc/MzNrQ7qLw9Ug6ANgV2D4tBg/ZN/u1qg4bCryYntcrNzOzHHTqDkDSTsAxwBci4r2q\nXdcCe0vqJ2ldYDhwP/AAMFzSupL6kjUUX9u10M3MrCvavQOQdDnwWWCQpBbgRLJeP/2AmyUB3BsR\nh0XEY5KuAB4nqxo6PCIWpPN8C7gR6A1MjojHlsHnMTOzDmo3AUTEPjWKJ7Vx/A+BH9YovwG4Yami\nMzOzZcYjgc3MSsoJwMyspJwAzMxKygnAzKyknADMzErKCcDMrKScAMzMSsoJwMyspJwAzMxKygnA\nzKyknADMzErKCcDMrKScAMzMSsoJwMyspJwAzMxKygnAzKyk2k0AkiZLelXSo1VlAyXdLOnJ9HO1\nVC5JZ0maIelhSaOqXnNAOv7JtJ6wmZnlqCN3ABcDO7UqOxa4NSKGA7embYCdydYBHg5MAM6HLGGQ\nLSU5DhgLnFhJGmZmlo92E0BE3AXMblU8HrgkPb8E2K2q/NLI3AusKmlNYEfg5oiYHRFvADezZFIx\nM7MG6mwbwEci4iWA9HONVD4EeL7quJZUVq98CZImSJoiacqsWbM6GZ6ZmbWnuxuBVaMs2ihfsjDi\nwogYHRGjBw8e3K3BmZnZIp1NAK+kqh3Sz1dTeQuwVtVxQ4EX2yg3M7OcdDYBXAtUevIcAFxTVb5/\n6g20BfBWqiK6EdhB0mqp8XeHVGZmZjlZrr0DJF0OfBYYJKmFrDfPj4ErJB0MPAd8KR1+A7ALMAN4\nDzgIICJmSzoFeCAdd3JEtG5YNjOzBmo3AUTEPnV2bV/j2AAOr3OeycDkpYrOzMyWGY8ENjMrKScA\nM7OScgIwMyspJwAzs5JyAjAzKyknADOzknICMDMrKScAM7OScgIwMyspJwAzs5JyAjAzKyknADOz\nknICMDMrKScAM7OScgIwMyupLiUASd+R9JikRyVdLqm/pHUl3SfpSUm/l9Q3Hdsvbc9I+4d1xwcw\nM7PO6XQCkDQE+E9gdERsCvQG9gZ+ApwZEcOBN4CD00sOBt6IiA2AM9NxZmaWk65WAS0HLC9pOWAF\n4CVgO+APaf8lwG7p+fi0Tdq/vSR18f3NzKyTOp0AIuIF4HSyNYFfAt4CpgJvRsT8dFgLMCQ9HwI8\nn147Px2/euvzSpogaYqkKbNmzepseGZm1o6uVAGtRvatfl3gY8CKwM41Do3KS9rYt6gg4sKIGB0R\nowcPHtzZ8MzMrB1dqQL6d+DpiJgVEfOAq4BPAaumKiGAocCL6XkLsBZA2r8KMLsL729mZl3QlQTw\nHLCFpBVSXf72wOPA7cAX0zEHANek59embdL+2yJiiTsAMzNrjK60AdxH1pj7IPBIOteFwDHAUZJm\nkNXxT0ovmQSsnsqPAo7tQtxmZtZFy7V/SH0RcSJwYqvip4CxNY6dC3ypK+9nZmbdxyOBzcxKygnA\nzKyknADMzErKCcDMrKScAMzMSqpLvYCs5xpxyYhuP+cjBzzS7ec0s85zAjAzWwZOOumkwp/TVUBm\nZiXlBGBmVlJOAGZmJeUEYGZWUk4AZmYl5QRgZlZSTgBmZiXlBGBmVlJOAGZmJdWlBCBpVUl/kPSE\npOmStpQ0UNLNkp5MP1dLx0rSWZJmSHpY0qju+QhmZtYZXb0D+AXwvxGxMTASmE621OOtETEcuJVF\nSz/uDAxPjwnA+V18bzMz64JOJwBJA4CtSWv+RsQHEfEmMB64JB12CbBbej4euDQy9wKrSlqz05Gb\nmVmXdOUOYD1gFnCRpL9L+rWkFYGPRMRLAOnnGun4IcDzVa9vSWWLkTRB0hRJU2bNmtWF8MzMrC1d\nSQDLAaOA8yPik8C7LKruqUU1ymKJgogLI2J0RIwePHhwF8IzM7O2dCUBtAAtEXFf2v4DWUJ4pVK1\nk36+WnX8WlWvHwq82IX3NzOzLuh0AoiIl4HnJW2UirYHHgeuBQ5IZQcA16Tn1wL7p95AWwBvVaqK\nzMys8bq6IMy3gcsk9QWeAg4iSypXSDoYeA74Ujr2BmAXYAbwXjrWzMxy0qUEEBHTgNE1dm1f49gA\nDu/K+5mZWffxSGAzs5JyAjAzKykvCm9mTaXl2Lu7/ZxDf/yZbj9nM/AdgJlZSTkBmJmVlBOAmVlJ\nOQGYmZWUE4CZWUk5AZiZlZQTgJlZSXkcgNkydu5ht3X7OQ+/YLtuP6eVj+8AzMxKygnAzKyknADM\nzErKCcDMrKS6nAAk9U6Lwl+XtteVdJ+kJyX9Pi0Wg6R+aXtG2j+sq+9tZmad1x13AEcA06u2fwKc\nGRHDgTeAg1P5wcAbEbEBcGY6zszMctKlBCBpKPB54NdpW8B2ZAvEA1wC7Jaej0/bpP3bp+PNzCwH\nXb0D+DlwNPBh2l4deDMi5qftFmBIej4EeB4g7X8rHb8YSRMkTZE0ZdasWV0Mz8zM6ul0ApC0K/Bq\nREytLq5xaHRg36KCiAsjYnREjB48eHBnwzMzs3Z0ZSTwVsAXJO0C9AcGkN0RrCppufQtfyjwYjq+\nBVgLaJG0HLAKMLsL729m3eiML+/a7ef87u+v6/ZzWvfp9B1ARBwXEUMjYhiwN3BbROwL3A58MR12\nAHBNen5t2ibtvy0ilrgDMDOzxlgW4wCOAY6SNIOsjn9SKp8ErJ7KjwKOXQbvbWZmHdQtk8FFxB3A\nHen5U8DYGsfMBb7UHe9nZmZd55HAZmYl5QRgZlZSTgBmZiXlBWGsaU3feJNuP+cmT0xv/yCzHsJ3\nAGZmJeUEYGZWUk4AZmYl5QRgZlZSTgBmZiXlBGBmVlJOAGZmJeUEYGZWUk4AZmYl5QRgZlZSTgBm\nZiXVlTWB15J0u6Tpkh6TdEQqHyjpZklPpp+rpXJJOkvSDEkPSxrVXR/CzMyWXlfuAOYD342ITYAt\ngMMlfZxspa9bI2I4cCuLVv7aGRieHhOA87vw3mZm1kVdWRP4pYh4MD2fA0wHhgDjgUvSYZcAu6Xn\n44FLI3Mv2eLxa3Y6cjMz65JuaQOQNAz4JHAf8JGIeAmyJAGskQ4bAjxf9bKWVNb6XBMkTZE0Zdas\nWd0RnpmZ1dDlBCBpJeCPwJER8XZbh9YoiyUKIi6MiNERMXrw4MFdDc/MzOroUgKQ1Ifs4n9ZRFyV\nil+pVO2kn6+m8hZgraqXDwVe7Mr7m5lZ53WlF5CAScD0iPhZ1a5rgQPS8wOAa6rK90+9gbYA3qpU\nFZmZWeN1ZUnIrYD9gEckTUtl/wX8GLhC0sHAc8CX0r4bgF2AGcB7wEFdeG8zM+uiTieAiLiH2vX6\nANvXOD6Awzv7fmZm1r08EtjMrKScAMzMSsoJwMyspJwAzMxKygnAzKyknADMzErKCcDMrKScAMzM\nSsoJwMyspJwAzMxKygnAzKyknADMzErKCcDMrKScAMzMSsoJwMyspJwAzMxKquEJQNJOkv4haYak\nYxv9/mZmlmloApDUGzgX2Bn4OLCPpI83MgYzM8s0+g5gLDAjIp6KiA+A3wHjGxyDmZkBypbqbdCb\nSV8EdoqIQ9L2fsC4iPhW1TETgAlpcyPgH90cxiDgtW4+57LgOLuX4+xezRBnM8QIyybOdSJicHsH\ndXpR+E6qtYj8YhkoIi4ELlxmAUhTImL0sjp/d3Gc3ctxdq9miLMZYoR842x0FVALsFbV9lDgxQbH\nYGZmND4BPAAMl7SupL7A3sC1DY7BzMxocBVQRMyX9C3gRqA3MDkiHmtkDCzD6qVu5ji7l+PsXs0Q\nZzPECDnG2dBGYDMzKw6PBDYzKyknADOzkurxCUDSpnnH0BHNEqeZ9Rw9PgEAF0i6X9I3Ja2adzBt\naJY4m5KkXpIG5B1Hs5K0laQV0/OvSvqZpHXyjsu6pscngIj4NLAv2fiDKZL+R9Lncg5rCc0SJ4Ck\nIyQNUGaSpAcl7ZB3XK2l3+GAdOF6HPiHpIl5x9WkzgfekzQSOBp4Frg035CWJGlUjcf6kho96LVN\nKaHeLOmfkp6S9LSkpxoeSESU4kHW7XRP4AVgOvAEsEfecTVjnMBD6eeOZOM4RgIP5h1XjTinpZ/7\nAj8D+gAP5x1XjTi3IBsj8w7wAbAAeDvvuFrF+GD6eQJwcHVZkR7Avel3OAWYCryffrdPATvkHV9V\nnE+QTYq5BrB65dHoOHr8HYCkzSSdSXYx3Q74j4jYJD0/M9fgqjRLnEllSo9dgIsi4iFqT/ORtz6S\n+gC7AddExLy8A6rjHGAf4ElgeeAQ4OxcI1rSHEnHAfsB16eZffvkHFMtzwCfjIjREfFvwCeBR4F/\nB36aZ2CtvBURf4mIVyPi9cqj0UH0+ARA9p/rQWBkRBweEQ8CRMSLwPG5Rra4ZokTYKqkm8gSwI2S\nVgY+zDmmWn5JdkFYEbgr1Vm/lWtEdUTEDKB3RCyIiIuAbfOOqZUvk32b/lpEvAwMAU7LN6SaNo6q\nwaUR8ThZQmh89Urbbpd0mqQtq6urGh2EB4IVhKQjI+LnrcqOiIhf5BVTPZJ6AZsDT0XEm5JWB4ZE\nxMM5h7YYSetGxNNV2wI2iIgncwxrCZLuIvuG+mvgZeAl4MCIGJlrYK2kBDo8Im6RtAJZwpqTd1zV\nJP0emE021TxkiWsQ2Z3LPRExJq/Yqkm6vUZxRMR2DY2jpycAScOBU8kWoOlfKY+I9XILqgZJD0bE\nqFZlf4+IT+YVUz3pQrovsF5EnCxpbeCjEXF/zqEtps7vdGqqGiiMdGF9BegLfAdYBTg3ImbmGlgV\nSYeSTdM+MCLWT/+vLoiI7XMObTGSlge+CXyarFryHuA8YC6wQkS8k2N4hVOolvFl5CLgRLJ69G2B\ngyhQfbWkfYCvAOtKqp4Yb2Wg4XWCHXQeWZXPdsDJwBzgj0BRvl1tDHwCWEXSHlW7BlD1JaBAdkt3\nenOBH0B29wcU6e7vcLIFne4DiIgnJa2Rb0hLioh/AWekR2uFufhLWoXsurR1KroTODkiGlpFWYYE\nsHxE3CpJEfEscJKku8l++UXwV7Jb/kEs/kc7ByhUlUqVcRExStLfASLijTS7a1FsBOwKrAr8R1X5\nHODQXCJq2wEsebE/sEZZnt6PiA+ymz9I3SoLV30gaSvgJGAdqq5vRbvjByaTNU7vlbb3I/uyukfd\nVywDZUgAc1Od9ZNpJtIXyLolzboPAAAT30lEQVReFUJKSs8CW+Ydy1KYl3qBBICkwRSoETgirgGu\nkbRlRPwt73jqabK7vzsl/RewfBqf8k3gzznHVMsksmq0qWTdaYtq/YjYs2r7B5KmNTqIMiSAI4EV\ngP8ETiGrtjgg14iqSLonIj4taQ6Lf6MSWaNQEUevngX8CVhD0g+BLwL/L9+Qanpd0q3ARyJiU0mb\nAV+IiP/OO7Ckme7+jgUOBh4Bvg7cQNZoXTRvRcRf8g6iA/4l6dMRcQ8svHP5V6OD6PGNwLZspHr2\n7ckS1a0RMT3nkJYg6U5gIvDLSmO6pEcjwvMu9VCSfkw2mPIqsm6rAFS6VReFpM2BS8ga/EXWc+nA\nNKamYXrsHYCkP9NGHWVEfKGB4dQlaWBb+yNidqNi6ShJv4mI/chGM7YuK5IVIuL+Sr11Mj+vYOqR\ntAXZwK9NyHoC9QbeLcLdn6RHaPv/0WYNDKcjxqWf1WvsBtmdf2FExDRgZGV+qoh4O484emwCAE7P\nO4AOmkr2B1qrZ1IARWu8gqyHzUKpPaBQXSuT1yStz6K2ii+SVbkUzTlky6NeSXbh2h/YINeIFtk1\n7wCWRkQUbQDdYiR9NSJ+K+moVuUARMTPGhlPj00AEXFn5XnqobIx2YXgHxHxQW6BtRIR6+YdQ0el\nqQAqDYFvsyhpfUAxl987nCyujSW9ADwNfDXfkGqLiBmSekfEAuAiSX/NOyZY2Emh8OpdWCsafWFt\nw4rp58o19jW8Pr7HJoAKSZ8HLgBmkl2w1pX09aI0FEnaOCKeqDcMvEh1lxFxKnCqpFMj4ri842lP\nGv7/72k20F5FG7Va5b30JWWapJ+S3aWs2M5rGqpVJ4W+ZPMAFaKaKmnrwloYEfHL9PSWiPi/6n2p\nIbihenwjsKQngF3TXCukKoHrI2LjfCPLSLowIiYUZWh4R0laDRjO4qOr78ovoiVJ+hHw04h4M22v\nBnw3Igo1t1KdkcDnVf5mi0jSbsDYiPivvGNpRnVGqS9RtszjKEECuCsitq7aFnBndZktHUmHAEcA\nQ4FpZNMZ/61oyarWVBp5/CfriDSWgoiYlXcsHSXp3ojYIu84qklaj2wA3RZkdyx/A75TlMngJG0J\nfIqse3r1LL8DgN0bPf9Tj68CAh6TdANwBdkfxJeABypTBETEVXkGVyGpP4vmMAngbrK5VubmGlht\nR5BN+3BvRGybuoT+IOeYauktqV9EvA8L54npl3NMC6UvIycC3yKrnuwlaT5wdkScnGtwrbSaUqMX\nWWN1Eb89/g9wLrB72t4buJxFvYPy1hdYiezaW11d9TbZeJqGKkMC6E92e71N2p4FDCSbIiDI+gsX\nwaVkA4Aq88DvA/yGLGEVzdyImCuJdIF9QtJGeQdVw2+BWyVdRPZv/TWyvtdFcSSwFTCmMmtp+gZ7\nvqTvRESR1oGonlJjPtk02+PzCaVNiojfVG3/Ns0AUAipc8qdki4uQgN7j68CahaSHmp9+1errAgk\n/YlsUr0jyfpXvwH0iYhdcg2sBkk7s2jA2k0RcWPOIS2U5lL6XES81qp8MFmshZsJtpqkFSPi3bzj\nqJYGgr1JNh10kE0H3Y/srqAw42rSv/HRZF2qq9vRPB10d0q9Kv6bbJj1/5ItX3hkRPw218BakXQx\nWZXPvWl7HHBARHwz18DaIWkbskbL/y1S99pm0Nao5CKNWJY0BFiTbDnND9IsoEeSjVz9WL7RLU7S\n023sjqJMCqdsQaXfA98DDiObnmZWRBzTyDjKUAW0Q0QcLWl3oIWsSuV2suqB3FWNtOwD7C/pubS9\nDtlC5oVRZ9TyI+nnSmTD2QujyCNsk7YSZiGSqaQjge8DM4B+kn5Btr7ypRRw8F8TjatZPSImKVv0\nqVItdGe7r+pmZUgAlXVLdwEuj4jZraYGyFszjbSsHrW8NlnVj8imXX4OKNp/viKPsIVsKoBaUwCI\n4qxbMAHYKP2/WZssEWxduVMtGkn71yqPiEsbHUs7KutTv5TGKr1I1quuocqQAP6cxgL8C/hmqnsr\nTM+a1g1B6fa6KP/5F1P5diXpAuDaiLghbe9MtqRh4RR1hC1ARPTOO4YOmFupN4+I5yT9s6gX/6R6\nUaL+ZO0/D5LdsRTJfytbFOa7ZHepA8jGgDRUj28DgIUDgN6OiAXK1jIdENnC1oUh6QtkUwJ/DHiV\nrApoekR8os0X5kA1llWUNCUiRtd7TR7UJGvtFpmkV1m0vi5kd1QLtyPiPxse1FJIF9nfFGXyR1g4\nd9Z/FqGXV4+9A5C0XUTcVt1/uVXVT1G6f1acQjZ45ZaI+KSkbcm6ghbRa5KOJ2tHCbL5dYq2gAlk\nqyz1Iutn/x1gLWDPNl9hrU1stT01lyg67z2yEeuFkb6IfoHFB4LloscmALJ+/7exeP/liiL1/6+Y\nFxGvS+olqVdE3C7pJ3kHVcc+ZAOY/pS276JAyUrS2hHxXFX12sK1dm3pRESRxk20S4tPA9+brAPA\nFflFVNdfJZ1D1hNoYVfaRs/9VYoqoGYg6RZgN+BUshWiXiUbIPSpXANrQ7q9/rBok6xVT/cg6Y+x\n+NJ71oOlbskV84FnI6Ilr3jqKcrcXz02AdSbFraiQNPDAtmgGrJvqgL2Jetbf1lEFK5qRdIYskWt\nK0PZ3wK+FhGFqB6ongOo1nxA1rNJ+giLGoPvj4hX84ynFknrtZ6fqFbZstarkW/WYCunx2jgG8CQ\n9DgM+HiOcdUUEe9GxIKImB8Rl0TEWUW8+CeTgG9GxLCIGEY27/5F+Ya0mKjz3Ho4SXsB95ON99kL\nuE/ZQkBF84caZVc2Ooge2wYQET+AhSPuRlWqKSSdRA6/6Hq05GLwC3dR3EXh50TE3ZWNiLgnfY6i\nqPSvF4sWr4Fi/04LLXWfPhQYRtV1IyK+lldMdXyfrOr0VVgY9y3UvuA2XJo48RPAKq0m2BtADt2/\ne2wCqLI2i4+q/IDsj7gQIqLQC1jUcb+kX5LNsliZb+UOpUVtGt2Q1VqT9K9vNteQzVB7C7Ag51ja\n0qtVlc/rFKumYyOywZ+rsngHlTlkCbahemwbQIWk75PdCv6J7GK1O/D7yFa3yp2kARHxdp1pFgoz\neVW1qgasyh9Pdf/ahjdk2bInaVpEbJ53HO2RdBqwGdmXE8i+nDzc6Dl22iNpy4j4W+5x9PQEAJC+\nmX4mbd4VEX/PM55qkq6LiF3TJFatF4cvzORVsFjDeiXGIJte+57KdMbWM0n6b+CvldHfRZaqVj5N\n9nd6V0T8qZ2XNFxRJqksRQKw7iHpxBrFA4EdgZMi4nc19lsPkNp4ViSrQq3MY1Oo9pQ0wvbGiCjk\ntCTVKndUaZLK3cgGKt7e6FHqZWgDaBoq+Dq7lYb11lL11S0sPmWA9SDN0FaVRti+J2mViHgr73ja\nUYhJKp0ACkJ11tklW3Cl0NIfb6GmWLXul6YvqKylfUdEXJdnPHXMBR6RdDOLj7At2pxFhZik0lVA\nBZHWBaiss7t56i72g4j4cs6htUvSdsDxbvztudJKW2OAy1LRPsDUiDg2v6iWJOmAqs2FnRSKOKVF\nESap7PF3AE2wKEhF4dfZrVq8ptpAsrnMa87Dbj3GLsDmEfEhgKRLgL8DhUgAksYDQyPi3LR9PzCY\n7O+1MD2AijZJZY9PABR/UZCKFkmrAlcDN0t6g+zCWiStF68J4PUo2LqwtsysyqJV31bJM5Aajib7\nf17Rl2zFspXIRqkXZfBnoSapLEMCKPSiIBURsXt6elLqZ78KWfewwmi9eI2VyqnA39PfpsjaAo7L\nN6TF9I2I56u270ljaGanebYKISJOTD8PyjsWKEEbQLMsCpLWWv19RBQuOZkBSFqTrB1AwH1FWlRJ\n0oyIqHlnL2lmRKzf6JjaUmeyyrfI2lWmNSqOIg2RXlaqFwV5l2xRkD3afEU+HgSOlzRD0mmSCrW6\nlpVT6oxQGUy5JtACPA98rDL1R0HcJ2mJqRQkfZ1scriiGU02MWVlksoJwGeBX0k6ulFBlOEO4IiI\n+EV7ZUWR+tTvSVafuXZEFGo1IysXSRdGxISizF9fj7K1tK8G3if7MgVZG0A/YLeIeCWv2GqRdCOw\nZ0S8k7ZXIpuwbneyu4CGzFhchgSwcHGQqrLCzhEvaSzZ/CW7AY9HRK3GIrOGktQ/Iua2V5a31CW5\nso72YxFxW57x1CNpOjAyIj5I2/2AaRGxSSOvTz22EVjSPsBXgHUlXVu1a2UKuH5tWv5xD2Am2RJ2\np0TEm/lGZbbQX4HWVT61ynKVLviFvOi38j/AvZKuSdv/AVyeGqwfb1QQPTYBkP1xvkS2vOIZVeVz\ngIdziahtTwNbRsRreQdiViHpo2R11MtL+iSLJgIcAKyQW2BNLiJOkXQDiyatOywipqTd+zYqjh5f\nBdRMij4XkJVPGll7IFmj5ZSqXXOAiyOiof3WexJJnwaGR8RFaSqIlRo9q26PTwDNMhK43lxARWlk\ns3KTtGdE/DHvOHqKNLPuaGCjiNhQ0seAKyNiq0bG0ZOrgCqaZSTwESyaC2jbylxAOcdkBkBE/FHS\n58kaWKvvUE/OL6qmtjvwSVKPpYh4UVLDZ1wtwzgAImIG0Dstun4RsG3eMdUwt9KjojIXENnycWa5\nk3QBWe+0b5PVWX8JWCfXoJrbB5FVvwRAXqOVy3AH8J6kvsC0tArPS2QLWxRNM8wFZOX1qYjYTNLD\nEfEDSWfQ4Hlrepgr0rraq6YBbF8jm62gocrQBrAO8ApZ/f93yObYOS/dFRSSpG1IcwFV+gmb5UnS\nfRExTtK9ZN2VXwce9UDFzpP0OWAHsjuqGyPi5obH0NMTAEBqYSciZuUdi1kzkvT/yDpTbA+cS1Z1\n8euI+H+5BtZDpOUs946Iy9o9uDvft6cmgLRC1YlkcwCJrL1jPnC2G67MOi+NWu3fBMsuFo6kAcDh\nZGMrrgVuTtsTyUYCj29kPD25EfhIYCtgTESsHhGrAeOArSR9J9/QzJqLpJmSDgOIiPcj4i1JRVwS\nsuh+Q9a54xHgEOAmsgb18Y2++EPPvgP4O/C51iNrU3XQTUWdC8isiNL6tQ8B7wFfj4gPijynVlFJ\neiQiRqTnvYHXyCZ9nJNHPD35DqBPrWkVUjtAnxziaZOkLSQ9IOkdSR9IWiDp7bzjMkveS+tTTwfu\nTp0reua3x2VrXuVJWqDq6bwu/tCzu4G21XumiD1rmmXAmpWTACLip5KmAjeSrQdtS2dk1Rc7kc2x\n9HZ6Ho2eoaAnJ4CRdb5Bi6qRjEXSDEtXWmmdUHkSEbdK2hE4IMd4mlJE9M47hmo9NgEU7RfdAc0y\nYM1KRNLGaVT6CzVWAHMjcJPrsY3AzabOgLVzI2JmroFZqUn6VUQcWvQVwaxznAAKotmWrjSz5ucE\nUBDNtnSllYOkPdra7/UAmluPbQNoFs22dKWVTltrUgeeEK6p+Q4gZ6nuf13gVODYql1zgIcjYn4u\ngZlZj+cEYGYd4gVhep6ePBK4qXgksBWZF4TpmZwAiuMcYB/gSWB5somizs41IrNFPhUR+wNvRMQP\ngC2BtXKOybrICaBAmmTpSiunf6Wf76UFzOeRtV1ZE3MvoOLwSGArsuvSkqWnkS1kHuSwhKF1LzcC\nF0QzLl1p5eQFYXoOJ4AC8dKVVlRp7vrPA8OoqjmIiJ/lFZN1nauAclZr6UpJXrrSiubPwFyylaw+\nzDkW6yZOAPmrXrryaQBJ6wHnS/pORJyZa3RmmaERsVneQVj3chVQzrx0pTUDST8Bbo2Im/KOxbqP\n7wDyV3fpSkmFW7rSSute4E+SepF1Ac1lBSvrXk4A+Wu2pSutnM4gG/z1SLjaoMdwAshf0y1daaX0\nJPCoL/49ixNAzppw6Uorp5eAOyT9BXi/UuhuoM3NCcDMOuLp9OibHtYDOAGYWZvSILCVImJi3rFY\n9/JkcGbWpohYAIxq90BrOr4DMLOOmJaWLL0SeLdS6DWBm5sTgJl1xECyNaq3qyrzmsBNziOBzcxK\nym0AZtYuSUMl/UnSq5JekfRHSUPzjsu6xgnAzDriIuBa4GPAELLZQS/KNSLrMlcBmVm7JE2LiM3b\nK7Pm4jsAM+uI1yR9VVLv9PgqWaOwNTHfAZhZuyStDZxDNiFcAH8FjoiIZ3MNzLrECcDMrKQ8DsDM\n6pJ0Qhu7IyJOaVgw1u18B2BmdUn6bo3iFYGDgdUjYqUGh2TdyAnAzDpE0srAEWQX/yuAMyLi1Xyj\nsq5wFZCZtUnSQOAoYF/gEmBURLyRb1TWHZwAzKwuSacBewAXAiMi4p2cQ7Ju5CogM6tL0odkK4DN\nJ+v+uXAXXhS+6TkBmJmVlEcCm5mVlBOAmVlJOQGYmZWUE4CVnqTVJU1Lj5clvVC13Xcpz3WRpI2W\nVaxm3cmNwGZVJJ0EvBMRp+cdi9my5jsAszZIOlrSo+nx7VS2gaTHJP1G0iOSrpC0fNp3j6TN0/PP\nS3pQ0kOSbsrzc5jV4gRgVoeksWSjX8eSTYP8TUmbpd0fB86NiBHAXODrrV77UeB8YPeIGAns3bDA\nzTrICcCsvs8Af4yI9yJiDnA18Om07+mIuDc9/21VecWWwO2V+fIjYnYjAjZbGk4AZvWpjX2tG89a\nb6tGmVmhOAGY1XcXsLuk5SWtBIwH7k771pU0Jj3fB7in1Wv/D9hO0jqwcEI1s0JxAjCrIyLuBy4H\nHgDuBc6PiEfS7seAQyU9TDY//oWtXvsK8A3gGkkPAZc1LHCzDnI3ULOlJGkD4A8RsXnesZh1he8A\nzMxKyncAZmYl5TsAM7OScgIwMyspJwAzs5JyAjAzKyknADOzkvr/tmQtbvf3kooAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plotting the \"Others\" topics\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "from collections import Counter\n",
    "\n",
    "casess_counter=dict(Counter(not_linking['Categorie']))\n",
    "df_4=pd.DataFrame(list(casess_counter.items()),\n",
    "                      columns=['Topic','Number of cases'])\n",
    "\n",
    "\n",
    "ax = df_4.plot.bar(x='Topic', y='Number of cases', rot=0)\n",
    "plt.title('Cases by categories')\n",
    "plt.xticks(rotation=90)\n",
    "plt.show()\n",
    "fname=\"label.png\"\n",
    "plt.savefig(fname, bbox_inches=\"tight\")\n",
    "plt.close(\"all\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(13092, 48547)\n"
     ]
    }
   ],
   "source": [
    "# Split the data to test and train\n",
    "\n",
    "features_1 = tfidf.fit_transform(not_linking.Mixed_Comments).toarray()\n",
    "labels_1 = not_linking.problem_id\n",
    "print(features.shape)\n",
    "\n",
    "X_train_1, X_test_1, y_train_1, y_test_1 = train_test_split(not_linking['Mixed_Comments'], not_linking['Categorie'], random_state = 42,test_size=0.3)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Case Number</th>\n",
       "      <th>Mixed_Comments</th>\n",
       "      <th>Categorie</th>\n",
       "      <th>problem_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10087</td>\n",
       "      <td>status st line wi dear christian this is a de...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10087</td>\n",
       "      <td>rodopi ingentaconnect special character  migra...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10099</td>\n",
       "      <td>hi rachel olga performed an in house analysis...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10099</td>\n",
       "      <td>directory of open access journals doaj wrong r...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11750</td>\n",
       "      <td>status st line wi dear kasper thank you for r...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Case Number                                     Mixed_Comments Categorie  \\\n",
       "0       10087   status st line wi dear christian this is a de...     Other   \n",
       "1       10087  rodopi ingentaconnect special character  migra...     Other   \n",
       "2       10099   hi rachel olga performed an in house analysis...     Other   \n",
       "3       10099  directory of open access journals doaj wrong r...     Other   \n",
       "4       11750   status st line wi dear kasper thank you for r...     Other   \n",
       "\n",
       "   problem_id  \n",
       "0           0  \n",
       "1           0  \n",
       "2           0  \n",
       "3           0  \n",
       "4           0  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Crating new dataframe of \"Data Linking\" and \"Others\" \n",
    "\n",
    "linking_or_not_linking=merged_cases_A\n",
    "Topic_copy = linking_or_not_linking['Categorie']    \n",
    "for i in linking_or_not_linking.index:\n",
    "    if linking_or_not_linking.at[i, 'Categorie'] != 'Data linking':\n",
    "        linking_or_not_linking.at[i, 'Categorie'] = 'Other'  \n",
    "    \n",
    "linking_or_not_linking.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAFJCAYAAACSIPrDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xu8VXWd//HXWy5eEUEODoGCJhkq\ngsRFh7KUCZAuoCZ5GQVC6eI4lQ1pjb/RcHSybCwrNQsKHCdHydDRUvFCao0KKnnDBBXliBcUIdRU\nwM/vj/09sDmeyz5w2AvP9/18PPbjrPVd37X2Zx1xv8/6rstWRGBmZvnZrugCzMysGA4AM7NMOQDM\nzDLlADAzy5QDwMwsUw4AM7NMOQAsG5JC0r5F11EUSSdKurXoOmzb4QCwViXpBEkLJL0u6QVJv5f0\n0aLr2hZImijpnqLePyKuioiRRb2/bXscANZqJJ0B/BC4ANgD2Au4FBhbZF0GktoXXYNtexwA1iok\ndQamAadFxHUR8UZErI2I/42IqanPUEn/J2lVOjr4iaSOaZkkXSzpZUmrJT0s6cC0bHtJF0l6TtJL\nki6XtGNa1k3SjWmbKyXdLampf9djJD0t6RVJ35e0Xdr+Skn9y/anu6S/SappZH9PlbRI0hpJj0sa\nlNrPkvRUWftRqb0fcDlwaDo6WtXcvqXl30y/q+WSTikfxpLUWdIsSSskPSvp7Lp9T0cbf0y/05XA\nufWPQCR9WNLctO9/kTS+bNmYVP8aSc9L+pdK/y3Y+0hE+OXXFr+A0cA6oH0TfT4CHAK0B/oAi4Cv\npWWjgAeA3QAB/YAeadkPgRuArkAn4H+B/0jL/oPSB2uH9PoYoEbeP4A703b2Ap4ETknLLgUuLOv7\nVeB/G9nOscDzwJBU675A77JlH6D0x9XngTfK9mMicE+9bTW1b6OBF4EDgJ2AK9M+7JuWzwKuT+v1\nSfszuey91gGnp9/3juXvD+wMLAMmpeWDgFeAA9LyF4CPpekuwKCi/4351fqvwgvwq228gBOBF1u4\nzteA36bpI9IH2CHAdmV9lD5EP1jWdijwTJqelj4E963g/QIYXTb/FeD2ND0sfSBul+YXAOMb2c4t\nwFcr3MeFwNg0vUkAVLBvM+rCIM3vWxcAQDvgbWD/suVfBOaVvddz9WopD4DPA3fXW/4z4Jw0/Vza\n3q5F/9vya+u9PARkreVVoFtTY82SPpSGa16U9FdK5wq6AUTEHcBPgJ8CL0m6QtKuQA2lv34fSMM8\nq4CbUzvA94ElwK1paOesZupcVjb9LKW/1omI+yh9GH9c0ocpfcje0Mg29gSeamQfT5a0sKzWA+v2\nsQHN7dsH6tVbPt0N6Jj2oXx/ejbSv77ewLC6903vfSLwd2n5McAY4FlJf5B0aBPbsvcpB4C1lv8D\n3gLGNdHnMuAJoG9E7Ap8m9JfwQBExCUR8RFKQx4fAqZSGpb4G6Whid3Sq3NE7JLWWRMR34iIfYDP\nAGdIGtFEDXuWTe8FLC+bnwn8I3ASMDsi3mpkG8uAD9ZvlNQb+DnwT8DuEbEb8GjZPtZ/9G6T+0Zp\nGKZXI7W/Aqyl9EFevj/Pl8039ajfZcAfyt53t4jYJSK+DBAR8yNiLNAdmANc08S27H3KAWCtIiJW\nA/8G/FTSOEk7Seog6UhJ30vdOgF/BV5Pf2V/uW59SUMkDZPUgdJf4m8B6yPiXUofqhdL6p769pQ0\nKk1/WtK+kpS2vT69GjNVUhdJe1Ia5/+fsmVXAkdRCoFZTWzjF8C/SPpIOnm9b/rw35nSh+6KVNsk\nSkcAdV4Ceimd+G5u3yh96E6S1E/STun3W/f7Xp+Wny+pU3r/M4D/aqLucjcCH5J0Uvrv1CH9N+gn\nqaNK9wx0joi1bPy9WhvjALBWExH/SelD6GxKH4LLKP01PCd1+RfgBGANpQ++8g/fXVPba5SGMl4F\nLkrLzqQ0zHNvGjq6DdgvLeub5l+ndBRyaUTMa6LM6ymdbF4I3ARML6u/FniQ0of43U3s57XA+cB/\np32ZA3SNiMeBH6Q6XgL6A38sW/UO4DHgRUmvNLdvEfF74BJKJ66XpO1CaewfSid43wCeBu5J9cxo\nYt/L92ENMBI4jtJR0IvAhcD2qctJwNJU05cohaK1MYrwF8KY1ZE0A1geEWcXXUt96VLSR4HtI2Jd\n0fXY+58DwCyR1IfSkcHBEfFMsdWUpPsIbqI0vDQTeDcimjrPYlYxDwGZAZLOo/TX9fe3lQ//5IuU\nhtOeojQO/+Wmu5tVzkcAZmaZ8hGAmVmmHABmZpnapp8Q2K1bt+jTp0/RZZiZva888MADr0REgw8y\nLLdNB0CfPn1YsGBB0WWYmb2vSHq2+V4eAjIzy5YDwMwsUw4AM7NMbdPnAMxsy61du5ba2lreequx\nh5va+9UOO+xAr1696NChw2at7wAwa+Nqa2vp1KkTffr0ofTQVGsLIoJXX32V2tpa9t57783ahoeA\nzNq4t956i913390f/m2MJHbfffctOrJzAJhlwB/+bdOW/nd1AJjZVieJb3zjGxvmL7roIs4999xW\n2fbEiROZPXt2q2yrKddeey39+vXj8MMP3+rvVS0+B2DW1q16DpZvHCboc8nyJjq33NJ//kCzfbbf\nviPXXXs135r0Kbp17QJ/fR7eeBOWP7TlBby5ElY+s1nbWr9+Pe3atauo7/RLL+bSSy9tUwHgIwAz\n2+rat2vHlBOP5uIrrnrPsolfO4fZN962YX6XvsMBmPenBXz8mFMY/8Uz+dBHx3HWBZdw1XW/Y+in\nTqL/iPE8tXTjd97fdvd9fOyoL/Chj47jxrl3AaUP96nnXcyQMf/IQf8wnp9dOXvDdg//3BROOO3b\n9B8x/j31/HrOzfQfMZ4DjziWM8//EQDTLr6Ce+5fyJe+9CWmTp36nnW+973v0b9/fwYMGMBZZ50F\nwM9//nOGDBnCgAEDOOaYY3jzzTeB0pHEgQceyIABAzjssMM21jp1KkOGDOGggw7iZz/7GQAvvPAC\nhx12GAMHDuTAAw/k7rsb/aK6zeIjADOritMmjuegf/g83/zKhIrX+fPjT7Jo3m/oultn9vn7z3DK\n8eO4/6Yr+dEv/psfz7iaH04rfRgvrV3OH37zC55aWsvhx05hyceGMWv2jXTu1In5v/sv3n77HYaP\nm8TIjx8KwP0LH+XRO65l7716bvJ+y19cwZnnX8IDN19Fl86dGHn8V5hz853829encMcf53PRJZcz\nePDgTdb5/e9/z5w5c7jvvvvYaaedWLlyJQBHH300p556KgBnn30206dP5/TTT2fatGnccsst9OzZ\nk1WrVgEwffp0OnfuzPz583n77bcZPnw4I0eO5LrrrmPUqFH867/+K+vXr98QIq3FAWBmVbFrp104\n+XOf5pLpV7Pjjts3vwIwZMAB9Nij9EyzD/buteEDvP+H9+XOP218Ttj4z3yS7bbbjr777MU+vXvy\nxJKl3PqHe3l40WJm31Q6uli95nUWP/McHTt0YOjAA9/z4Q8w/8+P8YlDP0LN7l0AOPHoMdx174OM\nG934sM9tt93GpEmT2GmnnQDo2rUrAI8++ihnn302q1at4vXXX2fUqFEADB8+nIkTJzJ+/HiOPvpo\nAG699VYefvjhDecyVq9ezeLFixkyZAhf+MIXWLt2LePGjWPgwIEV/d4q5QBoBX3OuqnoEtqUpd/9\nVNEl2FbytVNOYNDoE5j0+c9uaGvfvh3vvvsuULq2/Z21azcs277jxhuctttuO7bv2HHD9Lp1G78W\nuf7VMJIIgh//+zcZ9Ym/32TZvD8tYOeddmiwvs35gqyIaPBqnIkTJzJnzhwGDBjAr371K+bNmwfA\n5Zdfzn333cdNN93EwIEDWbhwIRHBj3/84w0hUe6uu+7ipptu4qSTTmLq1KmcfPLJLa6xMT4HYGZV\n07VLZ8Z/5pNM//X1G9r69PoADzyyCIDrb5nH2rUt/777a2+8jXfffZenli7j6WefZ78P9mbUxw/l\nslmzWZsC5cmnnuWNN//W5HaGHXwgf7j3AV5Z+Rrr16/n13Nu5uOHfqTJdUaOHMmMGTM2DM/UDQGt\nWbOGHj16sHbtWq66auO5j6eeeophw4Yxbdo0unXrxrJlyxg1ahSXXXbZxlqffJI33niDZ599lu7d\nu3PqqacyefJkHnzwwRb/bpriIwAzq6pvfPEkfvLLazbMn3riUYyd9HWGfuokRnx0KDvvtGOLt7nf\nPr35+DGn8NKKlVz+3W+zww7bc8oJR7F02XIGjT6RiKCmaxfmzPhBk9vpsUcN//Gt0zn82C8SEYw5\nYjhjR32iyXVGjx7NwoULGTx4MB07dmTMmDFccMEFnHfeeQwbNozevXvTv39/1qxZA8DUqVNZvHgx\nEcGIESMYMGAABx10EEuXLmXQoEGlWmtqmDNnDvPmzeP73/8+HTp0YJdddmHWrFkt/t00ZZv+TuDB\ngwfH++H7ADwE1Lo8BNS6Fv3fLfTr3b3oMtqGDxxcdAXvsWjRIvr167dJm6QHImJwI6ts4CEgM7NM\nOQDMzDLlADAzy5QDwKzNi826vNG2fVv639UBYNbG7bD6aV59Y51DoI2p+z6AHXZo+J6GSvgyULM2\nrteDF1LLmazovA/gx0JvkdWLiq5gE3XfCLa5HABmbVyHd1ax973fKrqMtuHc1UVX0KoqGgKStJuk\n2ZKekLRI0qGSukqaK2lx+tkl9ZWkSyQtkfSwpEFl25mQ+i+WVPkToczMrNVVeg7gR8DNEfFhYACw\nCDgLuD0i+gK3p3mAI4G+6TUFuAxAUlfgHGAYMBQ4py40zMys+poNAEm7AocB0wEi4p2IWAWMBWam\nbjOBcWl6LDArSu4FdpPUAxgFzI2IlRHxGjAXGN2qe2NmZhWr5AhgH2AF8EtJD0n6haSdgT0i4gWA\n9LPuXvOewLKy9WtTW2PtZmZWgEoCoD0wCLgsIg4G3mDjcE9DGrrMIJpo33RlaYqkBZIWrFixooLy\nzMxsc1QSALVAbUTcl+ZnUwqEl9LQDunny2X99yxbvxewvIn2TUTEFRExOCIG19TUtGRfzMysBZoN\ngIh4EVgmab/UNAJ4HLgBqLuSZwJQ94DvG4CT09VAhwCr0xDRLcBISV3Syd+Rqc3MzApQ6X0ApwNX\nSeoIPA1MohQe10iaDDwHHJv6/g4YAywB3kx9iYiVks4D5qd+0yJiZavshZmZtVhFARARC4GGni09\nooG+AZzWyHZmADNaUqCZmW0dfhaQmVmmHABmZplyAJiZZcoBYGaWKQeAmVmmHABmZplyAJiZZcoB\nYGaWKQeAmVmmHABmZplyAJiZZcoBYGaWKQeAmVmmHABmZplyAJiZZcoBYGaWKQeAmVmmHABmZply\nAJiZZcoBYGaWKQeAmVmmHABmZplyAJiZZaqiAJC0VNIjkhZKWpDaukqaK2lx+tkltUvSJZKWSHpY\n0qCy7UxI/RdLmrB1dsnMzCrRkiOAwyNiYEQMTvNnAbdHRF/g9jQPcCTQN72mAJdBKTCAc4BhwFDg\nnLrQMDOz6tuSIaCxwMw0PRMYV9Y+K0ruBXaT1AMYBcyNiJUR8RowFxi9Be9vZmZboNIACOBWSQ9I\nmpLa9oiIFwDSz+6pvSewrGzd2tTWWLuZmRWgfYX9hkfEckndgbmSnmiirxpoiybaN125FDBTAPba\na68KyzMzs5aq6AggIpanny8Dv6U0hv9SGtoh/Xw5da8F9ixbvRewvIn2+u91RUQMjojBNTU1Ldsb\nMzOrWLMBIGlnSZ3qpoGRwKPADUDdlTwTgOvT9A3AyelqoEOA1WmI6BZgpKQu6eTvyNRmZmYFqGQI\naA/gt5Lq+v93RNwsaT5wjaTJwHPAsan/74AxwBLgTWASQESslHQeMD/1mxYRK1ttT8zMrEWaDYCI\neBoY0ED7q8CIBtoDOK2Rbc0AZrS8TDMza22+E9jMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOz\nTDkAzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DM\nLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOzTFUcAJLaSXpI0o1pfm9J90laLOl/JHVM7dun+SVp\neZ+ybXwrtf9F0qjW3hkzM6tcS44AvgosKpu/ELg4IvoCrwGTU/tk4LWI2Be4OPVD0v7AccABwGjg\nUknttqx8MzPbXBUFgKRewKeAX6R5AUcAs1OXmcC4ND02zZOWj0j9xwJXR8TbEfEMsAQY2ho7YWZm\nLVfpEcAPgW8C76b53YFVEbEuzdcCPdN0T2AZQFq+OvXf0N7AOmZmVmXNBoCkTwMvR8QD5c0NdI1m\nljW1Tvn7TZG0QNKCFStWNFeemZltpkqOAIYDn5W0FLia0tDPD4HdJLVPfXoBy9N0LbAnQFreGVhZ\n3t7AOhtExBURMTgiBtfU1LR4h8zMrDLNBkBEfCsiekVEH0once+IiBOBO4HPpW4TgOvT9A1pnrT8\njoiI1H5cukpob6AvcH+r7YmZmbVI++a7NOpM4GpJ/w48BExP7dOBKyUtofSX/3EAEfGYpGuAx4F1\nwGkRsX4L3t/MzLZAiwIgIuYB89L00zRwFU9EvAUc28j65wPnt7RIMzNrfb4T2MwsUw4AM7NMOQDM\nzDLlADAzy5QDwMwsUw4AM7NMOQDMzDLlADAzy5QDwMwsUw4AM7NMOQDMzDLlADAzy5QDwMwsUw4A\nM7NMOQDMzDLlADAzy5QDwMwsUw4AM7NMOQDMzDLlADAzy5QDwMwsUw4AM7NMOQDMzDLVbABI2kHS\n/ZL+LOkxSd9J7XtLuk/SYkn/I6ljat8+zS9Jy/uUbetbqf0vkkZtrZ0yM7PmVXIE8DZwREQMAAYC\noyUdAlwIXBwRfYHXgMmp/2TgtYjYF7g49UPS/sBxwAHAaOBSSe1ac2fMzKxyzQZAlLyeZjukVwBH\nALNT+0xgXJoem+ZJy0dIUmq/OiLejohngCXA0FbZCzMza7GKzgFIaidpIfAyMBd4ClgVEetSl1qg\nZ5ruCSwDSMtXA7uXtzewTvl7TZG0QNKCFStWtHyPzMysIhUFQESsj4iBQC9Kf7X3a6hb+qlGljXW\nXv+9roiIwRExuKamppLyzMxsM7ToKqCIWAXMAw4BdpPUPi3qBSxP07XAngBpeWdgZXl7A+uYmVmV\nVXIVUI2k3dL0jsA/AIuAO4HPpW4TgOvT9A1pnrT8joiI1H5cukpob6AvcH9r7YiZmbVM++a70AOY\nma7Y2Q64JiJulPQ4cLWkfwceAqan/tOBKyUtofSX/3EAEfGYpGuAx4F1wGkRsb51d8fMzCrVbABE\nxMPAwQ20P00DV/FExFvAsY1s63zg/JaXaWZmrc13ApuZZcoBYGaWKQeAmVmmHABmZplyAJiZZcoB\nYGaWKQeAmVmmHABmZplyAJiZZcoBYGaWKQeAmVmmHABmZplyAJiZZcoBYGaWKQeAmVmmHABmZply\nAJiZZcoBYGaWKQeAmVmmHABmZplyAJiZZcoBYGaWqWYDQNKeku6UtEjSY5K+mtq7SporaXH62SW1\nS9IlkpZIeljSoLJtTUj9F0uasPV2y8zMmlPJEcA64BsR0Q84BDhN0v7AWcDtEdEXuD3NAxwJ9E2v\nKcBlUAoM4BxgGDAUOKcuNMzMrPqaDYCIeCEiHkzTa4BFQE9gLDAzdZsJjEvTY4FZUXIvsJukHsAo\nYG5ErIyI14C5wOhW3RszM6tYi84BSOoDHAzcB+wRES9AKSSA7qlbT2BZ2Wq1qa2xdjMzK0DFASBp\nF+A3wNci4q9NdW2gLZpor/8+UyQtkLRgxYoVlZZnZmYtVFEASOpA6cP/qoi4LjW/lIZ2SD9fTu21\nwJ5lq/cCljfRvomIuCIiBkfE4Jqampbsi5mZtUAlVwEJmA4sioj/LFt0A1B3Jc8E4Pqy9pPT1UCH\nAKvTENEtwEhJXdLJ35GpzczMCtC+gj7DgZOARyQtTG3fBr4LXCNpMvAccGxa9jtgDLAEeBOYBBAR\nKyWdB8xP/aZFxMpW2QszM2uxZgMgIu6h4fF7gBEN9A/gtEa2NQOY0ZICzcxs6/CdwGZmmXIAmJll\nygFgZpYpB4CZWaYcAGZmmXIAmJllygFgZpYpB4CZWaYcAGZmmXIAmJllygFgZpYpB4CZWaYcAGZm\nmXIAmJllygFgZpYpB4CZWaYcAGZmmXIAmJllygFgZpYpB4CZWaYcAGZmmXIAmJllqtkAkDRD0suS\nHi1r6ypprqTF6WeX1C5Jl0haIulhSYPK1pmQ+i+WNGHr7I6ZmVWqkiOAXwGj67WdBdweEX2B29M8\nwJFA3/SaAlwGpcAAzgGGAUOBc+pCw8zMitFsAETEXcDKes1jgZlpeiYwrqx9VpTcC+wmqQcwCpgb\nESsj4jVgLu8NFTMzq6LNPQewR0S8AJB+dk/tPYFlZf1qU1tj7WZmVpDWPgmsBtqiifb3bkCaImmB\npAUrVqxo1eLMzGyjzQ2Al9LQDunny6m9FtizrF8vYHkT7e8REVdExOCIGFxTU7OZ5ZmZWXM2NwBu\nAOqu5JkAXF/WfnK6GugQYHUaIroFGCmpSzr5OzK1mZlZQdo310HSr4FPAN0k1VK6mue7wDWSJgPP\nAcem7r8DxgBLgDeBSQARsVLSecD81G9aRNQ/sWxmZlXUbABExPGNLBrRQN8ATmtkOzOAGS2qzszM\nthrfCWxmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwA\nZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlikH\ngJlZphwAZmaZqnoASBot6S+Slkg6q9rvb2ZmJVUNAEntgJ8CRwL7A8dL2r+aNZiZWUm1jwCGAksi\n4umIeAe4Ghhb5RrMzAxoX+X36wksK5uvBYaVd5A0BZiSZl+X9Jcq1ZaDbsArRRfRHF1YdAVWgPfF\nv02+o6IrqFTvSjpVOwAa+u3FJjMRVwBXVKecvEhaEBGDi67DrD7/2yxGtYeAaoE9y+Z7AcurXIOZ\nmVH9AJgP9JW0t6SOwHHADVWuwczMqPIQUESsk/RPwC1AO2BGRDxWzRoy56E121b532YBFBHN9zIz\nszbHdwKbmWXKAWBmlikHgJlZpqp9H4BVkaTtgEMi4k9F12JWTtIlDTSvBhZExPXVridXPgJowyLi\nXeAHRddh1oAdgIHA4vQ6COgKTJb0wyILy4mPANq+WyUdA1wXvuTLth37AkdExDoASZcBtwKfBB4p\nsrCcOADavjOAnYH1kv5G6XEcERG7FluWZa4npX+Xq9P8zsAHImK9pLeLKysvDoA2LiI6FV2DWQO+\nByyUNI/SHyWHARdI2hm4rcjCcuIbwdo4SQJOBPaOiPMk7Qn0iIj7Cy7NMiepB6VHxAu4PyL8XLAq\ncwC0cWls9V1K4639JHUBbo2IIQWXZpmT1JPSY4s3jERExF3FVZQfDwG1fcMiYpCkhwAi4rX0ID6z\nwki6EPg88BilP1Cg9Gh4B0AVOQDavrXpqzgDQFING/+HMyvKOGC/iPAJ3wL5PoC27xLgt0B3SecD\n9wAXFFuSGU8DHYouInc+B5ABSR8GRlA62XZ7RCwquCTLnKTfAAOA24ENRwER8c+FFZUhB0AG0hDQ\nHmx6su254iqy3Ema0FB7RMysdi05cwC0cZJOB84BXgLWs/FGsIMKLczMCucAaOMkLaF0JdCrRddi\nJumaiBgv6RHShQnl/IdJdfkqoLZvGRtvtzcr2lfTz08XWoUBPgJosySdkSYPAPYDbmLTk23/WURd\nZgCSjoyI39dr+1JEXF5UTTnyZaBtV6f0eg6YC3Qsa9ulwLrMAP6fpCPqZiSdCYwtsJ4s+QigjZN0\nbERc21ybWTVJ6gbcCEwFRgMfBo6LiLWFFpYZB0AbJ+nBiBjUXJtZtUnqTunJnw8AX/D3VVSfTwK3\nUZKOBMYAPet9/d6uwLpiqrLcSVrDplf/dAT2AT4nyd9TUWUOgLZrObAAOBZ4ktL/dOsp3Q/w9QLr\nsoz5+ym2LQ6AtutxSt8D0BH4AqUbwPYEfklp7NWsUH4cdPF8DqCNknQxpat9zoiINaltV+Ai4G8R\n8dWm1jfbmsoeB/04pSNTKN2h/tniqsqPA6CNkrQY+FD9E2vpuUBPRETfYiozA0l/AQ7y46CL5fsA\n2q5o6KqKiFhPA7fgm1WZHwe9DfA5gLbrcUknR8Ss8kZJ/wg8UVBNZnXepPSl8H4cdIE8BNRGpRNs\n1wF/o3SddQBDgB2BoyLi+QLLs8z5cdDbBgdAG5dutz+A0lVAj0XE7QWXZGbbCAeAmVWNHwe9bXEA\nmFnVSOoRES9I6t3Q8oh4tto15cwBYGaWKV8FZGZV08CzgDYsonTpsp8FVEU+AjAzy5RvBDMzy5QD\nwMwsUw4AM7NMOQDMzDLlADCzqpN0iKT5kl6X9I6k9ZL+WnRduXEAmFkRfgIcDyym9HyqU4AfF1pR\nhnwfgJkVIiKWSGqXHlH+S0l/Krqm3DgAzKwIb0rqSOmR0N8DXgB2Lrim7HgIyMyKcBKlz59/At6g\n9H3VRxdaUYYcAGZWhHER8VZE/DUivhMRZwCfLrqo3DgAzKwIDX0hzMRqF5E7nwMws6qRdDxwArC3\npBvKFnUCXi2mqnw5AMysmv5E6YRvN+AHZe1rgIcLqShjfhqomVmmfA7AzKrOdwJvGxwAZlYE3wm8\nDfA5ADMrhO8ELp4DwMyK4DuBtwEeAjKzIjR0J/AxhVaUIV8FZGaFkFQDEBEriq4lVz4CMLOqUcm5\nkl4BngCelLRC0r8VXVuOHABmVk1fA4YDQyJi94joAgwDhkv6erGl5cdDQGZWNZIeAj4ZEa/Ua68B\nbo2Ig4upLE8+AjCzaupQ/8MfNpwH6FBAPVlzAJhZNb2zmctsK/AQkJlVjaT1lC77fM8iYIeI8FFA\nFTkAzMwy5SEgM7NMOQDMzDLlADAzy5QDwLInaXdJC9PrRUnPl813bOG2filpv61Vq1lr8klgszKS\nzgVej4iLiq7FbGvzEYBZEySkwFPuAAABM0lEQVR9U9Kj6XV6attX0mOSrpT0iKRrJO2Ylt0jaWCa\n/pSkByX9WdKtRe6HWUMcAGaNkDQUOBEYChwKfEXSQWnx/sBPI6I/8BbwxXrr/h1wGXBURAwAjqta\n4WYVcgCYNe5jwG8i4s2IWAPMAT6alj0TEfem6f8qa69zKHBnRDwLEBErq1GwWUs4AMwapyaW1T95\nVn9eDbSZbVMcAGaNuws4StKOknYBxgJ3p2V7SxqSpo8H7qm37h+BIyT1BpDUtRoFm7WEA8CsERFx\nP/BrYD5wL3BZRDySFj8GnCrpYUrfZXtFvXVfAr4MXC/pz8BVVSvcrEK+DNSshSTtC8yOiIFF12K2\nJXwEYGaWKR8BmJllykcAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXq/wPmgEA5KNI8rgAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plotting \"Others\" vs \"Data Linking\" topics\n",
    "\n",
    "casess_counter=dict(Counter(linking_or_not_linking['Categorie']))\n",
    "df_5=pd.DataFrame(list(casess_counter.items()),\n",
    "                      columns=['Topic','Number of cases'])\n",
    "\n",
    "\n",
    "ax = df_5.plot.bar(x='Topic', y='Number of cases', rot=0)\n",
    "plt.title('Cases by categories')\n",
    "plt.xticks(rotation=90)\n",
    "plt.show()\n",
    "fname=\"label.png\"\n",
    "plt.savefig(fname, bbox_inches=\"tight\")\n",
    "plt.close(\"all\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 248,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Data linking    6604\n",
       "Other           6488\n",
       "Name: Categorie, dtype: int64"
      ]
     },
     "execution_count": 248,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "linking_or_not_linking['Categorie'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(13092, 48547)\n"
     ]
    }
   ],
   "source": [
    "# Split the data to test and train\n",
    "\n",
    "features_2 = tfidf.fit_transform(not_linking.Mixed_Comments).toarray()\n",
    "labels_2 = not_linking.problem_id\n",
    "print(features.shape)\n",
    "\n",
    "X_train_2, X_test_2, y_train_2, y_test_2 = train_test_split(linking_or_not_linking['Mixed_Comments'], linking_or_not_linking['Categorie'], random_state = 42,test_size=0.3)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Linking / Others Classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 253,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "Data linking       0.80      0.87      0.83      1999\n",
      "       Other       0.85      0.77      0.81      1929\n",
      "\n",
      " avg / total       0.83      0.82      0.82      3928\n",
      "\n",
      "\n",
      "\n",
      "[[1738  261]\n",
      " [ 435 1494]]\n"
     ]
    }
   ],
   "source": [
    "#Naive Bayes\n",
    "\n",
    "pipeline_MultinomialNB_linking = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean,stop_words='english', ngram_range=(1, 2))),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', MultinomialNB())\n",
    "])\n",
    "pipeline_MultinomialNB_linking.fit(X_train_2, y_train_2)\n",
    "predictions_MultinomialNB_linking = pipeline_MultinomialNB_linking.predict(X_test_2)   \n",
    "\n",
    "print(classification_report(y_test_2, predictions_MultinomialNB_linking))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_2, predictions_MultinomialNB_linking))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 254,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "Data linking       0.85      0.84      0.84      1999\n",
      "       Other       0.83      0.85      0.84      1929\n",
      "\n",
      " avg / total       0.84      0.84      0.84      3928\n",
      "\n",
      "\n",
      "\n",
      "[[1671  328]\n",
      " [ 291 1638]]\n"
     ]
    }
   ],
   "source": [
    "#Random Forest\n",
    "\n",
    "pipeline_rf_linking = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean,stop_words='english', ngram_range=(1, 2))),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', RandomForestClassifier(n_estimators=300))\n",
    "])\n",
    "pipeline_rf_linking.fit(X_train_2, y_train_2)\n",
    "predictions_rf_linking = pipeline_rf_linking.predict(X_test_2)   \n",
    "\n",
    "print(classification_report(y_test_2, predictions_rf_linking))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_2, predictions_rf_linking))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "Data linking       0.86      0.85      0.86      1999\n",
      "       Other       0.85      0.86      0.85      1929\n",
      "\n",
      " avg / total       0.86      0.86      0.86      3928\n",
      "\n",
      "\n",
      "\n",
      "[[1709  290]\n",
      " [ 277 1652]]\n"
     ]
    }
   ],
   "source": [
    "#Logistic Regression\n",
    "\n",
    "pipeline_lr_linking = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer='word', stop_words='english')),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', LogisticRegression())\n",
    "])\n",
    "pipeline_lr_linking.fit(X_train_2, y_train_2)\n",
    "predictions_lr_linking = pipeline_lr_linking.predict(X_test_2)\n",
    "print(classification_report(y_test_2, predictions_lr_linking))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_2, predictions_lr_linking))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 266,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "Data linking       0.86      0.83      0.84      1999\n",
      "       Other       0.83      0.86      0.84      1929\n",
      "\n",
      " avg / total       0.84      0.84      0.84      3928\n",
      "\n",
      "\n",
      "\n",
      "[[1659  340]\n",
      " [ 278 1651]]\n"
     ]
    }
   ],
   "source": [
    "#SGD\n",
    "\n",
    "pipeline_SGD_linking = Pipeline([('vect', CountVectorizer()),\n",
    "                    ('tfidf', TfidfTransformer()),\n",
    "                    ('clf', SGDClassifier(loss='hinge', penalty='l2',\n",
    "                    alpha=1e-3, random_state=42,max_iter=5, tol=None)),\n",
    "                    ])\n",
    "pipeline_SGD_linking.fit(X_train_2, y_train_2)\n",
    "predictions_SGD_linking = pipeline_SGD_linking.predict(X_test_2)\n",
    "print(classification_report(y_test_2, predictions_SGD_linking))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_2, predictions_SGD_linking))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "Data linking       0.51      1.00      0.67      1999\n",
      "       Other       0.00      0.00      0.00      1929\n",
      "\n",
      " avg / total       0.26      0.51      0.34      3928\n",
      "\n",
      "\n",
      "\n",
      "[[1999    0]\n",
      " [1929    0]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "#SVC\n",
    "\n",
    "pipeline_svm_linking = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean)),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', SVC())\n",
    "])\n",
    "pipeline_svm_linking.fit(X_train_2, y_train_2)\n",
    "predictions_svm_linking = pipeline_svm_linking.predict(X_test_2)\n",
    "print(classification_report(y_test_2, predictions_svm_linking))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_2, predictions_svm_linking))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 268,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "Data linking       0.74      0.52      0.61      1999\n",
      "       Other       0.62      0.81      0.70      1929\n",
      "\n",
      " avg / total       0.68      0.67      0.66      3928\n",
      "\n",
      "\n",
      "\n",
      "[[1047  952]\n",
      " [ 362 1567]]\n"
     ]
    }
   ],
   "source": [
    "#KNN\n",
    "\n",
    "pipeline_knn_linking = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean)),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', KNeighborsClassifier())\n",
    "])\n",
    "pipeline_knn_linking.fit(X_train_2, y_train_2)\n",
    "predictions_knn_linking = pipeline_knn_linking.predict(X_test_2)\n",
    "print(classification_report(y_test_2, predictions_knn_linking))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_2, predictions_knn_linking))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "inputs (InputLayer)          (None, 200)               0         \n",
      "_________________________________________________________________\n",
      "embedding_3 (Embedding)      (None, 200, 50)           75000     \n",
      "_________________________________________________________________\n",
      "lstm_3 (LSTM)                (None, 64)                29440     \n",
      "_________________________________________________________________\n",
      "FC1 (Dense)                  (None, 256)               16640     \n",
      "_________________________________________________________________\n",
      "activation_5 (Activation)    (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "out_layer (Dense)            (None, 1)                 257       \n",
      "_________________________________________________________________\n",
      "activation_6 (Activation)    (None, 1)                 0         \n",
      "=================================================================\n",
      "Total params: 121,337\n",
      "Trainable params: 121,337\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "#LSTM\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from keras.models import Model\n",
    "from keras.layers import LSTM, Activation, Dense, Dropout, Input, Embedding\n",
    "from keras.optimizers import RMSprop\n",
    "from keras.preprocessing.text import Tokenizer\n",
    "from keras.preprocessing import sequence\n",
    "from keras.utils import to_categorical\n",
    "from keras.callbacks import EarlyStopping\n",
    "%matplotlib inline\n",
    "\n",
    "# Create a new data frame for the LSTM classification\n",
    "linking_or_not_linking_lstm=linking_or_not_linking\n",
    "linking_or_not_linking_lstm['Categorie'].replace('Data linking', 0, inplace=True)\n",
    "linking_or_not_linking_lstm['Categorie'].replace('Other', 1, inplace=True)\n",
    "linking_or_not_linking_lstm\n",
    "\n",
    "\n",
    "X = linking_or_not_linking.Mixed_Comments\n",
    "Y = linking_or_not_linking.Categorie\n",
    "le = LabelEncoder()\n",
    "Y = le.fit_transform(Y)\n",
    "Y = Y.reshape(-1,1)\n",
    "\n",
    "# Spliting the data to train and test\n",
    "X_train_lstm,X_test_lstm,Y_train_lstm,Y_test_lstm = train_test_split(X,Y,test_size=0.15)\n",
    "\n",
    "max_words = 1500\n",
    "max_len = 200\n",
    "tok = Tokenizer(num_words=max_words)\n",
    "tok.fit_on_texts(X_train_lstm)\n",
    "sequences = tok.texts_to_sequences(X_train_lstm)\n",
    "sequences_matrix = sequence.pad_sequences(sequences,maxlen=max_len)\n",
    "\n",
    "def RNN():\n",
    "    inputs = Input(name='inputs',shape=[max_len])\n",
    "    layer = Embedding(max_words,50,input_length=max_len)(inputs)\n",
    "    layer = LSTM(64)(layer)\n",
    "    layer = Dense(256,name='FC1')(layer)\n",
    "    layer = Activation('relu')(layer)\n",
    "    layer = Dropout(0.5)(layer)\n",
    "    layer = Dense(1,name='out_layer')(layer)\n",
    "    layer = Activation('sigmoid')(layer)\n",
    "    model_lstm = Model(inputs=inputs,outputs=layer)\n",
    "    return model_lstm\n",
    "\n",
    "model_lstm = RNN()\n",
    "model_lstm.summary()\n",
    "model_lstm.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 8902 samples, validate on 2226 samples\n",
      "Epoch 1/10\n",
      "8902/8902 [==============================] - 32s 4ms/step - loss: 0.5971 - acc: 0.6795 - val_loss: 0.4573 - val_acc: 0.8037\n",
      "Epoch 2/10\n",
      "8902/8902 [==============================] - 28s 3ms/step - loss: 0.4134 - acc: 0.8308 - val_loss: 0.4468 - val_acc: 0.8140\n",
      "Epoch 3/10\n",
      "8902/8902 [==============================] - 28s 3ms/step - loss: 0.3815 - acc: 0.8461 - val_loss: 0.4402 - val_acc: 0.8113\n",
      "Epoch 4/10\n",
      "8902/8902 [==============================] - 27s 3ms/step - loss: 0.3804 - acc: 0.8503 - val_loss: 0.4242 - val_acc: 0.8266\n",
      "Epoch 5/10\n",
      "8902/8902 [==============================] - 33s 4ms/step - loss: 0.3458 - acc: 0.8649 - val_loss: 0.4135 - val_acc: 0.8356\n",
      "Epoch 6/10\n",
      "8902/8902 [==============================] - 30s 3ms/step - loss: 0.3369 - acc: 0.8680 - val_loss: 0.4135 - val_acc: 0.8351\n",
      "Epoch 7/10\n",
      "8902/8902 [==============================] - 27s 3ms/step - loss: 0.3216 - acc: 0.8742 - val_loss: 0.4225 - val_acc: 0.8311\n",
      "Epoch 8/10\n",
      "8902/8902 [==============================] - 27s 3ms/step - loss: 0.3077 - acc: 0.8798 - val_loss: 0.4263 - val_acc: 0.8257\n",
      "Epoch 9/10\n",
      "8902/8902 [==============================] - 37s 4ms/step - loss: 0.2958 - acc: 0.8851 - val_loss: 0.4424 - val_acc: 0.8297\n",
      "Epoch 10/10\n",
      "8902/8902 [==============================] - 34s 4ms/step - loss: 0.2823 - acc: 0.8916 - val_loss: 0.4575 - val_acc: 0.8270\n"
     ]
    }
   ],
   "source": [
    "# Traing the network\n",
    "history=model_lstm.fit(sequences_matrix,Y_train_lstm,batch_size=128,epochs=10,\n",
    "          validation_split=0.2,)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1964/1964 [==============================] - 3s 2ms/step\n",
      "Test set\n",
      "  Loss: 0.431\n",
      "  Accuracy: 0.838\n"
     ]
    }
   ],
   "source": [
    "test_sequences = tok.texts_to_sequences(X_test_lstm)\n",
    "test_sequences_matrix = sequence.pad_sequences(test_sequences,maxlen=max_len)\n",
    "accr = model_lstm.evaluate(test_sequences_matrix,Y_test_lstm)\n",
    "print('Test set\\n  Loss: {:0.3f}\\n  Accuracy: {:0.3f}'.format(accr[0],accr[1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAETCAYAAADZHBoWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl4ZFdh5/1vlaqkkkr71mpJvbi3\n427HuO1uY7exceMN2+DEBJInEBIwcYCZzMsEhuQ1TNgTSHjiCQFCIAMemwTyDmRiYpNgYLDxbjC2\n27uP1Ivb2lp7aSutVff9494qldSSulrdpVJV/T7P04+q7q1bdeq2dH/3nHPPuT7HcRAREQHwZ7sA\nIiKyfigUREQkSaEgIiJJCgUREUlSKIiISJJCQUREkhQKIoAx5ofGmPee4jUHjTEvrFGRRLJCoSAi\nIkmBbBdA5HQZYw4CXwBeAwwwAfwl8CHv+f+x1n7Ye+37veUxoBf4L9baNmNMM3AX0AwcBxpT3n83\n8LdAHVAEfNlae8cK5fEDfwNcClQAPuBWa+2jxphy4CvAG4A54AfAfwfCyyz/X8AL1tq/9t77zsRz\nY8yrwC+A1wEfB2a9n8Ve+e+y1n7C2+59wH/zvvcA8B7gk0Cftfa/e695N/B2a+3b0tnvUhhUU5Bc\ndTHwl9bavcAo8DHgLcBFwB8ZY5qNMVcBfwq8yVp7AfBd4AfGGB/wd8AT1trzcEPjXABjTAD4F+A2\na+0+4Ergo8aYS1coyyW44XLAWrsHN2xu89Z9FggBu4G9uCFw5QrLT+UFa+1u3BD5b8B7rLX7cQPp\nY8aYemPMBcBfAddba18H3IMbOH8H3OJ9R4D3A19P4zOlgKimILnqmLX2Ge/xEWDEWjsDDBhjRoFa\n4Hrgf1tr+wGstXcaY/4W2ApcA3zUW37YGHO/9167gO3AHcaYxGeVAhcCLy9VEGvt48aYPwM+YIzZ\nDhwExrzV1wAfsdbGcM/arwQwxnx5meXvPcX3ftj7TMcYcxPwVmPMu3DDxYdbA7ka+LG1tsN77ZcS\nGxtjjgFvMca04QbZT07xeVJgFAqSq6YXPZ9d4jVFwMyiZT4gCDje44S5lG1GvBoIAMaYDcAI7tn4\nSYwxb8Ftbrod+DfgFeDdKe/rpLx2ExBdYfnichUv+rhx7/Vh4BngbtyguAO42dt28XuXAlusta/g\n1hbeB7QB/2Ct1eRnsoCajySf3Qf8jjGmAcAYcwswCBz21r3fW74ZeJO3jQUmvfb2xMH6BWDfCp9z\nLXCvtfbvgV/hHpyLvHX/F3iPMcZvjCnBbZq6coXl/cB+77ObWb5JaSdQCfyZtfZe3NpJife5DwDX\nGGM2eq/9APBF7/G/4NZ63oEbJCILKBQkb1lrf4rbAXy/MeZF3M7Wt1pr48AfAXuMMS8D3wIOedvM\nAL8B3GqMeQ63eeUT1tpHV/iorwMHjTHPA0/jNmed43VAfwa3tvIs7pn9f1hr/3WF5V8BNhpjLG6n\n8/2LP8zzHPBD4BXvO9wEvATssNY+D/wJcJ8x5lncZrQPpny/fwEes9YOpLkrpYD4NHW2SOHwmp0e\nAv7IWvtEtssj649qCiIFwhjzZqAD+JECQZajmoKIiCSppiAiIkkKBRERScrYOAXvyouvARfgXlN+\nq7X2cMr6/xd4J+5o1C9aa39ojKnHHXVaCnQDt1hroyt9Tn//2Krbv2pqyhgeXvHtC4b2xULaHwtp\nf8zLh33R0FDhW25dJmsKNwMha+0B3CH/tydWGGPOB96FOxjoOuCzxpgy3LlZvmutvQL3Mr0PZLB8\nBAJFp35RgdC+WEj7YyHtj3n5vi8yOaL5ctwBQlhrnzDG7E9Ztxv4ubV2CsAY0447ydflwOe91/zI\ne/w3K31ITU3ZGf0nNTRUrHrbfKN9sZD2x0LaH/PyeV9kMhQqcacGSIgZYwLW2jngedzJuypwh/Ff\nBvzDom3GgKpTfciZVOMaGiro7x879QsLgPbFQtofC2l/zMuHfbFSqGWy+WgUdxrh5Gd5gYC19mXg\nq7i1gdtxpwMeWLRNBRDJYPlERGSRTIbCo8CNAN60w88nVnhz0dRbay8H/iuQmF8muQ1wA96MkCIi\nsjYy2Xx0N3CtMeYx3JkbbzHGfAR3MrJ7gW3GmCdx53/5E2ttzBjz58Bdxpg/xK05vCuD5RMRkUUy\nFgrepGMfXLT4lZTHJ11ZZK3txZ28S0REskCD10REJEk32RERyQFzsTi9Q1E6+sfp7JugKlzMtRdv\nOuufo1DIgOnpaX7ykx9x0003n/K1//Ef91JZWcnll6dze14RyXeO4zA6MZM8+Hf0jdPZP07P4ARz\nsfkJHGorS7hmfys+37KDk1dFoZABQ0OD3HvvD9IKhRtvvGkNSiQi69HsXIzugWjywJ/4ORZdeHfZ\n4oCfTY3ltDaU09pYzqaGcrY0VZz1QIACCIXv3X+YJ1/pW3JdUZGPWOz0p066+NxGfvuqHcuu//a3\n7+DVV49xxRUXs3//65mcnOS22z7Bfff9O6+88hLRaJStW8/h4x//FN/61jeoq6tj8+atfOc73yYY\nDNDT081VV13Le97zB6ddNhFZfxzHYWh02jv7nw+A3qFJ4otuX9BQHWJHS9WCEGisLsXvP/sBsJS8\nD4Vs+P3ffx9HjhzmkksOMDY2xh//8UeZmBinoqKCL33pa8TjcX7v936b/v6FYdXb28Odd/4zs7Oz\n3Hzz9QoFkRw0NTNHV//EfAD0jdPRP8Hk9NyC15WWFLG9pTJ55t/aWE5LfZjSkuwelvM+FH77qh3L\nntWvxXD1zZu3AFBSEmJ4eJhPferjlJWVMTk5ydzcwl+Sbdt2EAgECAQClJSEMlouETkzccehPzLp\nHvT7xunsn6Czb5y+yOSC1/l80FRbxq+dU5sSAGHqKkMZaf45U3kfCtng8/lxnDhAssr3xBOP0tfX\ny2c/+wWGh4d56KEHWHzXu3X4+yFS8OZicfojk5wYitI7NEkkOsvhjmE6+8eZmY0veG15aZDdW2q8\nZp8wmxrLaa4LUxzMnZlVFQoZUFNTw+zsHNPT08llu3efx513fov3v/+9FBcX09zcwsBAfxZLKSIJ\nsXicgZEpeocm6R2O0jc0yYnhKL1DUQZHp1h81+Iiv4+NdWE2NYYXNP9UhYvX5dn/6cj5ezSfyU12\n8mG2w7NF+2Ih7Y+F8mF/xB2HodEpeocn6fXO+nuHo/QOTzIQmSQWP/lQUhUuZkNNKY21ZWyoKaWp\ntozd2xso9jkEinJ37O9KN9lRTUFE8objOETGZ9yDvnfAdx9P0jc8yVwsftI25aVBtjZVsME78Ls/\ny2isKV2y0zcfAnIlCgURySmO4zAWnXUP+omz/ZQD//Rs7KRtSkuKaG0Izx/4a8rcx7WlhEPBLHyL\n9UuhICLr0vRsjN6hKCeGopwYjNIzNH/wX3x5J0Bx0O8e7FPO9jfUugFQURbM+bb+taJQEJGsSTT3\nuAf+CXoG3RDoGYwyNDrF4lb+QJGfxppSzt1cfdJZf3V57nfyrgcKBRHJuNm5GL3Dk8kz/hODE8mD\n/9TMyc09VeXFmM3VbKwL01Rbxsa6Mppqy6itDK3ZyN5CpVAQkbMi0dbfMzjhHfgTZ/0TDIycfFln\noMjHhpoymurmD/qJEMj2qN5Cpj2fAaczS2rCoUNPU15ewY4dOzNYMpEzlxjM1TPoHvCTbf6DUaJL\ntPVXlgXZ2VqdPONPBEB91drN5yPpUyhkwOnMkprw7/9+D1dffZ1CQdaNqZk5ugeidPWPMzI5x9HO\nCD1DUfqHT57Ercjvo7GmFLO52j3zrw27B/+6Ml3dk2PyPhT+9fAPeabv+SXXFfl9Sw5YOZULG8/n\nN3e8ddn1iVlS77jjHzh69DAjIyMA/PEf/wnbt+/gL/7i03R1dTIzM8M73/luWlo28YtfPE5b2yts\n3bqNpqam0y6TyGrNzMboGYzSNTBO18AEXf0TdA+4TT6LhUMBtjVXntTkU18VyunBXDIv70MhGxKz\npE5NTbFv3+t529veQUfHa3z+85/h9tu/zNNP/4pvfvMf8fl8/PKXT3Duubu55JIDXH31dQoEyZjE\nnbsSB373pzuB2+L2/qpwMbu31NDSEKalPszu7Q2UFkFFWXF2Ci9rJu9D4Td3vHXZs/pMj0w8evQw\nTz/9K372s58AMDY2RllZmA9/+E/54hf/gmh0guuuuyFjny+FKR536ItMegf+8eSZ/4mh6Ek143Ao\nwM7Walrqw8kAaK4Pn3Twz/dRvDIv70MhGxKzpG7ZspXrrtvDddddz/DwEPfe+wMGBgaw9mW+8IW/\nZnp6mre//S28+c034vP5kjOriqQj7jgMjUzROeAe9Lv63QDoGYoyO7fwdylUXMTWpgpaGsI015fT\n0hCmtT5MZR5M4CZnl0IhAxKzpEajUR544Kfcc8+/Eo1O8L73vZ+6ujqGhga55ZZ3UVpaxu/8zrsJ\nBALs2fNrfP3rX2Xjxha2bj0n219B1pHEAK/EWX9qu//iKR2KA36a68MpZ/7ujVtqK0t08Je0aJZU\nVYkB7YvF1mp/xOJxxqKzjE7MuD+jM4xNzDDqLesdjtLVP3HSpZ7u1M1ltDS4B/1ECGTqMk/9fszL\nh32hWVJF1ojjOEzNxBidmGE0OsPoxCxj0cRj92A/5q0bi84yPjm74vv5fT421JayZ2sNzfVhWhvK\naa4P01hTqqt9JCMUCiKnMBdLPZtfdLBPnNVHvXUTs0tOz5zKB4RLg1SGi2ltcDt1K8uKqQwHqQh7\nj8uKqQgHqa0IEQzo4C9rR6EgkqJncIIHD3XTNRhlMDLJWHSGiamTR+kuFgz4qSwrZlNj2DugJw7u\n3oE+5Xl5WZAivw70sj4pFKTgzc7Fecr28fND3bR1RAD3ftnlpUGqy0vY1FiePKi7B/vg/GPveUmw\nSB25khcUClKweoeiPHiom0ee70m27e/eUsOVe5u57rJziAxHs1xCkbWnUJCCMheL83RbPw8e6ubl\n48OAWyO4/pLNXHlBMxtqywAIBoqyWUyRrFEoSEHoG47y4LPdPPpcD6NRt1ZgNlVz5YXN7NvVqM5c\nEY9CQfLWXCzOofYBHjzUxYuvurWCcCjAdRdv4sq9zWysC2e5hCLrj0JB8s5AZJIHn+3mked6GJmY\nAWBnaxUH97aw/9wGNQ2JrEChIHkhFo/z7OFBfn6oixePDuEAZSUBrtnXypV7m2lpKM92EUVygkJB\nctrgyBQPPdvNw891Exl3awXbWyq9WkEjJUHVCkROh0JBck487vDcEbdW8PzRQRwHSkuKuOqiFg7u\nbaG1UbUCkdVSKEjOGBqd4uHnenjo2W6Gx6YBOGdjJQf3NvP63RsoKVatQORMKRRkXYvHHV44NsjP\nn+nm2SMDOI57b4CDF7ZwcG8zmzdUZLuIInklY6FgjPEDXwMuAKaBW621h1PWfxR4JxAHPm+tvdsY\n4wM6gXbvZY9baz+WqTLK+hUZn+bhZ7t56NkeBkfdewVvaarg4N5mLtmzgVCxzmdEMiGTf1k3AyFr\n7QFjzKXA7cBvABhjqoEPATuAMHAIuBvYDjxtrb0pg+WSdWp2LobtiPDgM9080z5A3HEoCRbxxgua\nOXhhM1ubKrNdRJG8l8lQuBy4D8Ba+4QxZn/KugngOG4ghHFrCwD7gBZjzAPAJPBha63NYBlljcTj\nDpHxaYbGphkanWJo1Ps5Ns3g6BTDo1PJkcYAmxvLufLCFi7ds4HSEtUKRNZKJv/aKoGRlOcxY0zA\nWpuYh7gDeAkoAr7gLesBvmCt/b4x5nLgn4CLV/qQmpoyAmcwGKmhQW3SCavdF47jMDoxw0Bkkv7I\nJAPev/5h7/nIJIMjU8TjS98krzjgp766lK3NVbQ2lnP1xZvZuak667OO6ndjIe2Pefm8LzIZCqNA\n6p7zpwTCDcBGIHEz4h8bYx4FfgXMAVhrHzHGtBhjfNbaZW+5OXwGM1nmw231zpaV9sXUzNyCM/uh\n0SkGE2f7Y9MMj04xM7f0jWX8Ph/VFcVs21hJbWUJtZUhaitKqKsMUVsZoqayhIrS4EkBMDAwfta/\n4+nQ78ZC2h/z8mFfrBRqmQyFR4GbgO95fQrPp6wbxm0emrbWOsaYCFANfAoYBL5ojLkAeG2lQJCz\nYy4W58Wjgxw5PuQe7JNNPO6Bf/H9gVNVlAXZWBeeP+BXllBbEfIO+iVUlRfrhjIiOSSToXA3cK0x\n5jHcOxDeYoz5CHDYWnuPMeYa4AljTBx4BPgp8CTwT8aYt+DWGN6bwfIVvMj4NA8d6uaBQ12MeKOB\nU5UUF1FXGWJbc+WiA74bADUVJRRrxLBIXvE5Tm6fiPf3j636C+RDNfB0OY7D4a4RfvZUJ0/ZfmJx\nh9KSIg7u20R9RUlK004JpSWBrLfrZ0sh/m6sRPtjXj7si4aGimX/sHVZR4GYno3xi5d6uf+pTl7r\nc9vrW+rDXLWvlQPnbWBTS03O/6KLyJlTKOS5vuEoDzzTxSPP9TAxNYff52OfaeDqi1oxm7N/hY+I\nrC8KhTwUdxxeODrE/U938vyRQRygsizIWy/bysG9zdRWhrJdRBFZpxQKeWRiapZHn+vh/me66Bue\nBGB7cyVX7Wtlv9EtJ0Xk1BQKeaCjb5z7n+7k8RdPMDMbJ1Dk5/LzN3LVvhZNDSEip0WhkKPmYnGe\nbuvn/qc6aet0B47XV4V404UtXHFBM+WlwSyXUERykUIhxyw1tuC8c2q5+qJWXre9Dr9fHccisnoK\nhRyw3NiCa/a3ctVFrTTVlmW7iHljYjbK4cgx5iJTRCdm8OPH5/Pj9/nw+/zz//ClLC9yf+I+9/n8\nFHmv8yWXJ/657+Njiff0+bzl7vY+b73IWlIorGOnGlugewqcufGZCQ5HjtLu/eseP4HD+hnQGSoK\nUROqorqkipqSaqpD7s+akqrk8lBAV5PJ2aOjyjq01NiC/aaBqzS24IyNzYy7ATB8lMORo3RPnEiu\nC/oD7Kzexo6abexq2szISJQ4DnEnnvLPIU4cx1luufc4sRz3ubPodYn1yffxPsdx4sS85TEnxsRs\nlMj0CD0Tvct+p9JAaD40kmFRTU2oipoS93EoULIWu1fygEJhndDYgswYmR7jcOQI7ZFjtEeOciLl\n4Br0BzE1O9hZvY2dNdvZUrmJoN/9k1hvUxlMzU0zMj3C8PQIw1MRItMjDE9HGJ4eITLlLl85OErd\ngPCCYj5A5n+WFBWv4TeS9UqhkGVLji1oqeSqizS2YDUi0yMcHp5vDuqN9ifXFfuD7K7dxY7qbeyq\n2cbmilYC/tz4EwgFSggFGtkQblz2NVNz08mwcIPCC4+pkeTy1JrRYguDo9p77P50SjfjOEH1cRSA\n3PiLyFMDkUk+c+eTTEzNEQxobMFqDE9FFjQH9U0OJNeVFBWzp9aws2YbO6vdECjy5++srqFACU2B\nRppWDI4pLyASYRFhOCVAhqaWCY5DEPAHaCito7GsgcbSevdnWT0byhooD4bVrJknFApZ9HT7ABNT\nc1x1UQs3X7FNYwvSMDQ1THuiJjB8hIGpoeS6UFGIX6s7lx3V29hZs41N5S15HQKrEQqEaAqEaApv\nWPY1k15wJGobQ1MRxpxROoZ66I32L9lMVRoI0VjqhoT7z3tcWp9THeGxeIzRmbGTm+kSta2pCJOx\nSUqLQoSDYcLBspR/3vPAoufBMsoCpTnzu6hQyKL2jggA179+swJhCY7jMDg1nAyAw5GjDE4NJ9eX\nBko5v343O6u3s7N6G60VzWreOAtKAyFKAyE2pgRHoo/FcRzGZyfojfbTFx2gL9pPX7Sf3skBusa7\nOT7WcdL7VRVXzIdEWQMbvJpGXWntmjbfxZ04I9Ojbj9M6kE/efAfYWR6dNmrz4p8RVSXVNISbmJs\ncuKUzXGLlQZKF4ZIIEz5glApWxQ0YYr9J9+VMNMUClniOA5tnRFqK0uoq8qdM6lMchyHgckhrz/g\nCO3DRxmejiTXlwVKuaD+PHZ4zUEt5RsVAmvM5/NRUVxORXE5O6rPWbAu7sQZmop4QTFA3+R8cBz2\nOvpT+X1+6kI1KbWK+eaoqpLK0/q/jTtx9wx/KtH57v4cTqnxjM6MEXeWu22sn+qSKrZVbTmpAz5x\nBVdFcRi/z7/gIoRYPEZ0bpKJ2QnGZ6NMJP9NzD+eW/h8eCpCzIml9b0C/kBKzWNhaGyr2sL59XvS\n3kfpUihkyYmhKGPRWS7ds6Hg22JnYjP8qvcQD3U+Rsd4d3J5eTDM3oZfc2sCNdvYGN6gEFjH/D4/\n9aW11JfWsqfOLFg3G5ulf3KQvsmB+dDwfr44+AovDi58r6A/mGx+SoRGbajGPaimdKQnmnVGZkZP\necDfWrn5pI70xMG/orh8Vb9bRf6iZEimy3EcpmPTKQHihsb43DKhMhtdsq8nHCzjry7/1Fk/figU\nsqTNazratak6yyXJnoHJQR7qepzHu58kOjeJ3+fn/Po97K7dxc7qbTSFGxUCeSJYFKS5vInm8qaT\n1kVnJ+mfHFimSapnxff1+/xUFVcuecBPPF7tAT9TfD4foUCIUCBEXWlt2tvF4jGv1uH+qyyuyMgJ\npUIhSxKhsLPAQiHuxHl5qJ2HOh/lxUGLg0NFsJzrt17N5c2XUBMqrP0hUBYsZUtwE1sqNy1Y7jgO\nIzOjyaAYnooQLg4vGKRXWVyxrg74mVTkL6KyuILK4oqMfo5CIUvaOkYoLw3SXFcY8xZFZyd5oudJ\nHup6nP5Jt63gnMotXNl6GXsbz08OGhNJ8Pl8VJe4U3nsqtme7eIUDP0lZsHgyBSDo1NcuLM+7/sT\nusZ7eLDzMZ488TQz8VmC/gCXbtzPlS2XsbmyNdvFE5FFFApZ0NaZ3/0JsXiMQ/0v8GDnYxwZOQZA\nXaiGK1oOcKD5YsqD4SyXUESWo1DIgvY87WQemR7l0e5f8EjXE4zMuJfs7a7dxZWtl3Fe3bkF0/Yr\nkssUCllgOyKUBIvYvCH9y9jWK8dxODpynAc7H+WZ/ueJO3FCRSHe1Ho5V7QeYENZQ7aLKCKnQaGw\nxkajM/QMRjlvaw1F/tw9c56JzfBk7zM81Pk4nd7YguZwE29sPcDFGy7SVM0iOUqhsMbaO9z7Kedq\n01F/dJCHux7n8Z75sQUXNpzPla2XsaN6W953nIvkO4XCGmvPwU5md2xBGw92PsZLKWMLbth6NW/Q\n2AKRvKJQWGNtHRGK/D7O2VhJdHaSY6OvUeTzEw7OT45VvE5udhKdjfJEz694sOtxBjS2QKQg6K96\nDUWnZnltrIuGXWN89blvcGz0tSXnawn6A0tMw7t4BsWTp+Y9W1f3dI5181DXY/zyxDPMemMLDmy8\nmDe2HmBzhcYWiOQzhUKGRWejvDzUzkuDluf6X6bkvAlGgbERH1sqN3Fu7U78Pv+Sk2ANTg7TFVt5\n7pcEHz7KFk/Nu1SILAqZ4iJ3yu65eIyneg/xYOfjGlsgUsAUCmdZ3InTOdbNi4OWl4Ze4djIa8n5\n2YOEmBto5s3n7uO63ReldaBdPAnW4uBIPB9PmaJ3YGpo2RkjFwv6g4SDZcSIMTY9DmhsgUghUyic\nBROzUV4eauOlQctLQ5axGffg6sPH1srNnFdn2FNn+O69fRzuGOH6Xz9AWTC9Xb+aSbAcx2EqOTVv\nYp73pcMkETg+HxpbICIKhdWIO3E6xrp4adDy4qDl1dH52kBFcTmXNO3jvDrDubW7CAfdCe9m5+Ic\n6z7CpsZyykKZ3e0+ny9596z6NKfmTb1xiIgULoVCmsZnJnh5qI0XBy0vD1nGZycAtzZwTtWWZG2g\ntXzpW0Ie6xllLhbPqUtRRaTwKBSWEXfivDbW6fYNDFqOj3YkawOVxRVc2rSfPXWG3bU7KQueevrr\nXByfICKFR6GQYmxmPNk38PJQW7I24Pf52Va11asNnEtr+cbTHrlrC/SmOiKSW9IKBWPMi8CdwD9a\na0+c4uU5Ix6Pc2zkeLI28NpYZ7I2UFVcyYGNF7OnznBuzU7KgqVn8DkOR7pG2FBbRlV4fQxMExFZ\nSro1hRuB3wceMMYcBf4X8G/W2tmMlSzDvt/2bzzVd4ixmfnawI7qc9hTZziv7lyaw01nbR6fjr5x\nJqdj7DdVZ+X9REQyJa1QsNYeBz4HfM4Y8zbgy8A3jDH/CHzOWjuYwTKedXEnzrP9LxIsCnLZxtdz\nXp3B1O6gNLD62sBK2vL0/gkikn/SbT4qB94B/B7QAvw98P8B1wM/BvYvsY0f+BpwATAN3GqtPZyy\n/qPAO4E48Hlr7d3GmFLgn4BGYAx4j7W2f9Xfbhl+n5/PXfYxGhoqGBgYP9tvf5J8v9OaiOSPdIer\nHgOuBD5jrT3XWvt5a+1R3HDoWGabm4GQtfYAcBtwe2KFMaYa+BBwALgO+JK36j8Bz1trrwC+DfzZ\naX6ftPl8vjWZ5tlxHNo6ItRUlFBfFcr454mInIl0Q2Eb8GVr7UPGmCpjzFUA1lrHWvu2Zba5HLjP\ne90TLKxNTADHgbD3L754G+BHwDXpfpH16sRQlLHoLLs2VeteAyKy7qXb0fxxYB/uWX0Z8EljzBut\ntZ9eYZtKYCTlecwYE7DWznnPO4CXgCLgC0tsMwacsme2pqaMQKAoza9xsoaG9KePWI2njwwBcNHu\nDRn/rDO13su31rQ/FtL+mJfP+yLdULgJt28Aa22PMeYa4Bng0ytsMwqk7jl/SiDcAGwEzvGe/9gY\n8+iibSqAyKkKNjwcTfMrnGwtpnZ46iX3Ct7m6tC6nkZC01wspP2xkPbHvHzYFyuFWrrNRwEg9dKc\nYvAu6F/eo7iXsmKMuRR4PmXdMDAJTFtrp3AP/tWp2+AGx8Nplm/dau+MEA4F2FivqadFZP1Lt6bw\nDeApY8y9uGFwI/B3p9jmbuBaY8xjgA+4xRjzEeCwtfYer7bxhDEmDjwC/NT7eZcx5hFgBnjXaX+j\ndWRodIqBkSn27qjHr/4EEckB6Y5T+BtjzMO4VyDNAu+21j5zim3iwAcXLX4lZf2ngE8tWh8Ffiud\nMuUCjU8QkVyTVvORMaYE2AT04Tb17DXGfDaTBcsHbZ1un7lCQURyRbrNR/8M1AA7cNv534Tb1CMr\naOuIUBIsYvOG8mwXRUQkLek4upjfAAAPOElEQVR2NL8OuAq3n+CLwBuArRkqU14Yn5yle2CC7S2V\nBIp0S0sRyQ3pHq36rLUObp/A67zRzJrucwXtif6EVjUdiUjuSLf56AVjzFdwp7X4jjGmGfeKIlmG\nVSeziOSgdGsK/xn4nrX2JdwrhjaS45eLZlp7Z4Qiv49tzZXZLoqISNrSrSn80lp7EYC19h7gnswV\nKfdNzcxx/MQ45zRXUBxc/RQcIiJrLd2awgljzBXepalyCke6Rok7jpqORCTnpFtTuBh4EMAYk1jm\nWGt1GryENnUyi0iOSndEc0OmC5JP2joi+ICdrbr9pojklnTvvPbJpZZbazWqeZHZuThHe0ZpbSyn\nLBTMdnFERE5Lun0KvpR/xcCvAxsyVahc9uqJUWbn4mo6EpGclG7z0WdSnxtjPgf8JCMlynHJ/oTN\nCgURyT2rnX+hHNh8NguSL9oTk+CpP0FEclC6fQrHmL+pjh93crwvZqpQuSoed2jvjLChppSqcl29\nKyK5J91LUg+mPHaAiLV29OwXJ7d19o8zOR1jn1HTkYjkpnSbjyqAv7LWHgfCwA9NyoAFcVmNTxCR\nHJduKHwTuAvAWvsy8DngW5kqVK5qVyeziOS4dEMhbK39UeKJtfanuDUG8TiOQ1vnCNXlxTRUhbJd\nHBGRVUm3T6HPGPNB4J+85+8EejNTpNzUOzzJ6MQMr9/diM+nWcVFJDelW1O4BXgr0AMcB24Ebs1U\noXJRm+6fICJ5IK1QsNa+BnzCWlsBbAO+Yq3tzGjJcowmwRORfJBWKBhj/hL4K+9pGfBJY8ynM1Wo\nXNTWESEcCtDcoK4WEcld6TYfvRW4AcBa2wNcA7w9U4XKNUOjUwyMTLGztRq/+hNEJIelGwoBoDTl\neTHzI5wLXlun23S0c5OmthCR3Jbu1UffAJ4yxtzrPb8B+GpmipR72ju8+Y7UySwiOS7dUPh7IAiU\nABHcgWsbM1WoXNPWGaE46GfLhopsF0VE5IykGwrfxZ0EbwfwMPAm4JFMFSqXjE/O0tU/we4tNQSK\nVjvprIjI+pDuUex1wFXA3bizo74B2JqhMuWU9k6NTxCR/JFuKPRZax3gFeB11tqjuJ3NBW9+fII6\nmUUk96XbfPSCMeYruH0L3zHGNOPemrPgtXWMUOT3sa1FoSAiuS/dmsJ/Ar5nrX0J+BRuJ/O7Mlaq\nHDE1M8drvWNsbaqgJFiU7eKIiJyxdO/RHMPtYMZaew9wTyYLlSuOdI8SizvsVH+CiOQJXS5zBto1\nCZ6I5BmFwhlo64jgA3aqk1lE8oRCYZXmYnGOdI/S0lBOOBTMdnFERM4KhcIqvXpijNm5OLs035GI\n5BGFwirppjoiko8UCquUCIWduqmOiOSRdAevnTZjjB/4GnABMA3caq097K3bC3wp5eWXAjcDvwTa\ngBe85Xdba/82U2VcrXjcob1zhMbqUmoqSrJdHBGRsyZjoYB7kA9Zaw8YYy4Fbgd+A8Baewg4CGCM\n+S2g21p7nzHmGuCfrbX/TwbLdcY6+8eZnJ5j366GbBdFROSsymQoXA7cB2CtfcIYs3/xC4wxYeAz\nwBu9RfuAi4wxDwJ9wIe8O70tq6amjEBg9aOJGxpOf7rrX9h+APbt2bCq7derfPouZ4P2x0LaH/Py\neV9kMhQqgZGU5zFjTMBaO5ey7A+A71trB7znrwBPWWv/rzHmd4GvAO9Y6UOGh6OrLmBDQwX9/WOn\nvd1TL/cC0FQdWtX269Fq90W+0v5YSPtjXj7si5VCLZMdzaNA6if7FwUCwO8C30x5fj/wgPf4buDC\nzBVvdRzHob0jQlV5MY3VpafeQEQkh2QyFB4FbgTw+hSeT11pjKkCSqy1HSmLvwm83Xt8NfBUBsu3\nKn3Dk4xMzLCrtRqfTxPFikh+yWTz0d3AtcaYx3Cn2b7FGPMR4LA3qd4u4NVF29wG3GGM+c/ABHBr\nBsu3KhqfICL5LGOhYK2NAx9ctPiVlPVP4l6hlLrNMdxbfa5bbbrTmojkMQ1eO01tHRHKSgK0NISz\nXRQRkbNOoXAahsem6Y9MsbO1Cr/6E0QkDykUTkO7mo5EJM8pFE6DTcx3pFAQkTylUDgN7R0RigN+\ntjbl72hGESlsCoU0jU/O0tk/wbbmSgJF2m0ikp90dEvT4U53xg71J4hIPlMopEnjE0SkECgU0tTW\nEaHI72N7s26/KSL5S6GQhumZGMdPjLGlqYKS4tVP0y0ist4pFNJwpHuEWNxhl269KSJ5TqGQBk2C\nJyKFQqGQhnbvyqMdrepPEJH8plA4hblYnCNdI7Q0hCkvDWa7OCIiGaVQOIXjJ8aYmYur6UhECoJC\n4RSS/QnqZBaRAqBQOAV1MotIIVEorCDuOLR3jtBQHaKmoiTbxRERyTiFwgq6+ieITs+p6UhECoZC\nYQVqOhKRQqNQWIFCQUQKjUJhGY7j0NYZoTJcTGNNabaLIyKyJhQKy+iPTDIyPsOuTdX4fL5sF0dE\nZE0oFJZhk+MTNLWFiBQOhcIy2jt0pzURKTwKhWW0dUQoLQnQ2lCe7aKIiKwZhcISIuPT9EUm2dla\nhd+v/gQRKRwKhSXoUlQRKVQKhSVoEjwRKVQKhSW0dYwQDPjZurEi20UREVlTCoVFJqZm6eofZ3tz\nJYEi7R4RKSw66i3S3jmCA+xU05GIFCCFwiLtif6EzQoFESk8CoVF2joi+H0+tjdXZrsoIiJrTqGQ\nYno2xqsnxtjSVE6oOJDt4oiIrDmFQoqj3aPE4o7GJ4hIwVIopND4BBEpdAqFFIlQ2KmagogUqIw1\nnBtj/MDXgAuAaeBWa+1hb91e4EspL78UuBn4FfBdoBToBm6x1kYzVcZUc7E4R7pHaKkPU14aXIuP\nFBFZdzJZU7gZCFlrDwC3AbcnVlhrD1lrD1prDwJ/B/yrtfY+4JPAd621VwDPAB/IYPkWON47xsxs\nXLUEESlomQyFy4H7AKy1TwD7F7/AGBMGPgN8aPE2wI+AazJYvgXm75+gm+qISOHK5HWXlcBIyvOY\nMSZgrZ1LWfYHwPettQNLbDMGnPIIXVNTRiBQtOpCNjS48xu92jsOwIELWqmvLsx7Mif2hbi0PxbS\n/piXz/sik6EwCqTuOf+iQAD4XeAdS2wz6f2MnOpDhodX3+XQ0FBBf/8YccfhxaMD1FeFcGbn6O8f\nW/V75qrEvhCX9sdC2h/z8mFfrBRqmWw+ehS4EcAYcynwfOpKY0wVUGKt7VhqG+AG4OEMli+pu3+C\niak5jU8QkYKXyZrC3cC1xpjHAB9wizHmI8Bha+09wC7g1UXb/DlwlzHmD4EB4F0ZLF9SW6duqiMi\nAhkMBWttHPjgosWvpKx/EvcKpdRteoHrM1Wm5ehOayIiroIfvOY4Dm0dESrLgmyoKcwOZhGRhIIP\nhf6RKSLjM+zcVI3P58t2cUREsqrgQ6HtNTUdiYgkKBQ6NQmeiEhCwYdCe0eE0pIiNjWWZ7soIiJZ\nV9ChMDw6Re/wJDtaqvH71Z8gIlLQofDisUFA8x2JiCQUdigcTYSC+hNEREChQKDIz9amymwXRURk\nXSjYUIhOzfJqzyjbmysJBgp2N4iILFCwR8P2zhEcR7feFBFJVbChMD8JnjqZRUQSCjYU2jtG8Pt9\nbG9WKIiIJBRkKMzF4hzrGWVbSxWlJZmcPVxEJLcU5BGxyO/j0vM2cMWFm7JdFBGRdaUgQ8Hn8/EH\nb9mTF7fVExE5mwqy+UhERJamUBARkSSFgoiIJCkUREQkSaEgIiJJCgUREUlSKIiISJJCQUREknyO\n42S7DCIisk6opiAiIkkKBRERSVIoiIhIkkJBRESSFAoiIpKkUBARkSSFgoiIJBXcTXaMMX7ga8AF\nwDRwq7X2cHZLlT3GmCBwB7AVKAH+3Fp7T1YLlWXGmEbgKeBaa+0r2S5PNhljPgb8OlAMfM1a+60s\nFylrvL+Vu3D/VmLAH+bj70ch1hRuBkLW2gPAbcDtWS5Ptr0bGLTWXgHcAHw1y+XJKu8P/xvAZLbL\nkm3GmIPAZcAbgCuBQr9/7Y1AwFp7GfBZ4C+yXJ6MKMRQuBy4D8Ba+wSwP7vFybrvA59IeT6XrYKs\nE38NfB3oznZB1oE3A88DdwP3Aj/MbnGyrg0IeK0NlcBslsuTEYUYCpXASMrzmDGm4JrREqy149ba\nMWNMBfAvwJ9lu0zZYox5L9Bvrf1xtsuyTtTjnjT9FvBB4DvGGF92i5RV47hNR68A/xP4clZLkyGF\nGAqjQEXKc7+1tqDPjo0xm4AHgH+01n432+XJovcB1xpjfg7sBb5tjGnKbpGyahD4sbV2xlprgSmg\nIctlyqYP4+6PXbh9kncZY0JZLtNZV4hnyI8CNwHfM8Zcils9LljGmA3AT4D/Yq39WbbLk03W2jcm\nHnvB8EFr7YnslSjrHgH+qzHmfwAbgTBuUBSqYeabjIaAIFCUveJkRiGGwt24Z4OPAT7gliyXJ9s+\nDtQAnzDGJPoWbrDWFnxHa6Gz1v7QGPNG4Je4rQp/ZK2NZblY2fQ3wB3GmIdxr8b6uLV2IstlOus0\ndbaIiCQVYp+CiIgsQ6EgIiJJCgUREUlSKIiISJJCQUREkhQKIllijHmvMebObJdDJJVCQUREkjRO\nQeQUjDG3Ab+NO3r1x8DfA/+GOwfOecBx4N3W2iFjzFuBP8c94ToKfMBa22uMuQZ3Rl6/9/p3Ab8J\n3Io7CeFm4GfW2j9cy+8msphqCiIrMMZcD+wDLgYuBFqA3wXOx72/wHnAy8CnvfswfAO42Vr7Otwp\nVb5qjCkBvgO8x1p7Pu7UKu/xPmIzbjjsBm4wxpy3Zl9OZAmFOM2FyOm4BrgE96Y7AKW4J1Nt1tqf\ne8vuAr6LO4fUL621r3rL/wH4GG6AdFlrDwFYaz8GyVlZH7LWDnnPj+DOTCqSNQoFkZUVAV+y1v4P\nAGNMNdAK/O+U1/hxm4AW17x9uH9js0CyndYYU8X8TL2pM/Q63jYiWaPmI5GV3Q/8njGm3Lvvxg9w\n7zFgjDF7vdfcAvwI+AVwqTFmq7f8/bhTklug0Rizx1v+p7j3JxBZdxQKIiuw1t4L/B/cA/4LwCHg\nQdypkz9jjHkRaMS9t3UvbhDc7S0/iDv99hTubU+/bYx5DtgD/OVafxeRdOjqI5HT5NUEfm6t3Zrl\nooicdaopiIhIkmoKIiKSpJqCiIgkKRRERCRJoSAiIkkKBRERSVIoiIhI0v8PVjpvTkkdVG8AAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize history for accuracy\n",
    "plt.plot(history.history['acc'])\n",
    "plt.plot(history.history['val_acc'])\n",
    "plt.title('model accuracy')\n",
    "plt.ylabel('accuracy')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train', 'test'], loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAETCAYAAADZHBoWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8XNV99/HPnUXLjLbRaJctS5bs\nY8u25A1sErOYLUBCAglZShYgITRt8qQJTdMkDdkeAi0pSUMJTdNAgQLpQxJMCARDEgiLwTbG2JYt\n6ciSbEvWvox2WdvM88edGUnGlmRZ49GMfu/XSy9m7jL36CDf75xz7z3H8Pl8CCGEEACWcBdACCHE\n/CGhIIQQIkhCQQghRJCEghBCiCAJBSGEEEESCkIIIYIkFISYJaXUs0qpm6fZ5hKl1MGZLhci3CQU\nhBBCBNnCXQAhzgWl1CXA3UAdoIB+4J+BL/vf/1Zr/VX/trf5l48BLcCXtNZVSqkc4BEgBzgGZEz4\n/JXATwE3YAXu01o/NMOyJQM/A9YCPuB54Fta61Gl1PeB64FhoAO4WWvddLrls6sdIcZJS0EsJOcB\n/6y1Xgv0AN8E3g+sB76olMpRSl0KfB3YqrUuBZ4AnlZKGZgn7p1a61WYobECQCllA34DfENrvQG4\nGPiaUmrzDMt1H+aJfQ2wESj1778Y+ApwntZ6I/AisOl0y8+mYoQIkFAQC8kRrfU7/tc1wMta62Gt\ndTtmSKQCVwH/T2vdBqC1fhjIBfKBy4GH/curgZf8n7UcKAQeUkrtA14B4oF1MyzX1cD9Wmuf1noI\n+Ll/WQOwH9irlPpXYJ/W+ukplgtx1iQUxEIydNL7kVNsY8XswpnIAOz+5caE5aMT9unWWq8N/ACb\ngf+eYbksJx3TAti11l7MVsfNmC2Jnyil7jnd8hkeS4gpSSgIMdl24BNKqXQApdQtmCfeav+62/zL\n84Ct/n00MKiU+pR/3WLgILBhhsd8AfiSUspQSsX6j/FHpVSp/3MqtNZ3Az8Bzjvd8rP7tYUwSSgI\nMYHW+o+YJ9mXlFKHgJuAD/i/nX8RKFZKVQAPAvv8+wwDHwJuVUodwOzjv0NrvWOGh/0y5kXrMv+P\nBn6otd4PPAnsUUrtAT4L3H665Wf/2wsBhgydLYQQIkBaCkIIIYIkFIQQQgRJKAghhAiSUBBCCBEU\n8cNctLX1zvpKucvlwOMZmMviRCypi8mkPiaT+hgXDXWRnp5onG7dgm4p2GzWcBdh3pC6mEzqYzKp\nj3HRXhcLOhSEEEJMJqEghBAiSEJBCCFEkISCEEKIoJDdfaSUsgAPYI4NPwTc6h9uOLD+auC7/rd7\nMceViQMewxwHphe4KTCEsRBCiNALZUvhOiBOa30B8A3g3sAKpVQi8CPMgcY2A0eBNOBvgDKt9YXA\no8C3Q1g+IYQQJwnlcwpbMIcaRmu9Uym1ccK692COBnmvUmop8EutdZtSagsQGBf+eeCO6Q7icjnO\n6hax9PTEWe8bbaQuJpP6mEzqY1w010UoQyEJ6J7wfkwpZdNaj2K2CrZizknbB7ymlHrzpH16geTp\nDjLbh0j+sPMY56/OIS3BPqv9o016eiJtbb3hLsa8IfUxmdTHuGioi6lCLZTdRz3AxCNb/IEA5qQl\nb2mtm7XWfcCrmAExcZ9EoCsUBRvzennqlVoefOZgKD6eoaEhfv/7mc2O+Ic//J7XX38lJOUQQogz\nFcpQ2AFcA+CfwLxswrq3gdVKqTT/pOebgfKJ+2DOUftaKApmtVgoWpRMVb2HnoHhOf/8zs6OGYfC\nNddcy5YtF895GYQQYjZC2X20DbhCKfUG5ry2tyilbgeqtdbPKKW+iTkNIcCTWuuDSqla4BGl1OvA\nMHDj2RbiyZeqeauy9V3LB4dG8fng2/+1i1j7mV2TOG9FBh+7tOi06x999CGOHj3ChReex8aN5zM4\nOMg3vnEH27c/R2VlOQMDA+TnF/Ctb32XBx/8T9xuN3l5+Tz++KPY7Taamhq59NIruOmmz53x7yuE\nEGcjZKHgn77wCyctrpyw/n+B/z1pnwHgo6Eq00QxdgsDQzA8MnbGoTCdz3zms9TUVLNp0wX09vby\nla98jf7+PhITE/m3f3sAr9fLpz/9MdraJodVS0sTDz/8K0ZGRrjuuqskFIQQ51zEj5I6nY9dWnTK\nb/U+n49v/mInvQMj3P3Xm7FZQ9OTlpe3BIDY2Dg8Hg/f/e63cDgcDA4OMjo6OmnbpUuLsNls2Gw2\nYmPjQlIeIYSYyoJ9otkwDDauzGRwaJSahu7pdzijz7bg83kBsFjMEWp37txBa2sL3//+Xdx22xcZ\nGjrByfNjG6cdzFYIIc6NBRsKAOcVZwGwv6ZjTj/X5XIxMjLK0NBQcNnKlatobGzgtttu5o47/pGc\nnFza2+VhbSHE/GKc/G010pzNJDtJKQ5u/PYfSEuJ585bN81lsSJONNx7PZekPiaT+hgXDXUhk+yc\nRqzdyoolLhrb+2nvGgx3cYQQIuwWdCgAlBa6gbnvQhJCiEi04EOhpDANgAMSCkIIIaHgTo5jUbqT\nimMehobHwl0cIYQIqwUfCmC2FkbHvFQc84S7KEIIEVYSCkCJ/7rCgZr2MJdECCHCS0IBKMxNwhln\nY39Nx7seKJuNMxklNWDfvr1UVx8+62MLIcTZkFDAHDV19VI3nt4hjrf1n/XnnckoqQHPPfeMPMwm\nhAi7qB/76KnqZ3mnteyU66wWgzGv2TIYShojtnSYnx7aQXz11NWyLmMNHy76wGnXB0ZJfeihX1Bb\nW013tzmMxle+8g8UFhbxwx9+j4aG4wwPD/NXf/UpcnMXs2vXm1RVVZKfv5SsrKxZ/rZCCHF2oj4U\nZirGbsEYNEdNjY89u2oJjJJ64sQJNmw4n+uvv4H6+jruuuv73Hvvfezdu4df/vJ/MAyD3bt3smLF\nSjZtuoDLLrtSAkEIEVZRHwofLvrAab/Vn/y4+l3/8zY1jd3c9eULSYg/+2k6a2ur2bt3D3/+84sA\n9Pb24nA4+epXv8499/yQgYF+rrzy6rM+jhBCzJWoD4UzUVLoprqhm4O1HWxeNftv7IFRUpcsyefK\nK4u58sqr8Hg6+f3vn6a9vR2tK7j77n9laGiIj3zk/bzvfddgGEZwZFUhhAgXCYUJSgrdPPVqLQdq\nzi4UAqOkDgwM8PLLf+SZZ55iYKCfz372NtxuN52dHdxyy43Exzv4xCc+hc1mo7h4NT//+f1kZ+eS\nn18wh7+VEELM3IIeJfXk7iOfz8fXHniD4ZExfvrlC4NzISwE0TDy41yS+phM6mNcNNSFjJI6Q4Zh\nUFLopv/EKDWNczvxjhBCRAIJhZOMP90sA+QJIRYeCYWTFC9JxWa1sL9aQkEIsfBIKJwkNsbKiiUp\nHG/ro7PnRLiLI4QQ55SEwimUyhwLQogFKmS3pCqlLMADQCkwBNyqta6esP4+4L1A4DL+hwArUAUc\n9C/bprX+aajKeDolhW4e/yPsr27nknW55/rwQggRNqF8TuE6IE5rfYFSajNwL+aJP2A98D6tdXC8\naqXU5cCvtNb/J4TlmlZ6SjzZbgcVxzwMj4wRY7eGszhCCHHOhLL7aAuwHUBrvRPYGFjhb0UsA36h\nlNqhlPqsf9UGYL1S6hWl1K+VUtkhLN+USgvTGB71UlnXFa4iCCHEORfKlkISMPFm/zGllE1rPQo4\ngX8HfozZZfSyUmoPUAm8rbX+k1Lqk/5tbpjqIC6XA5tt9t/k09MTT7n8og2L2b67jsONPVy2OX/W\nnx9JTlcXC5XUx2RSH+OiuS5CGQo9wMSas/gDAWAA+KnWegBAKfUS5rWHp/3rALYBP5juIB7PwHSb\nnNZUTyamJdiJj7Wx62ATH96Sj2FE99PN0fCU5lyS+phM6mNcNNTFVKEWyu6jHcA1AP5rChMnNVgO\nvK6Usiql7JhdTXuBXwIf8W9zGfB2CMs3JZvVwqqCVNq7T9DYMfvgEUKISBLKUNgGnFBKvQH8BPiq\nUup2pdQHtdYVwOPATuAV4FGt9SHgG8DfKKX+AnwB+LsQlm9apTJ3sxBigQlZ95HW2ot5Yp+ocsL6\ne4B7TtrnCLA1VGU6U2uWujGAA9UdXL1pSbiLI4QQIScPr00hyRlDfnYSh493M3BiJNzFEUKIkJNQ\nmEZpoRuvz8fBI53hLooQQoSchMI0Sopk1FQhxMIhoTCNvMxEkp0xlNV24I3wCYmEEGI6EgrTsBgG\nawrd9A6McKSpJ9zFEUKIkJJQmIHgqKkyx4IQIspJKMxAcb4Lq8WQ6wpCiKgnoTAD8bE2VF4Kx1p6\n8fQOhbs4QggRMhIKM1Ti70Iqq5XWghAiekkozFBgyIv91TLkhRAiekkozFBmqoNMVzzlRz2MjHrD\nXRwhhAgJCYUzUFKYxtDIGFX1MvGOECI6SSicgcDTzftl1FQhRJSSUDgDanEKsTFWuTVVCBG1JBTO\ngM1qYVV+Kq2eQZo7ZeIdIUT0kVA4QyWBiXfkLiQhRBSSUDhDgVDYL11IQogoJKFwhlISYlmSmUhV\nfReDQ6PhLo4QQswpCYVZKCl0M+b1UX5UJt4RQkQXCYVZGL81VbqQhBDRRUJhFgqyk0h02CmrkYl3\nhBDRRUJhFiyGQclSN939w9S19Ia7OEIIMWckFGappEgm3hFCRB8JhVlalZ+K1WLIkBdCiKhiC9UH\nK6UswANAKTAE3Kq1rp6w/j7gvUCg/+VDgB14AogHGoFbtNbz8tFhR5yNZYuSqazrort/mGRnTLiL\nJIQQZy2ULYXrgDit9QXAN4B7T1q/Hnif1voS/0838B3gCa31hcA7wF+HsHxnLTjxjtyFJISIEiFr\nKQBbgO0AWuudSqmNgRX+VsQy4BdKqUzgQa31Q/597vJv9rz/9U+mOojL5cBms866kOnpibPe95Lz\n8njy5Wr08W6uv2z5rD9nvjibuohGUh+TSX2Mi+a6CGUoJAHdE96PKaVsWutRwAn8O/BjwAq8rJTa\nc9I+vUDydAfxeGbfu5Senkhb2+zvHoo1fKQlx7FXt9DU3I3NGrmXaM62LqKN1MdkUh/joqEupgq1\nUJ7FeoCJR7b4AwFgAPip1npAa90LvIR57WHiPonAvJ7NxjAMSgvTGBwa4/Dx7ul3EEKIeS6UobAD\nuAZAKbUZKJuwbjnwulLKqpSyY3Yb7Z24D3A18FoIyzcnAk83H5C7kIQQUSCUobANOKGUegPzusBX\nlVK3K6U+qLWuAB4HdgKvAI9qrQ8BdwKfUErtAC4A7g9h+ebEirwUYuwWmXhHCBEVQnZNQWvtBb5w\n0uLKCevvAe45aZ8W4KpQlSkU7DYrxUtS2VfdTmvXIBkp8eEukhBCzFrkXhmdR2TiHSFEtJBQmAPB\nUJAuJCFEhJNQmAOpSXEsSk+gsq6LoeGxcBdHCCFmTUJhjpQWuRkd81J+TCbeEUJELgmFOVLqH/JC\nupCEEJFMQmGOLM1JIiHezoGaDnwy8Y4QIkJJKMwRi8Vg9dJUPL1D1Lf2hbs4QggxKxIKcyhwF5LM\n3SyEiFQSCnNodYEbw5AhL4QQkUtCYQ4lxNspyk2mtqGH3oHhcBdHCCHOmITCHCspdOMDDtbKralC\niMgjoTDHAremytzNQohIJKEwx3LTnaQmxXKwtpMxrzfcxRFCiDMioTDHDMOgpDCNgaFRahp6wl0c\nIYQ4IxIKITB+a6p0IQkhIouEQgisXOLCbpOJd4QQkUdCIQRi7VZW5LloaOuno/tEuIsjhBAzJqEQ\nIuNzLEgXkhAickgohEipDHkhhIhAEgohkpYST26ak8pjHoZHZOIdIURksIW7AOHg9Xn5RdmjeC2j\nZMVmsTgxl8WJuWQ40rAYc5eTJYVunt9VR2WdhxL/Q21CCDEbI2MjVHcfobxDU95ZRbw1lq9t/NKc\nH2dBhoKBwYnRExzuquUQVcHlMdYYFiXkBEMiLzGXLEcGVot1VscJhML+mg4JBSHEGfH5fLQNtlPe\nUUV5p6bKU8OIdwSAGIud87LWh+S4CzMUDIOvrP8CzhQb+48epr63IfhzpPsYtd1Hg9vaLDZyndks\nThwPixxnFnarfdrjFC1KxhFr40B1O74rlmMYRgh/KyFEpDsxOsThrhqzNdChaT8xPoZatjOT4lRF\nsVtRmJw/o3PQbIQsFJRSFuABoBQYAm7VWlefYpvngN9prX+ulDKA48Bh/yZvaq2/GaoyOuzxFKUU\nUJRSEFw2PDZMQ1/zeFD0NXC8r5FjvfXBbSyGhWxn5qQWRW5CDrHWmEmfb7VYWL00ld0VrTS097Mo\nPSFUv4oQIgL5fD6a+ls41FFJeWcVNV1HGPOZ1yDjrHGsTV9DsXs5xakKV1zKOSlTKFsK1wFxWusL\nlFKbgXuBD520zZ1A6oT3hcBerfW1ISzXlGKsMRQk51GQnBdcNuodpam/ZVKL4nhfEw19Texs2gOY\nXVKZjvRgUJg/OZQUutld0cqBmg4JBSEEAyODVHoOU+G/NtA11B1ctzgxl1WpipVuRUFS3qy7rs9G\nKENhC7AdQGu9Uym1ceJKpdQNgBd4fsLiDUCuUuplYBD4qtZah7CMM2Kz2IIn+oAx7xgtA23B1kR9\nbwPHextpHmjlrZZ3gtulxqYSUxjD682NFHQMsTgxl4QYZzh+DSFEGHh9Xup7G4LXBo721OH1mYNl\nOu0ONmaupThVsdK9nKSYxDCXdoahoJQ6H/Mkfz/wLLAO+LTWevsUuyUB3RPejymlbFrrUaXUauBG\n4AbgOxO2aQLu1lr/Wim1BXgMOG+qsrlcDmy22adpevrs/ydkkUIpy4LvvT4vLX3tHPHUUeup54in\njiOeeqzuTrpp5v79BwBwO1wUuPJY6lpMfspishMzSHOkEmuLOd2hzomzqYtoJPUxmdTHuOnqovtE\nD/ubK9jXXM7+5nJ6h8x52w3DYHlqAaXZq1ibVcxSVx4Wy/x6MmCmLYX7ME/eNwADwHrgKfwtgdPo\nASbWnEVrPep//RkgF3gJyAeGlVJHgVeBUQCt9etKqVyllKG19p3uIB7PwAx/hXdLT0+kra131vuf\nio14lsUrlsUryDH7DH+9o4wXDx1i49pYvHFd1PUeZ0/DfvY07J+0b4LdiTsuldS4FFLjXKTGu3DH\nuczXcS7ibXFzWtaJQlEXkUzqYzKpj3Gnqosx7xhHe+op76ikvFNT19sQXJcck8QF2edR7FascBXh\nsDvMFV7o6Og/l0UPmirUZhoKFq31i0qpx4Hfaq3rlVLT7bsDuBZ40n9NoSywQmv99cBrpdT3gGat\n9Xal1L8AHcA9SqlSoG6qQIgEhmGwqaiA7a+3Y23N5AvXrgKge6jHf22ikY7BTjpPdNFxopOGky5q\nTxRvi58QEimTAiM13oXT5pA7nIQ4RzwnuqjorKK8Q1PpOczgqDnOmdWwsjylkGK3eadQjjMrov5d\nzjQUBpRSfw9cCnxJKfVlYLqvDduAK5RSbwAGcItS6nagWmv9zGn2+WfgMaXU+zFbDDfPsHzzWl5m\nAikJMZTVduL1+rBYDJJjk0iOTWJ12spJ23p9XnqH++g44aHT/zP+uovWgTaO9zWe8jgx1hhS41yn\nDY7EmIQ5fThPiIVkzDtGTfdRtjfU8PbxgzT2NwfXueNcbMxcR3Hqcpa7CokLYas+1GYaCp8EPgd8\nRGvtUUrlYl4TOC2ttRf4wkmLK0+x3fcmvPYA759hmSKGOfGOm1f3N1Hb1ENRbvJpt7UYlmBgLE1e\n8q71Pp+P/pGBk8JicnA097ec8rNtFhupsSnjrYtAcMSnYk3IA879nQ5CzGf9IwMc6qjkYHsF5Z1V\nDI4OAmC32MyWgP+5gYz4tIhqDUxlpqHQBjyttT6glLoRc8wkGRP6DJQUpvHq/iYO1LRPGQrTMQyD\nhBgnCTFO8pIWnXKbwdFBOk90mWExOB4cgS6qVs8pRm7dC1mODFanrWRNWnHYbocTIpx8Ph8tA22U\ntZdzsKOC2u5jwTuFXLEpnJe5ji2F60k3sokJ0cNj4TbTUHgMOKKUigO+DzwKPAx8IETlijrF+S5s\nVoMD1R18+KLCkB4r3hZPbkI8uQnZp1w/PDbsD4jxwGgfaaesuZI/1b3Cn+pewWlzUOxWrE5bSXHq\n8vGLY0JEmTHvGNVdRyjrKKesvYL2QXNkYwOD/KQ81qStZHXayuC1gWi/6D7TUCjQWn/MfyH4l1rr\nf1FKvRXKgkWbuBgbanEKh4568PQO4UqMDVtZYqwxZDkzyHJmBJelpyfS0NxJlaeag/7m8lst7/BW\nyztYDAuFyflmK8K9kswJ+wkRifpG+jnUXsnBjgrKO6o4MWZ2fMRaY1ibvoY1aStZ5V5BYszCe+B0\npqFgU0qlAdcDH1ZKZQHxoStWdCopTOPQUQ8Hatq5eG3u9DucYzFWO6v934p8y6+joa+Jgx0VlLVX\nUN11hMNdtWyrfo6M+DR/N9NKCpMLpJtJzHs+n4/mgVazW6jd7BbyYd7Y6I5zsSl7A2vSVlKUshS7\nZUEOCRc009/+R8Au4Bmt9UGlVBVwR+iKFZ1Ki9z86s+HOVDTMS9DYSLDMFiUmMOixByuyr+MnuFe\nDnVoDraXU9FZxUv1r/FS/WvEWeModi9nTVoxxW5Fgl2e1hbzw6h31OwW8gdBYHA5A4OC5CWscZtf\ngLKdmVFzkXguzCgUtNZPKKV+AyxXSq0Fiic8iCZmKMPlICvVQflRDyOjXuy2yLk9NCkmkQuyN3JB\n9kZGvKNUe2op66jgYHs5e1sPsLf1wPg/trSVrHbLPzZx7vUO9wXvFqrorOLE2BAAcdZY1mWUsMZt\ndgvJUDOnN9NhLjYCv8F8sMwCZCqlrtda7wpl4aJRSaGbF9+qR9d7WF3gDndxZsVusbHSvZyV7uV8\ndNkHaepvCXYzBYYe/13N87jjUoPXIYpc0iwXcy8wymjgbqEj3XXBbqG0uFQuyD6P1WkrKUopwCZ/\nfzMy01r6KfDxQAj4n1D+d+D8UBUsWpX6Q2F/dUfEhsJEhmGQk5BFTkIWVy7ZSt9wP+WdmrL2cso7\nqnjl+A5eOb6DWGsMK1OXszqtmNUL9AKemBvjLVWzW6jjhAcwu4WWJi9hTVoxa9JWkunIkJbqLMw0\nFBImtgr8o55G7iN7YbRscQpxMVYO1LRz4+XLou6PNiHGyflZ6zk/a33wVj+zFVHOvraD7Gs7iIHB\nkqTFwW6m3ITsqKsHcXa8Pi8nRocYGB2gf2SAgZFBOoc8lHdoKjqrGBobBiDeFseGjFLz1mm5pjUn\nZhoKnUqpD2mtfweglLoesytJnCGb1cKqglTe1m00dw6Q7Y7eP2KrxYpKLUKlFvGRZdfS0t/qvw5R\nQU33UY721PH72hdwxaaYdz25V6BcRSGbUUqce2PeMQZGB82fEf8JfnTQf6IfoD+wfNQ88U98HegG\nOll6vJs1acWsdpvdQnL329yaaSjchjkm0YP+97XAp0JTpOhXUujmbd3G/uqOqA6Fk2U6M8h0ZnB5\n3sUMjAxQ3lnl72bSvNbwJq81vInNsJKTkMXixEXBWe1yErLlekSYjYyN0D3Uc4qTunkCN/87+XX/\nyGDw/v+ZsBpWHPZ4Eu0JZDoycNrjcdgcOO0OHDYHCTFOlqUsJdORLi3LEJryX5p/sptAXA8ARzAv\nNPcDP8ccIE+coZKl5rWEAzXtXLUpb5qto5PDP7nIxsy1jHnHONJTR1l7OVWeGhr7mycNPWw1rOQE\npz9dRF5SLjnO6B1mIFx8Ph9dQ9009rfQ3N9Ck/+nub/1jE7uMRY7DrsDd7wLhy0+eFJ32h047PE4\nbQ4cdsfkk77dQYzFLif7eWC6r1/fOxeFWGiSE2LJz0rk8PFuBk6M4ohb2N+CrRbrpLmyx7xjNAan\nPz1OXW8DDX2N1Pc1QpP5IP2758lexKKEbGKs4Z2oKBL4fD48Q1009bfS1N9Mc3+r/+TfEryFM8Bq\nWEl3pJGRUIDdF2OezG3xOPwn8sBr88Qfj8MWL91/EW7Ks5HW+pVzVZCFpqTQzdHmXsqPdrJxhQwb\nMZHVYmVxYg6LE3MITLw35h2jeaCVOn9QBKY/PXme7CxnBnnBrqdF5CZkE2cL35Ai4TR+8p/8rf90\nJ/8MRxrZzkyynZlkOTPJcWaSHp+G1WKN+vF+xLiF/RU1jEqL0nhmx1H217RLKMyA1WIlNyGb3IRs\nLsg2p/v2+rw097f6WxQN1PUep76vkab+FnY1vw2YQZHpSDe7nRJz/NcqciJ6vPuTnerkH/jmH7hL\nJ8BqWMl0pJPlzPAHQBbZzozgyV8ICYUwWZKVSJIzhrKaDrw+HxbpSz1jFsMSfEZiU/YGwAyK1oF2\nMyD8YVHf20DzQCtvtewFzKBId7gntCjMLqh42/wezsvn85nzZQzM/ORvfuvPkJO/mDEJhTCxGAZr\nlqayo6yZY829FGQnhbtIUcFiWIIjwJ6ftR4wg6J9sIO6QGuip4H6vgb2tOxjT8u+4L7p8e5gt1P+\nYA69vbOfMsTnm4tZZH14hrpp6muhaeBMTv6ZpMe75eQvZsWYmz/e8Glr6531LxDuftK3dSs/23YQ\nd1Icn7hsGeuXh2/2pnDXxbnm8/loH+yc1KKo6z3OgH9mrflm4sk/0Od/Lk/+C+3vYyrRUBfp6Ymn\nPdFISyGM1i1P55rNS3hhdx0/21bGqoJUbrx82YJ6diFcDMPsQkp3uNmQWQoEumc81PU24I0Zpq9v\naJpPmeYYc1DOxJhE+eYvzilpKcyDxG/q6OeJP1Zx6KgHq8XgyvMW84H35BMfe+4ye77UxXwh9TGZ\n1Me4aKiLqVoKkTN2cxTLdju5/eNr+eL1a0hJiOX5XXX803/tZGd58xz1TQshxMxIKMwThmGwQaVz\n5+c38cH35tM3OMovninnX554h+OtfeEunhBigZBQmGdi7Vauu3Apd35+E2uL0qiq7+J7//0WT/yx\nioETI+EunhAiyoWs01opZQEeAEqBIeBWrXX1KbZ5Dvid1vrnSql44DEgA+gFbtJat4WqjPNZRko8\nX76hhAM17Tzxp8P86e3j7Kpo4YZLCnnvmmx5rkEIERKhbClcB8RprS8AvgHce4pt7gRSJ7z/G6BM\na30h8Cjw7RCWLyKUFKbxfz+ysEP9AAAVK0lEQVS3iY9cvJShkTH++w+V3PU/b3OkqSfcRRNCRKFQ\nhsIWYDuYk/IAGyeuVErdAHiB50+1j3/55SEsX8Sw2yy8/4J87vr8Zs5bkUFtYw93PrKHR7ZX0jsw\nPP0HCCHEDIXynsckoHvC+zGllE1rPaqUWg3cCNwAfOc0+/QCydMdxOVyYLPN/v7t9PTEWe97rqWn\nJ/KdwnT2H27jP7eV8cq+Rt7WbXz6mpW8b3M+VsvZdSlFUl2cC1Ifk0l9jIvmughlKPQAE2vOorUe\n9b/+DJALvATkA8NKqaMn7ZMIdE13EI9nYNYFjNT7jXNS4rjjMxt46e3jPP36Ef7jtwd47rVaPnWl\nomjRtDl6SpFaF6Ei9TGZ1Me4aKiLqUItlN1HO4BrAJRSm4GywAqt9de11pu01pcADwM/1lpvn7gP\ncDXwWgjLF9FsVgtXnp/H3bdt5j2rs6hr7eOux97ml8+W032WT+IKIRauULYUtgFXKKXewHzi/xal\n1O1Atdb6mdPs8x/AI0qp14FhzC4mMYXkhFhu/UAxF6/N4fEXq3jjYDN7q9q4bksBl25YhM0qdx0L\nIWZOhrmI8GbgRF6vj1f2NfDUq7X0nxglJ83JJ69Yzsolrmn3jba6OFtSH5NJfYyLhrqQYS4WCIvF\nYOv6Rdx122YuXptDU3s/P/rVO/zH0wfp7Jn9MNBCiIVDRkmNQomOGG66agUXlebw+B+reKuylf01\n7Vz7nnyuPC8Pu02+CwghTk3ODlGsIDuJb316A7dcs4JYu5XfvlLLdx7cRVltR7iLJoSYpyQUopzF\nMLiwJIe7b9vM5RsW0do1yE+e3M99vzlAa9f8nFBGCBE+0n20QDji7Nx4xXIu9Hcp7atu5+CRTq7Z\nnMfVm5eEu3hCiHlCQmGBWZyRwD/euI5dFS08+VI1z+w4yo6yZm65dhVOuwWr1cBqMbBaLdgs46+t\nFgOb1cBqsWA5yyenhRDzl4TCAmQYBpuLsygtTOPZN47y4lv1/Ovjb5/B/mC1mAFycnC8K0ym3WZC\n6FgsuBJjuXhtDjF2mXpSiHCQUFjA4mNtfHRrEVtKstlX24mne5Axr4+xMR9jXm/w9eiY/7XXx5j/\n9ehJ2wReDw+NBrcJrPOe4bMwf957nJuvWsGKGTxfIYSYWxIKgmy3k5IVWSF7IMfre3fQmMHifVfQ\n7K5o4Y976rnnV+9wUWkOH9taiCPOHpJyCSHeTUJBhJzFMLDYDOwzuNltaU4Sm4oz+e8/VPLq/kb2\n17TzqSsUG1T6OSipEEJuSRXzTkF2Et+5eSMfvmgp/YOj/GxbGT97qgxPrwz0J0SoSUtBzEs2q4UP\nvCefDSqdR56v5O2qNsqPefjY1kIuKs3BkOlIhQgJaSmIeS3b7eTrn1zPZ96nAB+PbNf86Ffv0NI5\n+3k0hBCnJ6Eg5j2LYXDJulzuvHUz65alUVnXxXce2s0fdh5jdMwb7uIJEVUkFETEcCXG8qUPr+Fv\nr1tNfKyN3/ylhjsf2cOx5sgexliI+URCQUQUwzDYuCKDO2/dxJaSbOpa+/i/j+zhyZerGRoZC3fx\nhIh4EgoiIiXE2/nsNSv52ifW4k6OZfuuOr774G4qjnaGu2hCRDQJBRHRivNT+cHnNnHVpjzaugf5\n0f/u46E/VNB/YiTcRRMiIkkoiIgXa7fysa1F3HHTRvIyEnj9QBP/9F+72FPZSqRPNyvEuSahIKJG\nflYS375pIzdcUsjg0CgPPH2Q++WhNyHOiDy8JqKKzWrhms1L2LA8nYefr+Sdw+1U1nn46CVFXLQ2\nB4s89CbElKSlIKJSZqqDf7hxHTdfvQIwePQFzT1PvENTR3+4iybEvCahIKKWxTC4qDSHO2/dxPrl\n6VTVd/Hdh97i2TeOykNvQpyGhIKIeoGH3r54/WqccTaeerWWHzy8hyNNPeEumhDzTsiuKSilLMAD\nQCkwBNyqta6esP6LwM2AD/iB1vpZpZQBHAcO+zd7U2v9zVCVUSwsG1QGK5e4ePLlGl7d38idj+7h\nio2Luf7CpcTGyExvQkBoLzRfB8RprS9QSm0G7gU+BKCUSgP+FlgLxAHlSqnngEJgr9b62hCWSyxg\njjg7N1+9gk3FmTyyvZIX36pnb1UbN121glUFqeEunhBhZ4TqPm6l1I+B3Vrr//W/b9Ba505Yb9Na\njyqllgFPaa3XKKU+Dvwj0A0MAl/VWuupjjM6Ouaz2eRbnjhzQyNj/OqFSra9UoPX6+PSjYv53AdX\nk+SMCXfRhAi1096GF8qWQhLmyT1gLBAEAP5A+BLwfeA+/zZNwN1a618rpbYAjwHnTXUQj2f2Qyin\npyeGbArKSLNQ6+L9m/JYvcTFw89X8tKeevaUN3PjFcu55sJC2tv7wl28eWOh/n2cSjTURXp64mnX\nhfJCcw8w8ciWQCAEaK3vB7KBi5RSW4E9wO/8614Hcv3XGYQImSVZiXz7pg18dGshg8Nj/Px3h/j7\nn77Koy9o/rSnnoqjnXT3DcnT0WJBCGVLYQdwLfCk/5pCWWCFUkoBdwMfAUYwL0R7ge8CHcA9SqlS\noE5rLf8SRchZLRau3rSE9cvTefzFKg4d7eRwfdekbZxxNrLTnOS4neSkOclJc5DjduJKjJWZ4ETU\nCGUobAOuUEq9gdl/dYtS6nagWmv9jFJqP/Am5t1Hz2utX1FKHQAeU0q9HxjFvDtJiHMm0+Xg9o+v\nJSnFwUHdQmN7P40d/TS2D9DY3k9tQw/Vx7sn7RMfayXH7XxXYKQmxckT1CLihOxC87nS1tY7618g\nGvoG54rUxWSnq4+RUS8tnQP+oPD/dAzQ0jnAmHfyn2Ks3Uq220FumhkU2f7/piVHXljI38e4aKiL\n9PTEsFxoFiLq2G0WFmUksCgjYdLy0TEvrZ7BCUFh/vd4Wx9HT5oZLsZmIcvtICfNaQaGv3WRnhKP\nxRJZYSGij4SCEHPAZrX4u42ck5aPeb20dZ14V1g0dQxQ19L3rs/ISnWY1yr8YbE0J4nUpLhz+auI\nBU5CQYgQslrME31WqoP1y9ODy71eH+3dg+a1ikldUWbrYqLcdCdrlrpZs9TNskXJ2KwyOo0IHQkF\nIcLAYjHIcDnIcDlYuywtuNzr89HZc4LG9gEa2vqoqPOg67rYvquO7bvqiIuxsnKJizWFbkqWuqUV\nIeachIIQ84jFMEhLjictOZ6SQjdXb17C8MgYlXVdlNV2UFbbwTuH23nncDsAuWmBVkQqyxanSCtC\nnDUJBSHmuRi7lZJCNyWFbgBaPAOU1XRQVttJZZ2H7bvr2L67jtgYK8VLXMGuJneytCLEmZNQECLC\nZLocZG50cPnGxQyPjKHru/whMbkVkZPmZM3SVP+1iBTsNmlFiOlJKAgRwWLs1mDLAKDVM0BZbSdl\ntR1UHvPwwu56XthdT6x9/FrEmqWppCXHh7nkYr6SUBAiimS4HFy2wcFlGxYxMhpoRZghsa+6nX3V\nZisi2+0IhsnyxdKKEOMkFISIUnabldUFblYXuPkrltHWNWherK7poKLOw4tv1fPiW/XE2C2szAu0\nItykp0grYiGTUBBigUhPiefS9Yu4dL3Ziqiq7w7e0bS/xvwByEr1tyIKU1GLU7DLfCULiox9FOFj\nmMwVqYvJFlp9tHcNUnak02xFHPMwNDIGQIzdwoo8F5tWZ7PY7SAn3Rlx4zbNtWj425Cxj4QQU0pL\niWfruly2rstlZNTL4eOB5yI6OVDTwQF/KyLRYWdFnouVS1ysWOIi0xUvw4ZHGQkFIcQkdpuF4vxU\nivNT+fil0NF9ggbPILsPNlFxzMNbla28VdkKgCsxNhgSK5e45NmIKCChIISYkjs5jhVF6ZTku/D5\nfLR6Bqk45qHimIfKOg9vHmrmzUPNAGSkxLNiyXhLIlnmu444EgpCiBkzDIPMVAeZqQ4uWZeL1+ej\nsa2fijoPlcc8VNZ18er+Rl7d3wiYD9CtzDMDQuWlkBBvD/NvIKYjoSCEmDWLYQTnl7hi42K8Xh/H\nWnqp9Lckqo538ee9/fx573EMIC8zMdiKWLYomfhYOQXNN/J/RAgxZywWg4LsJAqyk7h68xJGx7zU\nNvYEQ6KmsZtjLb1s312HxTAoyDFDYmWei8LcZGLscvtruEkoCCFCxma1sHxxCssXp/DBLQUMj4xR\n3dBtXo845uFIYy81DT08+8YxbFYLRblJ/ovWqeRnJ8qor2EgoSCEOGdi7NbgnU0Ag0OjVNV3BUOi\nsq6Lyroutr12hFi7lWWLk4N3NuVlJMp0peeAhIIQImziY22UFqVRWmRONNQ3OIKu8wTvbjpY28nB\n2k4AHLE2luYmUZSTTGFuMgXZSTji5BQ216RGhRDzRkK8nQ0qgw0qA4CuvqHg9Qhd1zUpJAzMu5sK\nc5Mo9AdFltux4J+4PlsSCkKIeSslIZbNq7LYvCoLgJ7+YWoau6lp6KGmoZsjzT00tPfz6v4mwN+a\nyEmiMDeZwtwklmYnS2viDIWstpRSFuABoBQYAm7VWldPWP9F4GbAB/xAa/2sUioeeAzIAHqBm7TW\nbaEqoxAisiQ5Y1i3LJ11y9IBGPN6Od7a7w8KMywOHunk4JHx1kR2mpPCYFAkky2tiSmFMkKvA+K0\n1hcopTYD9wIfAlBKpQF/C6wF4oBypdRzwN8AZVrr7ymlPgF8G/i7EJZRCBHBrBYLS7ISWZKVyKXr\nFwFma6K2sScYFLVNPTS29/PaAbM1ER9oTeQkUZSbzNKcJBxx8lBdQChDYQuwHUBrvVMptTGwQmvd\nrpQq1VqPKqXygS6ttU8ptQW4x7/Z88AdISyfECIKJTljWLssjbXLzIvXY14vDW391DR0U91ghsWh\nI50c8rcmwJx0qDA3maLcZApzkshOW7ijwYYyFJKA7gnvx5RSNq31KIA/EL4EfB+47xT79ALJ0x3E\n5XJgO4vx3tPTE2e9b7SRuphM6mOySK6PrMxkNqzOCb7v7htC13moPNqJPuahqs7D6weaeN3fmnDE\n2Vie52LFklRW5LtQeS4SHOPjOEVyXUwnlKHQA0ysOUsgEAK01vcrpX4BPK+U2nrSPolA13QH8XgG\nZl3AaBgXfa5IXUwm9TFZNNZHQbqTgnQnV5+3eLw10djjvzbRzb6qNvZVjV/SzHY7KMxJZu2KTHJd\ncWRE8LDhU4VaKENhB3At8KT/mkJZYIVSSgF3Ax8BRjAvRHv9+1wD7AauBl4LYfmEEAIwr03kZSaS\nl5nI1nW5APQODFPT2EOt/26n2qYemsqaeL3MbE2kJMSwIs8c6G9FniuiQ2KikM28NuHuoxLMmwBu\nwTzhV2utn1FKfRfzxO8Dntda/0Ap5QAeAbKBYeBGrXXzVMeRmdfmhtTFZFIfk0l9gNfr43hbH81d\nJ9hT0YKu89A7MBJcH0khMdXMazId5wL/Qw+QuphM6mMyqY9xgbrw+Xw0dgyg68wH63Sdh54JIZF8\nUkjMp1nqZDpOIYSYY4ZhkJvmJDfNyaXrF+Hz+Wjyh0SlPyR2lbewq7wFmN8hMZGEghBCzAHDMMhJ\nc5KT5mRrBIeEhIIQQoTAVCGh683RYOdjSEgoCCHEOXCqkGjuHAi2IuZLSEgoCCFEGBiGQbbbSbbb\nydZ1udOHhDMmGBAqL4WsVEdIQkJCQQgh5oHpQkLXdbG7opXdFa0AFOYk8U+f2TjNp545CQUhhJiH\nThcSuq6LyjoPjjg7Pp9vzlsLEgpCCBEBJobEJf6nrkNBZsUWQggRJKEghBAiSEJBCCFEkISCEEKI\nIAkFIYQQQRIKQgghgiQUhBBCBEkoCCGECIr4SXaEEELMHWkpCCGECJJQEEIIESShIIQQIkhCQQgh\nRJCEghBCiCAJBSGEEEESCkIIIYIW3CQ7SikL8ABQCgwBt2qtq8NbqvBRStmBh4B8IBa4U2v9TFgL\nFWZKqQzgbeAKrXVluMsTTkqpbwIfBGKAB7TWD4a5SGHj/7fyCOa/lTHg89H497EQWwrXAXFa6wuA\nbwD3hrk84fYpoENrfSFwNXB/mMsTVv5/+P8JDIa7LOGmlLoEeA/wXuBiYHFYCxR+1wA2rfV7gB8A\nPwxzeUJiIYbCFmA7gNZ6JzD3M19Hll8Dd0x4PxqugswT/wr8HGgMd0HmgfcBZcA24PfAs+EtTthV\nATZ/b0MSMBLm8oTEQgyFJKB7wvsxpdSC60YL0Fr3aa17lVKJwG+Ab4e7TOGilLoZaNNavxDusswT\naZhfmj4KfAF4XCk1t7PER5Y+zK6jSuC/gPvCWpoQWYih0AMkTnhv0Vov6G/HSqnFwMvA/2itnwh3\necLos8AVSqm/AGuBR5VSWeEtUlh1AC9orYe11ho4AaSHuUzh9FXM+liOeU3yEaVUXJjLNOcW4jfk\nHcC1wJNKqc2YzeMFSymVCbwIfElr/edwlyectNYXBV77g+ELWuvm8JUo7F4H/k4p9WMgG3BiBsVC\n5WG8y6gTsAPW8BUnNBZiKGzD/Db4BmAAt4S5POH2LcAF3KGUClxbuFprveAvtC50WutnlVIXAbsx\nexW+qLUeC3OxwuknwENKqdcw78b6lta6P8xlmnMydLYQQoighXhNQQghxGlIKAghhAiSUBBCCBEk\noSCEECJIQkEIIUSQhIIQYaKUulkp9XC4yyHERBIKQgghguQ5BSGmoZT6BvAxzKdXXwD+A/gd5hg4\nq4BjwKe01p1KqQ8Ad2J+4aoF/lpr3aKUuhxzRF6Lf/sbgQ8Dt2IOQpgH/Flr/flz+bsJcTJpKQgx\nBaXUVcAG4DxgHZALfBJYgzm/wCqgAviefx6G/wSu01qXYA6pcr9SKhZ4HLhJa70Gc2iVm/yHyMMM\nh5XA1UqpVefslxPiFBbiMBdCnInLgU2Yk+4AxGN+marSWv/Fv+wR4AnMMaR2a62P+pf/AvgmZoA0\naK33AWitvwnBUVlf1Vp3+t/XYI5MKkTYSCgIMTUr8G9a6x8DKKVSgEXA/5uwjQWzC+jklreB+W9s\nBAj20yqlkhkfqXfiCL0+/z5ChI10HwkxtZeATyulEvzzbjyNOceAUkqt9W9zC/A8sAvYrJTK9y+/\nDXNIcg1kKKWK/cu/jjk/gRDzjoSCEFPQWv8e+C3mCf8gsA94BXPo5O8rpQ4BGZhzW7dgBsE2//JL\nMIffPoE57emjSqkDQDHwz+f6dxFiJuTuIyHOkL8l8BetdX6YiyLEnJOWghBCiCBpKQghhAiSloIQ\nQoggCQUhhBBBEgpCCCGCJBSEEEIESSgIIYQI+v+NUhqe9nVhiQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize history for loss\n",
    "plt.plot(history.history['loss'])\n",
    "plt.plot(history.history['val_loss'])\n",
    "plt.title('model loss')\n",
    "plt.ylabel('loss')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train', 'test'], loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Case Number</th>\n",
       "      <th>Mixed_Comments</th>\n",
       "      <th>Categorie</th>\n",
       "      <th>problem_id</th>\n",
       "      <th>Categorie_A</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10087</td>\n",
       "      <td>status st line wi dear christian this is a de...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10087</td>\n",
       "      <td>rodopi ingentaconnect special character  migra...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10099</td>\n",
       "      <td>hi rachel olga performed an in house analysis...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10099</td>\n",
       "      <td>directory of open access journals doaj wrong r...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11750</td>\n",
       "      <td>status st line wi dear kasper thank you for r...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Case Number                                     Mixed_Comments Categorie  \\\n",
       "0       10087   status st line wi dear christian this is a de...     Other   \n",
       "1       10087  rodopi ingentaconnect special character  migra...     Other   \n",
       "2       10099   hi rachel olga performed an in house analysis...     Other   \n",
       "3       10099  directory of open access journals doaj wrong r...     Other   \n",
       "4       11750   status st line wi dear kasper thank you for r...     Other   \n",
       "\n",
       "   problem_id Categorie_A  \n",
       "0           0       Other  \n",
       "1           0       Other  \n",
       "2           0       Other  \n",
       "3           0       Other  \n",
       "4           0       Other  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Preparing a data frame to test the accuracy of the two-stage model\n",
    "\n",
    "two_step_dataframe=merged_cases_A\n",
    "two_step_dataframe['Categorie_A']=linking_or_not_linking['Categorie']\n",
    "two_step_dataframe.head()  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Case Number</th>\n",
       "      <th>Mixed_Comments</th>\n",
       "      <th>Categorie</th>\n",
       "      <th>problem_id</th>\n",
       "      <th>Categorie_A</th>\n",
       "      <th>step_one</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10087</td>\n",
       "      <td>status st line wi dear christian this is a de...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10087</td>\n",
       "      <td>rodopi ingentaconnect special character  migra...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10099</td>\n",
       "      <td>hi rachel olga performed an in house analysis...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10099</td>\n",
       "      <td>directory of open access journals doaj wrong r...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11750</td>\n",
       "      <td>status st line wi dear kasper thank you for r...</td>\n",
       "      <td>Other</td>\n",
       "      <td>0</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Case Number                                     Mixed_Comments Categorie  \\\n",
       "0       10087   status st line wi dear christian this is a de...     Other   \n",
       "1       10087  rodopi ingentaconnect special character  migra...     Other   \n",
       "2       10099   hi rachel olga performed an in house analysis...     Other   \n",
       "3       10099  directory of open access journals doaj wrong r...     Other   \n",
       "4       11750   status st line wi dear kasper thank you for r...     Other   \n",
       "\n",
       "   problem_id Categorie_A step_one  \n",
       "0           0       Other    Other  \n",
       "1           0       Other    Other  \n",
       "2           0       Other    Other  \n",
       "3           0       Other    Other  \n",
       "4           0       Other    Other  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "two_step_dataframe['step_one']=pipeline_lr_linking.predict(two_step_dataframe['Mixed_Comments'])\n",
    "two_step_dataframe.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classifying all the topics except for \"Data Linking\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 296,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.00      0.00      0.00       153\n",
      "  Data availability       0.47      0.95      0.63       524\n",
      "       Data display       0.44      0.75      0.56       509\n",
      "             Facets       0.00      0.00      0.00        94\n",
      "           Grouping       1.00      0.01      0.03       155\n",
      "Normalization Rules       0.00      0.00      0.00       119\n",
      "       Registration       1.00      0.01      0.02       199\n",
      "            Updates       1.00      0.07      0.13       194\n",
      "\n",
      "        avg / total       0.52      0.46      0.33      1947\n",
      "\n",
      "\n",
      "\n",
      "[[  0  54  99   0   0   0   0   0]\n",
      " [  0 499  25   0   0   0   0   0]\n",
      " [  0 126 383   0   0   0   0   0]\n",
      " [  0  33  61   0   0   0   0   0]\n",
      " [  0  63  90   0   2   0   0   0]\n",
      " [  0  36  83   0   0   0   0   0]\n",
      " [  0 168  29   0   0   0   2   0]\n",
      " [  0  89  91   0   0   0   0  14]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "#Naive Bayes\n",
    "pipeline_MultinomialNB_others = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean,stop_words='english', ngram_range=(1, 2))),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', MultinomialNB())\n",
    "])\n",
    "pipeline_MultinomialNB_others.fit(X_train_1, y_train_1)\n",
    "predictions_MultinomialNB_others = pipeline_MultinomialNB_others.predict(X_test_1)   \n",
    "\n",
    "print(classification_report(y_test_1, predictions_MultinomialNB_others))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_1, predictions_MultinomialNB_others))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 297,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.74      0.19      0.30       153\n",
      "  Data availability       0.71      0.86      0.77       524\n",
      "       Data display       0.49      0.81      0.61       509\n",
      "             Facets       0.90      0.48      0.63        94\n",
      "           Grouping       0.90      0.52      0.66       155\n",
      "Normalization Rules       0.00      0.00      0.00       119\n",
      "       Registration       0.78      0.68      0.73       199\n",
      "            Updates       0.83      0.49      0.62       194\n",
      "\n",
      "        avg / total       0.65      0.64      0.61      1947\n",
      "\n",
      "\n",
      "\n",
      "[[ 29  20  95   1   1   0   2   5]\n",
      " [  0 449  65   0   0   0   8   2]\n",
      " [  6  60 414   2   5   0  14   8]\n",
      " [  0   6  39  45   1   0   3   0]\n",
      " [  1  18  53   1  81   0   0   1]\n",
      " [  3  16  94   1   0   0   3   2]\n",
      " [  0  32  30   0   1   0 135   1]\n",
      " [  0  34  55   0   1   0   8  96]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "#Random Forest\n",
    "\n",
    "pipeline_rf_others = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean,stop_words='english', ngram_range=(1, 2))),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', RandomForestClassifier(n_estimators=300))\n",
    "])\n",
    "pipeline_rf_others.fit(X_train_1, y_train_1)\n",
    "predictions_rf_others = pipeline_rf_others.predict(X_test_1)   \n",
    "\n",
    "print(classification_report(y_test_1, predictions_rf_others))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_1, predictions_rf_others))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.71      0.33      0.45       153\n",
      "  Data availability       0.79      0.83      0.81       524\n",
      "       Data display       0.55      0.80      0.65       509\n",
      "             Facets       0.81      0.72      0.76        94\n",
      "           Grouping       0.86      0.67      0.75       155\n",
      "Normalization Rules       0.57      0.03      0.06       119\n",
      "       Registration       0.78      0.81      0.80       199\n",
      "            Updates       0.78      0.64      0.70       194\n",
      "\n",
      "        avg / total       0.71      0.70      0.67      1947\n",
      "\n",
      "\n",
      "\n",
      "[[ 51  17  76   1   2   0   1   5]\n",
      " [  1 434  72   0   2   0  10   5]\n",
      " [ 12  44 407   7   8   2  16  13]\n",
      " [  0   1  18  68   2   1   4   0]\n",
      " [  1  12  31   3 104   0   2   2]\n",
      " [  5  16  78   5   1   4   4   6]\n",
      " [  0  10  20   0   2   0 162   5]\n",
      " [  2  18  42   0   0   0   8 124]]\n"
     ]
    }
   ],
   "source": [
    "#Logistic Regression\n",
    "\n",
    "pipeline_lr_others = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer='word', stop_words='english')),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', LogisticRegression())\n",
    "])\n",
    "pipeline_lr_others.fit(X_train_1, y_train_1)\n",
    "predictions_lr_others = pipeline_lr_others.predict(X_test_1)\n",
    "print(classification_report(y_test_1, predictions_lr_others))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_1, predictions_lr_others))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 299,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.73      0.31      0.43       153\n",
      "  Data availability       0.77      0.86      0.81       524\n",
      "       Data display       0.62      0.70      0.66       509\n",
      "             Facets       0.72      0.83      0.77        94\n",
      "           Grouping       0.75      0.77      0.76       155\n",
      "Normalization Rules       0.50      0.07      0.12       119\n",
      "       Registration       0.70      0.86      0.77       199\n",
      "            Updates       0.73      0.71      0.72       194\n",
      "\n",
      "        avg / total       0.69      0.70      0.68      1947\n",
      "\n",
      "\n",
      "\n",
      "[[ 47  22  59   2   9   1   5   8]\n",
      " [  1 449  43   3   3   1  18   6]\n",
      " [ 11  56 358  13  18   4  28  21]\n",
      " [  0   1   8  78   3   1   3   0]\n",
      " [  1  11  16   3 120   1   1   2]\n",
      " [  4  17  65   9   2   8   7   7]\n",
      " [  0  12   5   0   3   0 172   7]\n",
      " [  0  17  26   0   1   0  13 137]]\n"
     ]
    }
   ],
   "source": [
    "#SGD\n",
    "\n",
    "pipeline_SGD_others = Pipeline([('vect', CountVectorizer()),\n",
    "                    ('tfidf', TfidfTransformer()),\n",
    "                    ('clf', SGDClassifier(loss='hinge', penalty='l2',\n",
    "                    alpha=1e-3, random_state=42,max_iter=5, tol=None)),\n",
    "                    ])\n",
    "pipeline_SGD_others.fit(X_train_1, y_train_1)\n",
    "predictions_SGD_others = pipeline_SGD_others.predict(X_test_1)\n",
    "print(classification_report(y_test_1, predictions_SGD_others))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_1, predictions_SGD_others))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 300,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.00      0.00      0.00       153\n",
      "  Data availability       0.00      0.00      0.00       524\n",
      "       Data display       0.26      1.00      0.41       509\n",
      "             Facets       0.00      0.00      0.00        94\n",
      "           Grouping       0.00      0.00      0.00       155\n",
      "Normalization Rules       0.00      0.00      0.00       119\n",
      "       Registration       0.00      0.00      0.00       199\n",
      "            Updates       0.00      0.00      0.00       194\n",
      "\n",
      "        avg / total       0.07      0.26      0.11      1947\n",
      "\n",
      "\n",
      "\n",
      "[[  0   0 153   0   0   0   0   0]\n",
      " [  0   0 524   0   0   0   0   0]\n",
      " [  0   0 509   0   0   0   0   0]\n",
      " [  0   0  94   0   0   0   0   0]\n",
      " [  0   0 155   0   0   0   0   0]\n",
      " [  0   0 119   0   0   0   0   0]\n",
      " [  0   0 199   0   0   0   0   0]\n",
      " [  0   0 194   0   0   0   0   0]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\AlonPc\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "#SVC\n",
    "\n",
    "pipeline_svm_others = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean)),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', SVC())\n",
    "])\n",
    "pipeline_svm_others.fit(X_train_1, y_train_1)\n",
    "predictions_svm_others = pipeline_svm_others.predict(X_test_1)\n",
    "print(classification_report(y_test_1, predictions_svm_others))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_1, predictions_svm_others))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 301,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     precision    recall  f1-score   support\n",
      "\n",
      "               Data       0.20      0.44      0.28       153\n",
      "  Data availability       0.37      0.68      0.48       524\n",
      "       Data display       0.44      0.32      0.37       509\n",
      "             Facets       0.79      0.12      0.20        94\n",
      "           Grouping       0.83      0.13      0.22       155\n",
      "Normalization Rules       0.10      0.15      0.12       119\n",
      "       Registration       0.77      0.15      0.25       199\n",
      "            Updates       0.69      0.11      0.19       194\n",
      "\n",
      "        avg / total       0.49      0.35      0.33      1947\n",
      "\n",
      "\n",
      "\n",
      "[[ 68  49  25   0   0  11   0   0]\n",
      " [ 59 357  62   1   0  41   1   3]\n",
      " [ 82 225 162   1   2  33   3   1]\n",
      " [ 13  42  17  11   0  10   0   1]\n",
      " [ 23  70  24   0  20  16   0   2]\n",
      " [ 17  51  31   0   1  18   1   0]\n",
      " [ 36  82  29   1   0  18  30   3]\n",
      " [ 40  84  18   0   1  25   4  22]]\n"
     ]
    }
   ],
   "source": [
    "#KNN\n",
    "\n",
    "pipeline_knn_others = Pipeline([\n",
    "    ('bow', CountVectorizer(analyzer=text_clean)),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('classifier', KNeighborsClassifier())\n",
    "])\n",
    "pipeline_knn_others.fit(X_train_1, y_train_1)\n",
    "predictions_knn_others = pipeline_knn_others.predict(X_test_1)\n",
    "print(classification_report(y_test_1, predictions_knn_others))\n",
    "print(\"\\n\")\n",
    "print(confusion_matrix(y_test_1, predictions_knn_others))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8821417659639474\n"
     ]
    }
   ],
   "source": [
    "#linking_not_linking prediction score\n",
    "\n",
    "score=0\n",
    "two_step_dataframe = two_step_dataframe.reset_index(drop=True)\n",
    "\n",
    "two_step_dataframe.describe()\n",
    "for i in range(len(two_step_dataframe)):\n",
    "    if two_step_dataframe['Categorie_A'][i]==two_step_dataframe['step_one'][i]:\n",
    "        score+=1\n",
    "print(score/len(two_step_dataframe))    \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Mixed_Comments</th>\n",
       "      <th>Categorie</th>\n",
       "      <th>Categorie_A</th>\n",
       "      <th>step_one</th>\n",
       "      <th>step_two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>tnproquest is not giving the right information...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>journal title missing in many records delivere...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>hi paul thank you for reporting this issue we...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>dear sarah  thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>sfx doesnt reach correct article openurl targ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>makind our database and ejournal collections d...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>dear shahar thank you for your quick reply i ...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>search by title hi support team we encounter a...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>record full of authors the following record di...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>dear amanda i would like to update you that w...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>hello pco team\u000b",
       "\u000b",
       " please refer to the followin...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>dear karen thank you for your patience the ve...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>hey meg i would like to update you that we ha...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>dear amelia thank you for reporting this to p...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>dear galen i was able to reproduce the scenar...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>fw search link failing hello this looks like a...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>hi nitzan i can confirm that the article titl...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>inhouse record oxfordsgmlformodb with doi fml...</td>\n",
       "      <td>Data</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>dear louisa thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>hi pc team availability indicator for this re...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>hi marie i dont know if these particular reco...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>hello yoni it seems tnproquest is no longer i...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>linking problem with pci record why we all hav...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>holdings file not harvested okstats holdings f...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>proquests dissertations and thesis collection ...</td>\n",
       "      <td>Facets</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>hi daria thank you very much for your help ca...</td>\n",
       "      <td>Grouping</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>hi pc team this records received from crossre...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>capitalism socialism democracy ebook not grant...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>visual anthropology incorrect doi data in arti...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>article link leading to wrong resource hi exli...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3898</th>\n",
       "      <td>journals from emerald collections not displayi...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3899</th>\n",
       "      <td>hi kb this is a linking issue that was recrea...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3900</th>\n",
       "      <td>primo central record matching on title to unre...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3901</th>\n",
       "      <td>ej portfolios with year embargo have articles ...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3902</th>\n",
       "      <td>dear amelia thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3903</th>\n",
       "      <td>bad link to full text henderson g the franks c...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3904</th>\n",
       "      <td>hello jill it looks like the article is a res...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3905</th>\n",
       "      <td>dear jan i am checking this issue and will up...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3906</th>\n",
       "      <td>hello tomer alborg is reporting an incorrect ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3907</th>\n",
       "      <td>dear daniel thank you for contacting exlibris...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3908</th>\n",
       "      <td>sfx the annals sage we found some of the artic...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3909</th>\n",
       "      <td>incorrect link in pci metadata from nora the l...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3910</th>\n",
       "      <td>dear pauline thank you for contacting exlibri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3911</th>\n",
       "      <td>interpersonal communication and psychology f...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3912</th>\n",
       "      <td>dear roland i would like to update you that w...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3913</th>\n",
       "      <td>internal records cqvip it seems like althoug...</td>\n",
       "      <td>Grouping</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3914</th>\n",
       "      <td>pci records linking to article and revised ver...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3915</th>\n",
       "      <td>primo not enough metadata for fulltext links h...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3916</th>\n",
       "      <td>dear carolyn thank you for contacting ex libr...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3917</th>\n",
       "      <td>hi emmanuelle the title klimaeffektar på sval...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3918</th>\n",
       "      <td>records for journal of canadian studies in sfx...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3919</th>\n",
       "      <td>incorrect doi hi the record with recordid tnwo...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3920</th>\n",
       "      <td>dear lihong i would like to update you that t...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3921</th>\n",
       "      <td>dear gilbert since this customer is still und...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3922</th>\n",
       "      <td>dear bettina i would like to update you that ...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3923</th>\n",
       "      <td>an article in primo central articles more is s...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3924</th>\n",
       "      <td>hi rachel please let the vendor know that the...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3925</th>\n",
       "      <td>thanks for the update wendy dear wendy if the...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3926</th>\n",
       "      <td>dear staff thank you for contacting exlibris ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3927</th>\n",
       "      <td>dear paul thank you for your patience i am an...</td>\n",
       "      <td>Registration</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3928 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         Mixed_Comments            Categorie  \\\n",
       "0     tnproquest is not giving the right information...         Data linking   \n",
       "1     journal title missing in many records delivere...         Data linking   \n",
       "2      hi paul thank you for reporting this issue we...         Data linking   \n",
       "3      dear sarah  thank you for contacting ex libri...         Data linking   \n",
       "4      sfx doesnt reach correct article openurl targ...         Data linking   \n",
       "5     makind our database and ejournal collections d...    Data availability   \n",
       "6      dear shahar thank you for your quick reply i ...    Data availability   \n",
       "7     search by title hi support team we encounter a...  Normalization Rules   \n",
       "8     record full of authors the following record di...         Data display   \n",
       "9      dear amanda i would like to update you that w...         Data linking   \n",
       "10     hello pco team\n",
       "\n",
       " please refer to the followin...    Data availability   \n",
       "11     dear karen thank you for your patience the ve...  Normalization Rules   \n",
       "12     hey meg i would like to update you that we ha...         Data display   \n",
       "13     dear amelia thank you for reporting this to p...         Data linking   \n",
       "14     dear galen i was able to reproduce the scenar...         Data linking   \n",
       "15    fw search link failing hello this looks like a...         Data display   \n",
       "16     hi nitzan i can confirm that the article titl...         Data linking   \n",
       "17     inhouse record oxfordsgmlformodb with doi fml...                 Data   \n",
       "18     dear louisa thank you for contacting ex libri...         Data linking   \n",
       "19     hi pc team availability indicator for this re...    Data availability   \n",
       "20     hi marie i dont know if these particular reco...    Data availability   \n",
       "21     hello yoni it seems tnproquest is no longer i...         Data linking   \n",
       "22    linking problem with pci record why we all hav...         Data linking   \n",
       "23    holdings file not harvested okstats holdings f...    Data availability   \n",
       "24    proquests dissertations and thesis collection ...               Facets   \n",
       "25     hi daria thank you very much for your help ca...             Grouping   \n",
       "26     hi pc team this records received from crossre...         Data linking   \n",
       "27    capitalism socialism democracy ebook not grant...    Data availability   \n",
       "28    visual anthropology incorrect doi data in arti...         Data linking   \n",
       "29    article link leading to wrong resource hi exli...         Data linking   \n",
       "...                                                 ...                  ...   \n",
       "3898  journals from emerald collections not displayi...              Updates   \n",
       "3899   hi kb this is a linking issue that was recrea...         Data linking   \n",
       "3900  primo central record matching on title to unre...    Data availability   \n",
       "3901  ej portfolios with year embargo have articles ...         Data display   \n",
       "3902   dear amelia thank you for contacting ex libri...         Data linking   \n",
       "3903  bad link to full text henderson g the franks c...         Data linking   \n",
       "3904   hello jill it looks like the article is a res...         Data display   \n",
       "3905   dear jan i am checking this issue and will up...         Data linking   \n",
       "3906   hello tomer alborg is reporting an incorrect ...         Data linking   \n",
       "3907   dear daniel thank you for contacting exlibris...    Data availability   \n",
       "3908  sfx the annals sage we found some of the artic...         Data linking   \n",
       "3909  incorrect link in pci metadata from nora the l...         Data linking   \n",
       "3910   dear pauline thank you for contacting exlibri...         Data linking   \n",
       "3911    interpersonal communication and psychology f...         Data linking   \n",
       "3912   dear roland i would like to update you that w...         Data linking   \n",
       "3913    internal records cqvip it seems like althoug...             Grouping   \n",
       "3914  pci records linking to article and revised ver...         Data linking   \n",
       "3915  primo not enough metadata for fulltext links h...         Data linking   \n",
       "3916   dear carolyn thank you for contacting ex libr...         Data linking   \n",
       "3917   hi emmanuelle the title klimaeffektar på sval...              Updates   \n",
       "3918  records for journal of canadian studies in sfx...         Data display   \n",
       "3919  incorrect doi hi the record with recordid tnwo...         Data linking   \n",
       "3920   dear lihong i would like to update you that t...    Data availability   \n",
       "3921   dear gilbert since this customer is still und...         Data display   \n",
       "3922   dear bettina i would like to update you that ...  Normalization Rules   \n",
       "3923  an article in primo central articles more is s...         Data linking   \n",
       "3924   hi rachel please let the vendor know that the...              Updates   \n",
       "3925   thanks for the update wendy dear wendy if the...         Data linking   \n",
       "3926   dear staff thank you for contacting exlibris ...         Data linking   \n",
       "3927   dear paul thank you for your patience i am an...         Registration   \n",
       "\n",
       "     Categorie_A step_one step_two  \n",
       "0            NaN      NaN      NaN  \n",
       "1            NaN      NaN      NaN  \n",
       "2            NaN      NaN      NaN  \n",
       "3            NaN      NaN      NaN  \n",
       "4            NaN      NaN      NaN  \n",
       "5            NaN      NaN      NaN  \n",
       "6            NaN      NaN      NaN  \n",
       "7            NaN      NaN      NaN  \n",
       "8            NaN      NaN      NaN  \n",
       "9            NaN      NaN      NaN  \n",
       "10           NaN      NaN      NaN  \n",
       "11           NaN      NaN      NaN  \n",
       "12           NaN      NaN      NaN  \n",
       "13           NaN      NaN      NaN  \n",
       "14           NaN      NaN      NaN  \n",
       "15           NaN      NaN      NaN  \n",
       "16           NaN      NaN      NaN  \n",
       "17           NaN      NaN      NaN  \n",
       "18           NaN      NaN      NaN  \n",
       "19           NaN      NaN      NaN  \n",
       "20           NaN      NaN      NaN  \n",
       "21           NaN      NaN      NaN  \n",
       "22           NaN      NaN      NaN  \n",
       "23           NaN      NaN      NaN  \n",
       "24           NaN      NaN      NaN  \n",
       "25           NaN      NaN      NaN  \n",
       "26           NaN      NaN      NaN  \n",
       "27           NaN      NaN      NaN  \n",
       "28           NaN      NaN      NaN  \n",
       "29           NaN      NaN      NaN  \n",
       "...          ...      ...      ...  \n",
       "3898         NaN      NaN      NaN  \n",
       "3899         NaN      NaN      NaN  \n",
       "3900         NaN      NaN      NaN  \n",
       "3901         NaN      NaN      NaN  \n",
       "3902         NaN      NaN      NaN  \n",
       "3903         NaN      NaN      NaN  \n",
       "3904         NaN      NaN      NaN  \n",
       "3905         NaN      NaN      NaN  \n",
       "3906         NaN      NaN      NaN  \n",
       "3907         NaN      NaN      NaN  \n",
       "3908         NaN      NaN      NaN  \n",
       "3909         NaN      NaN      NaN  \n",
       "3910         NaN      NaN      NaN  \n",
       "3911         NaN      NaN      NaN  \n",
       "3912         NaN      NaN      NaN  \n",
       "3913         NaN      NaN      NaN  \n",
       "3914         NaN      NaN      NaN  \n",
       "3915         NaN      NaN      NaN  \n",
       "3916         NaN      NaN      NaN  \n",
       "3917         NaN      NaN      NaN  \n",
       "3918         NaN      NaN      NaN  \n",
       "3919         NaN      NaN      NaN  \n",
       "3920         NaN      NaN      NaN  \n",
       "3921         NaN      NaN      NaN  \n",
       "3922         NaN      NaN      NaN  \n",
       "3923         NaN      NaN      NaN  \n",
       "3924         NaN      NaN      NaN  \n",
       "3925         NaN      NaN      NaN  \n",
       "3926         NaN      NaN      NaN  \n",
       "3927         NaN      NaN      NaN  \n",
       "\n",
       "[3928 rows x 5 columns]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "\n",
    "two_step_dataframe=pd.DataFrame(\n",
    "                      columns=['Mixed_Comments','Categorie','Categorie_A','step_one','step_two'])\n",
    "two_step_dataframe['Mixed_Comments']=X_test\n",
    "two_step_dataframe['Categorie']=y_test\n",
    "two_step_dataframe = two_step_dataframe.reset_index(drop=True)\n",
    "two_step_dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Mixed_Comments</th>\n",
       "      <th>Categorie</th>\n",
       "      <th>Categorie_A</th>\n",
       "      <th>step_one</th>\n",
       "      <th>step_two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>tnproquest is not giving the right information...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>journal title missing in many records delivere...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>hi paul thank you for reporting this issue we...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>dear sarah  thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>sfx doesnt reach correct article openurl targ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>makind our database and ejournal collections d...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>dear shahar thank you for your quick reply i ...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>search by title hi support team we encounter a...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>record full of authors the following record di...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>dear amanda i would like to update you that w...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>hello pco team\u000b",
       "\u000b",
       " please refer to the followin...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>dear karen thank you for your patience the ve...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>hey meg i would like to update you that we ha...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>dear amelia thank you for reporting this to p...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>dear galen i was able to reproduce the scenar...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>fw search link failing hello this looks like a...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>hi nitzan i can confirm that the article titl...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>inhouse record oxfordsgmlformodb with doi fml...</td>\n",
       "      <td>Data</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>dear louisa thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>hi pc team availability indicator for this re...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>hi marie i dont know if these particular reco...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>hello yoni it seems tnproquest is no longer i...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>linking problem with pci record why we all hav...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>holdings file not harvested okstats holdings f...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>proquests dissertations and thesis collection ...</td>\n",
       "      <td>Facets</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>hi daria thank you very much for your help ca...</td>\n",
       "      <td>Grouping</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>hi pc team this records received from crossre...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>capitalism socialism democracy ebook not grant...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>visual anthropology incorrect doi data in arti...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>article link leading to wrong resource hi exli...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3898</th>\n",
       "      <td>journals from emerald collections not displayi...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3899</th>\n",
       "      <td>hi kb this is a linking issue that was recrea...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3900</th>\n",
       "      <td>primo central record matching on title to unre...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3901</th>\n",
       "      <td>ej portfolios with year embargo have articles ...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3902</th>\n",
       "      <td>dear amelia thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3903</th>\n",
       "      <td>bad link to full text henderson g the franks c...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3904</th>\n",
       "      <td>hello jill it looks like the article is a res...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3905</th>\n",
       "      <td>dear jan i am checking this issue and will up...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3906</th>\n",
       "      <td>hello tomer alborg is reporting an incorrect ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3907</th>\n",
       "      <td>dear daniel thank you for contacting exlibris...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3908</th>\n",
       "      <td>sfx the annals sage we found some of the artic...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3909</th>\n",
       "      <td>incorrect link in pci metadata from nora the l...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3910</th>\n",
       "      <td>dear pauline thank you for contacting exlibri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3911</th>\n",
       "      <td>interpersonal communication and psychology f...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3912</th>\n",
       "      <td>dear roland i would like to update you that w...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3913</th>\n",
       "      <td>internal records cqvip it seems like althoug...</td>\n",
       "      <td>Grouping</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3914</th>\n",
       "      <td>pci records linking to article and revised ver...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3915</th>\n",
       "      <td>primo not enough metadata for fulltext links h...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3916</th>\n",
       "      <td>dear carolyn thank you for contacting ex libr...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3917</th>\n",
       "      <td>hi emmanuelle the title klimaeffektar på sval...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3918</th>\n",
       "      <td>records for journal of canadian studies in sfx...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3919</th>\n",
       "      <td>incorrect doi hi the record with recordid tnwo...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3920</th>\n",
       "      <td>dear lihong i would like to update you that t...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3921</th>\n",
       "      <td>dear gilbert since this customer is still und...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3922</th>\n",
       "      <td>dear bettina i would like to update you that ...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3923</th>\n",
       "      <td>an article in primo central articles more is s...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3924</th>\n",
       "      <td>hi rachel please let the vendor know that the...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3925</th>\n",
       "      <td>thanks for the update wendy dear wendy if the...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3926</th>\n",
       "      <td>dear staff thank you for contacting exlibris ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3927</th>\n",
       "      <td>dear paul thank you for your patience i am an...</td>\n",
       "      <td>Registration</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3928 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         Mixed_Comments            Categorie  \\\n",
       "0     tnproquest is not giving the right information...         Data linking   \n",
       "1     journal title missing in many records delivere...         Data linking   \n",
       "2      hi paul thank you for reporting this issue we...         Data linking   \n",
       "3      dear sarah  thank you for contacting ex libri...         Data linking   \n",
       "4      sfx doesnt reach correct article openurl targ...         Data linking   \n",
       "5     makind our database and ejournal collections d...    Data availability   \n",
       "6      dear shahar thank you for your quick reply i ...    Data availability   \n",
       "7     search by title hi support team we encounter a...  Normalization Rules   \n",
       "8     record full of authors the following record di...         Data display   \n",
       "9      dear amanda i would like to update you that w...         Data linking   \n",
       "10     hello pco team\n",
       "\n",
       " please refer to the followin...    Data availability   \n",
       "11     dear karen thank you for your patience the ve...  Normalization Rules   \n",
       "12     hey meg i would like to update you that we ha...         Data display   \n",
       "13     dear amelia thank you for reporting this to p...         Data linking   \n",
       "14     dear galen i was able to reproduce the scenar...         Data linking   \n",
       "15    fw search link failing hello this looks like a...         Data display   \n",
       "16     hi nitzan i can confirm that the article titl...         Data linking   \n",
       "17     inhouse record oxfordsgmlformodb with doi fml...                 Data   \n",
       "18     dear louisa thank you for contacting ex libri...         Data linking   \n",
       "19     hi pc team availability indicator for this re...    Data availability   \n",
       "20     hi marie i dont know if these particular reco...    Data availability   \n",
       "21     hello yoni it seems tnproquest is no longer i...         Data linking   \n",
       "22    linking problem with pci record why we all hav...         Data linking   \n",
       "23    holdings file not harvested okstats holdings f...    Data availability   \n",
       "24    proquests dissertations and thesis collection ...               Facets   \n",
       "25     hi daria thank you very much for your help ca...             Grouping   \n",
       "26     hi pc team this records received from crossre...         Data linking   \n",
       "27    capitalism socialism democracy ebook not grant...    Data availability   \n",
       "28    visual anthropology incorrect doi data in arti...         Data linking   \n",
       "29    article link leading to wrong resource hi exli...         Data linking   \n",
       "...                                                 ...                  ...   \n",
       "3898  journals from emerald collections not displayi...              Updates   \n",
       "3899   hi kb this is a linking issue that was recrea...         Data linking   \n",
       "3900  primo central record matching on title to unre...    Data availability   \n",
       "3901  ej portfolios with year embargo have articles ...         Data display   \n",
       "3902   dear amelia thank you for contacting ex libri...         Data linking   \n",
       "3903  bad link to full text henderson g the franks c...         Data linking   \n",
       "3904   hello jill it looks like the article is a res...         Data display   \n",
       "3905   dear jan i am checking this issue and will up...         Data linking   \n",
       "3906   hello tomer alborg is reporting an incorrect ...         Data linking   \n",
       "3907   dear daniel thank you for contacting exlibris...    Data availability   \n",
       "3908  sfx the annals sage we found some of the artic...         Data linking   \n",
       "3909  incorrect link in pci metadata from nora the l...         Data linking   \n",
       "3910   dear pauline thank you for contacting exlibri...         Data linking   \n",
       "3911    interpersonal communication and psychology f...         Data linking   \n",
       "3912   dear roland i would like to update you that w...         Data linking   \n",
       "3913    internal records cqvip it seems like althoug...             Grouping   \n",
       "3914  pci records linking to article and revised ver...         Data linking   \n",
       "3915  primo not enough metadata for fulltext links h...         Data linking   \n",
       "3916   dear carolyn thank you for contacting ex libr...         Data linking   \n",
       "3917   hi emmanuelle the title klimaeffektar på sval...              Updates   \n",
       "3918  records for journal of canadian studies in sfx...         Data display   \n",
       "3919  incorrect doi hi the record with recordid tnwo...         Data linking   \n",
       "3920   dear lihong i would like to update you that t...    Data availability   \n",
       "3921   dear gilbert since this customer is still und...         Data display   \n",
       "3922   dear bettina i would like to update you that ...  Normalization Rules   \n",
       "3923  an article in primo central articles more is s...         Data linking   \n",
       "3924   hi rachel please let the vendor know that the...              Updates   \n",
       "3925   thanks for the update wendy dear wendy if the...         Data linking   \n",
       "3926   dear staff thank you for contacting exlibris ...         Data linking   \n",
       "3927   dear paul thank you for your patience i am an...         Registration   \n",
       "\n",
       "       Categorie_A step_one step_two  \n",
       "0     Data linking      NaN      NaN  \n",
       "1     Data linking      NaN      NaN  \n",
       "2     Data linking      NaN      NaN  \n",
       "3     Data linking      NaN      NaN  \n",
       "4     Data linking      NaN      NaN  \n",
       "5            Other      NaN      NaN  \n",
       "6            Other      NaN      NaN  \n",
       "7            Other      NaN      NaN  \n",
       "8            Other      NaN      NaN  \n",
       "9     Data linking      NaN      NaN  \n",
       "10           Other      NaN      NaN  \n",
       "11           Other      NaN      NaN  \n",
       "12           Other      NaN      NaN  \n",
       "13    Data linking      NaN      NaN  \n",
       "14    Data linking      NaN      NaN  \n",
       "15           Other      NaN      NaN  \n",
       "16    Data linking      NaN      NaN  \n",
       "17           Other      NaN      NaN  \n",
       "18    Data linking      NaN      NaN  \n",
       "19           Other      NaN      NaN  \n",
       "20           Other      NaN      NaN  \n",
       "21    Data linking      NaN      NaN  \n",
       "22    Data linking      NaN      NaN  \n",
       "23           Other      NaN      NaN  \n",
       "24           Other      NaN      NaN  \n",
       "25           Other      NaN      NaN  \n",
       "26    Data linking      NaN      NaN  \n",
       "27           Other      NaN      NaN  \n",
       "28    Data linking      NaN      NaN  \n",
       "29    Data linking      NaN      NaN  \n",
       "...            ...      ...      ...  \n",
       "3898         Other      NaN      NaN  \n",
       "3899  Data linking      NaN      NaN  \n",
       "3900         Other      NaN      NaN  \n",
       "3901         Other      NaN      NaN  \n",
       "3902  Data linking      NaN      NaN  \n",
       "3903  Data linking      NaN      NaN  \n",
       "3904         Other      NaN      NaN  \n",
       "3905  Data linking      NaN      NaN  \n",
       "3906  Data linking      NaN      NaN  \n",
       "3907         Other      NaN      NaN  \n",
       "3908  Data linking      NaN      NaN  \n",
       "3909  Data linking      NaN      NaN  \n",
       "3910  Data linking      NaN      NaN  \n",
       "3911  Data linking      NaN      NaN  \n",
       "3912  Data linking      NaN      NaN  \n",
       "3913         Other      NaN      NaN  \n",
       "3914  Data linking      NaN      NaN  \n",
       "3915  Data linking      NaN      NaN  \n",
       "3916  Data linking      NaN      NaN  \n",
       "3917         Other      NaN      NaN  \n",
       "3918         Other      NaN      NaN  \n",
       "3919  Data linking      NaN      NaN  \n",
       "3920         Other      NaN      NaN  \n",
       "3921         Other      NaN      NaN  \n",
       "3922         Other      NaN      NaN  \n",
       "3923  Data linking      NaN      NaN  \n",
       "3924         Other      NaN      NaN  \n",
       "3925  Data linking      NaN      NaN  \n",
       "3926  Data linking      NaN      NaN  \n",
       "3927         Other      NaN      NaN  \n",
       "\n",
       "[3928 rows x 5 columns]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "score=0\n",
    "for i in range(len(two_step_dataframe)):\n",
    "    if two_step_dataframe['Categorie'][i]=='Data linking':\n",
    "        two_step_dataframe['Categorie_A'][i]='Data linking'\n",
    "    else:\n",
    "        two_step_dataframe['Categorie_A'][i]='Other'\n",
    "# print(score/len(two_step_dataframe))\n",
    "two_step_dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 379,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8556517311608961\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Mixed_Comments</th>\n",
       "      <th>Categorie</th>\n",
       "      <th>Categorie_A</th>\n",
       "      <th>step_one</th>\n",
       "      <th>step_two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>tnproquest is not giving the right information...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>journal title missing in many records delivere...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>hi paul thank you for reporting this issue we...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>dear sarah  thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>sfx doesnt reach correct article openurl targ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>makind our database and ejournal collections d...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>dear shahar thank you for your quick reply i ...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>search by title hi support team we encounter a...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>record full of authors the following record di...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>dear amanda i would like to update you that w...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>hello pco team\u000b",
       "\u000b",
       " please refer to the followin...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>dear karen thank you for your patience the ve...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>hey meg i would like to update you that we ha...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>dear amelia thank you for reporting this to p...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>dear galen i was able to reproduce the scenar...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>fw search link failing hello this looks like a...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>hi nitzan i can confirm that the article titl...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>inhouse record oxfordsgmlformodb with doi fml...</td>\n",
       "      <td>Data</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>dear louisa thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>hi pc team availability indicator for this re...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>hi marie i dont know if these particular reco...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>hello yoni it seems tnproquest is no longer i...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>linking problem with pci record why we all hav...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>holdings file not harvested okstats holdings f...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>proquests dissertations and thesis collection ...</td>\n",
       "      <td>Facets</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>hi daria thank you very much for your help ca...</td>\n",
       "      <td>Grouping</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>hi pc team this records received from crossre...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>capitalism socialism democracy ebook not grant...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>visual anthropology incorrect doi data in arti...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>article link leading to wrong resource hi exli...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3898</th>\n",
       "      <td>journals from emerald collections not displayi...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3899</th>\n",
       "      <td>hi kb this is a linking issue that was recrea...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3900</th>\n",
       "      <td>primo central record matching on title to unre...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3901</th>\n",
       "      <td>ej portfolios with year embargo have articles ...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3902</th>\n",
       "      <td>dear amelia thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3903</th>\n",
       "      <td>bad link to full text henderson g the franks c...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3904</th>\n",
       "      <td>hello jill it looks like the article is a res...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3905</th>\n",
       "      <td>dear jan i am checking this issue and will up...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3906</th>\n",
       "      <td>hello tomer alborg is reporting an incorrect ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3907</th>\n",
       "      <td>dear daniel thank you for contacting exlibris...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3908</th>\n",
       "      <td>sfx the annals sage we found some of the artic...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3909</th>\n",
       "      <td>incorrect link in pci metadata from nora the l...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3910</th>\n",
       "      <td>dear pauline thank you for contacting exlibri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3911</th>\n",
       "      <td>interpersonal communication and psychology f...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3912</th>\n",
       "      <td>dear roland i would like to update you that w...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3913</th>\n",
       "      <td>internal records cqvip it seems like althoug...</td>\n",
       "      <td>Grouping</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3914</th>\n",
       "      <td>pci records linking to article and revised ver...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3915</th>\n",
       "      <td>primo not enough metadata for fulltext links h...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3916</th>\n",
       "      <td>dear carolyn thank you for contacting ex libr...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3917</th>\n",
       "      <td>hi emmanuelle the title klimaeffektar på sval...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3918</th>\n",
       "      <td>records for journal of canadian studies in sfx...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3919</th>\n",
       "      <td>incorrect doi hi the record with recordid tnwo...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3920</th>\n",
       "      <td>dear lihong i would like to update you that t...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3921</th>\n",
       "      <td>dear gilbert since this customer is still und...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3922</th>\n",
       "      <td>dear bettina i would like to update you that ...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3923</th>\n",
       "      <td>an article in primo central articles more is s...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3924</th>\n",
       "      <td>hi rachel please let the vendor know that the...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3925</th>\n",
       "      <td>thanks for the update wendy dear wendy if the...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3926</th>\n",
       "      <td>dear staff thank you for contacting exlibris ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3927</th>\n",
       "      <td>dear paul thank you for your patience i am an...</td>\n",
       "      <td>Registration</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3928 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Mixed_Comments</th>\n",
       "      <th>Categorie</th>\n",
       "      <th>Categorie_A</th>\n",
       "      <th>step_one</th>\n",
       "      <th>step_two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>tnproquest is not giving the right information...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>journal title missing in many records delivere...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>hi paul thank you for reporting this issue we...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>dear sarah  thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>sfx doesnt reach correct article openurl targ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>makind our database and ejournal collections d...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>dear shahar thank you for your quick reply i ...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>search by title hi support team we encounter a...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>record full of authors the following record di...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>dear amanda i would like to update you that w...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>hello pco team\u000b",
       "\u000b",
       " please refer to the followin...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>dear karen thank you for your patience the ve...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>hey meg i would like to update you that we ha...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>dear amelia thank you for reporting this to p...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>dear galen i was able to reproduce the scenar...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>fw search link failing hello this looks like a...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>hi nitzan i can confirm that the article titl...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>inhouse record oxfordsgmlformodb with doi fml...</td>\n",
       "      <td>Data</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>dear louisa thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>hi pc team availability indicator for this re...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>hi marie i dont know if these particular reco...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>hello yoni it seems tnproquest is no longer i...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>linking problem with pci record why we all hav...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>holdings file not harvested okstats holdings f...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>proquests dissertations and thesis collection ...</td>\n",
       "      <td>Facets</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>hi daria thank you very much for your help ca...</td>\n",
       "      <td>Grouping</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>hi pc team this records received from crossre...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>capitalism socialism democracy ebook not grant...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>visual anthropology incorrect doi data in arti...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>article link leading to wrong resource hi exli...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3898</th>\n",
       "      <td>journals from emerald collections not displayi...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3899</th>\n",
       "      <td>hi kb this is a linking issue that was recrea...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3900</th>\n",
       "      <td>primo central record matching on title to unre...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3901</th>\n",
       "      <td>ej portfolios with year embargo have articles ...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3902</th>\n",
       "      <td>dear amelia thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3903</th>\n",
       "      <td>bad link to full text henderson g the franks c...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3904</th>\n",
       "      <td>hello jill it looks like the article is a res...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3905</th>\n",
       "      <td>dear jan i am checking this issue and will up...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3906</th>\n",
       "      <td>hello tomer alborg is reporting an incorrect ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3907</th>\n",
       "      <td>dear daniel thank you for contacting exlibris...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3908</th>\n",
       "      <td>sfx the annals sage we found some of the artic...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3909</th>\n",
       "      <td>incorrect link in pci metadata from nora the l...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3910</th>\n",
       "      <td>dear pauline thank you for contacting exlibri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3911</th>\n",
       "      <td>interpersonal communication and psychology f...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3912</th>\n",
       "      <td>dear roland i would like to update you that w...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3913</th>\n",
       "      <td>internal records cqvip it seems like althoug...</td>\n",
       "      <td>Grouping</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3914</th>\n",
       "      <td>pci records linking to article and revised ver...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3915</th>\n",
       "      <td>primo not enough metadata for fulltext links h...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3916</th>\n",
       "      <td>dear carolyn thank you for contacting ex libr...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3917</th>\n",
       "      <td>hi emmanuelle the title klimaeffektar på sval...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3918</th>\n",
       "      <td>records for journal of canadian studies in sfx...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3919</th>\n",
       "      <td>incorrect doi hi the record with recordid tnwo...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3920</th>\n",
       "      <td>dear lihong i would like to update you that t...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3921</th>\n",
       "      <td>dear gilbert since this customer is still und...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3922</th>\n",
       "      <td>dear bettina i would like to update you that ...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3923</th>\n",
       "      <td>an article in primo central articles more is s...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3924</th>\n",
       "      <td>hi rachel please let the vendor know that the...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3925</th>\n",
       "      <td>thanks for the update wendy dear wendy if the...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3926</th>\n",
       "      <td>dear staff thank you for contacting exlibris ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3927</th>\n",
       "      <td>dear paul thank you for your patience i am an...</td>\n",
       "      <td>Registration</td>\n",
       "      <td>Other</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3928 rows × 5 columns</p>\n",
       "</div>"
      ]
     },
     "execution_count": 379,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "two_step_dataframe['step_one']=pipeline_lr_linking.predict(two_step_dataframe['Mixed_Comments'])\n",
    "# \n",
    "score=0\n",
    "\n",
    "for i in range(len(two_step_dataframe)):\n",
    "    if two_step_dataframe['Categorie_A'][i]==two_step_dataframe['step_one'][i]:\n",
    "        score+=1\n",
    "print(score/len(two_step_dataframe))    \n",
    "two_step_dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Mixed_Comments</th>\n",
       "      <th>Categorie</th>\n",
       "      <th>Categorie_A</th>\n",
       "      <th>step_one</th>\n",
       "      <th>step_two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>tnproquest is not giving the right information...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>journal title missing in many records delivere...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>hi paul thank you for reporting this issue we...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>dear sarah  thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>sfx doesnt reach correct article openurl targ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>makind our database and ejournal collections d...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>dear shahar thank you for your quick reply i ...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>search by title hi support team we encounter a...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>record full of authors the following record di...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>dear amanda i would like to update you that w...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>hello pco team\u000b",
       "\u000b",
       " please refer to the followin...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>dear karen thank you for your patience the ve...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>hey meg i would like to update you that we ha...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>dear amelia thank you for reporting this to p...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>dear galen i was able to reproduce the scenar...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>fw search link failing hello this looks like a...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>hi nitzan i can confirm that the article titl...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>inhouse record oxfordsgmlformodb with doi fml...</td>\n",
       "      <td>Data</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>dear louisa thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>hi pc team availability indicator for this re...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>hi marie i dont know if these particular reco...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>hello yoni it seems tnproquest is no longer i...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>linking problem with pci record why we all hav...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>holdings file not harvested okstats holdings f...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>proquests dissertations and thesis collection ...</td>\n",
       "      <td>Facets</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Facets</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>hi daria thank you very much for your help ca...</td>\n",
       "      <td>Grouping</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Grouping</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>hi pc team this records received from crossre...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>capitalism socialism democracy ebook not grant...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>visual anthropology incorrect doi data in arti...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>article link leading to wrong resource hi exli...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3898</th>\n",
       "      <td>journals from emerald collections not displayi...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3899</th>\n",
       "      <td>hi kb this is a linking issue that was recrea...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3900</th>\n",
       "      <td>primo central record matching on title to unre...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3901</th>\n",
       "      <td>ej portfolios with year embargo have articles ...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3902</th>\n",
       "      <td>dear amelia thank you for contacting ex libri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3903</th>\n",
       "      <td>bad link to full text henderson g the franks c...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3904</th>\n",
       "      <td>hello jill it looks like the article is a res...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3905</th>\n",
       "      <td>dear jan i am checking this issue and will up...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3906</th>\n",
       "      <td>hello tomer alborg is reporting an incorrect ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3907</th>\n",
       "      <td>dear daniel thank you for contacting exlibris...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3908</th>\n",
       "      <td>sfx the annals sage we found some of the artic...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3909</th>\n",
       "      <td>incorrect link in pci metadata from nora the l...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3910</th>\n",
       "      <td>dear pauline thank you for contacting exlibri...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3911</th>\n",
       "      <td>interpersonal communication and psychology f...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Grouping</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3912</th>\n",
       "      <td>dear roland i would like to update you that w...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3913</th>\n",
       "      <td>internal records cqvip it seems like althoug...</td>\n",
       "      <td>Grouping</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Grouping</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3914</th>\n",
       "      <td>pci records linking to article and revised ver...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Grouping</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3915</th>\n",
       "      <td>primo not enough metadata for fulltext links h...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3916</th>\n",
       "      <td>dear carolyn thank you for contacting ex libr...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3917</th>\n",
       "      <td>hi emmanuelle the title klimaeffektar på sval...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Updates</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3918</th>\n",
       "      <td>records for journal of canadian studies in sfx...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3919</th>\n",
       "      <td>incorrect doi hi the record with recordid tnwo...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3920</th>\n",
       "      <td>dear lihong i would like to update you that t...</td>\n",
       "      <td>Data availability</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3921</th>\n",
       "      <td>dear gilbert since this customer is still und...</td>\n",
       "      <td>Data display</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3922</th>\n",
       "      <td>dear bettina i would like to update you that ...</td>\n",
       "      <td>Normalization Rules</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Normalization Rules</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3923</th>\n",
       "      <td>an article in primo central articles more is s...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3924</th>\n",
       "      <td>hi rachel please let the vendor know that the...</td>\n",
       "      <td>Updates</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data display</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3925</th>\n",
       "      <td>thanks for the update wendy dear wendy if the...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3926</th>\n",
       "      <td>dear staff thank you for contacting exlibris ...</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>Data linking</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3927</th>\n",
       "      <td>dear paul thank you for your patience i am an...</td>\n",
       "      <td>Registration</td>\n",
       "      <td>Other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Data availability</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3928 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         Mixed_Comments            Categorie  \\\n",
       "0     tnproquest is not giving the right information...         Data linking   \n",
       "1     journal title missing in many records delivere...         Data linking   \n",
       "2      hi paul thank you for reporting this issue we...         Data linking   \n",
       "3      dear sarah  thank you for contacting ex libri...         Data linking   \n",
       "4      sfx doesnt reach correct article openurl targ...         Data linking   \n",
       "5     makind our database and ejournal collections d...    Data availability   \n",
       "6      dear shahar thank you for your quick reply i ...    Data availability   \n",
       "7     search by title hi support team we encounter a...  Normalization Rules   \n",
       "8     record full of authors the following record di...         Data display   \n",
       "9      dear amanda i would like to update you that w...         Data linking   \n",
       "10     hello pco team\n",
       "\n",
       " please refer to the followin...    Data availability   \n",
       "11     dear karen thank you for your patience the ve...  Normalization Rules   \n",
       "12     hey meg i would like to update you that we ha...         Data display   \n",
       "13     dear amelia thank you for reporting this to p...         Data linking   \n",
       "14     dear galen i was able to reproduce the scenar...         Data linking   \n",
       "15    fw search link failing hello this looks like a...         Data display   \n",
       "16     hi nitzan i can confirm that the article titl...         Data linking   \n",
       "17     inhouse record oxfordsgmlformodb with doi fml...                 Data   \n",
       "18     dear louisa thank you for contacting ex libri...         Data linking   \n",
       "19     hi pc team availability indicator for this re...    Data availability   \n",
       "20     hi marie i dont know if these particular reco...    Data availability   \n",
       "21     hello yoni it seems tnproquest is no longer i...         Data linking   \n",
       "22    linking problem with pci record why we all hav...         Data linking   \n",
       "23    holdings file not harvested okstats holdings f...    Data availability   \n",
       "24    proquests dissertations and thesis collection ...               Facets   \n",
       "25     hi daria thank you very much for your help ca...             Grouping   \n",
       "26     hi pc team this records received from crossre...         Data linking   \n",
       "27    capitalism socialism democracy ebook not grant...    Data availability   \n",
       "28    visual anthropology incorrect doi data in arti...         Data linking   \n",
       "29    article link leading to wrong resource hi exli...         Data linking   \n",
       "...                                                 ...                  ...   \n",
       "3898  journals from emerald collections not displayi...              Updates   \n",
       "3899   hi kb this is a linking issue that was recrea...         Data linking   \n",
       "3900  primo central record matching on title to unre...    Data availability   \n",
       "3901  ej portfolios with year embargo have articles ...         Data display   \n",
       "3902   dear amelia thank you for contacting ex libri...         Data linking   \n",
       "3903  bad link to full text henderson g the franks c...         Data linking   \n",
       "3904   hello jill it looks like the article is a res...         Data display   \n",
       "3905   dear jan i am checking this issue and will up...         Data linking   \n",
       "3906   hello tomer alborg is reporting an incorrect ...         Data linking   \n",
       "3907   dear daniel thank you for contacting exlibris...    Data availability   \n",
       "3908  sfx the annals sage we found some of the artic...         Data linking   \n",
       "3909  incorrect link in pci metadata from nora the l...         Data linking   \n",
       "3910   dear pauline thank you for contacting exlibri...         Data linking   \n",
       "3911    interpersonal communication and psychology f...         Data linking   \n",
       "3912   dear roland i would like to update you that w...         Data linking   \n",
       "3913    internal records cqvip it seems like althoug...             Grouping   \n",
       "3914  pci records linking to article and revised ver...         Data linking   \n",
       "3915  primo not enough metadata for fulltext links h...         Data linking   \n",
       "3916   dear carolyn thank you for contacting ex libr...         Data linking   \n",
       "3917   hi emmanuelle the title klimaeffektar på sval...              Updates   \n",
       "3918  records for journal of canadian studies in sfx...         Data display   \n",
       "3919  incorrect doi hi the record with recordid tnwo...         Data linking   \n",
       "3920   dear lihong i would like to update you that t...    Data availability   \n",
       "3921   dear gilbert since this customer is still und...         Data display   \n",
       "3922   dear bettina i would like to update you that ...  Normalization Rules   \n",
       "3923  an article in primo central articles more is s...         Data linking   \n",
       "3924   hi rachel please let the vendor know that the...              Updates   \n",
       "3925   thanks for the update wendy dear wendy if the...         Data linking   \n",
       "3926   dear staff thank you for contacting exlibris ...         Data linking   \n",
       "3927   dear paul thank you for your patience i am an...         Registration   \n",
       "\n",
       "       Categorie_A step_one             step_two  \n",
       "0     Data linking      NaN                 Data  \n",
       "1     Data linking      NaN         Data display  \n",
       "2     Data linking      NaN    Data availability  \n",
       "3     Data linking      NaN    Data availability  \n",
       "4     Data linking      NaN                 Data  \n",
       "5            Other      NaN    Data availability  \n",
       "6            Other      NaN         Data display  \n",
       "7            Other      NaN         Data display  \n",
       "8            Other      NaN         Data display  \n",
       "9     Data linking      NaN         Data display  \n",
       "10           Other      NaN    Data availability  \n",
       "11           Other      NaN         Data display  \n",
       "12           Other      NaN         Data display  \n",
       "13    Data linking      NaN         Data display  \n",
       "14    Data linking      NaN    Data availability  \n",
       "15           Other      NaN         Data display  \n",
       "16    Data linking      NaN         Data display  \n",
       "17           Other      NaN                 Data  \n",
       "18    Data linking      NaN         Data display  \n",
       "19           Other      NaN    Data availability  \n",
       "20           Other      NaN    Data availability  \n",
       "21    Data linking      NaN         Data display  \n",
       "22    Data linking      NaN    Data availability  \n",
       "23           Other      NaN    Data availability  \n",
       "24           Other      NaN               Facets  \n",
       "25           Other      NaN             Grouping  \n",
       "26    Data linking      NaN         Data display  \n",
       "27           Other      NaN    Data availability  \n",
       "28    Data linking      NaN         Data display  \n",
       "29    Data linking      NaN         Data display  \n",
       "...            ...      ...                  ...  \n",
       "3898         Other      NaN    Data availability  \n",
       "3899  Data linking      NaN         Data display  \n",
       "3900         Other      NaN    Data availability  \n",
       "3901         Other      NaN    Data availability  \n",
       "3902  Data linking      NaN    Data availability  \n",
       "3903  Data linking      NaN    Data availability  \n",
       "3904         Other      NaN         Data display  \n",
       "3905  Data linking      NaN    Data availability  \n",
       "3906  Data linking      NaN         Data display  \n",
       "3907         Other      NaN    Data availability  \n",
       "3908  Data linking      NaN         Data display  \n",
       "3909  Data linking      NaN         Data display  \n",
       "3910  Data linking      NaN         Data display  \n",
       "3911  Data linking      NaN             Grouping  \n",
       "3912  Data linking      NaN         Data display  \n",
       "3913         Other      NaN             Grouping  \n",
       "3914  Data linking      NaN             Grouping  \n",
       "3915  Data linking      NaN         Data display  \n",
       "3916  Data linking      NaN         Data display  \n",
       "3917         Other      NaN              Updates  \n",
       "3918         Other      NaN    Data availability  \n",
       "3919  Data linking      NaN         Data display  \n",
       "3920         Other      NaN    Data availability  \n",
       "3921         Other      NaN         Data display  \n",
       "3922         Other      NaN  Normalization Rules  \n",
       "3923  Data linking      NaN    Data availability  \n",
       "3924         Other      NaN         Data display  \n",
       "3925  Data linking      NaN    Data availability  \n",
       "3926  Data linking      NaN    Data availability  \n",
       "3927         Other      NaN    Data availability  \n",
       "\n",
       "[3928 rows x 5 columns]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "two_step_dataframe['step_two']=pipeline_lr_others.predict(two_step_dataframe['Mixed_Comments'])\n",
    "# two_step_dataframe\n",
    "for i in range(len(two_step_dataframe)):\n",
    "    if two_step_dataframe['step_one'][i]=='Data linking':\n",
    "        two_step_dataframe['step_two'][i]='Data linking'\n",
    "two_step_dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 383,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7731670061099797\n"
     ]
    }
   ],
   "source": [
    "#two steps score\n",
    "score=0\n",
    "\n",
    "for i in range(len(two_step_dataframe)):\n",
    "    if two_step_dataframe['Categorie'][i]==two_step_dataframe['step_two'][i]:\n",
    "        score+=1\n",
    "print(score/len(two_step_dataframe))    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Final Results:\n",
    "<br>\n",
    "One stage model: 74% accuricy\n",
    "<br>\n",
    "Two stage model: 77.3% accuricy"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}