Naive_Bayes_Model.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "CA02_Naive_Bayes Silvia Ji.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P1lH8YnduocD"
      },
      "source": [
        "# Spam eMail Detection using Naive Bayes Classification Algorithm\n",
        "\n",
        "In this project, a model is trained with set of emails labelled as either from Spam or Non-Spam. There are 702 emails equally divided into spam and non spam category.\n",
        "\n",
        "Next, the model is tested on 260 emails. The model is tasked to predict the category of the emails and compare the accuracy with known correct classifications.\n",
        "\n",
        "There are two folders: test-mails and train-mails. Train-mails are to train the model. Test-mails are used to test the accuracy of the model.\n",
        "\n",
        "Each email's first line is the subject; the content starts from the third line.\n",
        "\n",
        "If you navigate to any of the train-mails or test-mails, you shall see file names in two patterns:\n",
        "\n",
        "* *number-numbermsg[number].txt : example 3-1msg1.txt* (non spam emails)\n",
        "\n",
        "* *spmsg[Number].txt : example **spms**ga162.txt* (spam emails)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XbW07R6puOH5",
        "outputId": "7f5e3f38-060d-491b-e735-521dd13927d3"
      },
      "source": [
        "# Importing modules\n",
        "import os\n",
        "import numpy as np\n",
        "from collections import Counter\n",
        "from sklearn.naive_bayes import GaussianNB\n",
        "from sklearn.metrics import accuracy_score\n",
        "from google.colab import drive\n",
        "\n",
        "# Mounting google drive to link directories\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gjieAxhrvX3h"
      },
      "source": [
        "# Linking directories\n",
        "train_data = '/content/drive/MyDrive/MSBA_Colab_2020/ML_Algorithms/CA02/Data/train-mails'\n",
        "test_data = '/content/drive/MyDrive/MSBA_Colab_2020/ML_Algorithms/CA02/Data/test-mails'"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ixbsee-Iu6eA"
      },
      "source": [
        "## 1. Cleaning and Preparing Data\n",
        "\n",
        "First, the data is cleaned and prepared for the modelby removing non-required words, expressions and symbols from text."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AwCi4TII_DI8"
      },
      "source": [
        "### Building a Dictionary\n",
        "\n",
        "This function builds a dictionary using the most common 3000 words from all email content. It will also remove all non-alpha-numeric and single character alpha-numeric characters."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "boWEOmsRxXx3"
      },
      "source": [
        "def create_dict(root_folder):                                                         # root_folder = main folder where all email txt files are stored\n",
        "  all_email_words = []                                                                # creates an empty list to append all words in all emails\n",
        "  all_emails = [os.path.join(root_folder,file) for file in os.listdir(root_folder)]   # creates list of paths: root_folder/.../file\n",
        "  for email in all_emails:\n",
        "    with open(email) as e:                                                            # opens file and returns it as file object\n",
        "      for text in e:\n",
        "        email_words = text.split()                                                    # splits up email content into list of individual words\n",
        "        all_email_words += email_words                                                # adds each new word to email_words\n",
        "  words_dict = Counter(all_email_words)                                               # creates dictionary pairs: email_words and their counts\n",
        "  list_to_remove = list(words_dict)                                                   # transforms dictionary to list; will store unneeded words to remove \n",
        "  for word in list_to_remove:\n",
        "    if word.isalpha() == False:                                                       # checks if word is NOT alphanumeric\n",
        "      del words_dict[word]                                                            # removes non-alphanumeric word from \"words_dict\"\n",
        "    elif len(word) == 1:                                                              # checks if word is single character\n",
        "      del words_dict[word]                                                            # removes single character words from \"words_dict\"\n",
        "  words_dict = words_dict.most_common(3000)                                           # only keeps 3000 most kommon words in \"words_dict\"\n",
        "  return words_dict                                                                   # prints the final dictionary of 3000 most common words in all email"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DOwHzHfeuA69"
      },
      "source": [
        "## 2. Building the Functions\n",
        "\n",
        "In this section, we create the functions to extract the accurate data which is fed in the model to make the appropriate predictions. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ca4olTKRsDPT"
      },
      "source": [
        "### Extracting features and corresponding label frequency matrix\n",
        "\n",
        "A word frequency matrix and train labels are created. \n",
        "\n",
        "**Frequency matrix.** The 3000 columns represent features, in this case the 3000 most commonly used words that were previously identified. There are as many rows as the number of emails, each row representing one email. The values indicate how many times a word occurs in each email.\n",
        "\n",
        "**Train labels.** A one-dimensional array. The number of columns correspond to the number of emails. It is populated with 0 if it is believed to be non-spam and 1 if it is believed to be spam.\n",
        "\n",
        "It will analyze the file name of each email and decide if it is spam or not based on the naming convention in order to categorize it as spam or non-spam."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2U3fgfA1t63m",
        "outputId": "01b5ebcc-f8bb-4c9c-dc9e-91151a7a090f"
      },
      "source": [
        "arr = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10],[11, 12, 13, 14, 15]])\n",
        "print(arr)\n",
        "print(arr[2,3])"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ 1  2  3  4  5]\n",
            " [ 6  7  8  9 10]\n",
            " [11 12 13 14 15]]\n",
            "14\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A391-RAeu18B"
      },
      "source": [
        "def get_features(main_folder):\n",
        "  email_files = [os.path.join(main_folder,fil) for fil in os.listdir(main_folder)]      # creates list of paths of all email files: root_folder/.../file\n",
        "  frequency_matrix = np.zeros((len(email_files),3000))                                  # creates 2D array filled with zeros. # of rows = # emails; # of columns = 3000 (most common words)\n",
        "  train_labels = np.zeros(len(email_files))                                             # creates 1D awway filled with zeros. length = # of emails\n",
        "  spam_count = 0;\n",
        "  email_ID = 0;                                                                         # email_ID = 0 -> starts at first email (first row)\n",
        "  for f in email_files:\n",
        "    with open(f) as fi:                                                                 # opens file and returns it as file object\n",
        "      for pos, text in enumerate(fi):                                                   # enumerate() in for loop returns email position and value of each email\n",
        "        if pos ==2:                                                                     # x==2 -> third line (first line is subject and content is on third line)\n",
        "          email_body = text.split()                                                     # splits email text into individual words (returns list)\n",
        "          for word in email_body:\n",
        "            word_ID = 0\n",
        "            for pos, w in enumerate(words_dict):                                        # for every position and {word:count} pair of the 3000 most common words\n",
        "              if w[0] == word:                                                          # checks if word in dictionary occurs in email body\n",
        "                word_ID = pos                                                           # if so, word_ID = position of word in dictionary (distinguishes word)\n",
        "                frequency_matrix[email_ID,word_ID] = email_body.count(word)             # email_ID=row position, word_ID=column position. populates value with number of occurances of word in all emails\n",
        "      train_labels[email_ID] = 0;                                                       # creates training labels to predict spam or non-spam; fills row with 0s (assuming there is no spam)                                                    \n",
        "      filepathTokens = f.split('/')                                                     # creates list with each path component name \n",
        "      lastToken = filepathTokens[len(filepathTokens)-1]                                 # last name in path -> email file name\n",
        "      if lastToken.startswith(\"spmsg\"):\n",
        "        train_labels[email_ID] = 1;                                                     # if file name name starts with \"smpsg\" (=spam), it's populated with 1 \n",
        "        spam_count = spam_count + 1                                                     # \"count\" counts spam emails, add 1 each time a new spam is detected\n",
        "      email_ID = email_ID + 1                                                           # moves on to next email (next row)      \n",
        "  return frequency_matrix, train_labels\n",
        "\n",
        "  # 0=no, 1=yes"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-orRvuIKuby9"
      },
      "source": [
        "## 3. Training the Model and Predicting Results\n",
        "\n",
        "In this section, we choose a method to train the model and use the previously defined functions to extract the input data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "by_QcjoGlO0m"
      },
      "source": [
        "### sklearn Naive Bayes\n",
        "\n",
        "sklearn Naive Bayes provides three alternatives for model training:\n",
        "\n",
        "1.   **Gaussian**\n",
        "2.   Multinominal\n",
        "3.   Bernoulli\n",
        "\n",
        "This model uses the Gaussian method, which is used in classification and  assumes that features follow a normal distribution."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2PsDdoq2bA0E",
        "outputId": "a69c0b5b-355f-4f82-f70f-7cdce5693947"
      },
      "source": [
        "words_dict = create_dict(train_data)                                    # specifies that dictionary is using train data\n",
        "print (\"reading and processing emails from TRAIN and TEST folders\")\n",
        "frequency_matrix, labels = get_features(train_data)                     # specifies that frequency_matrix and labels are derived from train data\n",
        "test_frequency_matrix, test_labels = get_features(test_data)            # stores test data frequency_matrix and labels into seperate variables\n",
        "\n",
        "model = GaussianNB()                                                    # specifies model type\n",
        "\n",
        "print (\"Training Model using Gaussian Naive Bayes algorithm .....\")\n",
        "model.fit(frequency_matrix, labels)                                     # trains model using train data \n",
        "print (\"Training completed\")        \n",
        "print (\"testing trained model to predict Test Data labels\")\n",
        "predicted_labels = model.predict(test_frequency_matrix)                 # model predicts labels based on test frequency matrix"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "reading and processing emails from TRAIN and TEST folders\n",
            "Training Model using Gaussian Naive Bayes algorithm .....\n",
            "Training completed\n",
            "testing trained model to predict Test Data labels\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mzEXHv5pxaTH"
      },
      "source": [
        "## 4. Evaluation\n",
        "\n",
        "Lastly, we evaluate the model's performance based on the calculated accuracy score."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RILwcCdzpj1t"
      },
      "source": [
        "### Accuracy score\n",
        "\n",
        "The accuracy score for predicting labels is calculated. The accuracy score represents the percentage of correct predictions, in this case whether an email is spam or not."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f7atY0Wh-I0B",
        "outputId": "8fe542da-c8ba-4e08-e039-9712d12c3cc3"
      },
      "source": [
        "print (\"Completed classification of the Test Data .... now printing Accuracy Score by comparing the Predicted Labels with the Test Labels:\")\n",
        "print (accuracy_score(test_labels, predicted_labels))                   # accuracy score = how close predicted labels are to test labels"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Completed classification of the Test Data .... now printing Accuracy Score by comparing the Predicted Labels with the Test Labels:\n",
            "0.9653846153846154\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a4DPuYDOw3FX"
      },
      "source": [
        "### Result\n",
        "\n",
        "The returned accuracy score is 0.9654, meaning that the model is 96.54% accurate when predicting whether an email is spam. This indicates a high quality model."
      ]
    }
  ]
}