venture_funding_with_deep_learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Venture Funding with Deep Learning\n",
    "\n",
    "You work as a risk management associate at Alphabet Soup, a venture capital firm. Alphabet Soup’s business team receives many funding applications from startups every day. This team has asked you to help them create a model that predicts whether applicants will be successful if funded by Alphabet Soup.\n",
    "\n",
    "The business team has given you a CSV containing more than 34,000 organizations that have received funding from Alphabet Soup over the years. With your knowledge of machine learning and neural networks, you decide to use the features in the provided dataset to create a binary classifier model that will predict whether an applicant will become a successful business. The CSV file contains a variety of information about these businesses, including whether or not they ultimately became successful.\n",
    "\n",
    "## Instructions:\n",
    "\n",
    "The steps for this challenge are broken out into the following sections:\n",
    "\n",
    "* Prepare the data for use on a neural network model.\n",
    "\n",
    "* Compile and evaluate a binary classification model using a neural network.\n",
    "\n",
    "* Optimize the neural network model.\n",
    "\n",
    "### Prepare the Data for Use on a Neural Network Model \n",
    "\n",
    "Using your knowledge of Pandas and scikit-learn’s `StandardScaler()`, preprocess the dataset so that you can use it to compile and evaluate the neural network model later.\n",
    "\n",
    "Open the starter code file, and complete the following data preparation steps:\n",
    "\n",
    "1. Read the `applicants_data.csv` file into a Pandas DataFrame. Review the DataFrame, looking for categorical variables that will need to be encoded, as well as columns that could eventually define your features and target variables.   \n",
    "\n",
    "2. Drop the “EIN” (Employer Identification Number) and “NAME” columns from the DataFrame, because they are not relevant to the binary classification model.\n",
    " \n",
    "3. Encode the dataset’s categorical variables using `OneHotEncoder`, and then place the encoded variables into a new DataFrame.\n",
    "\n",
    "4. Add the original DataFrame’s numerical variables to the DataFrame containing the encoded variables.\n",
    "\n",
    "> **Note** To complete this step, you will employ the Pandas `concat()` function that was introduced earlier in this course. \n",
    "\n",
    "5. Using the preprocessed data, create the features (`X`) and target (`y`) datasets. The target dataset should be defined by the preprocessed DataFrame column “IS_SUCCESSFUL”. The remaining columns should define the features dataset. \n",
    "\n",
    "6. Split the features and target sets into training and testing datasets.\n",
    "\n",
    "7. Use scikit-learn's `StandardScaler` to scale the features data.\n",
    "\n",
    "### Compile and Evaluate a Binary Classification Model Using a Neural Network\n",
    "\n",
    "Use your knowledge of TensorFlow to design a binary classification deep neural network model. This model should use the dataset’s features to predict whether an Alphabet Soup&ndash;funded startup will be successful based on the features in the dataset. Consider the number of inputs before determining the number of layers that your model will contain or the number of neurons on each layer. Then, compile and fit your model. Finally, evaluate your binary classification model to calculate the model’s loss and accuracy. \n",
    " \n",
    "To do so, complete the following steps:\n",
    "\n",
    "1. Create a deep neural network by assigning the number of input features, the number of layers, and the number of neurons on each layer using Tensorflow’s Keras.\n",
    "\n",
    "> **Hint** You can start with a two-layer deep neural network model that uses the `relu` activation function for both layers.\n",
    "\n",
    "2. Compile and fit the model using the `binary_crossentropy` loss function, the `adam` optimizer, and the `accuracy` evaluation metric.\n",
    "\n",
    "> **Hint** When fitting the model, start with a small number of epochs, such as 20, 50, or 100.\n",
    "\n",
    "3. Evaluate the model using the test data to determine the model’s loss and accuracy.\n",
    "\n",
    "4. Save and export your model to an HDF5 file, and name the file `AlphabetSoup.h5`. \n",
    "\n",
    "### Optimize the Neural Network Model\n",
    "\n",
    "Using your knowledge of TensorFlow and Keras, optimize your model to improve the model's accuracy. Even if you do not successfully achieve a better accuracy, you'll need to demonstrate at least two attempts to optimize the model. You can include these attempts in your existing notebook. Or, you can make copies of the starter notebook in the same folder, rename them, and code each model optimization in a new notebook. \n",
    "\n",
    "> **Note** You will not lose points if your model does not achieve a high accuracy, as long as you make at least two attempts to optimize the model.\n",
    "\n",
    "To do so, complete the following steps:\n",
    "\n",
    "1. Define at least three new deep neural network models (the original plus 2 optimization attempts). With each, try to improve on your first model’s predictive accuracy.\n",
    "\n",
    "> **Rewind** Recall that perfect accuracy has a value of 1, so accuracy improves as its value moves closer to 1. To optimize your model for a predictive accuracy as close to 1 as possible, you can use any or all of the following techniques:\n",
    ">\n",
    "> * Adjust the input data by dropping different features columns to ensure that no variables or outliers confuse the model.\n",
    ">\n",
    "> * Add more neurons (nodes) to a hidden layer.\n",
    ">\n",
    "> * Add more hidden layers.\n",
    ">\n",
    "> * Use different activation functions for the hidden layers.\n",
    ">\n",
    "> * Add to or reduce the number of epochs in the training regimen.\n",
    "\n",
    "2. After finishing your models, display the accuracy scores achieved by each model, and compare the results.\n",
    "\n",
    "3. Save each of your models as an HDF5 file.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Imports\n",
    "import pandas as pd\n",
    "from pathlib import Path\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras.layers import Dense\n",
    "from tensorflow.keras.models import Sequential\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler,OneHotEncoder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Prepare the data to be used on a neural network model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Read the `applicants_data.csv` file into a Pandas DataFrame. Review the DataFrame, looking for categorical variables that will need to be encoded, as well as columns that could eventually define your features and target variables.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EIN</th>\n",
       "      <th>NAME</th>\n",
       "      <th>APPLICATION_TYPE</th>\n",
       "      <th>AFFILIATION</th>\n",
       "      <th>CLASSIFICATION</th>\n",
       "      <th>USE_CASE</th>\n",
       "      <th>ORGANIZATION</th>\n",
       "      <th>STATUS</th>\n",
       "      <th>INCOME_AMT</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS</th>\n",
       "      <th>ASK_AMT</th>\n",
       "      <th>IS_SUCCESSFUL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10520599</td>\n",
       "      <td>BLUE KNIGHTS MOTORCYCLE CLUB</td>\n",
       "      <td>T10</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10531628</td>\n",
       "      <td>AMERICAN CHESAPEAKE CLUB CHARITABLE TR</td>\n",
       "      <td>T3</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C2000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Co-operative</td>\n",
       "      <td>1</td>\n",
       "      <td>1-9999</td>\n",
       "      <td>N</td>\n",
       "      <td>108590</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10547893</td>\n",
       "      <td>ST CLOUD PROFESSIONAL FIREFIGHTERS</td>\n",
       "      <td>T5</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C3000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10553066</td>\n",
       "      <td>SOUTHSIDE ATHLETIC ASSOCIATION</td>\n",
       "      <td>T3</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C2000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Trust</td>\n",
       "      <td>1</td>\n",
       "      <td>10000-24999</td>\n",
       "      <td>N</td>\n",
       "      <td>6692</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10556103</td>\n",
       "      <td>GENETIC RESEARCH INSTITUTE OF THE DESERT</td>\n",
       "      <td>T3</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>Heathcare</td>\n",
       "      <td>Trust</td>\n",
       "      <td>1</td>\n",
       "      <td>100000-499999</td>\n",
       "      <td>N</td>\n",
       "      <td>142590</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        EIN                                      NAME APPLICATION_TYPE  \\\n",
       "0  10520599              BLUE KNIGHTS MOTORCYCLE CLUB              T10   \n",
       "1  10531628    AMERICAN CHESAPEAKE CLUB CHARITABLE TR               T3   \n",
       "2  10547893        ST CLOUD PROFESSIONAL FIREFIGHTERS               T5   \n",
       "3  10553066            SOUTHSIDE ATHLETIC ASSOCIATION               T3   \n",
       "4  10556103  GENETIC RESEARCH INSTITUTE OF THE DESERT               T3   \n",
       "\n",
       "        AFFILIATION CLASSIFICATION      USE_CASE  ORGANIZATION  STATUS  \\\n",
       "0       Independent          C1000    ProductDev   Association       1   \n",
       "1       Independent          C2000  Preservation  Co-operative       1   \n",
       "2  CompanySponsored          C3000    ProductDev   Association       1   \n",
       "3  CompanySponsored          C2000  Preservation         Trust       1   \n",
       "4       Independent          C1000     Heathcare         Trust       1   \n",
       "\n",
       "      INCOME_AMT SPECIAL_CONSIDERATIONS  ASK_AMT  IS_SUCCESSFUL  \n",
       "0              0                      N     5000              1  \n",
       "1         1-9999                      N   108590              1  \n",
       "2              0                      N     5000              0  \n",
       "3    10000-24999                      N     6692              1  \n",
       "4  100000-499999                      N   142590              1  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Read the applicants_data.csv file from the Resources folder into a Pandas DataFrame\n",
    "applicant_data_df = pd.read_csv(\"./Resources/applicants_data.csv\")\n",
    "\n",
    "# Review the DataFrame\n",
    "applicant_data_df.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 34299 entries, 0 to 34298\n",
      "Data columns (total 12 columns):\n",
      " #   Column                  Non-Null Count  Dtype \n",
      "---  ------                  --------------  ----- \n",
      " 0   EIN                     34299 non-null  int64 \n",
      " 1   NAME                    34299 non-null  object\n",
      " 2   APPLICATION_TYPE        34299 non-null  object\n",
      " 3   AFFILIATION             34299 non-null  object\n",
      " 4   CLASSIFICATION          34299 non-null  object\n",
      " 5   USE_CASE                34299 non-null  object\n",
      " 6   ORGANIZATION            34299 non-null  object\n",
      " 7   STATUS                  34299 non-null  int64 \n",
      " 8   INCOME_AMT              34299 non-null  object\n",
      " 9   SPECIAL_CONSIDERATIONS  34299 non-null  object\n",
      " 10  ASK_AMT                 34299 non-null  int64 \n",
      " 11  IS_SUCCESSFUL           34299 non-null  int64 \n",
      "dtypes: int64(4), object(8)\n",
      "memory usage: 3.1+ MB\n"
     ]
    }
   ],
   "source": [
    "# Review the data types associated with the columns\n",
    "applicant_data_df.info()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Drop the “EIN” (Employer Identification Number) and “NAME” columns from the DataFrame, because they are not relevant to the binary classification model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>APPLICATION_TYPE</th>\n",
       "      <th>AFFILIATION</th>\n",
       "      <th>CLASSIFICATION</th>\n",
       "      <th>USE_CASE</th>\n",
       "      <th>ORGANIZATION</th>\n",
       "      <th>STATUS</th>\n",
       "      <th>INCOME_AMT</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS</th>\n",
       "      <th>ASK_AMT</th>\n",
       "      <th>IS_SUCCESSFUL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>T10</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>T3</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C2000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Co-operative</td>\n",
       "      <td>1</td>\n",
       "      <td>1-9999</td>\n",
       "      <td>N</td>\n",
       "      <td>108590</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>T5</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C3000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>T3</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C2000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Trust</td>\n",
       "      <td>1</td>\n",
       "      <td>10000-24999</td>\n",
       "      <td>N</td>\n",
       "      <td>6692</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>T3</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>Heathcare</td>\n",
       "      <td>Trust</td>\n",
       "      <td>1</td>\n",
       "      <td>100000-499999</td>\n",
       "      <td>N</td>\n",
       "      <td>142590</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  APPLICATION_TYPE       AFFILIATION CLASSIFICATION      USE_CASE  \\\n",
       "0              T10       Independent          C1000    ProductDev   \n",
       "1               T3       Independent          C2000  Preservation   \n",
       "2               T5  CompanySponsored          C3000    ProductDev   \n",
       "3               T3  CompanySponsored          C2000  Preservation   \n",
       "4               T3       Independent          C1000     Heathcare   \n",
       "\n",
       "   ORGANIZATION  STATUS     INCOME_AMT SPECIAL_CONSIDERATIONS  ASK_AMT  \\\n",
       "0   Association       1              0                      N     5000   \n",
       "1  Co-operative       1         1-9999                      N   108590   \n",
       "2   Association       1              0                      N     5000   \n",
       "3         Trust       1    10000-24999                      N     6692   \n",
       "4         Trust       1  100000-499999                      N   142590   \n",
       "\n",
       "   IS_SUCCESSFUL  \n",
       "0              1  \n",
       "1              1  \n",
       "2              0  \n",
       "3              1  \n",
       "4              1  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Drop the 'EIN' and 'NAME' columns from the DataFrame\n",
    "applicant_data_df = applicant_data_df.drop(columns = [\"EIN\",\"NAME\"])\n",
    "\n",
    "# Review the DataFrame\n",
    "applicant_data_df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Encode the dataset’s categorical variables using `OneHotEncoder`, and then place the encoded variables into a new DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['APPLICATION_TYPE',\n",
       " 'AFFILIATION',\n",
       " 'CLASSIFICATION',\n",
       " 'USE_CASE',\n",
       " 'ORGANIZATION',\n",
       " 'INCOME_AMT',\n",
       " 'SPECIAL_CONSIDERATIONS']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a list of categorical variables \n",
    "categorical_variables = list(applicant_data_df .dtypes[applicant_data_df .dtypes == \"object\"].index)\n",
    "\n",
    "# Display the categorical variables list\n",
    "categorical_variables\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a OneHotEncoder instance\n",
    "enc = OneHotEncoder(sparse = False)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 0., 0., ..., 0., 1., 0.],\n",
       "       [0., 0., 0., ..., 0., 1., 0.],\n",
       "       [0., 0., 0., ..., 0., 1., 0.],\n",
       "       ...,\n",
       "       [0., 0., 0., ..., 0., 1., 0.],\n",
       "       [0., 0., 0., ..., 0., 1., 0.],\n",
       "       [0., 0., 0., ..., 0., 1., 0.]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Encode the categorcal variables using OneHotEncoder\n",
    "encoded_data = enc.fit_transform(applicant_data_df[categorical_variables])\n",
    "encoded_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>APPLICATION_TYPE_T10</th>\n",
       "      <th>APPLICATION_TYPE_T12</th>\n",
       "      <th>APPLICATION_TYPE_T13</th>\n",
       "      <th>APPLICATION_TYPE_T14</th>\n",
       "      <th>APPLICATION_TYPE_T15</th>\n",
       "      <th>APPLICATION_TYPE_T17</th>\n",
       "      <th>APPLICATION_TYPE_T19</th>\n",
       "      <th>APPLICATION_TYPE_T2</th>\n",
       "      <th>APPLICATION_TYPE_T25</th>\n",
       "      <th>APPLICATION_TYPE_T29</th>\n",
       "      <th>...</th>\n",
       "      <th>INCOME_AMT_1-9999</th>\n",
       "      <th>INCOME_AMT_10000-24999</th>\n",
       "      <th>INCOME_AMT_100000-499999</th>\n",
       "      <th>INCOME_AMT_10M-50M</th>\n",
       "      <th>INCOME_AMT_1M-5M</th>\n",
       "      <th>INCOME_AMT_25000-99999</th>\n",
       "      <th>INCOME_AMT_50M+</th>\n",
       "      <th>INCOME_AMT_5M-10M</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_N</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 114 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   APPLICATION_TYPE_T10  APPLICATION_TYPE_T12  APPLICATION_TYPE_T13  \\\n",
       "0                   1.0                   0.0                   0.0   \n",
       "1                   0.0                   0.0                   0.0   \n",
       "2                   0.0                   0.0                   0.0   \n",
       "3                   0.0                   0.0                   0.0   \n",
       "4                   0.0                   0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T14  APPLICATION_TYPE_T15  APPLICATION_TYPE_T17  \\\n",
       "0                   0.0                   0.0                   0.0   \n",
       "1                   0.0                   0.0                   0.0   \n",
       "2                   0.0                   0.0                   0.0   \n",
       "3                   0.0                   0.0                   0.0   \n",
       "4                   0.0                   0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T19  APPLICATION_TYPE_T2  APPLICATION_TYPE_T25  \\\n",
       "0                   0.0                  0.0                   0.0   \n",
       "1                   0.0                  0.0                   0.0   \n",
       "2                   0.0                  0.0                   0.0   \n",
       "3                   0.0                  0.0                   0.0   \n",
       "4                   0.0                  0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T29  ...  INCOME_AMT_1-9999  INCOME_AMT_10000-24999  \\\n",
       "0                   0.0  ...                0.0                     0.0   \n",
       "1                   0.0  ...                1.0                     0.0   \n",
       "2                   0.0  ...                0.0                     0.0   \n",
       "3                   0.0  ...                0.0                     1.0   \n",
       "4                   0.0  ...                0.0                     0.0   \n",
       "\n",
       "   INCOME_AMT_100000-499999  INCOME_AMT_10M-50M  INCOME_AMT_1M-5M  \\\n",
       "0                       0.0                 0.0               0.0   \n",
       "1                       0.0                 0.0               0.0   \n",
       "2                       0.0                 0.0               0.0   \n",
       "3                       0.0                 0.0               0.0   \n",
       "4                       1.0                 0.0               0.0   \n",
       "\n",
       "   INCOME_AMT_25000-99999  INCOME_AMT_50M+  INCOME_AMT_5M-10M  \\\n",
       "0                     0.0              0.0                0.0   \n",
       "1                     0.0              0.0                0.0   \n",
       "2                     0.0              0.0                0.0   \n",
       "3                     0.0              0.0                0.0   \n",
       "4                     0.0              0.0                0.0   \n",
       "\n",
       "   SPECIAL_CONSIDERATIONS_N  SPECIAL_CONSIDERATIONS_Y  \n",
       "0                       1.0                       0.0  \n",
       "1                       1.0                       0.0  \n",
       "2                       1.0                       0.0  \n",
       "3                       1.0                       0.0  \n",
       "4                       1.0                       0.0  \n",
       "\n",
       "[5 rows x 114 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a DataFrame with the encoded variables\n",
    "encoded_df = pd.DataFrame(encoded_data,\n",
    "                          columns = enc.get_feature_names_out(categorical_variables)\n",
    "                         )\n",
    "\n",
    "# Review the DataFrame\n",
    "encoded_df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Add the original DataFrame’s numerical variables to the DataFrame containing the encoded variables.\n",
    "\n",
    "> **Note** To complete this step, you will employ the Pandas `concat()` function that was introduced earlier in this course. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>APPLICATION_TYPE_T10</th>\n",
       "      <th>APPLICATION_TYPE_T12</th>\n",
       "      <th>APPLICATION_TYPE_T13</th>\n",
       "      <th>APPLICATION_TYPE_T14</th>\n",
       "      <th>APPLICATION_TYPE_T15</th>\n",
       "      <th>APPLICATION_TYPE_T17</th>\n",
       "      <th>APPLICATION_TYPE_T19</th>\n",
       "      <th>APPLICATION_TYPE_T2</th>\n",
       "      <th>APPLICATION_TYPE_T25</th>\n",
       "      <th>APPLICATION_TYPE_T29</th>\n",
       "      <th>...</th>\n",
       "      <th>INCOME_AMT_100000-499999</th>\n",
       "      <th>INCOME_AMT_10M-50M</th>\n",
       "      <th>INCOME_AMT_1M-5M</th>\n",
       "      <th>INCOME_AMT_25000-99999</th>\n",
       "      <th>INCOME_AMT_50M+</th>\n",
       "      <th>INCOME_AMT_5M-10M</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_N</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_Y</th>\n",
       "      <th>STATUS</th>\n",
       "      <th>ASK_AMT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>108590</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>6692</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>142590</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 116 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   APPLICATION_TYPE_T10  APPLICATION_TYPE_T12  APPLICATION_TYPE_T13  \\\n",
       "0                   1.0                   0.0                   0.0   \n",
       "1                   0.0                   0.0                   0.0   \n",
       "2                   0.0                   0.0                   0.0   \n",
       "3                   0.0                   0.0                   0.0   \n",
       "4                   0.0                   0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T14  APPLICATION_TYPE_T15  APPLICATION_TYPE_T17  \\\n",
       "0                   0.0                   0.0                   0.0   \n",
       "1                   0.0                   0.0                   0.0   \n",
       "2                   0.0                   0.0                   0.0   \n",
       "3                   0.0                   0.0                   0.0   \n",
       "4                   0.0                   0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T19  APPLICATION_TYPE_T2  APPLICATION_TYPE_T25  \\\n",
       "0                   0.0                  0.0                   0.0   \n",
       "1                   0.0                  0.0                   0.0   \n",
       "2                   0.0                  0.0                   0.0   \n",
       "3                   0.0                  0.0                   0.0   \n",
       "4                   0.0                  0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T29  ...  INCOME_AMT_100000-499999  INCOME_AMT_10M-50M  \\\n",
       "0                   0.0  ...                       0.0                 0.0   \n",
       "1                   0.0  ...                       0.0                 0.0   \n",
       "2                   0.0  ...                       0.0                 0.0   \n",
       "3                   0.0  ...                       0.0                 0.0   \n",
       "4                   0.0  ...                       1.0                 0.0   \n",
       "\n",
       "   INCOME_AMT_1M-5M  INCOME_AMT_25000-99999  INCOME_AMT_50M+  \\\n",
       "0               0.0                     0.0              0.0   \n",
       "1               0.0                     0.0              0.0   \n",
       "2               0.0                     0.0              0.0   \n",
       "3               0.0                     0.0              0.0   \n",
       "4               0.0                     0.0              0.0   \n",
       "\n",
       "   INCOME_AMT_5M-10M  SPECIAL_CONSIDERATIONS_N  SPECIAL_CONSIDERATIONS_Y  \\\n",
       "0                0.0                       1.0                       0.0   \n",
       "1                0.0                       1.0                       0.0   \n",
       "2                0.0                       1.0                       0.0   \n",
       "3                0.0                       1.0                       0.0   \n",
       "4                0.0                       1.0                       0.0   \n",
       "\n",
       "   STATUS  ASK_AMT  \n",
       "0       1     5000  \n",
       "1       1   108590  \n",
       "2       1     5000  \n",
       "3       1     6692  \n",
       "4       1   142590  \n",
       "\n",
       "[5 rows x 116 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add the numerical variables from the original DataFrame to the one-hot encoding DataFrame\n",
    "encoded_df = pd.concat([encoded_df, applicant_data_df.filter([\"STATUS\",\"ASK_AMT\"])], axis = 1)\n",
    "\n",
    "# Review the Dataframe\n",
    "encoded_df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5: Using the preprocessed data, create the features (`X`) and target (`y`) datasets. The target dataset should be defined by the preprocessed DataFrame column “IS_SUCCESSFUL”. The remaining columns should define the features dataset. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    1\n",
       "2    0\n",
       "3    1\n",
       "4    1\n",
       "Name: IS_SUCCESSFUL, dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the target set y using the IS_SUCCESSFUL column\n",
    "y = applicant_data_df[\"IS_SUCCESSFUL\"]\n",
    "\n",
    "# Display a sample of y\n",
    "y[:5]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>APPLICATION_TYPE_T10</th>\n",
       "      <th>APPLICATION_TYPE_T12</th>\n",
       "      <th>APPLICATION_TYPE_T13</th>\n",
       "      <th>APPLICATION_TYPE_T14</th>\n",
       "      <th>APPLICATION_TYPE_T15</th>\n",
       "      <th>APPLICATION_TYPE_T17</th>\n",
       "      <th>APPLICATION_TYPE_T19</th>\n",
       "      <th>APPLICATION_TYPE_T2</th>\n",
       "      <th>APPLICATION_TYPE_T25</th>\n",
       "      <th>APPLICATION_TYPE_T29</th>\n",
       "      <th>...</th>\n",
       "      <th>INCOME_AMT_100000-499999</th>\n",
       "      <th>INCOME_AMT_10M-50M</th>\n",
       "      <th>INCOME_AMT_1M-5M</th>\n",
       "      <th>INCOME_AMT_25000-99999</th>\n",
       "      <th>INCOME_AMT_50M+</th>\n",
       "      <th>INCOME_AMT_5M-10M</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_N</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_Y</th>\n",
       "      <th>STATUS</th>\n",
       "      <th>ASK_AMT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>108590</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>6692</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>142590</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 116 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   APPLICATION_TYPE_T10  APPLICATION_TYPE_T12  APPLICATION_TYPE_T13  \\\n",
       "0                   1.0                   0.0                   0.0   \n",
       "1                   0.0                   0.0                   0.0   \n",
       "2                   0.0                   0.0                   0.0   \n",
       "3                   0.0                   0.0                   0.0   \n",
       "4                   0.0                   0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T14  APPLICATION_TYPE_T15  APPLICATION_TYPE_T17  \\\n",
       "0                   0.0                   0.0                   0.0   \n",
       "1                   0.0                   0.0                   0.0   \n",
       "2                   0.0                   0.0                   0.0   \n",
       "3                   0.0                   0.0                   0.0   \n",
       "4                   0.0                   0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T19  APPLICATION_TYPE_T2  APPLICATION_TYPE_T25  \\\n",
       "0                   0.0                  0.0                   0.0   \n",
       "1                   0.0                  0.0                   0.0   \n",
       "2                   0.0                  0.0                   0.0   \n",
       "3                   0.0                  0.0                   0.0   \n",
       "4                   0.0                  0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T29  ...  INCOME_AMT_100000-499999  INCOME_AMT_10M-50M  \\\n",
       "0                   0.0  ...                       0.0                 0.0   \n",
       "1                   0.0  ...                       0.0                 0.0   \n",
       "2                   0.0  ...                       0.0                 0.0   \n",
       "3                   0.0  ...                       0.0                 0.0   \n",
       "4                   0.0  ...                       1.0                 0.0   \n",
       "\n",
       "   INCOME_AMT_1M-5M  INCOME_AMT_25000-99999  INCOME_AMT_50M+  \\\n",
       "0               0.0                     0.0              0.0   \n",
       "1               0.0                     0.0              0.0   \n",
       "2               0.0                     0.0              0.0   \n",
       "3               0.0                     0.0              0.0   \n",
       "4               0.0                     0.0              0.0   \n",
       "\n",
       "   INCOME_AMT_5M-10M  SPECIAL_CONSIDERATIONS_N  SPECIAL_CONSIDERATIONS_Y  \\\n",
       "0                0.0                       1.0                       0.0   \n",
       "1                0.0                       1.0                       0.0   \n",
       "2                0.0                       1.0                       0.0   \n",
       "3                0.0                       1.0                       0.0   \n",
       "4                0.0                       1.0                       0.0   \n",
       "\n",
       "   STATUS  ASK_AMT  \n",
       "0       1     5000  \n",
       "1       1   108590  \n",
       "2       1     5000  \n",
       "3       1     6692  \n",
       "4       1   142590  \n",
       "\n",
       "[5 rows x 116 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define features set X by selecting all columns but IS_SUCCESSFUL\n",
    "X = encoded_df.copy()\n",
    "\n",
    "# Review the features DataFrame\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 6: Split the features and target sets into training and testing datasets.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split the preprocessed data into a training and testing dataset\n",
    "# Assign the function a random_state equal to 1\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 7: Use scikit-learn's `StandardScaler` to scale the features data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a StandardScaler instance\n",
    "scaler = StandardScaler()\n",
    "\n",
    "# Fit the scaler to the features training dataset\n",
    "X_scaler = scaler.fit (X_train)\n",
    "\n",
    "# Fit the scaler to the features training dataset\n",
    "X_train_scaled = X_scaler.transform(X_train)\n",
    "X_test_scaled = X_scaler.transform(X_test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Compile and Evaluate a Binary Classification Model Using a Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Step 1: Create a deep neural network by assigning the number of input features, the number of layers, and the number of neurons on each layer using Tensorflow’s Keras.\n",
    "\n",
    "> **Hint** You can start with a two-layer deep neural network model that uses the `relu` activation function for both layers.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "116"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the the number of inputs (features) to the model\n",
    "number_input_features = len(X_train.iloc[0])\n",
    "\n",
    "# Review the number of features\n",
    "number_input_features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the number of neurons in the output layer\n",
    "number_output_neurons = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "58"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the number of hidden nodes for the first hidden layer\n",
    "hidden_nodes_layer1 = (number_input_features + 1)//2\n",
    "\n",
    "# Review the number hidden nodes in the first layer\n",
    "hidden_nodes_layer1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the number of hidden nodes for the second hidden layer\n",
    "hidden_nodes_layer2 = (hidden_nodes_layer1 + 1)//2\n",
    "\n",
    "# Review the number hidden nodes in the second layer\n",
    "hidden_nodes_layer2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the Sequential model instance\n",
    "nn = Sequential()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add the first hidden layer\n",
    "nn.add(Dense(units = hidden_nodes_layer1, activation = \"relu\", input_dim = number_input_features))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add the second hidden layer\n",
    "nn.add(Dense(units = hidden_nodes_layer2, activation = \"relu\"))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add the output layer to the model specifying the number of output neurons and activation function\n",
    "nn.add(Dense(units = number_output_neurons, activation = \"sigmoid\"))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense (Dense)               (None, 58)                6786      \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 29)                1711      \n",
      "                                                                 \n",
      " dense_2 (Dense)             (None, 1)                 30        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 8,527\n",
      "Trainable params: 8,527\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Display the Sequential model summary\n",
    "nn.summary()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Step 2: Compile and fit the model using the `binary_crossentropy` loss function, the `adam` optimizer, and the `accuracy` evaluation metric.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compile the Sequential model\n",
    "nn.compile(loss = \"binary_crossentropy\", optimizer = \"adam\", metrics =[\"accuracy\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "804/804 [==============================] - 1s 832us/step - loss: 0.5791 - accuracy: 0.7187\n",
      "Epoch 2/50\n",
      "804/804 [==============================] - 1s 840us/step - loss: 0.5537 - accuracy: 0.7309\n",
      "Epoch 3/50\n",
      "804/804 [==============================] - 1s 916us/step - loss: 0.5496 - accuracy: 0.7299\n",
      "Epoch 4/50\n",
      "804/804 [==============================] - 1s 905us/step - loss: 0.5478 - accuracy: 0.7319\n",
      "Epoch 5/50\n",
      "804/804 [==============================] - 1s 840us/step - loss: 0.5468 - accuracy: 0.7336\n",
      "Epoch 6/50\n",
      "804/804 [==============================] - 1s 836us/step - loss: 0.5450 - accuracy: 0.7326\n",
      "Epoch 7/50\n",
      "804/804 [==============================] - 1s 861us/step - loss: 0.5442 - accuracy: 0.7353\n",
      "Epoch 8/50\n",
      "804/804 [==============================] - 1s 838us/step - loss: 0.5434 - accuracy: 0.7330\n",
      "Epoch 9/50\n",
      "804/804 [==============================] - 1s 872us/step - loss: 0.5432 - accuracy: 0.7338\n",
      "Epoch 10/50\n",
      "804/804 [==============================] - 1s 889us/step - loss: 0.5417 - accuracy: 0.7352\n",
      "Epoch 11/50\n",
      "804/804 [==============================] - 1s 881us/step - loss: 0.5419 - accuracy: 0.7341\n",
      "Epoch 12/50\n",
      "804/804 [==============================] - 1s 858us/step - loss: 0.5409 - accuracy: 0.7352\n",
      "Epoch 13/50\n",
      "804/804 [==============================] - 1s 890us/step - loss: 0.5404 - accuracy: 0.7350\n",
      "Epoch 14/50\n",
      "804/804 [==============================] - 1s 891us/step - loss: 0.5399 - accuracy: 0.7357\n",
      "Epoch 15/50\n",
      "804/804 [==============================] - 1s 899us/step - loss: 0.5398 - accuracy: 0.7353\n",
      "Epoch 16/50\n",
      "804/804 [==============================] - 1s 904us/step - loss: 0.5394 - accuracy: 0.7364\n",
      "Epoch 17/50\n",
      "804/804 [==============================] - 1s 863us/step - loss: 0.5390 - accuracy: 0.7367\n",
      "Epoch 18/50\n",
      "804/804 [==============================] - 1s 826us/step - loss: 0.5381 - accuracy: 0.7368\n",
      "Epoch 19/50\n",
      "804/804 [==============================] - 1s 849us/step - loss: 0.5385 - accuracy: 0.7359\n",
      "Epoch 20/50\n",
      "804/804 [==============================] - 1s 813us/step - loss: 0.5383 - accuracy: 0.7375\n",
      "Epoch 21/50\n",
      "804/804 [==============================] - 1s 844us/step - loss: 0.5376 - accuracy: 0.7378\n",
      "Epoch 22/50\n",
      "804/804 [==============================] - 1s 902us/step - loss: 0.5368 - accuracy: 0.7387\n",
      "Epoch 23/50\n",
      "804/804 [==============================] - 1s 913us/step - loss: 0.5372 - accuracy: 0.7383\n",
      "Epoch 24/50\n",
      "804/804 [==============================] - 1s 916us/step - loss: 0.5369 - accuracy: 0.7387\n",
      "Epoch 25/50\n",
      "804/804 [==============================] - 1s 919us/step - loss: 0.5362 - accuracy: 0.7385\n",
      "Epoch 26/50\n",
      "804/804 [==============================] - 1s 905us/step - loss: 0.5355 - accuracy: 0.7396\n",
      "Epoch 27/50\n",
      "804/804 [==============================] - 1s 886us/step - loss: 0.5358 - accuracy: 0.7382\n",
      "Epoch 28/50\n",
      "804/804 [==============================] - 1s 931us/step - loss: 0.5354 - accuracy: 0.7381\n",
      "Epoch 29/50\n",
      "804/804 [==============================] - 1s 873us/step - loss: 0.5350 - accuracy: 0.7385\n",
      "Epoch 30/50\n",
      "804/804 [==============================] - 1s 858us/step - loss: 0.5353 - accuracy: 0.7376\n",
      "Epoch 31/50\n",
      "804/804 [==============================] - 1s 821us/step - loss: 0.5353 - accuracy: 0.7381\n",
      "Epoch 32/50\n",
      "804/804 [==============================] - 1s 847us/step - loss: 0.5342 - accuracy: 0.7384\n",
      "Epoch 33/50\n",
      "804/804 [==============================] - 1s 820us/step - loss: 0.5343 - accuracy: 0.7392\n",
      "Epoch 34/50\n",
      "804/804 [==============================] - 1s 910us/step - loss: 0.5342 - accuracy: 0.7399\n",
      "Epoch 35/50\n",
      "804/804 [==============================] - 1s 893us/step - loss: 0.5345 - accuracy: 0.7391\n",
      "Epoch 36/50\n",
      "804/804 [==============================] - 1s 889us/step - loss: 0.5342 - accuracy: 0.7394\n",
      "Epoch 37/50\n",
      "804/804 [==============================] - 1s 914us/step - loss: 0.5339 - accuracy: 0.7403\n",
      "Epoch 38/50\n",
      "804/804 [==============================] - 1s 905us/step - loss: 0.5341 - accuracy: 0.7386\n",
      "Epoch 39/50\n",
      "804/804 [==============================] - 1s 897us/step - loss: 0.5336 - accuracy: 0.7401\n",
      "Epoch 40/50\n",
      "804/804 [==============================] - 1s 887us/step - loss: 0.5330 - accuracy: 0.7402\n",
      "Epoch 41/50\n",
      "804/804 [==============================] - 1s 856us/step - loss: 0.5331 - accuracy: 0.7390\n",
      "Epoch 42/50\n",
      "804/804 [==============================] - 1s 879us/step - loss: 0.5332 - accuracy: 0.7407\n",
      "Epoch 43/50\n",
      "804/804 [==============================] - 1s 855us/step - loss: 0.5337 - accuracy: 0.7406\n",
      "Epoch 44/50\n",
      "804/804 [==============================] - 1s 873us/step - loss: 0.5327 - accuracy: 0.7407\n",
      "Epoch 45/50\n",
      "804/804 [==============================] - 1s 900us/step - loss: 0.5329 - accuracy: 0.7404\n",
      "Epoch 46/50\n",
      "804/804 [==============================] - 1s 917us/step - loss: 0.5325 - accuracy: 0.7416\n",
      "Epoch 47/50\n",
      "804/804 [==============================] - 1s 854us/step - loss: 0.5317 - accuracy: 0.7411\n",
      "Epoch 48/50\n",
      "804/804 [==============================] - 1s 927us/step - loss: 0.5320 - accuracy: 0.7395\n",
      "Epoch 49/50\n",
      "804/804 [==============================] - 1s 854us/step - loss: 0.5325 - accuracy: 0.7407\n",
      "Epoch 50/50\n",
      "804/804 [==============================] - 1s 953us/step - loss: 0.5322 - accuracy: 0.7412\n"
     ]
    }
   ],
   "source": [
    "# Fit the model using 50 epochs and the training data\n",
    "model = nn.fit(X_train_scaled, y_train, epochs = 50)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Evaluate the model using the test data to determine the model’s loss and accuracy.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "268/268 - 0s - loss: 0.5541 - accuracy: 0.7300 - 286ms/epoch - 1ms/step\n",
      "Loss: 0.5540708303451538, Accuracy: 0.7300291657447815\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss, model_accuracy = nn.evaluate(X_test_scaled, y_test, verbose = 2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss}, Accuracy: {model_accuracy}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Save and export your model to an HDF5 file, and name the file `AlphabetSoup.h5`. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the model's file path\n",
    "file_path =  Path(\"./Resources/AlphabetSoup.h5\")\n",
    "\n",
    "# Export your model to a HDF5 file\n",
    "nn.save_weights(file_path)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Optimize the neural network model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Define at least three new deep neural network models (resulting in the original plus 3 optimization attempts). With each, try to improve on your first model’s predictive accuracy.\n",
    "\n",
    "> **Rewind** Recall that perfect accuracy has a value of 1, so accuracy improves as its value moves closer to 1. To optimize your model for a predictive accuracy as close to 1 as possible, you can use any or all of the following techniques:\n",
    ">\n",
    "> * Adjust the input data by dropping different features columns to ensure that no variables or outliers confuse the model.\n",
    ">\n",
    "> * Add more neurons (nodes) to a hidden layer.\n",
    ">\n",
    "> * Add more hidden layers.\n",
    ">\n",
    "> * Use different activation functions for the hidden layers.\n",
    ">\n",
    "> * Add to or reduce the number of epochs in the training regimen.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Alternative Model 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "116"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the the number of inputs (features) to the model\n",
    "number_input_features = len(X_train.iloc[0])\n",
    "\n",
    "# Review the number of features\n",
    "number_input_features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the number of neurons in the output layer\n",
    "number_output_neurons_A1 = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "117"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the number of hidden nodes for the first hidden layer\n",
    "hidden_nodes_layer1_A1 = number_input_features + 1\n",
    "\n",
    "# Review the number of hidden nodes in the first layer\n",
    "hidden_nodes_layer1_A1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "59"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the number of hidden nodes for the second hidden layer\n",
    "hidden_nodes_layer2_A1 = (hidden_nodes_layer1_A1 + 1)//2\n",
    "\n",
    "# Review the number of hidden nodes in the first layer\n",
    "hidden_nodes_layer2_A1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "30"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the number of hidden nodes for the third hidden layer\n",
    "hidden_nodes_layer3_A1 = (hidden_nodes_layer2_A1 + 1)//2\n",
    "\n",
    "# Review the number of hidden nodes in the first layer\n",
    "hidden_nodes_layer3_A1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the Sequential model instance\n",
    "nn_A1 =  Sequential()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense_3 (Dense)             (None, 117)               13689     \n",
      "                                                                 \n",
      " dense_4 (Dense)             (None, 59)                6962      \n",
      "                                                                 \n",
      " dense_5 (Dense)             (None, 30)                1800      \n",
      "                                                                 \n",
      " dense_6 (Dense)             (None, 1)                 31        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 22,482\n",
      "Trainable params: 22,482\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# First hidden layer\n",
    "nn_A1.add(Dense(units = hidden_nodes_layer1_A1, activation = \"relu\", input_dim = number_input_features))\n",
    "\n",
    "# Second hidden layer\n",
    "nn_A1.add(Dense(units = hidden_nodes_layer2_A1, activation = \"relu\"))\n",
    "\n",
    "# third hidden layer\n",
    "nn_A1.add(Dense(units = hidden_nodes_layer3_A1, activation = \"relu\"))\n",
    "\n",
    "# Output layer\n",
    "nn_A1.add(Dense(units = number_output_neurons_A1, activation = \"sigmoid\"))\n",
    "\n",
    "# Check the structure of the model\n",
    "nn_A1.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compile the Sequential model\n",
    "nn_A1.compile(loss = \"binary_crossentropy\", optimizer = \"adam\", metrics =[\"accuracy\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5688 - accuracy: 0.7226\n",
      "Epoch 2/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5513 - accuracy: 0.7292\n",
      "Epoch 3/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5482 - accuracy: 0.7329\n",
      "Epoch 4/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5464 - accuracy: 0.7327\n",
      "Epoch 5/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5450 - accuracy: 0.7346\n",
      "Epoch 6/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5435 - accuracy: 0.7344\n",
      "Epoch 7/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5432 - accuracy: 0.7341\n",
      "Epoch 8/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5425 - accuracy: 0.7358\n",
      "Epoch 9/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5411 - accuracy: 0.7360\n",
      "Epoch 10/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5415 - accuracy: 0.7367\n",
      "Epoch 11/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5402 - accuracy: 0.7371\n",
      "Epoch 12/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5401 - accuracy: 0.7363\n",
      "Epoch 13/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5398 - accuracy: 0.7371\n",
      "Epoch 14/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5391 - accuracy: 0.7365\n",
      "Epoch 15/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5384 - accuracy: 0.7383\n",
      "Epoch 16/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5387 - accuracy: 0.7381\n",
      "Epoch 17/25\n",
      "804/804 [==============================] - 1s 967us/step - loss: 0.5379 - accuracy: 0.7379\n",
      "Epoch 18/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5375 - accuracy: 0.7387\n",
      "Epoch 19/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5369 - accuracy: 0.7388\n",
      "Epoch 20/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5364 - accuracy: 0.7388\n",
      "Epoch 21/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5363 - accuracy: 0.7388\n",
      "Epoch 22/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5362 - accuracy: 0.7381\n",
      "Epoch 23/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5363 - accuracy: 0.7392\n",
      "Epoch 24/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5356 - accuracy: 0.7390\n",
      "Epoch 25/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5349 - accuracy: 0.7404\n"
     ]
    }
   ],
   "source": [
    "# Fit the model using 50 epochs and the training data\n",
    "fit_model_A1 = nn_A1.fit(X_train_scaled, y_train, epochs = 25)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Alternative Model 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "116"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the the number of inputs (features) to the model\n",
    "number_input_features = len(X_train.iloc[0])\n",
    "\n",
    "# Review the number of features\n",
    "number_input_features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the number of neurons in the output layer\n",
    "number_output_neurons_A2 = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "117"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the number of hidden nodes for the first hidden layer\n",
    "hidden_nodes_layer1_A2 =  number_input_features + 1\n",
    "\n",
    "# Review the number of hidden nodes in the first layer\n",
    "hidden_nodes_layer1_A2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "80"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the number of hidden nodes for the second hidden layer\n",
    "hidden_nodes_layer2_A2 = 80\n",
    "\n",
    "# Review the number of hidden nodes in the second layer\n",
    "hidden_nodes_layer2_A2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the Sequential model instance\n",
    "nn_A2 = Sequential()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense_7 (Dense)             (None, 117)               13689     \n",
      "                                                                 \n",
      " dense_8 (Dense)             (None, 80)                9440      \n",
      "                                                                 \n",
      " dense_9 (Dense)             (None, 1)                 81        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 23,210\n",
      "Trainable params: 23,210\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# First hidden layer\n",
    "nn_A2.add(Dense(units = hidden_nodes_layer1_A2, activation = \"tanh\", input_dim = number_input_features))\n",
    "\n",
    "# Second hidden layer\n",
    "nn_A2.add(Dense(units = hidden_nodes_layer2_A2, activation = \"tanh\"))\n",
    "# YOUR CODE HERE\n",
    "\n",
    "# Output layer\n",
    "nn_A2.add(Dense(units = number_output_neurons_A2, activation = \"sigmoid\"))\n",
    "\n",
    "# Check the structure of the model\n",
    "nn_A2.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compile the model\n",
    "nn_A2.compile(loss = \"binary_crossentropy\", optimizer = \"adam\", metrics =[\"accuracy\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5732 - accuracy: 0.7190\n",
      "Epoch 2/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5556 - accuracy: 0.7287\n",
      "Epoch 3/25\n",
      "804/804 [==============================] - 1s 980us/step - loss: 0.5508 - accuracy: 0.7291\n",
      "Epoch 4/25\n",
      "804/804 [==============================] - 1s 954us/step - loss: 0.5478 - accuracy: 0.7314\n",
      "Epoch 5/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5467 - accuracy: 0.7313\n",
      "Epoch 6/25\n",
      "804/804 [==============================] - 1s 956us/step - loss: 0.5454 - accuracy: 0.7329\n",
      "Epoch 7/25\n",
      "804/804 [==============================] - 1s 933us/step - loss: 0.5438 - accuracy: 0.7339\n",
      "Epoch 8/25\n",
      "804/804 [==============================] - 1s 971us/step - loss: 0.5426 - accuracy: 0.7354\n",
      "Epoch 9/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5414 - accuracy: 0.7361\n",
      "Epoch 10/25\n",
      "804/804 [==============================] - 1s 930us/step - loss: 0.5410 - accuracy: 0.7341\n",
      "Epoch 11/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5399 - accuracy: 0.7374\n",
      "Epoch 12/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5390 - accuracy: 0.7371\n",
      "Epoch 13/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5384 - accuracy: 0.7361\n",
      "Epoch 14/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5381 - accuracy: 0.7379\n",
      "Epoch 15/25\n",
      "804/804 [==============================] - 1s 997us/step - loss: 0.5370 - accuracy: 0.7380\n",
      "Epoch 16/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5368 - accuracy: 0.7368\n",
      "Epoch 17/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5360 - accuracy: 0.7386\n",
      "Epoch 18/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5350 - accuracy: 0.7378\n",
      "Epoch 19/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5347 - accuracy: 0.7387\n",
      "Epoch 20/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5344 - accuracy: 0.7400\n",
      "Epoch 21/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5339 - accuracy: 0.7409\n",
      "Epoch 22/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5334 - accuracy: 0.7395\n",
      "Epoch 23/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5331 - accuracy: 0.7402\n",
      "Epoch 24/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5328 - accuracy: 0.7403\n",
      "Epoch 25/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5321 - accuracy: 0.7392\n"
     ]
    }
   ],
   "source": [
    "# Fit the model\n",
    "fit_model_A2 = nn_A2.fit(X_train_scaled, y_train, epochs = 25)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: After finishing your models, display the accuracy scores achieved by each model, and compare the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original Model Results\n",
      "268/268 - 0s - loss: 0.5541 - accuracy: 0.7300 - 168ms/epoch - 627us/step\n",
      "Loss: 0.5540708303451538, Accuracy: 0.7300291657447815\n"
     ]
    }
   ],
   "source": [
    "print(\"Original Model Results\")\n",
    "\n",
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss, model_accuracy = nn.evaluate(X_test_scaled, y_test, verbose = 2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss}, Accuracy: {model_accuracy}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Alternative Model 1 Results\n",
      "268/268 - 0s - loss: 0.5531 - accuracy: 0.7329 - 239ms/epoch - 893us/step\n",
      "Loss: 0.5530715584754944, Accuracy: 0.7329446077346802\n"
     ]
    }
   ],
   "source": [
    "print(\"Alternative Model 1 Results\")\n",
    "\n",
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss, model_accuracy =nn_A1.evaluate(X_test_scaled, y_test, verbose = 2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss}, Accuracy: {model_accuracy}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Alternative Model 2 Results\n",
      "268/268 - 0s - loss: 0.5509 - accuracy: 0.7310 - 248ms/epoch - 927us/step\n",
      "Loss: 0.5508823990821838, Accuracy: 0.7309620976448059\n"
     ]
    }
   ],
   "source": [
    "print(\"Alternative Model 2 Results\")\n",
    "\n",
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss, model_accuracy = nn_A2.evaluate(X_test_scaled, y_test, verbose = 2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss}, Accuracy: {model_accuracy}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Save each of your alternative models as an HDF5 file.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the file path for the first alternative model\n",
    "file_path_A1 = Path(\"./Resources/AlphabetSoup_A1.h5\")\n",
    "\n",
    "# Export your model to a HDF5 file\n",
    "nn_A1.save_weights(file_path_A1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the file path for the second alternative model\n",
    "file_path_A2 = Path(\"./Resources/AlphabetSoup_A2.h5\")\n",
    "\n",
    "# Export your model to a HDF5 file\n",
    "nn_A2.save_weights(file_path_A2)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}