ComparingRegressionModels.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 6:  Train Various Regression Models and Compare Their Performances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import os \n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV\n",
    "from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.metrics import mean_squared_error, r2_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this lab assignment, you will train various regression models (regressors) and compare their performances. You will train, test and evaluate individual models as well as ensemble models. You will:\n",
    "\n",
    "1. Build your DataFrame and define your ML problem:\n",
    "    * Load the Airbnb \"listings\" data set\n",
    "    * Define the label - what are you predicting?\n",
    "    * Identify the features\n",
    "2. Create labeled examples from the data set.\n",
    "3. Split the data into training and test data sets.\n",
    "4. Train, test and evaluate two individual regressors.\n",
    "5. Use the stacking ensemble method to train the same regressors.\n",
    "6. Train, test and evaluate Gradient Boosted Decision Trees.\n",
    "7. Train, test and evaluate Random Forest.\n",
    "8. Visualize and compare the performance of all of the models.\n",
    "\n",
    "<font color='red'><b>Note:</font><br> \n",
    "<font color='red'><b>1. Some of the code cells in this notebook may take a while to run.</font><br>\n",
    "<font color='red'><b>2. Ignore warning messages that pertain to deprecated packages.</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1. Build Your DataFrame and Define Your ML Problem\n",
    "\n",
    "#### Load a Data Set and Save it as a Pandas DataFrame\n",
    "\n",
    "We will work with the data set ``airbnbData_train``. This data set already has all the necessary preprocessing steps implemented, including one-hot encoding of the categorical variables, scaling of all numerical variable values, and imputing missing values. It is ready for modeling.\n",
    "\n",
    "<b>Task</b>: In the code cell below, use the same method you have been using to load the data using `pd.read_csv()` and save it to DataFrame `df`.\n",
    "\n",
    "You will be working with the file named \"airbnbData_train.csv\" that is located in a folder named \"data_regressors\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "file = 'data_regressors/airbnbData_train.csv'\n",
    "df = pd.read_csv(file)\n",
    "# YOUR CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Define the Label\n",
    "\n",
    "Your goal is to train a machine learning model that predicts the price of an Airbnb listing. This is an example of supervised learning and is a regression problem. In our dataset, our label will be the `price` column and the label contains continuous values.\n",
    "\n",
    "#### Evaluation Metrics for Regressors\n",
    "\n",
    "So far, we have mostly focused on classification problems. For this assignment, we will focus on a regression problem and predict a continuous outcome. There are different evaluation metrics that are used to determine the performance of a regressor. We will use two metrics to evaluate our regressors: RMSE (root mean square error) and $R^2$ (coefficient of determination).\n",
    "\n",
    "RMSE:<br>\n",
    "RMSE finds the average difference between the predicted values and the actual values. We will compute the RMSE on the test set.  To compute the RMSE, we will use the scikit-learn ```mean_squared_error()``` function. Since RMSE finds the difference between the predicted and actual values, lower RMSE values indicate good performance - the model fits the data well and makes more accurate predictions. On the other hand, higher RSME values indicate that the model is not performing well.\n",
    "\n",
    "$R^2$:<br>\n",
    "$R^2$ is a measure of the proportion of variability in the prediction that the model was able to make using the test data. An $R^2$ value of 1 is perfect and 0 implies no explanatory value. We can use scikit-learn's ```r2_score()``` function to compute it. Since $R^2$ measures how well the model fits the data, a higher $R^2$ value indicates that good performance and a lower $R^2$ indicates that poor performance.\n",
    "\n",
    "#### Identify Features\n",
    "\n",
    "Our features will be all of the remaining columns in the dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2. Create Labeled Examples from the Data Set \n",
    "\n",
    "<b>Task</b>: In the code cell below, create labeled examples from DataFrame `df`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.drop('price', axis=1)\n",
    "y = df['price']# YOUR CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3. Create Training and Test Data Sets\n",
    "\n",
    "<b>Task</b>: In the code cell below, create training and test sets out of the labeled examples. Create a test set that is 30 percent of the size of the data set. Save the results to variables `X_train, X_test, y_train, y_test`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)# YOUR CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4: Train, Test and Evaluate Two Regression Models: Linear Regression and Decision Tree\n",
    "\n",
    "### a. Train, Test and Evaluate a Linear Regression\n",
    "\n",
    "You will use the scikit-learn `LinearRegression` class to create a linear regression model. For more information, consult the online [documentation](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html).\n",
    "\n",
    "First let's import `LinearRegression`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Task</b>: Initialize a scikit-learn `LinearRegression` model object with no arguments, and fit the model to the training data. The model object should be named `lr_model`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-1 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-1 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-1 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-1 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-1 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;LinearRegression<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.4/modules/generated/sklearn.linear_model.LinearRegression.html\">?<span>Documentation for LinearRegression</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>LinearRegression()</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "LinearRegression()"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_model = LinearRegression()\n",
    "lr_model.fit(X_train, y_train)\n",
    "# YOUR CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Task:</b> Test your model on the test set (`X_test`). Call the ``predict()`` method  to use the fitted model to generate a vector of predictions on the test set. Save the result to the variable ``y_lr_pred``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.33195527, -1.01239832, -0.13252108, ...,  0.38126116,\n",
       "       -0.4907086 , -0.94779962])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_lr_pred = lr_model.predict(X_test)\n",
    "y_lr_pred\n",
    "# Call predict() to use the fitted model to make predictions on the test data\n",
    "# YOUR CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To compute the RMSE, we will use the scikit-learn ```mean_squared_error()``` function, which computes the mean squared error between the predicted values and the actual values: ```y_lr_pred``` and```y_test```. In order to obtain the root mean squared error, we will specify the parameter `squared=False`. \n",
    "\n",
    "To compute the $R^2$, we will use the scikit-learn ```r2_score()``` function. \n",
    "\n",
    "<b>Task</b>: In the code cell below, do the following:\n",
    "\n",
    "1. Call the `mean_squared_error()` function with arguments `y_test` and `y_lr_pred` and the parameter `squared=False` to find the RMSE. Save your result to the variable `lr_rmse`.\n",
    "\n",
    "2. Call the `r2_score()` function with the arguments `y_test` and `y_lr_pred`.  Save the result to the variable `lr_r2`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LR] Root Mean Squared Error: 0.7057693639474409\n",
      "[LR] R2: 0.4771223060789187\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/.pyenv/versions/3.9.19/lib/python3.9/site-packages/sklearn/metrics/_regression.py:483: FutureWarning: 'squared' is deprecated in version 1.4 and will be removed in 1.6. To calculate the root mean squared error, use the function'root_mean_squared_error'.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "# 1. Compute the RMSE using mean_squared_error()\n",
    "# YOUR CODE HERE\n",
    "lr_rmse = mean_squared_error(y_test, y_lr_pred, squared=False)\n",
    "\n",
    "# 2. Compute the R2 score using r2_score()\n",
    "# YOUR CODE HERE\n",
    "lr_r2 = r2_score(y_test, y_lr_pred)\n",
    "\n",
    "print('[LR] Root Mean Squared Error: {0}'.format(lr_rmse))\n",
    "print('[LR] R2: {0}'.format(lr_r2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### b. Train, Test and Evaluate a Decision Tree Using GridSearch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will use the scikit-learn `DecisionTreeRegressor` class to create a decision tree regressor. For more information, consult the online [documentation](https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html).\n",
    "\n",
    "First let's import `DecisionTreeRegressor`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set Up a Parameter Grid \n",
    "\n",
    "<b>Task</b>: Create a dictionary called `param_grid` that contains possible hyperparameter values for `max_depth` and `min_samples_leaf`. The dictionary should contain the following key/value pairs:\n",
    "\n",
    "* a key called 'max_depth' with a value which is a list consisting of the integers 4 and 8\n",
    "* a key called 'min_samples_leaf' with a value which is a list consisting of the integers 25 and 50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "param_grid = {\n",
    "    'max_depth': [4, 8],\n",
    "    'min_samples_leaf': [25, 50]\n",
    "}# YOUR CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Task:</b> Use `GridSearchCV` to fit a grid of decision tree regressors and search over the different values of hyperparameters `max_depth` and `min_samples_leaf` to find the ones that results in the best 3-fold cross-validation (CV) score.\n",
    "\n",
    "\n",
    "You will pass the following arguments to `GridSearchCV()`:\n",
    "\n",
    "1. A decision tree **regressor** model object.\n",
    "2. The `param_grid` variable.\n",
    "3. The number of folds (`cv=3`).\n",
    "4. The scoring method `scoring='neg_root_mean_squared_error'`. Note that `neg_root_mean_squared_error` returns the negative RMSE.\n",
    "\n",
    "\n",
    "Complete the code in the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running Grid Search...\n",
      "Done\n"
     ]
    }
   ],
   "source": [
    "print('Running Grid Search...')\n",
    "\n",
    "# 1. Create a DecisionTreeRegressor model object without supplying arguments. \n",
    "#    Save the model object to the variable 'dt_regressor'\n",
    "\n",
    "dt_regressor = DecisionTreeRegressor()# YOUR CODE HERE\n",
    "\n",
    "\n",
    "# 2. Run a Grid Search with 3-fold cross-validation and assign the output to the object 'dt_grid'.\n",
    "#    * Pass the model and the parameter grid to GridSearchCV()\n",
    "#    * Set the number of folds to 3\n",
    "#    * Specify the scoring method\n",
    "\n",
    "dt_grid = GridSearchCV(\n",
    "    estimator=dt_regressor,\n",
    "    param_grid=param_grid,\n",
    "    cv=3,\n",
    "    scoring='neg_root_mean_squared_error'\n",
    ")# YOUR CODE HERE\n",
    "\n",
    "\n",
    "# 3. Fit the model (use the 'grid' variable) on the training data and assign the fitted model to the \n",
    "#    variable 'dt_grid_search'\n",
    "\n",
    "dt_grid_search = dt_grid.fit(X_train, y_train)# YOUR CODE HERE\n",
    "\n",
    "print('Done')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code cell below prints the RMSE score of the best model using the `best_score_` attribute of the fitted grid search object `dt_grid_search`. Note that specifying a scoring method of `neg_root_mean_squared_error` will result in the negative RMSE, so we will multiply `dt_grid_search.best_score` by -1 to obtain the RMSE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[DT] RMSE for the best model is : 0.73\n"
     ]
    }
   ],
   "source": [
    "rmse_DT = -1 * dt_grid_search.best_score_\n",
    "print(\"[DT] RMSE for the best model is : {:.2f}\".format(rmse_DT) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Task</b>: In the code cell below, obtain the best model hyperparameters identified by the grid search and save them to the variable `dt_best_params`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'max_depth': 8, 'min_samples_leaf': 25}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dt_best_params = dt_grid_search.best_params_# YOUR CODE HERE\n",
    "\n",
    "dt_best_params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Task</b>: In the code cell below, initialize a `DecisionTreeRegressor` model object, supplying the best values of hyperparameters `max_depth` and `min_samples_leaf` as arguments.  Name the model object `dt_model`. Then fit the model `dt_model` to the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-2 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-2 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-2 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-2 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-2 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-2 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-2 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-2 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-2 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>DecisionTreeRegressor(max_depth=8, min_samples_leaf=25)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;DecisionTreeRegressor<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.4/modules/generated/sklearn.tree.DecisionTreeRegressor.html\">?<span>Documentation for DecisionTreeRegressor</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>DecisionTreeRegressor(max_depth=8, min_samples_leaf=25)</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "DecisionTreeRegressor(max_depth=8, min_samples_leaf=25)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_max_depth = 8\n",
    "best_min_samples_leaf = 25\n",
    "dt_model = DecisionTreeRegressor(max_depth=best_max_depth, min_samples_leaf=best_min_samples_leaf)\n",
    "dt_model.fit(X_train, y_train)\n",
    "# YOUR CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Task:</b> Test your model `dt_model` on the test set `X_test`. Call the ``predict()`` method  to use the fitted model to generate a vector of predictions on the test set. Save the result to the variable ``y_dt_pred``. Evaluate the results by computing the RMSE and R2 score in the same manner as you did above. Save the results to the variables `dt_rmse` and `dt_r2`.\n",
    "\n",
    "Complete the code in the cell below to accomplish this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[DT] Root Mean Squared Error: 0.7071794065452084\n",
      "[DT] R2: 0.4750309250756932\n"
     ]
    }
   ],
   "source": [
    "# 1. Use the fitted model to make predictions on the test data\n",
    "# YOUR CODE HERE\n",
    "\n",
    "y_dt_pred = dt_model.predict(X_test)\n",
    "# 2. Compute the RMSE using mean_squared_error()\n",
    "# YOUR CODE HERE\n",
    "dt_rmse = np.sqrt(mean_squared_error(y_test, y_dt_pred))\n",
    "\n",
    "# 3. Compute the R2 score using r2_score()\n",
    "# YOUR CODE HERE\n",
    "\n",
    "dt_r2 = r2_score(y_test, y_dt_pred)\n",
    "print('[DT] Root Mean Squared Error: {0}'.format(dt_rmse))\n",
    "print('[DT] R2: {0}'.format(dt_r2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 5: Train, Test and Evaluate Ensemble Models: Stacking "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will use the stacking ensemble method to train two regression models. You will use the scikit-learn `StackingRegressor` class. For more information, consult the online [documentation](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.StackingRegressor.html).\n",
    "\n",
    "First let's import `StackingRegressor`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import StackingRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this part of the assignment, we will use two models jointly. In the code cell below, we creates a list of tuples, each consisting of a scikit-learn model function and the corresponding shorthand name that we choose. We will specify the hyperparameters for the decision tree that we determined through the grid search above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "estimators = [(\"DT\", DecisionTreeRegressor(max_depth=8, min_samples_leaf=25)),\n",
    "              (\"LR\", LinearRegression())\n",
    "             ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Task</b>: \n",
    "\n",
    "\n",
    "1. Create a `StackingRegressor` model object. Call `StackingRegressor()` with the following parameters:\n",
    "    * Assign the list `estimators` to the parameter `estimators`.\n",
    "    * Use the parameter 'passthrough=False'. \n",
    "Assign the results to the variable `stacking_model`.\n",
    "\n",
    "2. Fit `stacking_model` to the training data.\n",
    "\n",
    "As you read up on the definition of the `StackingRegressor` class, you will notice that by default, the results of each model are combined using a ridge regression (a \"final regressor\")."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Implement Stacking...\n",
      "End\n"
     ]
    }
   ],
   "source": [
    "print('Implement Stacking...')\n",
    "\n",
    "# YOUR CODE HERE\n",
    "base_estimators = [\n",
    "    ('dt', DecisionTreeRegressor()),\n",
    "    ('rf', RandomForestRegressor())\n",
    "]\n",
    "stacking_model = StackingRegressor(\n",
    "    estimators=base_estimators,\n",
    "    final_estimator=LinearRegression(),\n",
    "    passthrough=False\n",
    ")\n",
    "stacking_model.fit(X_train, y_train)\n",
    "\n",
    "print('End')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Task:</b> Use the `predict()` method to test your ensemble model `stacking_model` on the test set (`X_test`). Save the result to the variable `stacking_pred`. Evaluate the results by computing the RMSE and R2 score. Save the results to the variables `stack_rmse` and `stack_r2`.\n",
    "\n",
    "Complete the code in the cell below to accomplish this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Root Mean Squared Error: 0.6100138723273814\n",
      "R2: 0.6093804882710125\n"
     ]
    }
   ],
   "source": [
    "# 1. Use the fitted model to make predictions on the test data\n",
    "# YOUR CODE HERE\n",
    "stacking_pred = stacking_model.predict(X_test)\n",
    "\n",
    "# 2. Compute the RMSE \n",
    "# YOUR CODE HERE\n",
    "stack_rmse = np.sqrt(mean_squared_error(y_test, stacking_pred))\n",
    "\n",
    "# 3. Compute the R2 score\n",
    "# YOUR CODE HERE\n",
    "stack_r2 = r2_score(y_test, stacking_pred)\n",
    "   \n",
    "print('Root Mean Squared Error: {0}'.format(stack_rmse))\n",
    "print('R2: {0}'.format(stack_r2))                       "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 6: Train, Test and Evaluate  Evaluate Ensemble Models: Gradient Boosted Decision Trees \n",
    "\n",
    "You will use the scikit-learn `GradientBoostingRegressor` class to create a gradient boosted decision tree. For more information, consult the online [documentation](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html).\n",
    "\n",
    "First let's import `GradientBoostingRegressor`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import GradientBoostingRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's assume you already performed a grid search to find the best model hyperparameters for your gradient boosted decision tree. (We are omitting this step to save computation time.) The best values are: `max_depth=2`, and `n_estimators = 300`. \n",
    "\n",
    "<b>Task</b>: Initialize a `GradientBoostingRegressor` model object with the above values as arguments. Save the result to the variable `gbdt_model`. Fit the `gbdt_model` model to the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Begin GBDT Implementation...\n",
      "End\n"
     ]
    }
   ],
   "source": [
    "print('Begin GBDT Implementation...')\n",
    "best_max_depth = 2\n",
    "best_n_estimators = 300\n",
    "\n",
    "gbdt_model = GradientBoostingRegressor(max_depth=best_max_depth, n_estimators=best_n_estimators)\n",
    "gbdt_model.fit(X_train, y_train)\n",
    "# YOUR CODE HERE\n",
    "\n",
    "print('End')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Task:</b> Use the `predict()` method to test your model `gbdt_model` on the test set `X_test`. Save the result to the variable ``y_gbdt_pred``. Evaluate the results by computing the RMSE and R2 score in the same manner as you did above. Save the results to the variables `gbdt_rmse` and `gbdt_r2`.\n",
    "\n",
    "Complete the code in the cell below to accomplish this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[GBDT] Root Mean Squared Error: 0.6304269561103651\n",
      "[GBDT] R2: 0.5828002309839375\n"
     ]
    }
   ],
   "source": [
    "# 1. Use the fitted model to make predictions on the test data\n",
    "# YOUR CODE HERE\n",
    "y_gbdt_pred = gbdt_model.predict(X_test)\n",
    "\n",
    "# 2. Compute the RMSE \n",
    "# YOUR CODE HERE\n",
    "gbdt_rmse = np.sqrt(mean_squared_error(y_test, y_gbdt_pred))\n",
    "\n",
    "\n",
    "# 3. Compute the R2 score \n",
    "# YOUR CODE HERE\n",
    "gbdt_r2 = r2_score(y_test, y_gbdt_pred)\n",
    "\n",
    "print('[GBDT] Root Mean Squared Error: {0}'.format(gbdt_rmse))\n",
    "print('[GBDT] R2: {0}'.format(gbdt_r2))                 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 7: Train, Test and Evaluate  Ensemble Models: Random Forest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will use the scikit-learn `RandomForestRegressor` class to create a gradient boosted decision tree. For more information, consult the online [documentation](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html).\n",
    "\n",
    "First let's import `RandomForestRegressor`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's assume you already performed a grid search to find the best model hyperparameters for your random forest model. (We are omitting this step to save computation time.) The best values are: `max_depth=32`, and `n_estimators = 300`. \n",
    "\n",
    "<b>Task</b>: Initialize a `RandomForestRegressor` model object with the above values as arguments. Save the result to the variable `rf_model`. Fit the `rf_model` model to the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Begin RF Implementation...\n",
      "End\n"
     ]
    }
   ],
   "source": [
    "print('Begin RF Implementation...')\n",
    "\n",
    "# YOUR CODE HERE\n",
    "rf_model = RandomForestRegressor(max_depth=32, n_estimators=300)\n",
    "rf_model.fit(X_train, y_train)\n",
    "print('End')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Task:</b> Use the `predict()` method to test your model `rf_model` on the test set `X_test`. Save the result to the variable ``y_rf_pred``. Evaluate the results by computing the RMSE and R2 score in the same manner as you did above. Save the results to the variables `rf_rmse` and `rf_r2`.\n",
    "\n",
    "Complete the code in the cell below to accomplish this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[RF] Root Mean Squared Error: 0.6069616061769155\n",
      "[RF] R2: 0.6132797172351814\n"
     ]
    }
   ],
   "source": [
    "# 1. Use the fitted model to make predictions on the test data\n",
    "# YOUR CODE HERE\n",
    "\n",
    "y_rf_pred = rf_model.predict(X_test)\n",
    "# 2. Compute the RMSE \n",
    "# YOUR CODE HERE\n",
    "\n",
    "rf_rmse = np.sqrt(mean_squared_error(y_test, y_rf_pred))\n",
    "# 3. Compute the R2 score \n",
    "# YOUR CODE HERE\n",
    "rf_r2 = r2_score(y_test, y_rf_pred)\n",
    "\n",
    "print('[RF] Root Mean Squared Error: {0}'.format(rf_rmse))\n",
    "print('[RF] R2: {0}'.format(rf_r2))                 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 8: Visualize and Compare Model Performance\n",
    "\n",
    "The code cell below will plot the RMSE and R2 score for each regressor. \n",
    "\n",
    "<b>Task:</b> Complete the code in the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHHCAYAAABDUnkqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9UUlEQVR4nO3df3xP9f//8ftrL/aamc2PzWaMYSWK+ZXlV+I9bWjqnbeUt8xolVpk/ZDSiN5+h8qvyOgH7yQ/3pUilvWLUrRKsRAfks2Ejc02tvP9w9crr7axse21Hbfr5XIudZ7nec55nPN62e47Py2GYRgCAAAwCRdnFwAAAFCaCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAHFgsFo0fP77E8x04cEAWi0VLly4t9ZquxltvvaUbbrhBVatWVc2aNZ1dDoByQLgBKqClS5fKYrHIYrHoyy+/LDDdMAwFBATIYrHojjvucEKFVy4xMdG+bRaLRVWrVlWTJk00ePBg/fbbb6W6rt27d2vIkCFq2rSpFi1apIULF5bq8gFUTFWcXQCAorm5uWn58uXq0qWLQ/tnn32m33//XTabzUmVXb0RI0bo5ptv1tmzZ7Vjxw4tXLhQ69at008//SR/f/9SWUdiYqLy8/P18ssvKygoqFSWCaDi48gNUIH17t1bK1eu1Llz5xzaly9frnbt2snPz89JlV29rl27atCgQYqKitKrr76qGTNm6Pjx43rjjTeuetmZmZmSpKNHj0pSqZ6OysrKKrVlASgbhBugArvvvvv0559/auPGjfa23Nxcvffeexo4cGCh82RmZuqJJ55QQECAbDabmjVrphkzZsgwDId+OTk5GjVqlHx8fFSjRg317dtXv//+e6HLPHz4sIYOHSpfX1/ZbDbdeOONio+PL70NldSjRw9J0v79++1tH3/8sbp27arq1aurRo0a6tOnj37++WeH+YYMGSIPDw/t27dPvXv3Vo0aNfTvf/9bgYGBGjdunCTJx8enwLVE8+bN04033iibzSZ/f389+uijOnnypMOyb7vtNt10003avn27br31Vrm7u+vZZ5+1X180Y8YMzZ07V02aNJG7u7tuv/12HTp0SIZhaOLEiWrQoIGqVaumO++8U8ePH3dY9v/+9z/16dNH/v7+stlsatq0qSZOnKi8vLxCa/jll1/UvXt3ubu7q379+po2bVqBfZidna3x48fr+uuvl5ubm+rVq6e7775b+/bts/fJz8/X7NmzdeONN8rNzU2+vr566KGHdOLEieJ/WEAFx2kpoAILDAxUx44d9d///le9evWSdP4Xfnp6uu6991698sorDv0Nw1Dfvn21efNmDRs2TK1bt9aGDRv01FNP6fDhw5o1a5a97wMPPKC3335bAwcOVKdOnfTpp5+qT58+BWpITU3VLbfcIovFopiYGPn4+Ojjjz/WsGHDlJGRoccff7xUtvXCL+A6depIOn8hcGRkpMLCwjR16lRlZWVp/vz56tKli77//nsFBgba5z137pzCwsLUpUsXzZgxQ+7u7hoyZIjefPNNrVmzRvPnz5eHh4datWolSRo/frxeeOEFhYaGavjw4UpOTtb8+fP17bff6quvvlLVqlXty/7zzz/Vq1cv3XvvvRo0aJB8fX3t05YtW6bc3Fw99thjOn78uKZNm6Z77rlHPXr0UGJiokaPHq29e/fq1Vdf1ZNPPukQCJcuXSoPDw/FxsbKw8NDn376qeLi4pSRkaHp06c77JsTJ04oPDxcd999t+655x699957Gj16tFq2bGn/XuTl5emOO+5QQkKC7r33Xo0cOVKnTp3Sxo0btXPnTjVt2lSS9NBDD2np0qWKiorSiBEjtH//fs2ZM0fff/99gW0HKi0DQIWzZMkSQ5Lx7bffGnPmzDFq1KhhZGVlGYZhGP379ze6d+9uGIZhNGrUyOjTp499vrVr1xqSjBdffNFhef/6178Mi8Vi7N271zAMw0hKSjIkGY888ohDv4EDBxqSjHHjxtnbhg0bZtSrV884duyYQ997773X8PLyste1f/9+Q5KxZMmSS27b5s2bDUlGfHy8kZaWZvzxxx/GunXrjMDAQMNisRjffvutcerUKaNmzZpGdHS0w7wpKSmGl5eXQ3tkZKQhyXjmmWcKrGvcuHGGJCMtLc3edvToUcPV1dW4/fbbjby8PHv7nDlz7HVd0K1bN0OSsWDBAoflXthWHx8f4+TJk/b2MWPGGJKM4OBg4+zZs/b2++67z3B1dTWys7PtbRf228Ueeughw93d3aHfhRrefPNNe1tOTo7h5+dn9OvXz94WHx9vSDJmzpxZYLn5+fmGYRjGF198YUgyli1b5jB9/fr1hbYDlRWnpYAK7p577tGZM2f04Ycf6tSpU/rwww+LPCX10UcfyWq1asSIEQ7tTzzxhAzD0Mcff2zvJ6lAv78fhTEMQ6tWrVJERIQMw9CxY8fsQ1hYmNLT07Vjx44r2q6hQ4fKx8dH/v7+6tOnjzIzM/XGG2+offv22rhxo06ePKn77rvPYZ1Wq1UhISHavHlzgeUNHz68WOvdtGmTcnNz9fjjj8vF5a8fgdHR0fL09NS6desc+ttsNkVFRRW6rP79+8vLy8s+HhISIkkaNGiQqlSp4tCem5urw4cP29uqVatm//9Tp07p2LFj6tq1q7KysrR7926H9Xh4eGjQoEH2cVdXV3Xo0MHh7rJVq1bJ29tbjz32WIE6LRaLJGnlypXy8vJSz549HfZru3bt5OHhUeh+BSojTksBFZyPj49CQ0O1fPlyZWVlKS8vT//6178K7ft///d/8vf3V40aNRzamzdvbp9+4b8uLi72UxUXNGvWzGE8LS1NJ0+e1MKFC4u8jfrCRbslFRcXp65du8pqtcrb21vNmze3B4I9e/ZI+us6nL/z9PR0GK9SpYoaNGhQrPVe2Ad/31ZXV1c1adLEPv2C+vXry9XVtdBlNWzY0GH8QtAJCAgotP3i61p+/vlnjR07Vp9++qkyMjIc+qenpzuMN2jQwB5QLqhVq5Z+/PFH+/i+ffvUrFkzh1D1d3v27FF6errq1q1b6PQr/SyBioZwA1QCAwcOVHR0tFJSUtSrV69yexhdfn6+pPNHIiIjIwvtc+E6lpJq2bKlQkNDL7net956q9A7wv7+C9xmszkchSlNFx9h+Tur1VqiduP/X9R98uRJdevWTZ6enpowYYKaNm0qNzc37dixQ6NHj7Zvf3GXV1z5+fmqW7euli1bVuh0Hx+fEi0PqKgIN0Al8M9//lMPPfSQvv76a61YsaLIfo0aNdKmTZt06tQph6M3F05zNGrUyP7f/Px8+1/7FyQnJzss78KdVHl5eUUGkbJw4YhS3bp1S329F/ZBcnKymjRpYm/Pzc3V/v37y2U7ExMT9eeff2r16tW69dZb7e0X3ylWUk2bNtU333yjs2fPFnlRcNOmTbVp0yZ17tz5kqENqOy45gaoBDw8PDR//nyNHz9eERERRfbr3bu38vLyNGfOHIf2WbNmyWKx2O+sufDfv99tNXv2bIdxq9Wqfv36adWqVdq5c2eB9aWlpV3J5lxWWFiYPD09NWnSJJ09e7ZU1xsaGipXV1e98sorDkc+Fi9erPT09ELvGCttF47EXLz+3NxczZs374qX2a9fPx07dqzAZ3/xeu655x7l5eVp4sSJBfqcO3euwK3wQGXFkRugkijqtNDFIiIi1L17dz333HM6cOCAgoOD9cknn+h///ufHn/8cfsRkdatW+u+++7TvHnzlJ6erk6dOikhIUF79+4tsMwpU6Zo8+bNCgkJUXR0tFq0aKHjx49rx44d2rRpU4Hnt5QGT09PzZ8/X/fff7/atm2re++9Vz4+Pjp48KDWrVunzp07F/pLvDh8fHw0ZswYvfDCCwoPD1ffvn2VnJysefPm6eabb3a4cLesdOrUSbVq1VJkZKRGjBghi8Wit956q8SnmS42ePBgvfnmm4qNjdW2bdvUtWtXZWZmatOmTXrkkUd05513qlu3bnrooYc0efJkJSUl6fbbb1fVqlW1Z88erVy5Ui+//HKR13MBlQnhBjARFxcXvf/++4qLi9OKFSu0ZMkSBQYGavr06XriiScc+sbHx8vHx0fLli3T2rVr1aNHD61bt67AxbC+vr7atm2bJkyYoNWrV2vevHmqU6eObrzxRk2dOrXMtmXgwIHy9/fXlClTNH36dOXk5Kh+/frq2rVrkXcvFdf48ePl4+OjOXPmaNSoUapdu7YefPBBTZo0qVye81KnTh19+OGHeuKJJzR27FjVqlVLgwYN0j/+8Q+FhYVd0TKtVqs++ugj/ec//9Hy5cu1atUq1alTR126dFHLli3t/RYsWKB27drptdde07PPPqsqVaooMDBQgwYNUufOnUtrEwGnshhX86cCAABABcM1NwAAwFQINwAAwFQINwAAwFScGm4+//xzRUREyN/fXxaLRWvXrr3sPImJiWrbtq1sNpuCgoK0dOnSMq8TAABUHk4NN5mZmQoODtbcuXOL1X///v3q06ePunfvrqSkJD3++ON64IEHtGHDhjKuFAAAVBYV5m4pi8WiNWvW6K677iqyz+jRo7Vu3TqHh4nde++9OnnypNavX18OVQIAgIquUj3nZuvWrQUejR4WFlbgTcYXy8nJUU5Ojn08Pz9fx48fV506dQq8iA4AAFRMhmHo1KlT8vf3v+y75CpVuElJSZGvr69Dm6+vrzIyMnTmzJlC35UyefJkvfDCC+VVIgAAKEOHDh1SgwYNLtmnUoWbKzFmzBjFxsbax9PT09WwYUMdOnRInp6eTqwMAAAUV0ZGhgICAhxeClyUShVu/Pz8lJqa6tCWmpoqT0/PIt9wa7PZZLPZCrR7enoSbgAAqGSKc0lJpXrOTceOHZWQkODQtnHjRnXs2NFJFQEAgIrGqeHm9OnTSkpKUlJSkqTzt3onJSXp4MGDks6fUho8eLC9/8MPP6zffvtNTz/9tHbv3q158+bp3Xff1ahRo5xRPgAAqICcGm6+++47tWnTRm3atJEkxcbGqk2bNoqLi5MkHTlyxB50JKlx48Zat26dNm7cqODgYL300kt6/fXXr/gtugAAwHwqzHNuyktGRoa8vLyUnp5+yWtu8vLydPbs2XKsDLgyrq6ul70tEgAqu+L+/pYq2QXF5cEwDKWkpOjkyZPOLgUoFhcXFzVu3Fiurq7OLgUAKgTCzd9cCDZ169aVu7s7D/pDhZafn68//vhDR44cUcOGDfm+AoAINw7y8vLswaZOnTrOLgcoFh8fH/3xxx86d+6cqlat6uxyAMDpOFF/kQvX2Li7uzu5EqD4LpyOysvLc3IlAFAxEG4KwaF9VCZ8XwHAEeEGAACYCuEGAACYChcUF1PgM+vKdX0HpvQpUf8hQ4bojTfekCRVqVJFDRo0UP/+/TVhwgS5ublJ+uv0xdatW3XLLbfY583JyZG/v7+OHz+uzZs367bbbpMkffbZZ3rhhReUlJSk7Oxs1a9fX506ddKiRYvk6uqqxMREde/evdB6jhw5Ij8/v5Ju9tUb71WO60ov8SyX+5wOHDigiRMn6tNPP1VKSor8/f01aNAgPffcc9zqDQDFRLgxkfDwcC1ZskRnz57V9u3bFRkZKYvFoqlTp9r7BAQEaMmSJQ7hZs2aNfLw8NDx48ftbb/88ovCw8P12GOP6ZVXXlG1atW0Z88erVq1qsCFq8nJyQUeqFS3bt0y2srK71Kf0+7du5Wfn6/XXntNQUFB2rlzp6Kjo5WZmakZM2Y4u3QAqBQINyZis9nsR0sCAgIUGhqqjRs3OoSbyMhIvfLKK5o9e7b9Terx8fGKjIzUxIkT7f0++eQT+fn5adq0afa2pk2bKjw8vMB669atq5o1a5bRVpnPpT6n8PBwh33cpEkTJScna/78+YQbACgmrrkxqZ07d2rLli0FTmW0a9dOgYGBWrVqlSTp4MGD+vzzz3X//fc79PPz89ORI0f0+eefl1vN16KiPqeLpaenq3bt2uVYFQBUbhy5MZEPP/xQHh4eOnfunHJycuTi4qI5c+YU6Dd06FDFx8dr0KBBWrp0qXr37i0fHx+HPv3799eGDRvUrVs3+fn56ZZbbtE//vEPDR48uMApqAYNGjiMN2rUSD///HPpb6BJFPdzkqS9e/fq1Vdf5agNAJQA4cZEunfvrvnz5yszM1OzZs1SlSpV1K9fvwL9Bg0apGeeeUa//fabli5dqldeeaVAH6vVqiVLlujFF1/Up59+qm+++UaTJk3S1KlTtW3bNtWrV8/e94svvlCNGjXs4zwl99KK+zkdPnxY4eHh6t+/v6Kjo51QKQBUTpyWMpHq1asrKChIwcHBio+P1zfffKPFixcX6FenTh3dcccdGjZsmLKzs9WrV68il1m/fn3df//9mjNnjn7++WdlZ2drwYIFDn0aN26soKAg+9CoUaNS3zYzKc7n9Mcff6h79+7q1KmTFi5c6KRKAaByItyYlIuLi5599lmNHTtWZ86cKTB96NChSkxM1ODBg2W1Wou1zFq1aqlevXrKzMws7XKvWYV9TocPH9Ztt92mdu3aacmSJXJx4Z8pAJQEPzVNrH///rJarZo7d26BaeHh4UpLS9OECRMKnfe1117T8OHD9cknn2jfvn36+eefNXr0aP3888+KiIhw6Hv06FGlpKQ4DBfe04XLu/hzuhBsGjZsqBkzZigtLc2+TwEAxcM1NyZWpUoVxcTEaNq0aRo+fLjDNIvFIm9v7yLn7dChg7788ks9/PDD+uOPP+Th4aEbb7xRa9euVbdu3Rz6NmvWrMD8f39QIIp28edUrVo17d27V3v37i1wobZhGE6qEAAqF4txjf3EzMjIkJeXl9LT0wvc9ZOdna39+/ercePG9qf6AhUd31sA14JL/f7+O05LAQAAUyHcAAAAUyHcAAAAUyHcAAAAUyHcFOIau8YalRzfVwBwRLi5yIXXBmRlZTm5EqD4cnNzJanYD2MEALPjOTcXsVqtqlmzpo4ePSpJcnd3l8VicXJVQNHy8/OVlpYmd3d3VanCP2cAkAg3Bfj5+UmSPeAAFZ2Li4saNmxIEAeA/49w8zcWi0X16tVT3bp1eYUAKgVXV1fePwUAFyHcFMFqtXINAwAAlRB/7gEAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFMh3AAAAFNxeriZO3euAgMD5ebmppCQEG3btu2S/WfPnq1mzZqpWrVqCggI0KhRo5SdnV1O1QIAgIrOqeFmxYoVio2N1bhx47Rjxw4FBwcrLCxMR48eLbT/8uXL9cwzz2jcuHHatWuXFi9erBUrVujZZ58t58oBAEBF5dRwM3PmTEVHRysqKkotWrTQggUL5O7urvj4+EL7b9myRZ07d9bAgQMVGBio22+/Xffdd99lj/YAAIBrh9PCTW5urrZv367Q0NC/inFxUWhoqLZu3VroPJ06ddL27dvtYea3337TRx99pN69exe5npycHGVkZDgMAADAvKo4a8XHjh1TXl6efH19Hdp9fX21e/fuQucZOHCgjh07pi5dusgwDJ07d04PP/zwJU9LTZ48WS+88EKp1g4AACoup19QXBKJiYmaNGmS5s2bpx07dmj16tVat26dJk6cWOQ8Y8aMUXp6un04dOhQOVYMAADKm9OO3Hh7e8tqtSo1NdWhPTU1VX5+foXO8/zzz+v+++/XAw88IElq2bKlMjMz9eCDD+q5556Ti0vBrGaz2WSz2Up/AwAAQIXktCM3rq6uateunRISEuxt+fn5SkhIUMeOHQudJysrq0CAsVqtkiTDMMquWAAAUGk47ciNJMXGxioyMlLt27dXhw4dNHv2bGVmZioqKkqSNHjwYNWvX1+TJ0+WJEVERGjmzJlq06aNQkJCtHfvXj3//POKiIiwhxwAAHBtc2q4GTBggNLS0hQXF6eUlBS1bt1a69evt19kfPDgQYcjNWPHjpXFYtHYsWN1+PBh+fj4KCIiQv/5z3+ctQkAAKCCsRjX2PmcjIwMeXl5KT09XZ6ens4uBwAAFENJfn9XqrulAAAALodwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATKWKswsAYB6Bz6xzdgk6MKWPs0sA4GSEG5gKv1wBAJyWAgAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAApkK4AQAAplLF2QUAAEpH4DPrnF2CDkzp4+wSAI7cAAAAcyHcAAAAUyHcAAAAUyHcAAAAUyHcAAAAU+FuKQAAzGK8l7MrOG98ulNX7/RwM3fuXE2fPl0pKSkKDg7Wq6++qg4dOhTZ/+TJk3ruuee0evVqHT9+XI0aNdLs2bPVu3fvcqy6aBXiVky3gc4u4Twnf7kBANcmp4abFStWKDY2VgsWLFBISIhmz56tsLAwJScnq27dugX65+bmqmfPnqpbt67ee+891a9fX//3f/+nmjVrln/xAABcpGL8cevsCioGp4abmTNnKjo6WlFRUZKkBQsWaN26dYqPj9czzzxToH98fLyOHz+uLVu2qGrVqpKkwMDA8iwZAABUcE67oDg3N1fbt29XaGjoX8W4uCg0NFRbt24tdJ73339fHTt21KOPPipfX1/ddNNNmjRpkvLy8opcT05OjjIyMhwGAABgXk4LN8eOHVNeXp58fX0d2n19fZWSklLoPL/99pvee+895eXl6aOPPtLzzz+vl156SS+++GKR65k8ebK8vLzsQ0BAQKluBwAAqFgq1a3g+fn5qlu3rhYuXKh27dppwIABeu6557RgwYIi5xkzZozS09Ptw6FDh8qxYgAAUN6cds2Nt7e3rFarUlNTHdpTU1Pl5+dX6Dz16tVT1apVZbVa7W3NmzdXSkqKcnNz5erqWmAem80mm81WusUDAIAKy2lHblxdXdWuXTslJCTY2/Lz85WQkKCOHTsWOk/nzp21d+9e5efn29t+/fVX1atXr9BgAwAArj1OPS0VGxurRYsW6Y033tCuXbs0fPhwZWZm2u+eGjx4sMaMGWPvP3z4cB0/flwjR47Ur7/+qnXr1mnSpEl69NFHnbUJAACggnHqreADBgxQWlqa4uLilJKSotatW2v9+vX2i4wPHjwoF5e/8ldAQIA2bNigUaNGqVWrVqpfv75Gjhyp0aNHO2sTAABABeP0JxTHxMQoJiam0GmJiYkF2jp27Kivv/66jKsCAFwRHv+PCqBS3S0FAABwOYQbAABgKoQbAABgKlcUbj766CM98MADevrpp7V7926HaSdOnFCPHj1KpTgAAICSKnG4Wb58ufr27auUlBRt3bpVbdq00bJly+zTc3Nz9dlnn5VqkQAAAMVV4rulpk+frpkzZ2rEiBGSpHfffVdDhw5Vdna2hg0bVuoFAgAAlESJw82ePXsUERFhH7/nnnvk4+Ojvn376uzZs/rnP/9ZqgUCAACURInDjaenp1JTU9W4cWN7W/fu3fXhhx/qjjvu0O+//16qBQIAAJREia+56dChgz7++OMC7d26ddMHH3yg2bNnl0ZdAAAAV6TE4WbUqFFyc3MrdNptt92mDz74QIMHD77qwgAAAK5EiU9LdevWTd26dStyevfu3dW9e/erKgoAAOBKlfpD/Hbs2KE77rijtBcLAABQLFcUbjZs2KAnn3xSzz77rH777TdJ0u7du3XXXXfp5ptvVn5+fqkWCQAAUFwlPi21ePFiRUdHq3bt2jpx4oRef/11zZw5U4899pgGDBignTt3qnnz5mVRKwAAwGWV+MjNyy+/rKlTp+rYsWN69913dezYMc2bN08//fSTFixYQLABAABOVeJws2/fPvXv31+SdPfdd6tKlSqaPn26GjRoUOrFAQAAlFSJw82ZM2fk7u4uSbJYLLLZbKpXr16pFwYAAHAlSnzNjSS9/vrr8vDwkCSdO3dOS5culbe3t0OfC++eAgAAKE8lDjcNGzbUokWL7ON+fn566623HPpYLBbCDQAAcIoSh5sDBw6UQRkAAAClo8TX3AwePFirVq1SZmZmWdQDAABwVUocboKCgjRp0iR5e3urV69emj9/vg4fPlwWtQEAAJRYicNNXFyctm/frj179igiIkJr165V06ZN1a5dO02YMEFJSUllUCYAAEDxXPG7pRo0aKBHHnlEGzZsUFpamkaPHq3k5GT16NFDjRo1UkxMjH7++efSrBUAAOCySuXFmTVq1NA999yjZcuWKS0tTfHx8bJardq6dWtpLB4AAKDYSny31NGjR1W3bt1L9qlRo4ZefvnlKy4KAADgSpX4yE29evV09OhR+3jLli116NAh+/ixY8fUsWPH0qkOAACghEocbgzDcBg/cOCAzp49e8k+AAAA5aVUrrn5O4vFUhaLBQAAuKwyCTcAAADOUuILii0Wi06dOiU3NzcZhiGLxaLTp08rIyNDkuz/BQAAcIYShxvDMHT99dc7jLdp08ZhnNNSAADAWUocbjZv3lwWdQAAAJSKEoebbt26lUUdAAAApaLE4ebcuXPKy8uTzWazt6WmpmrBggXKzMxU37591aVLl1ItEgAAoLhKHG6io6Pl6uqq1157TZJ06tQp3XzzzcrOzla9evU0a9Ys/e9//1Pv3r1LvVgAAIDLKfGt4F999ZX69etnH3/zzTeVl5enPXv26IcfflBsbKymT59eqkUCAAAUV4nDzeHDh3XdddfZxxMSEtSvXz95eXlJkiIjI3kbOAAAcJoSn5Zyc3PTmTNn7ONff/21w5EaNzc3nT59unSqAyqj8V7OruC88enOrgAAnKLE4aZ169Z66623NHnyZH3xxRdKTU1Vjx497NP37dsnf3//Ui0SAIqNcAlc80ocbuLi4tSrVy+9++67OnLkiIYMGaJ69erZp69Zs0adO3cu1SIBAACK64qec7N9+3Z98skn8vPzU//+/R2mt27dWh06dCi1AgEAAEqixOFGkpo3b67mzZsXOu3BBx+8qoIAAACuRonDzeeff16sfrfeemuJiwEAALhaJQ43t912m/3FmIZhFNrHYrEoLy/v6ioDAAC4AiUON7Vq1VKNGjU0ZMgQ3X///fL29i6LugAAAK5IiR/id+TIEU2dOlVbt25Vy5YtNWzYMG3ZskWenp7y8vKyDwAAAM5Q4nDj6uqqAQMGaMOGDdq9e7datWqlmJgYBQQE6LnnntO5c+fKok4AAIBiKXG4uVjDhg0VFxenTZs26frrr9eUKVOUkZFRWrUBAACU2BWHm5ycHC1fvlyhoaG66aab5O3trXXr1ql27dqlWR8AAECJlPiC4m3btmnJkiV65513FBgYqKioKL377ruEGgAAUCGUONzccsstatiwoUaMGKF27dpJkr788ssC/fr27Xv11QEAAJTQFT2h+ODBg5o4cWKR03nODQAAcJYSh5v8/PzL9snKyrqiYgAAAK7WVd0t9Xc5OTmaOXOmmjRpUpqLBQAAKLYSh5ucnByNGTNG7du3V6dOnbR27VpJUnx8vBo3bqxZs2Zp1KhRpV0nAABAsZT4tFRcXJxee+01hYaGasuWLerfv7+ioqL09ddfa+bMmerfv7+sVmtZ1AoAAHBZJQ43K1eu1Jtvvqm+fftq586datWqlc6dO6cffvjB/kJNAAAAZynxaanff//dfgv4TTfdJJvNplGjRhFsAABAhVDicJOXlydXV1f7eJUqVeTh4VGqRQEAAFypEp+WMgxDQ4YMkc1mkyRlZ2fr4YcfVvXq1R36rV69unQqBAAAKIESh5vIyEiH8UGDBpVaMQAAAFerxOFmyZIlZVEHAABAqSjVh/hdqblz5yowMFBubm4KCQnRtm3bijXfO++8I4vForvuuqtsCwQAAJWG08PNihUrFBsbq3HjxmnHjh0KDg5WWFiYjh49esn5Dhw4oCeffFJdu3Ytp0oBAEBl4PRwM3PmTEVHRysqKkotWrTQggUL5O7urvj4+CLnycvL07///W+98MILvOoBAAA4cGq4yc3N1fbt2xUaGmpvc3FxUWhoqLZu3VrkfBMmTFDdunU1bNiwy64jJydHGRkZDgMAADAvp4abY8eOKS8vT76+vg7tvr6+SklJKXSeL7/8UosXL9aiRYuKtY7JkyfLy8vLPgQEBFx13QAAoOJy+mmpkjh16pTuv/9+LVq0SN7e3sWaZ8yYMUpPT7cPhw4dKuMqAQCAM5X4VvDS5O3tLavVqtTUVIf21NRU+fn5Fei/b98+HThwQBEREfa2/Px8SeeflJycnKymTZs6zGOz2ewPHAQAAObn1CM3rq6uateunRISEuxt+fn5SkhIUMeOHQv0v+GGG/TTTz8pKSnJPvTt21fdu3dXUlISp5wAAIBzj9xIUmxsrCIjI9W+fXt16NBBs2fPVmZmpqKioiRJgwcPVv369TV58mS5ubnppptucpi/Zs2aklSgHQAAXJucHm4GDBigtLQ0xcXFKSUlRa1bt9b69evtFxkfPHhQLi6V6tIgAADgRE4PN5IUExOjmJiYQqclJiZect6lS5eWfkEAAKDS4pAIAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwlQoRbubOnavAwEC5ubkpJCRE27ZtK7LvokWL1LVrV9WqVUu1atVSaGjoJfsDAIBri9PDzYoVKxQbG6tx48Zpx44dCg4OVlhYmI4ePVpo/8TERN13333avHmztm7dqoCAAN1+++06fPhwOVcOAAAqIqeHm5kzZyo6OlpRUVFq0aKFFixYIHd3d8XHxxfaf9myZXrkkUfUunVr3XDDDXr99deVn5+vhISEcq4cAABURE4NN7m5udq+fbtCQ0PtbS4uLgoNDdXWrVuLtYysrCydPXtWtWvXLnR6Tk6OMjIyHAYAAGBeTg03x44dU15ennx9fR3afX19lZKSUqxljB49Wv7+/g4B6WKTJ0+Wl5eXfQgICLjqugEAQMXl9NNSV2PKlCl65513tGbNGrm5uRXaZ8yYMUpPT7cPhw4dKucqAQBAearizJV7e3vLarUqNTXVoT01NVV+fn6XnHfGjBmaMmWKNm3apFatWhXZz2azyWazlUq9AACg4nPqkRtXV1e1a9fO4WLgCxcHd+zYscj5pk2bpokTJ2r9+vVq3759eZQKAAAqCaceuZGk2NhYRUZGqn379urQoYNmz56tzMxMRUVFSZIGDx6s+vXra/LkyZKkqVOnKi4uTsuXL1dgYKD92hwPDw95eHg4bTsAAEDF4PRwM2DAAKWlpSkuLk4pKSlq3bq11q9fb7/I+ODBg3Jx+esA0/z585Wbm6t//etfDssZN26cxo8fX56lAwCACsjp4UaSYmJiFBMTU+i0xMREh/EDBw6UfUEAAKDSqtR3SwEAAPwd4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJgK4QYAAJhKhQg3c+fOVWBgoNzc3BQSEqJt27Zdsv/KlSt1ww03yM3NTS1bttRHH31UTpUCAICKzunhZsWKFYqNjdW4ceO0Y8cOBQcHKywsTEePHi20/5YtW3Tfffdp2LBh+v7773XXXXfprrvu0s6dO8u5cgAAUBE5PdzMnDlT0dHRioqKUosWLbRgwQK5u7srPj6+0P4vv/yywsPD9dRTT6l58+aaOHGi2rZtqzlz5pRz5QAAoCJyarjJzc3V9u3bFRoaam9zcXFRaGiotm7dWug8W7dudegvSWFhYUX2BwAA15Yqzlz5sWPHlJeXJ19fX4d2X19f7d69u9B5UlJSCu2fkpJSaP+cnBzl5OTYx9PT0yVJGRkZV1N6kfJzsspkuSWRYTGcXcJ5ZbSPL4X9fxH2v3Ox/52L/e9cZbD/L/zeNozLb6NTw015mDx5sl544YUC7QEBAU6opnx4ObuAC6ZUmErKVYXZava/c7H/nYv971xluP9PnTolL69LL9+p4cbb21tWq1WpqakO7ampqfLz8yt0Hj8/vxL1HzNmjGJjY+3j+fn5On78uOrUqSOLxXKVW1DxZGRkKCAgQIcOHZKnp6ezy7nmsP+di/3vXOx/5zL7/jcMQ6dOnZK/v/9l+zo13Li6uqpdu3ZKSEjQXXfdJel8+EhISFBMTEyh83Ts2FEJCQl6/PHH7W0bN25Ux44dC+1vs9lks9kc2mrWrFka5Vdonp6epvxyVxbsf+di/zsX+9+5zLz/L3fE5gKnn5aKjY1VZGSk2rdvrw4dOmj27NnKzMxUVFSUJGnw4MGqX7++Jk+eLEkaOXKkunXrppdeekl9+vTRO++8o++++04LFy505mYAAIAKwunhZsCAAUpLS1NcXJxSUlLUunVrrV+/3n7R8MGDB+Xi8tdNXZ06ddLy5cs1duxYPfvss7ruuuu0du1a3XTTTc7aBAAAUIE4PdxIUkxMTJGnoRITEwu09e/fX/379y/jqionm82mcePGFTgVh/LB/ncu9r9zsf+di/3/F4tRnHuqAAAAKgmnP6EYAACgNBFuAACAqRBuAACAqRBuTCoxMVEWi0UnT54sdPqBAwdksViUlJRUrnUBAFDWCDflIC0tTcOHD1fDhg1ls9nk5+ensLAwffXVV5Iki8WitWvXlmtNAQEBOnLkCLfQX6EhQ4bYHzz5d4GBgbJYLLJYLHJ3d1fLli31+uuvl2+BJjZkyBD7/q1atap8fX3Vs2dPxcfHKz8/3x7sLzUUdhcmzr+7b+TIkQoKCpKbm5t8fX3VuXNnzZ8/X1lZ59+bdPH322q1yt/fX8OGDdOJEyfsy7n4M3BxcZGXl5fatGmjp59+WkeOHLH3u3hZhQ1Dhgwp711QKfz930Djxo319NNPKzs7296nsP3ZpUsXJ1ZdvirEreBm169fP+Xm5uqNN95QkyZNlJqaqoSEBP35559Oq8lqtRb5ygpcvQkTJig6OlpZWVlauXKloqOjVb9+ffXq1cvZpZlCeHi4lixZory8PKWmpmr9+vUaOXKk3nvvPa1du9bhF+jIkSOVkZGhJUuW2Ntq167tjLIrtN9++02dO3dWzZo1NWnSJLVs2VI2m00//fSTFi5cqPr166tv376S/vp+5+Xl6ddff9WDDz6oESNG6K233nJYZnJysjw9PZWRkaEdO3Zo2rRpWrx4sRITE9WyZUt9++23ysvLkyRt2bJF/fr1s88jSdWqVSvfnVCJXPg3cPbsWW3fvl2RkZGyWCyaOnWqvc+SJUsUHh5uH3d1dXVGqc5hoEydOHHCkGQkJiYWOr1Ro0aGJPvQqFEjwzAMY+/evUbfvn2NunXrGtWrVzfat29vbNy40WHe7Oxs4+mnnzYaNGhguLq6Gk2bNjVef/11wzAMY/PmzYYk48SJE4ZhGEZmZqYRHh5udOrUyThx4oSxf/9+Q5Lx/fffO/TftGmT0a5dO6NatWpGx44djd27dzusc+LEiYaPj4/h4eFhDBs2zBg9erQRHBxcavursoiMjDTuvPPOQqc1atTImDVrlkNb7dq1jVGjRpV9YdeAovZ9QkKCIclYtGhRsfrDUVhYmNGgQQPj9OnThU7Pz883DKPw7/fEiRONFi1a2Mf//vPngqysLKNZs2ZG586dCyy/qHlQUGHf6bvvvtto06aNfVySsWbNmvItrALhtFQZ8/DwkIeHh9auXaucnJwC07/99ltJ5xP2kSNH7OOnT59W7969lZCQoO+//17h4eGKiIjQwYMH7fMOHjxY//3vf/XKK69o165deu211+Th4VFgHSdPnlTPnj2Vn5+vjRs3XvLdWs8995xeeuklfffdd6pSpYqGDh1qn7Zs2TL95z//0dSpU7V9+3Y1bNhQ8+fPv9Jdc03Iz8/XqlWrdOLEiWvrryYn6NGjh4KDg7V69Wpnl1Lp/Pnnn/rkk0/06KOPqnr16oX2KepFw4cPH9YHH3ygkJCQy66nWrVqevjhh/XVV1/p6NGjV1Uz/rJz505t2bKFnzEXc3a6uha89957Rq1atQw3NzejU6dOxpgxY4wffvjBPl3FTNg33nij8eqrrxqGYRjJycmGpAJHcy648FfQrl27jFatWhn9+vUzcnJy7NMvdeTmgnXr1hmSjDNnzhiGYRghISHGo48+6rCezp07c+Tmbxo1amS4uroa1atXN6pUqWJIMmrXrm3s2bOnfIs0qUvt+wEDBhjNmzcvdn+c9/XXXxuSjNWrVzu016lTx6hevbpRvXp14+mnnzYMw/H77ebmZkgyQkJCHI64XOoozMcff2xIMr755huHdo7cFF9kZKRhtVqN6tWrGzabzZBkuLi4GO+99569jyTDzc3N/vlVr179mjqSw5GbctCvXz/98ccfev/99xUeHq7ExES1bdtWS5cuLXKe06dP68knn1Tz5s1Vs2ZNeXh4aNeuXfYjN0lJSbJarerWrdsl192zZ08FBQVpxYoVxUr1rVq1sv9/vXr1JMn+F1ZycrI6dOjg0P/v4zjvqaeeUlJSkj799FOFhIRo1qxZCgoKcnZZpmcYRpFHGFBy27ZtU1JSkm688UaHI88Xvt8//vijEhISJEl9+vSxXz9zKcb/fyg+n9PV6d69u5KSkvTNN98oMjJSUVFR6tevn0OfWbNmKSkpyT707NnTSdWWP8JNOXFzc1PPnj31/PPPa8uWLRoyZIjGjRtXZP8nn3xSa9as0aRJk/TFF18oKSlJLVu2VG5urqTiX2jXp08fff755/rll1+K1b9q1ar2/7/wwyc/P79Y8+Iv3t7eCgoKUteuXbVy5UqNGDGi2J8BrtyuXbvUuHFjZ5dR6QQFBclisSg5OdmhvUmTJgoKCirw8+bC9/u6665Tjx49NHv2bG3ZskWbN2++7Lp27dol6fydUrhy1atXV1BQkIKDgxUfH69vvvlGixcvdujj5+enoKAg+1DUKUczItw4SYsWLZSZmSnpfKD4+188X331lYYMGaJ//vOfatmypfz8/HTgwAH79JYtWyo/P1+fffbZJdczZcoURUZG6h//+MdV/3Jt1qyZ/ZqgC/4+joICAgI0YMAAjRkzxtmlmNqnn36qn376qcBfr7i8OnXqqGfPnpozZ47951JJWK1WSdKZM2cu2e/MmTNauHChbr31Vvn4+FxRrSjIxcVFzz77rMaOHXvZz+BaQbgpY3/++ad69Oiht99+Wz/++KP279+vlStXatq0abrzzjslnf8LJiEhQSkpKfZnRVx33XVavXq1kpKS9MMPP2jgwIEOR1ACAwMVGRmpoUOHau3atdq/f78SExP17rvvFqhhxowZ+ve//60ePXpo9+7dV7wtjz32mBYvXqw33nhDe/bs0Ysvvqgff/zxmj28nJ6e7nDINykpSYcOHSq078iRI/XBBx/ou+++K+cqzSknJ0cpKSk6fPiwduzYoUmTJunOO+/UHXfcocGDBzu7vEpp3rx5OnfunNq3b68VK1Zo165dSk5O1ttvv63du3fbA4wknTp1SikpKTpy5Ii2bdump556Sj4+PurUqZPDMo8ePaqUlBTt2bNH77zzjjp37qxjx45xI0IZ6N+/v6xWq+bOnevsUioGZ1/0Y3bZ2dnGM888Y7Rt29bw8vIy3N3djWbNmhljx441srKyDMMwjPfff98ICgoyqlSpYr8VfP/+/Ub37t2NatWqGQEBAcacOXOMbt26GSNHjrQv+8yZM8aoUaOMevXqGa6urkZQUJARHx9vGEbhF+c99thjRr169Yzk5OQiLyi+uP/3339vSDL2799vb5swYYLh7e1teHh4GEOHDjVGjBhh3HLLLWWx6yq0yMhIh1v4LwzDhg0r9FZZwzh/q22vXr3Kv1iTuXjfV6lSxfDx8TFCQ0ON+Ph4Iy8vr9D+XFBcPH/88YcRExNjNG7c2Khatarh4eFhdOjQwZg+fbqRmZlpGEbBx1f4+PgYvXv3tv8sMYy/fp5IMiwWi1GjRg0jODjYeOqpp4wjR44Uum4uKC6+or7TkydPNnx8fIzTp09f87eCWwzj/1/dBVyBnj17ys/Pr8DDuwAAcBaeUIxiy8rK0oIFCxQWFiar1ar//ve/2rRpkzZu3Ojs0gAAsOPIDYrtzJkzioiI0Pfff6/s7Gw1a9ZMY8eO1d133+3s0gAAsCPcAAAAU+FuKQAAYCqEGwAAYCqEGwAAYCqEGwAAYCqEGwCmk5iYKIvFopMnTxZ7nsDAQM2ePbvMagJQfgg3AMrdkCFDZLFY9PDDDxeY9uijj8pisWjIkCHlXxgAUyDcAHCKgIAAvfPOOw4v+svOztby5cvVsGFDJ1YGoLIj3ABwirZt2yogIECrV6+2t61evVoNGzZUmzZt7G05OTkaMWKE6tatKzc3N3Xp0qXA2+g/+ugjXX/99apWrZq6d++uAwcOFFjfl19+qa5du6patWoKCAjQiBEjinwDtmEYGj9+vBo2bCibzSZ/f3+NGDGidDYcQJkj3ABwmqFDh2rJkiX28fj4eEVFRTn0efrpp7Vq1Sq98cYb2rFjh4KCghQWFqbjx49Lkg4dOqS7775bERERSkpK0gMPPKBnnnnGYRn79u1TeHi4+vXrpx9//FErVqzQl19+qZiYmELrWrVqlWbNmqXXXntNe/bs0dq1a9WyZctS3noAZcaJL+0EcI268Fbjo0ePGjabzThw4IBx4MABw83NzUhLSzPuvPNOIzIy0jh9+rRRtWpVY9myZfZ5c3NzDX9/f2PatGmGYRjGmDFjjBYtWjgsf/To0Q5vmB42bJjx4IMPOvT54osvDBcXF+PMmTOGYRgOb3N/6aWXjOuvv97Izc0toz0AoCxx5AaA0/j4+KhPnz5aunSplixZoj59+sjb29s+fd++fTp79qw6d+5sb6tatao6dOigXbt2SZJ27dqlkJAQh+V27NjRYfyHH37Q0qVL5eHhYR/CwsKUn5+v/fv3F6irf//+OnPmjJo0aaLo6GitWbNG586dK81NB1CGeCs4AKcaOnSo/fTQ3Llzy2Qdp0+f1kMPPVTodTOFXbwcEBCg5ORk+1vvH3nkEU2fPl2fffaZqlatWiY1Aig9HLkB4FTh4eHKzc3V2bNnFRYW5jCtadOmcnV11VdffWVvO3v2rL799lu1aNFCktS8eXNt27bNYb6vv/7aYbxt27b65ZdfFBQUVGBwdXUttK5q1aopIiJCr7zyihITE7V161b99NNPpbHJAMoYR24AOJXVarWfYrJarQ7TqlevruHDh+upp55S7dq11bBhQ02bNk1ZWVkaNmyYJOnhhx/WSy+9pKeeekoPPPCAtm/frqVLlzosZ/To0brlllsUExOjBx54QNWrV9cvv/yijRs3as6cOQVqWrp0qfLy8hQSEiJ3d3e9/fbbqlatmho1alQ2OwFAqeLIDQCn8/T0lKenZ6HTpkyZon79+un+++9X27ZttXfvXm3YsEG1atWSdP600qpVq7R27VoFBwdrwYIFmjRpksMyWrVqpc8++0y//vqrunbtqjZt2iguLk7+/v6FrrNmzZpatGiROnfurFatWmnTpk364IMPVKdOndLdcABlwmIYhuHsIgAAAEoLR24AAICpEG4AAICpEG4AAICpEG4AAICpEG4AAICpEG4AAICpEG4AAICpEG4AAICpEG4AAICpEG4AAICpEG4AAICpEG4AAICp/D87aQK54qBmDgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "RMSE_Results = [stack_rmse, lr_rmse, dt_rmse, gbdt_rmse, rf_rmse]\n",
    "R2_Results = [stack_r2, lr_r2, dt_r2, gbdt_r2, rf_r2]\n",
    "\n",
    "rg= np.arange(5)\n",
    "width = 0.35\n",
    "\n",
    "# 1. Create bar plot with RMSE results\n",
    "# YOUR CODE HERE\n",
    "plt.bar(rg, RMSE_Results, width, label='RMSE')\n",
    "\n",
    "# 2. Create bar plot with R2 results\n",
    "# YOUR CODE HERE\n",
    "\n",
    "plt.bar(rg + width, R2_Results, width, label='R2')\n",
    "\n",
    "labels = ['Stacking','LR', 'DT', 'GBDT', 'RF']\n",
    "plt.xticks(rg + width/2, labels)\n",
    "\n",
    "plt.xlabel(\"Models\")\n",
    "plt.ylabel(\"RMSE/R2\")\n",
    "\n",
    "\n",
    "plt.ylim([0,1])\n",
    "plt.title('Model Performance')\n",
    "plt.legend(loc='upper left', ncol=2)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Analysis</b>: Compare and contrast the resulting $R^2$ and RSME scores of the ensemble models and the individual models. Are the ensemble models performing better? Which is the best performing model? Explain."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ensemble models are performing better than the individual models. Among the ensemble models, the stacking model has the best performance. The best-performing model is the Stacking model, which has the lowest RMSE (0.25) and the highest R2 score (0.92). This indicates that the stacking model is the most accurate in predicting the target values and explains the highest proportion of variance."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.19"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}