8. SpaceX_Machine_Learning_Prediction_Part_5.jupyterlite.ipynb

{"cells":[{"cell_type":"markdown","id":"ceae7cd6-5733-45da-a934-b7f03b54f9d2","metadata":{},"source":["<p style=\"text-align:center\">\n","    <a href=\"https://skills.network/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDS0321ENSkillsNetwork26802033-2022-01-01\" target=\"_blank\">\n","    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\">\n","    </a>\n","</p>\n"]},{"cell_type":"markdown","id":"b85cf4a6-253c-40df-854c-fe6ef2afeae2","metadata":{},"source":["# **Space X  Falcon 9 First Stage Landing Prediction**\n"]},{"cell_type":"markdown","id":"c8df9486-4706-4c4f-9aa6-29612de59b54","metadata":{},"source":["## Assignment:  Machine Learning Prediction\n"]},{"cell_type":"markdown","id":"4127b9f4-7258-4ac1-9a71-110d17e31936","metadata":{},"source":["Estimated time needed: **60** minutes\n"]},{"cell_type":"markdown","id":"ed8eb893-2b88-4329-b315-f0833afc3244","metadata":{},"source":["Space X advertises Falcon 9 rocket launches on its website with a cost of 62 million dollars; other providers cost upward of 165 million dollars each, much of the savings is because Space X can reuse the first stage. Therefore if we can determine if the first stage will land, we can determine the cost of a launch. This information can be used if an alternate company wants to bid against space X for a rocket launch.   In this lab, you will create a machine learning pipeline  to predict if the first stage will land given the data from the preceding labs.\n"]},{"cell_type":"markdown","id":"a8bc0980-2128-4542-9937-b5c686af7b3a","metadata":{},"source":["![](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DS0701EN-SkillsNetwork/api/Images/landing_1.gif)\n"]},{"cell_type":"markdown","id":"3b2d2ffd-9439-446b-844c-c61823e23373","metadata":{},"source":["Several examples of an unsuccessful landing are shown here:\n"]},{"cell_type":"markdown","id":"e13b5b23-5f86-4b83-a8b7-e4e210eafcdb","metadata":{},"source":["![](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DS0701EN-SkillsNetwork/api/Images/crash.gif)\n"]},{"cell_type":"markdown","id":"8e5b7c7b-9774-441f-82fc-a21a1afc8f48","metadata":{},"source":["Most unsuccessful landings are planed. Space X; performs a controlled landing in the oceans.\n"]},{"cell_type":"markdown","id":"5862cf6c-a7fc-4f2b-a1d8-5f9939ec01e4","metadata":{},"source":["## Objectives\n"]},{"cell_type":"markdown","id":"d14a7f61-1f88-4b22-9392-b528c695381b","metadata":{},"source":["Perform exploratory  Data Analysis and determine Training Labels\n","\n","*   create a column for the class\n","*   Standardize the data\n","*   Split into training data and test data\n","\n","\\-Find best Hyperparameter for SVM, Classification Trees and Logistic Regression\n","\n","*   Find the method performs best using test data\n"]},{"cell_type":"markdown","id":"e2296c5c-b964-40aa-a4f1-4cb9e1413fe0","metadata":{},"source":["## Import Libraries and Define Auxiliary Functions\n"]},{"cell_type":"markdown","id":"d81029d7-c03c-48b9-a6ed-2e9cfcee87a0","metadata":{},"source":["We will import the following libraries for the lab\n"]},{"cell_type":"code","execution_count":5,"id":"3153de69-2f5c-474d-a4d0-0ce138c9e6a8","metadata":{},"outputs":[],"source":["# Pandas is a software library written for the Python programming language for data manipulation and analysis.\n","import pandas as pd\n","# NumPy is a library for the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays\n","import numpy as np\n","# Matplotlib is a plotting library for python and pyplot gives us a MatLab like plotting framework. We will use this in our plotter function to plot data.\n","import matplotlib.pyplot as plt\n","#Seaborn is a Python data visualization library based on matplotlib. It provides a high-level interface for drawing attractive and informative statistical graphics\n","import seaborn as sns\n","# Preprocessing allows us to standarsize our data\n","from sklearn import preprocessing\n","# Allows us to split our data into training and testing data\n","from sklearn.model_selection import train_test_split\n","# Allows us to test parameters of classification algorithms and find the best one\n","from sklearn.model_selection import GridSearchCV\n","# Logistic Regression classification algorithm\n","from sklearn.linear_model import LogisticRegression\n","# Support Vector Machine classification algorithm\n","from sklearn.svm import SVC\n","# Decision Tree classification algorithm\n","from sklearn.tree import DecisionTreeClassifier\n","# K Nearest Neighbors classification algorithm\n","from sklearn.neighbors import KNeighborsClassifier"]},{"cell_type":"markdown","id":"5aa4db43-3dc1-4339-a3a0-c0ae6e0ecbd6","metadata":{},"source":["This function is to plot the confusion matrix.\n"]},{"cell_type":"code","execution_count":6,"id":"de3dfbf2-cda3-4ec1-b897-a2a9878b65f2","metadata":{},"outputs":[],"source":["def plot_confusion_matrix(y,y_predict):\n","    \"this function plots the confusion matrix\"\n","    from sklearn.metrics import confusion_matrix\n","\n","    cm = confusion_matrix(y, y_predict)\n","    ax= plt.subplot()\n","    sns.heatmap(cm, annot=True, ax = ax); #annot=True to annotate cells\n","    ax.set_xlabel('Predicted labels')\n","    ax.set_ylabel('True labels')\n","    ax.set_title('Confusion Matrix'); \n","    ax.xaxis.set_ticklabels(['did not land', 'land']); ax.yaxis.set_ticklabels(['did not land', 'landed']) \n","    plt.show() "]},{"cell_type":"markdown","id":"d6f19751-5f52-46cb-9781-fe9fa237aa57","metadata":{},"source":["## Load the dataframe\n"]},{"cell_type":"markdown","id":"f54bc78e-aca6-46aa-a105-5ad9f5c3692a","metadata":{},"source":["Load the data\n"]},{"cell_type":"code","execution_count":null,"id":"0f804631-fdb0-4e2c-ac6c-078e77646d58","metadata":{},"outputs":[],"source":["# from js import fetch\n","# import io\n","\n","# URL1 = \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBM-DS0321EN-SkillsNetwork/datasets/dataset_part_2.csv\"\n","# resp1 = await fetch(URL1)\n","# text1 = io.BytesIO((await resp1.arrayBuffer()).to_py())\n","# data = pd.read_csv(text1)"]},{"cell_type":"code","execution_count":null,"id":"81b24478-6a8c-4a46-a0b0-5332186bdc5e","metadata":{},"outputs":[],"source":["# data.head()"]},{"cell_type":"code","execution_count":null,"id":"cd9c03d9-70f5-4c7f-867a-4012bc6f9dab","metadata":{},"outputs":[],"source":["# URL2 = 'https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBM-DS0321EN-SkillsNetwork/datasets/dataset_part_3.csv'\n","# resp2 = await fetch(URL2)\n","# text2 = io.BytesIO((await resp2.arrayBuffer()).to_py())\n","# X = pd.read_csv(text2)"]},{"cell_type":"code","execution_count":7,"metadata":{},"outputs":[],"source":["URL1 = \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBM-DS0321EN-SkillsNetwork/datasets/dataset_part_2.csv\"\n","URL2 = 'https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBM-DS0321EN-SkillsNetwork/datasets/dataset_part_3.csv'\n","\n","data = pd.read_csv(URL1)\n","X = pd.read_csv(URL2)"]},{"cell_type":"code","execution_count":8,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>FlightNumber</th>\n","      <th>Date</th>\n","      <th>BoosterVersion</th>\n","      <th>PayloadMass</th>\n","      <th>Orbit</th>\n","      <th>LaunchSite</th>\n","      <th>Outcome</th>\n","      <th>Flights</th>\n","      <th>GridFins</th>\n","      <th>Reused</th>\n","      <th>Legs</th>\n","      <th>LandingPad</th>\n","      <th>Block</th>\n","      <th>ReusedCount</th>\n","      <th>Serial</th>\n","      <th>Longitude</th>\n","      <th>Latitude</th>\n","      <th>Class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>2010-06-04</td>\n","      <td>Falcon 9</td>\n","      <td>6104.959412</td>\n","      <td>LEO</td>\n","      <td>CCAFS SLC 40</td>\n","      <td>None None</td>\n","      <td>1</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>NaN</td>\n","      <td>1.0</td>\n","      <td>0</td>\n","      <td>B0003</td>\n","      <td>-80.577366</td>\n","      <td>28.561857</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>2</td>\n","      <td>2012-05-22</td>\n","      <td>Falcon 9</td>\n","      <td>525.000000</td>\n","      <td>LEO</td>\n","      <td>CCAFS SLC 40</td>\n","      <td>None None</td>\n","      <td>1</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>NaN</td>\n","      <td>1.0</td>\n","      <td>0</td>\n","      <td>B0005</td>\n","      <td>-80.577366</td>\n","      <td>28.561857</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>3</td>\n","      <td>2013-03-01</td>\n","      <td>Falcon 9</td>\n","      <td>677.000000</td>\n","      <td>ISS</td>\n","      <td>CCAFS SLC 40</td>\n","      <td>None None</td>\n","      <td>1</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>NaN</td>\n","      <td>1.0</td>\n","      <td>0</td>\n","      <td>B0007</td>\n","      <td>-80.577366</td>\n","      <td>28.561857</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>4</td>\n","      <td>2013-09-29</td>\n","      <td>Falcon 9</td>\n","      <td>500.000000</td>\n","      <td>PO</td>\n","      <td>VAFB SLC 4E</td>\n","      <td>False Ocean</td>\n","      <td>1</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>NaN</td>\n","      <td>1.0</td>\n","      <td>0</td>\n","      <td>B1003</td>\n","      <td>-120.610829</td>\n","      <td>34.632093</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>5</td>\n","      <td>2013-12-03</td>\n","      <td>Falcon 9</td>\n","      <td>3170.000000</td>\n","      <td>GTO</td>\n","      <td>CCAFS SLC 40</td>\n","      <td>None None</td>\n","      <td>1</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>NaN</td>\n","      <td>1.0</td>\n","      <td>0</td>\n","      <td>B1004</td>\n","      <td>-80.577366</td>\n","      <td>28.561857</td>\n","      <td>0</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   FlightNumber        Date BoosterVersion  PayloadMass Orbit    LaunchSite  \\\n","0             1  2010-06-04       Falcon 9  6104.959412   LEO  CCAFS SLC 40   \n","1             2  2012-05-22       Falcon 9   525.000000   LEO  CCAFS SLC 40   \n","2             3  2013-03-01       Falcon 9   677.000000   ISS  CCAFS SLC 40   \n","3             4  2013-09-29       Falcon 9   500.000000    PO   VAFB SLC 4E   \n","4             5  2013-12-03       Falcon 9  3170.000000   GTO  CCAFS SLC 40   \n","\n","       Outcome  Flights  GridFins  Reused   Legs LandingPad  Block  \\\n","0    None None        1     False   False  False        NaN    1.0   \n","1    None None        1     False   False  False        NaN    1.0   \n","2    None None        1     False   False  False        NaN    1.0   \n","3  False Ocean        1     False   False  False        NaN    1.0   \n","4    None None        1     False   False  False        NaN    1.0   \n","\n","   ReusedCount Serial   Longitude   Latitude  Class  \n","0            0  B0003  -80.577366  28.561857      0  \n","1            0  B0005  -80.577366  28.561857      0  \n","2            0  B0007  -80.577366  28.561857      0  \n","3            0  B1003 -120.610829  34.632093      0  \n","4            0  B1004  -80.577366  28.561857      0  "]},"execution_count":8,"metadata":{},"output_type":"execute_result"}],"source":["data.head()"]},{"cell_type":"code","execution_count":9,"id":"5c11cd90-d12d-4470-b631-ba00359908b9","metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>FlightNumber</th>\n","      <th>PayloadMass</th>\n","      <th>Flights</th>\n","      <th>Block</th>\n","      <th>ReusedCount</th>\n","      <th>Orbit_ES-L1</th>\n","      <th>Orbit_GEO</th>\n","      <th>Orbit_GTO</th>\n","      <th>Orbit_HEO</th>\n","      <th>Orbit_ISS</th>\n","      <th>...</th>\n","      <th>Serial_B1058</th>\n","      <th>Serial_B1059</th>\n","      <th>Serial_B1060</th>\n","      <th>Serial_B1062</th>\n","      <th>GridFins_False</th>\n","      <th>GridFins_True</th>\n","      <th>Reused_False</th>\n","      <th>Reused_True</th>\n","      <th>Legs_False</th>\n","      <th>Legs_True</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>1.0</td>\n","      <td>6104.959412</td>\n","      <td>1.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>...</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>2.0</td>\n","      <td>525.000000</td>\n","      <td>1.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>...</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>3.0</td>\n","      <td>677.000000</td>\n","      <td>1.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>...</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>4.0</td>\n","      <td>500.000000</td>\n","      <td>1.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>...</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>5.0</td>\n","      <td>3170.000000</td>\n","      <td>1.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>...</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>85</th>\n","      <td>86.0</td>\n","      <td>15400.000000</td>\n","      <td>2.0</td>\n","      <td>5.0</td>\n","      <td>2.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>...</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","    </tr>\n","    <tr>\n","      <th>86</th>\n","      <td>87.0</td>\n","      <td>15400.000000</td>\n","      <td>3.0</td>\n","      <td>5.0</td>\n","      <td>2.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>...</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","    </tr>\n","    <tr>\n","      <th>87</th>\n","      <td>88.0</td>\n","      <td>15400.000000</td>\n","      <td>6.0</td>\n","      <td>5.0</td>\n","      <td>5.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>...</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","    </tr>\n","    <tr>\n","      <th>88</th>\n","      <td>89.0</td>\n","      <td>15400.000000</td>\n","      <td>3.0</td>\n","      <td>5.0</td>\n","      <td>2.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>...</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","    </tr>\n","    <tr>\n","      <th>89</th>\n","      <td>90.0</td>\n","      <td>3681.000000</td>\n","      <td>1.0</td>\n","      <td>5.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>...</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","      <td>1.0</td>\n","      <td>0.0</td>\n","      <td>0.0</td>\n","      <td>1.0</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>90 rows × 83 columns</p>\n","</div>"],"text/plain":["    FlightNumber   PayloadMass  Flights  Block  ReusedCount  Orbit_ES-L1  \\\n","0            1.0   6104.959412      1.0    1.0          0.0          0.0   \n","1            2.0    525.000000      1.0    1.0          0.0          0.0   \n","2            3.0    677.000000      1.0    1.0          0.0          0.0   \n","3            4.0    500.000000      1.0    1.0          0.0          0.0   \n","4            5.0   3170.000000      1.0    1.0          0.0          0.0   \n","..           ...           ...      ...    ...          ...          ...   \n","85          86.0  15400.000000      2.0    5.0          2.0          0.0   \n","86          87.0  15400.000000      3.0    5.0          2.0          0.0   \n","87          88.0  15400.000000      6.0    5.0          5.0          0.0   \n","88          89.0  15400.000000      3.0    5.0          2.0          0.0   \n","89          90.0   3681.000000      1.0    5.0          0.0          0.0   \n","\n","    Orbit_GEO  Orbit_GTO  Orbit_HEO  Orbit_ISS  ...  Serial_B1058  \\\n","0         0.0        0.0        0.0        0.0  ...           0.0   \n","1         0.0        0.0        0.0        0.0  ...           0.0   \n","2         0.0        0.0        0.0        1.0  ...           0.0   \n","3         0.0        0.0        0.0        0.0  ...           0.0   \n","4         0.0        1.0        0.0        0.0  ...           0.0   \n","..        ...        ...        ...        ...  ...           ...   \n","85        0.0        0.0        0.0        0.0  ...           0.0   \n","86        0.0        0.0        0.0        0.0  ...           1.0   \n","87        0.0        0.0        0.0        0.0  ...           0.0   \n","88        0.0        0.0        0.0        0.0  ...           0.0   \n","89        0.0        0.0        0.0        0.0  ...           0.0   \n","\n","    Serial_B1059  Serial_B1060  Serial_B1062  GridFins_False  GridFins_True  \\\n","0            0.0           0.0           0.0             1.0            0.0   \n","1            0.0           0.0           0.0             1.0            0.0   \n","2            0.0           0.0           0.0             1.0            0.0   \n","3            0.0           0.0           0.0             1.0            0.0   \n","4            0.0           0.0           0.0             1.0            0.0   \n","..           ...           ...           ...             ...            ...   \n","85           0.0           1.0           0.0             0.0            1.0   \n","86           0.0           0.0           0.0             0.0            1.0   \n","87           0.0           0.0           0.0             0.0            1.0   \n","88           0.0           1.0           0.0             0.0            1.0   \n","89           0.0           0.0           1.0             0.0            1.0   \n","\n","    Reused_False  Reused_True  Legs_False  Legs_True  \n","0            1.0          0.0         1.0        0.0  \n","1            1.0          0.0         1.0        0.0  \n","2            1.0          0.0         1.0        0.0  \n","3            1.0          0.0         1.0        0.0  \n","4            1.0          0.0         1.0        0.0  \n","..           ...          ...         ...        ...  \n","85           0.0          1.0         0.0        1.0  \n","86           0.0          1.0         0.0        1.0  \n","87           0.0          1.0         0.0        1.0  \n","88           0.0          1.0         0.0        1.0  \n","89           1.0          0.0         0.0        1.0  \n","\n","[90 rows x 83 columns]"]},"execution_count":9,"metadata":{},"output_type":"execute_result"}],"source":["X.head(100)"]},{"cell_type":"markdown","id":"3a4e605f-1cfe-43de-923a-a67a6b06a675","metadata":{},"source":["## TASK  1\n"]},{"cell_type":"markdown","id":"b6d883d4-ff87-4ceb-ae40-1c9df7161648","metadata":{},"source":["Create a NumPy array from the column <code>Class</code> in <code>data</code>, by applying the method <code>to_numpy()</code>  then\n","assign it  to the variable <code>Y</code>,make sure the output is a  Pandas series (only one bracket df\\['name of  column']).\n"]},{"cell_type":"code","execution_count":10,"id":"2d3ba84c-fa4a-4998-93d7-90c848010e38","metadata":{},"outputs":[],"source":["Y = data['Class'].to_numpy()"]},{"cell_type":"markdown","id":"5b454493-dcbf-42d1-a046-81e06448277f","metadata":{},"source":["## TASK  2\n"]},{"cell_type":"markdown","id":"c15b284d-f7ba-4dc3-aeb7-018255cdd468","metadata":{},"source":["Standardize the data in <code>X</code> then reassign it to the variable  <code>X</code> using the transform provided below.\n"]},{"cell_type":"code","execution_count":14,"id":"d3c3a495-a332-4ee7-9bbe-0e262e21440a","metadata":{},"outputs":[],"source":["# students get this \n","transform = preprocessing.StandardScaler()\n","\n","X = transform.fit_transform(X)"]},{"cell_type":"markdown","id":"3f3dba04-3959-4492-92cb-6a56104713bd","metadata":{},"source":["We split the data into training and testing data using the  function  <code>train_test_split</code>.   The training data is divided into validation data, a second set used for training  data; then the models are trained and hyperparameters are selected using the function <code>GridSearchCV</code>.\n"]},{"cell_type":"markdown","id":"d188f29b-129b-4a6d-846d-9458d7a69e79","metadata":{},"source":["## TASK  3\n"]},{"cell_type":"markdown","id":"3f0eeda1-00d1-41a3-9426-06d204c5cec3","metadata":{},"source":["Use the function train_test_split to split the data X and Y into training and test data. Set the parameter test_size to  0.2 and random_state to 2. The training data and test data should be assigned to the following labels.\n"]},{"cell_type":"markdown","id":"3b6ef0a6-19f2-4647-9a48-65c986b9a59b","metadata":{},"source":["<code>X_train, X_test, Y_train, Y_test</code>\n"]},{"cell_type":"code","execution_count":15,"id":"d8195594-5f62-425d-b1a0-df8e8d084804","metadata":{},"outputs":[],"source":["X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size =0.2, random_state= 2)"]},{"cell_type":"markdown","id":"3b4e7269-00fc-4392-b0a3-e9625e3e822b","metadata":{},"source":["we can see we only have 18 test samples.\n"]},{"cell_type":"code","execution_count":16,"id":"56da499a-e306-42c9-9d59-92d6df9b0202","metadata":{},"outputs":[{"data":{"text/plain":["(18,)"]},"execution_count":16,"metadata":{},"output_type":"execute_result"}],"source":["Y_test.shape"]},{"cell_type":"markdown","id":"a79ad4c5-c097-4eca-9efb-0c3e52a90c40","metadata":{},"source":["## TASK  4\n"]},{"cell_type":"markdown","id":"ac37da9c-8fc5-498f-b281-ffa0d4f41434","metadata":{},"source":["Create a logistic regression object  then create a  GridSearchCV object  <code>logreg_cv</code> with cv = 10.  Fit the object to find the best parameters from the dictionary <code>parameters</code>.\n"]},{"cell_type":"code","execution_count":27,"id":"15c3b431-1f27-425b-926e-69ace6cc903c","metadata":{},"outputs":[],"source":["parameters ={'C':[0.01,0.1,1],\n","             'penalty':['l2'],\n","             'solver':['lbfgs']}"]},{"cell_type":"code","execution_count":28,"id":"98182035-f928-4609-8bc5-267c1add00ac","metadata":{},"outputs":[{"data":{"text/html":["<style>#sk-container-id-3 {\n","  /* Definition of color scheme common for light and dark mode */\n","  --sklearn-color-text: black;\n","  --sklearn-color-line: gray;\n","  /* Definition of color scheme for unfitted estimators */\n","  --sklearn-color-unfitted-level-0: #fff5e6;\n","  --sklearn-color-unfitted-level-1: #f6e4d2;\n","  --sklearn-color-unfitted-level-2: #ffe0b3;\n","  --sklearn-color-unfitted-level-3: chocolate;\n","  /* Definition of color scheme for fitted estimators */\n","  --sklearn-color-fitted-level-0: #f0f8ff;\n","  --sklearn-color-fitted-level-1: #d4ebff;\n","  --sklearn-color-fitted-level-2: #b3dbfd;\n","  --sklearn-color-fitted-level-3: cornflowerblue;\n","\n","  /* Specific color for light theme */\n","  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n","  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n","  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n","  --sklearn-color-icon: #696969;\n","\n","  @media (prefers-color-scheme: dark) {\n","    /* Redefinition of color scheme for dark theme */\n","    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n","    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n","    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n","    --sklearn-color-icon: #878787;\n","  }\n","}\n","\n","#sk-container-id-3 {\n","  color: var(--sklearn-color-text);\n","}\n","\n","#sk-container-id-3 pre {\n","  padding: 0;\n","}\n","\n","#sk-container-id-3 input.sk-hidden--visually {\n","  border: 0;\n","  clip: rect(1px 1px 1px 1px);\n","  clip: rect(1px, 1px, 1px, 1px);\n","  height: 1px;\n","  margin: -1px;\n","  overflow: hidden;\n","  padding: 0;\n","  position: absolute;\n","  width: 1px;\n","}\n","\n","#sk-container-id-3 div.sk-dashed-wrapped {\n","  border: 1px dashed var(--sklearn-color-line);\n","  margin: 0 0.4em 0.5em 0.4em;\n","  box-sizing: border-box;\n","  padding-bottom: 0.4em;\n","  background-color: var(--sklearn-color-background);\n","}\n","\n","#sk-container-id-3 div.sk-container {\n","  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n","     but bootstrap.min.css set `[hidden] { display: none !important; }`\n","     so we also need the `!important` here to be able to override the\n","     default hidden behavior on the sphinx rendered scikit-learn.org.\n","     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n","  display: inline-block !important;\n","  position: relative;\n","}\n","\n","#sk-container-id-3 div.sk-text-repr-fallback {\n","  display: none;\n","}\n","\n","div.sk-parallel-item,\n","div.sk-serial,\n","div.sk-item {\n","  /* draw centered vertical line to link estimators */\n","  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n","  background-size: 2px 100%;\n","  background-repeat: no-repeat;\n","  background-position: center center;\n","}\n","\n","/* Parallel-specific style estimator block */\n","\n","#sk-container-id-3 div.sk-parallel-item::after {\n","  content: \"\";\n","  width: 100%;\n","  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n","  flex-grow: 1;\n","}\n","\n","#sk-container-id-3 div.sk-parallel {\n","  display: flex;\n","  align-items: stretch;\n","  justify-content: center;\n","  background-color: var(--sklearn-color-background);\n","  position: relative;\n","}\n","\n","#sk-container-id-3 div.sk-parallel-item {\n","  display: flex;\n","  flex-direction: column;\n","}\n","\n","#sk-container-id-3 div.sk-parallel-item:first-child::after {\n","  align-self: flex-end;\n","  width: 50%;\n","}\n","\n","#sk-container-id-3 div.sk-parallel-item:last-child::after {\n","  align-self: flex-start;\n","  width: 50%;\n","}\n","\n","#sk-container-id-3 div.sk-parallel-item:only-child::after {\n","  width: 0;\n","}\n","\n","/* Serial-specific style estimator block */\n","\n","#sk-container-id-3 div.sk-serial {\n","  display: flex;\n","  flex-direction: column;\n","  align-items: center;\n","  background-color: var(--sklearn-color-background);\n","  padding-right: 1em;\n","  padding-left: 1em;\n","}\n","\n","\n","/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n","clickable and can be expanded/collapsed.\n","- Pipeline and ColumnTransformer use this feature and define the default style\n","- Estimators will overwrite some part of the style using the `sk-estimator` class\n","*/\n","\n","/* Pipeline and ColumnTransformer style (default) */\n","\n","#sk-container-id-3 div.sk-toggleable {\n","  /* Default theme specific background. It is overwritten whether we have a\n","  specific estimator or a Pipeline/ColumnTransformer */\n","  background-color: var(--sklearn-color-background);\n","}\n","\n","/* Toggleable label */\n","#sk-container-id-3 label.sk-toggleable__label {\n","  cursor: pointer;\n","  display: block;\n","  width: 100%;\n","  margin-bottom: 0;\n","  padding: 0.5em;\n","  box-sizing: border-box;\n","  text-align: center;\n","}\n","\n","#sk-container-id-3 label.sk-toggleable__label-arrow:before {\n","  /* Arrow on the left of the label */\n","  content: \"▸\";\n","  float: left;\n","  margin-right: 0.25em;\n","  color: var(--sklearn-color-icon);\n","}\n","\n","#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {\n","  color: var(--sklearn-color-text);\n","}\n","\n","/* Toggleable content - dropdown */\n","\n","#sk-container-id-3 div.sk-toggleable__content {\n","  max-height: 0;\n","  max-width: 0;\n","  overflow: hidden;\n","  text-align: left;\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-3 div.sk-toggleable__content.fitted {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","#sk-container-id-3 div.sk-toggleable__content pre {\n","  margin: 0.2em;\n","  border-radius: 0.25em;\n","  color: var(--sklearn-color-text);\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-3 div.sk-toggleable__content.fitted pre {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n","  /* Expand drop-down */\n","  max-height: 200px;\n","  max-width: 100%;\n","  overflow: auto;\n","}\n","\n","#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n","  content: \"▾\";\n","}\n","\n","/* Pipeline/ColumnTransformer-specific style */\n","\n","#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-3 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Estimator-specific style */\n","\n","/* Colorize estimator box */\n","#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-3 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","#sk-container-id-3 div.sk-label label.sk-toggleable__label,\n","#sk-container-id-3 div.sk-label label {\n","  /* The background is the default theme color */\n","  color: var(--sklearn-color-text-on-default-background);\n","}\n","\n","/* On hover, darken the color of the background */\n","#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","/* Label box, darken color on hover, fitted */\n","#sk-container-id-3 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Estimator label */\n","\n","#sk-container-id-3 div.sk-label label {\n","  font-family: monospace;\n","  font-weight: bold;\n","  display: inline-block;\n","  line-height: 1.2em;\n","}\n","\n","#sk-container-id-3 div.sk-label-container {\n","  text-align: center;\n","}\n","\n","/* Estimator-specific */\n","#sk-container-id-3 div.sk-estimator {\n","  font-family: monospace;\n","  border: 1px dotted var(--sklearn-color-border-box);\n","  border-radius: 0.25em;\n","  box-sizing: border-box;\n","  margin-bottom: 0.5em;\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-3 div.sk-estimator.fitted {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","/* on hover */\n","#sk-container-id-3 div.sk-estimator:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-3 div.sk-estimator.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Specification for estimator info (e.g. \"i\" and \"?\") */\n","\n","/* Common style for \"i\" and \"?\" */\n","\n",".sk-estimator-doc-link,\n","a:link.sk-estimator-doc-link,\n","a:visited.sk-estimator-doc-link {\n","  float: right;\n","  font-size: smaller;\n","  line-height: 1em;\n","  font-family: monospace;\n","  background-color: var(--sklearn-color-background);\n","  border-radius: 1em;\n","  height: 1em;\n","  width: 1em;\n","  text-decoration: none !important;\n","  margin-left: 1ex;\n","  /* unfitted */\n","  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-unfitted-level-1);\n","}\n","\n",".sk-estimator-doc-link.fitted,\n","a:link.sk-estimator-doc-link.fitted,\n","a:visited.sk-estimator-doc-link.fitted {\n","  /* fitted */\n","  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-fitted-level-1);\n","}\n","\n","/* On hover */\n","div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",".sk-estimator-doc-link:hover,\n","div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",".sk-estimator-doc-link:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",".sk-estimator-doc-link.fitted:hover,\n","div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",".sk-estimator-doc-link.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","/* Span, style for the box shown on hovering the info icon */\n",".sk-estimator-doc-link span {\n","  display: none;\n","  z-index: 9999;\n","  position: relative;\n","  font-weight: normal;\n","  right: .2ex;\n","  padding: .5ex;\n","  margin: .5ex;\n","  width: min-content;\n","  min-width: 20ex;\n","  max-width: 50ex;\n","  color: var(--sklearn-color-text);\n","  box-shadow: 2pt 2pt 4pt #999;\n","  /* unfitted */\n","  background: var(--sklearn-color-unfitted-level-0);\n","  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n","}\n","\n",".sk-estimator-doc-link.fitted span {\n","  /* fitted */\n","  background: var(--sklearn-color-fitted-level-0);\n","  border: var(--sklearn-color-fitted-level-3);\n","}\n","\n",".sk-estimator-doc-link:hover span {\n","  display: block;\n","}\n","\n","/* \"?\"-specific style due to the `<a>` HTML tag */\n","\n","#sk-container-id-3 a.estimator_doc_link {\n","  float: right;\n","  font-size: 1rem;\n","  line-height: 1em;\n","  font-family: monospace;\n","  background-color: var(--sklearn-color-background);\n","  border-radius: 1rem;\n","  height: 1rem;\n","  width: 1rem;\n","  text-decoration: none;\n","  /* unfitted */\n","  color: var(--sklearn-color-unfitted-level-1);\n","  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n","}\n","\n","#sk-container-id-3 a.estimator_doc_link.fitted {\n","  /* fitted */\n","  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-fitted-level-1);\n","}\n","\n","/* On hover */\n","#sk-container-id-3 a.estimator_doc_link:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","#sk-container-id-3 a.estimator_doc_link.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-3);\n","}\n","</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=10, estimator=LogisticRegression(),\n","             param_grid={&#x27;C&#x27;: [0.01, 0.1, 1], &#x27;penalty&#x27;: [&#x27;l2&#x27;],\n","                         &#x27;solver&#x27;: [&#x27;lbfgs&#x27;]})</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-7\" type=\"checkbox\" ><label for=\"sk-estimator-id-7\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;GridSearchCV<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.model_selection.GridSearchCV.html\">?<span>Documentation for GridSearchCV</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>GridSearchCV(cv=10, estimator=LogisticRegression(),\n","             param_grid={&#x27;C&#x27;: [0.01, 0.1, 1], &#x27;penalty&#x27;: [&#x27;l2&#x27;],\n","                         &#x27;solver&#x27;: [&#x27;lbfgs&#x27;]})</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-8\" type=\"checkbox\" ><label for=\"sk-estimator-id-8\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">best_estimator_: LogisticRegression</label><div class=\"sk-toggleable__content fitted\"><pre>LogisticRegression(C=0.01)</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-9\" type=\"checkbox\" ><label for=\"sk-estimator-id-9\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;LogisticRegression<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.linear_model.LogisticRegression.html\">?<span>Documentation for LogisticRegression</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>LogisticRegression(C=0.01)</pre></div> </div></div></div></div></div></div></div></div></div>"],"text/plain":["GridSearchCV(cv=10, estimator=LogisticRegression(),\n","             param_grid={'C': [0.01, 0.1, 1], 'penalty': ['l2'],\n","                         'solver': ['lbfgs']})"]},"execution_count":28,"metadata":{},"output_type":"execute_result"}],"source":["lr=LogisticRegression()\n","logreg_cv = GridSearchCV(lr, parameters, cv=10)\n","logreg_cv.fit(X_train, Y_train)"]},{"cell_type":"markdown","id":"eddb648c-f42a-491a-aba0-e5b2f29d5ee9","metadata":{},"source":["We output the <code>GridSearchCV</code> object for logistic regression. We display the best parameters using the data attribute <code>best_params\\_</code> and the accuracy on the validation data using the data attribute <code>best_score\\_</code>.\n"]},{"cell_type":"code","execution_count":29,"id":"6e21922b-b38d-41d6-aba2-dd07c69bd8f5","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["tuned hpyerparameters :(best parameters)  {'C': 0.01, 'penalty': 'l2', 'solver': 'lbfgs'}\n","accuracy : 0.8464285714285713\n"]}],"source":["print(\"tuned hpyerparameters :(best parameters) \",logreg_cv.best_params_)\n","print(\"accuracy :\",logreg_cv.best_score_)"]},{"cell_type":"markdown","id":"7c73150d-d4a4-425c-8696-b2314ae73c83","metadata":{},"source":["## TASK  5\n"]},{"cell_type":"markdown","id":"f90dbb3d-176d-48e2-9b0a-2f19061be34e","metadata":{},"source":["Calculate the accuracy on the test data using the method <code>score</code>:\n"]},{"cell_type":"code","execution_count":34,"id":"1ee881e8-5161-4056-8545-4677f067d044","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["MSE =  0.16666666666666666\n","Variance score: 0.87\n"]}],"source":["y_hat = logreg_cv.predict(X_test)\n","print(\"MSE = \", np.mean((y_hat-Y_test)**2))\n","print('Variance score: %.2f' % logreg_cv.score(X, Y))\n"]},{"cell_type":"markdown","id":"0b0244e8-0d09-45c2-8cbd-99e0b15d6700","metadata":{},"source":["Lets look at the confusion matrix:\n"]},{"cell_type":"code","execution_count":35,"id":"ba33d67e-d377-4d82-8d24-93e3ce9e3a3a","metadata":{},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAhIAAAHHCAYAAADqJrG+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAABD30lEQVR4nO3dd3RU1fr/8c+EkEkISQg9USABFIJUQbh0uESKghSV6jU0vQpIL3IVSACNghRBEUEFRBS5IiCoKFX0ggiEANJLKNKlSguQOb8//DFfxySaDGc4k+H9cs1amX327P2cLAcedjs2wzAMAQAAuMHP6gAAAEDORSIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBeNDevXvVuHFjhYWFyWazaeHChaa2f/DgQdlsNs2cOdPUdnOyBg0aqEGDBlaHAdw1SCTg8/bv369///vfKlmypAIDAxUaGqratWvrzTff1NWrVz3ad1xcnLZt26ZXXnlFs2fPVrVq1Tza353UuXNn2Ww2hYaGZvh73Lt3r2w2m2w2m954441st3/s2DHFx8crOTnZhGgBeIq/1QEAnvTll1/qySeflN1u19NPP63y5cvr+vXr+uGHHzRo0CBt375d06ZN80jfV69e1bp16/TSSy+pV69eHumjRIkSunr1qnLnzu2R9v+Ov7+/rly5osWLF6tt27Yu1+bMmaPAwEBdu3bNrbaPHTumhIQERUVFqXLlyln+3LfffutWfwDcQyIBn5WSkqL27durRIkSWrlypSIiIpzXevbsqX379unLL7/0WP+nT5+WJOXLl89jfdhsNgUGBnqs/b9jt9tVu3ZtffLJJ+kSiY8//liPPvqo5s+ff0diuXLlivLkyaOAgIA70h+A3zG1AZ81ZswYXbp0Se+//75LEnFL6dKl1adPH+f7mzdvatSoUSpVqpTsdruioqL0n//8R6mpqS6fi4qKUvPmzfXDDz+oevXqCgwMVMmSJfXhhx8668THx6tEiRKSpEGDBslmsykqKkrS71MCt37+o/j4eNlsNpeyZcuWqU6dOsqXL5/y5s2rMmXK6D//+Y/zemZrJFauXKm6desqODhY+fLlU8uWLbVz584M+9u3b586d+6sfPnyKSwsTF26dNGVK1cy/8X+SceOHfX111/r/PnzzrINGzZo79696tixY7r6Z8+e1cCBA1WhQgXlzZtXoaGhatasmbZs2eKss3r1aj300EOSpC5dujinSG7dZ4MGDVS+fHlt2rRJ9erVU548eZy/lz+vkYiLi1NgYGC6+2/SpInCw8N17NixLN8rgPRIJOCzFi9erJIlS6pWrVpZqt+9e3cNHz5cDz74oCZMmKD69esrMTFR7du3T1d33759euKJJ/Twww9r3LhxCg8PV+fOnbV9+3ZJUps2bTRhwgRJUocOHTR79mxNnDgxW/Fv375dzZs3V2pqqkaOHKlx48bpscce0//+97+//Nzy5cvVpEkTnTp1SvHx8erfv7/Wrl2r2rVr6+DBg+nqt23bVr/99psSExPVtm1bzZw5UwkJCVmOs02bNrLZbPr888+dZR9//LHKli2rBx98MF39AwcOaOHChWrevLnGjx+vQYMGadu2bapfv77zL/WYmBiNHDlSkvTss89q9uzZmj17turVq+ds58yZM2rWrJkqV66siRMnqmHDhhnG9+abb6pQoUKKi4tTWlqaJOndd9/Vt99+q8mTJysyMjLL9wogAwbggy5cuGBIMlq2bJml+snJyYYko3v37i7lAwcONCQZK1eudJaVKFHCkGSsWbPGWXbq1CnDbrcbAwYMcJalpKQYkoyxY8e6tBkXF2eUKFEiXQwjRoww/viVnDBhgiHJOH36dKZx3+pjxowZzrLKlSsbhQsXNs6cOeMs27Jli+Hn52c8/fTT6frr2rWrS5utW7c2ChQokGmff7yP4OBgwzAM44knnjAaNWpkGIZhpKWlGUWLFjUSEhIy/B1cu3bNSEtLS3cfdrvdGDlypLNsw4YN6e7tlvr16xuSjKlTp2Z4rX79+i5l33zzjSHJGD16tHHgwAEjb968RqtWrf72HgH8PUYk4JMuXrwoSQoJCclS/a+++kqS1L9/f5fyAQMGSFK6tRTlypVT3bp1ne8LFSqkMmXK6MCBA27H/Ge31lYsWrRIDocjS585fvy4kpOT1blzZ+XPn99ZXrFiRT388MPO+/yj5557zuV93bp1debMGefvMCs6duyo1atX68SJE1q5cqVOnDiR4bSG9Pu6Cj+/3//oSUtL05kzZ5zTNklJSVnu0263q0uXLlmq27hxY/373//WyJEj1aZNGwUGBurdd9/Ncl8AMkciAZ8UGhoqSfrtt9+yVP/QoUPy8/NT6dKlXcqLFi2qfPny6dChQy7lxYsXT9dGeHi4zp0752bE6bVr1061a9dW9+7dVaRIEbVv317z5s37y6TiVpxlypRJdy0mJka//vqrLl++7FL+53sJDw+XpGzdyyOPPKKQkBB9+umnmjNnjh566KF0v8tbHA6HJkyYoPvuu092u10FCxZUoUKFtHXrVl24cCHLfd5zzz3ZWlj5xhtvKH/+/EpOTtakSZNUuHDhLH8WQOZIJOCTQkNDFRkZqZ9//jlbn/vzYsfM5MqVK8NywzDc7uPW/P0tQUFBWrNmjZYvX65//etf2rp1q9q1a6eHH344Xd3bcTv3covdblebNm00a9YsLViwINPRCEl69dVX1b9/f9WrV08fffSRvvnmGy1btkwPPPBAlkdepN9/P9mxefNmnTp1SpK0bdu2bH0WQOZIJOCzmjdvrv3792vdunV/W7dEiRJyOBzau3evS/nJkyd1/vx55w4MM4SHh7vscLjlz6MekuTn56dGjRpp/Pjx2rFjh1555RWtXLlSq1atyrDtW3Hu3r073bVdu3apYMGCCg4Ovr0byETHjh21efNm/fbbbxkuUL3ls88+U8OGDfX++++rffv2aty4sWJjY9P9TrKa1GXF5cuX1aVLF5UrV07PPvusxowZow0bNpjWPnA3I5GAzxo8eLCCg4PVvXt3nTx5Mt31/fv3680335T0+9C8pHQ7K8aPHy9JevTRR02Lq1SpUrpw4YK2bt3qLDt+/LgWLFjgUu/s2bPpPnvrYKY/b0m9JSIiQpUrV9asWbNc/mL++eef9e233zrv0xMaNmyoUaNG6a233lLRokUzrZcrV650ox3//e9/dfToUZeyWwlPRklXdg0ZMkSHDx/WrFmzNH78eEVFRSkuLi7T3yOArONAKvisUqVK6eOPP1a7du0UExPjcrLl2rVr9d///ledO3eWJFWqVElxcXGaNm2azp8/r/r16+unn37SrFmz1KpVq0y3Frqjffv2GjJkiFq3bq3evXvrypUreuedd3T//fe7LDYcOXKk1qxZo0cffVQlSpTQqVOnNGXKFN17772qU6dOpu2PHTtWzZo1U82aNdWtWzddvXpVkydPVlhYmOLj4027jz/z8/PTyy+//Lf1mjdvrpEjR6pLly6qVauWtm3bpjlz5qhkyZIu9UqVKqV8+fJp6tSpCgkJUXBwsGrUqKHo6OhsxbVy5UpNmTJFI0aMcG5HnTFjhho0aKBhw4ZpzJgx2WoPwJ9YvGsE8Lg9e/YYzzzzjBEVFWUEBAQYISEhRu3atY3Jkycb165dc9a7ceOGkZCQYERHRxu5c+c2ihUrZgwdOtSljmH8vv3z0UcfTdfPn7cdZrb90zAM49tvvzXKly9vBAQEGGXKlDE++uijdNs/V6xYYbRs2dKIjIw0AgICjMjISKNDhw7Gnj170vXx5y2Sy5cvN2rXrm0EBQUZoaGhRosWLYwdO3a41LnV35+3l86YMcOQZKSkpGT6OzUM1+2fmcls++eAAQOMiIgIIygoyKhdu7axbt26DLdtLlq0yChXrpzh7+/vcp/169c3HnjggQz7/GM7Fy9eNEqUKGE8+OCDxo0bN1zq9evXz/Dz8zPWrVv3l/cA4K/ZDCMbK6oAAAD+gDUSAADAbSQSAADAbSQSAADAbSQSAAD4qDVr1qhFixaKjIyUzWbTwoULnddu3LihIUOGqEKFCgoODlZkZKSefvrpbD8Rl0QCAAAfdfnyZVWqVElvv/12umtXrlxRUlKShg0bpqSkJH3++efavXu3HnvssWz1wa4NAADuAjabTQsWLFCrVq0yrbNhwwZVr15dhw4dyvCZQhnhQCoAAHKI1NTUdCey2u122e12U9q/cOGCbDab8+nDWeGTiUTXqCesDgEAkEN8cPAzj/dx49cDprST+NaHSkhIcCkbMWKEKafWXrt2TUOGDFGHDh2cT1DOCp9MJAAA8EVDhw5V//79XcrMGI24ceOG2rZtK8Mw9M4772TrsyQSAAB4miPNlGbMnMa45VYScejQIa1cuTJboxESiQQAAJ5nOKyOIEO3koi9e/dq1apVKlCgQLbbIJEAAMDTHNYkEpcuXdK+ffuc71NSUpScnKz8+fMrIiJCTzzxhJKSkrRkyRKlpaXpxIkTkqT8+fMrICAgS3345PZPFlsCALLqjiy2PL7TlHZyR8Rkq/7q1avVsGHDdOVxcXGKj49XdHR0hp9btWqVGjRokKU+GJEAAMDDDIumNho0aKC/Gi8wYyyBRAIAAE+zaGrjTuCIbAAA4DZGJAAA8DQv3bVhBhIJAAA8zaRzJLwRUxsAAMBtjEgAAOBpTG0AAAC3sWsDAAAgPUYkAADwMKsOpLoTSCQAAPA0H57aIJEAAMDTfHhEgjUSAADAbYxIAADgaT58IBWJBAAAnsbUBgAAQHqMSAAA4Gns2gAAAG5jagMAACA9RiQAAPA0pjYAAIC7DMN3t38ytQEAANzGiAQAAJ7mw4stSSQAAPA01kgAAAC3+fCIBGskAACA2xiRAADA03hoFwAAcBtTGwAAAOkxIgEAgKexawMAALiNqQ0AAID0GJEAAMDTmNoAAABu8+FEgqkNAADgNkYkAADwMF9+jDiJBAAAnubDUxskEgAAeBrbPwEAANJjRAIAAE9jagMAALiNqQ0AAID0GJEAAMDTmNoAAABuY2oDAAAgPUYkAADwNKY2AACA20gkzBUeHi6bzZalumfPnvVwNAAAwF2WJBITJ050/nzmzBmNHj1aTZo0Uc2aNSVJ69at0zfffKNhw4ZZER4AAOby4cWWNsMwDCsDePzxx9WwYUP16tXLpfytt97S8uXLtXDhwmy32TXqCZOiAwD4ug8OfubxPq5+8YYp7QQ9NtCUdsxk+a6Nb775Rk2bNk1X3rRpUy1fvtyCiAAAMJnhMOflhSxPJAoUKKBFixalK1+0aJEKFChgQUQAAPiGNWvWqEWLFoqMjJTNZks3ym8YhoYPH66IiAgFBQUpNjZWe/fuzVYflu/aSEhIUPfu3bV69WrVqFFDkrR+/XotXbpU06dPtzg6AABMYNGujcuXL6tSpUrq2rWr2rRpk+76mDFjNGnSJM2aNUvR0dEaNmyYmjRpoh07digwMDBLfVieSHTu3FkxMTGaNGmSPv/8c0lSTEyMfvjhB2diAQBAjmbRtESzZs3UrFmzDK8ZhqGJEyfq5ZdfVsuWLSVJH374oYoUKaKFCxeqffv2WerD8kRCkmrUqKE5c+ZYHQYAAHeNlJQUnThxQrGxsc6ysLAw1ahRQ+vWrctZiYTD4dC+fft06tQpOf40/FOvXj2LogIAwCQmTW2kpqYqNTXVpcxut8tut2e7rRMnTkiSihQp4lJepEgR57WssDyR+PHHH9WxY0cdOnRIf96JarPZlJaWZlFkAACYxKREIjExUQkJCS5lI0aMUHx8vCntu8PyROK5555TtWrV9OWXXyoiIiLLJ14CAHC3GTp0qPr37+9S5s5ohCQVLVpUknTy5ElFREQ4y0+ePKnKlStnuR3LE4m9e/fqs88+U+nSpa0OBQAAzzDp7Ed3pzEyEh0draJFi2rFihXOxOHixYtav369nn/++Sy3Y3kiUaNGDe3bt49EAgDguyza/nnp0iXt27fP+T4lJUXJycnKnz+/ihcvrr59+2r06NG67777nNs/IyMj1apVqyz3YXki8cILL2jAgAE6ceKEKlSooNy5c7tcr1ixokWRAQCQs23cuFENGzZ0vr81LRIXF6eZM2dq8ODBunz5sp599lmdP39ederU0dKlS7N8hoTkBc/a8PNLf7imzWaTYRhuL7bkWRsAgKy6I8/amGPOQyiDOo0ypR0zWT4ikZKSYnUIAAB4lpc+J8MMlicSJUqUsDoEAAA8y6I1EneC5YnELTt27NDhw4d1/fp1l/LHHnvMoogAAMDfsTyROHDggFq3bq1t27Y510ZIcp4nwYFUAIAcz9rliB5l+WPE+/Tpo+joaJ06dUp58uTR9u3btWbNGlWrVk2rV6+2OjwAAG6fw2HOywtZPiKxbt06rVy5UgULFpSfn5/8/PxUp04dJSYmqnfv3tq8ebPVIQIAgExYPiKRlpamkJAQSVLBggV17NgxSb8vwty9e7eVoQEAYA5GJDynfPny2rJli6Kjo1WjRg2NGTNGAQEBmjZtmkqWLGl1eAAA3D62f3rOyy+/rMuXL0uSRo4cqebNm6tu3boqUKCAPv30U4ujAwAAf8XyRKJJkybOn0uXLq1du3bp7NmzCg8P50mgAACfYDh8d9eG5YlERvLnz291CAAAmMdL1zeYwZJEok2bNlmu+/nnn3swEgAAcDssSSTCwsKs6BYAAGuw2NJcM2bMsKJbAACswRoJAADgNh9eI2H5gVQAACDnYkQCAABP8+ERCRIJAAA8jad/es6HH36o1NTUdOXXr1/Xhx9+aEFEAAAgq2yGYW2alCtXLh0/flyFCxd2KT9z5owKFy6stLS0bLfZNeoJs8KDGxo81VgNOzVRwXsLSZKO7j2ixZM+07bVPMkVdze+G97pg4OfebyPK+OfMaWdPP2nm9KOmSyf2jAMI8OjsH/55RfOm8ihzh0/o89e/0gnDx6XzWZT7ccb6IVpgxX/6CAd2/uL1eEBluG7cRdj+6f5qlSpIpvNJpvNpkaNGsnf//9CSUtLU0pKipo2bWpVeLgNW1Zscnn/+RufqMFTjVWqyv38YYm7Gt8N+CLLEolWrVpJkpKTk9WkSRPlzZvXeS0gIEBRUVF6/PHHLYoOZrH5+emhR2vKHhSo/Ul7rA4H8Bp8N+4ynGxpvhEjRkiSoqKi1K5dOwUGBloVCjzgnjLF9dLnryi3PUCpV67prX+P0bF9/IsL4Ltxl/LhqQ3LF1vesmnTJu3cuVOS9MADD6hKlSpZ+lxqamq6XR8vVIhTLlsu02NE1uXK7a8CkQUVFJJH1R75h+q1a6TX243gD0zc9fhueJ87stjy9S6mtJNniPc9YsLyxZanTp1S+/bttXr1auXLl0+SdP78eTVs2FBz585VoUKF/vLziYmJSkhIcCmrHBajKvnKeSpkZEHajZs6deiEJOnQzwcUXbG0Yrs+og//M83iyABr8d24Oxk+fCCV5edIvPDCC/rtt9+0fft2nT17VmfPntXPP/+sixcvqnfv3n/7+aFDh+rChQsur4phZe5A5MgOm59N/gG5rQ4D8Dp8N+4SDsOclxeyfERi6dKlWr58uWJiYpxl5cqV09tvv63GjRv/7eftdrvsdrtLGdMa1np8cEdtW71ZZ479qsDgIP2jZR2V+ccDGv/0aKtDAyzFd+MuxmJLz3E4HMqdO302njt3bjl8eCjIl4UWCFP38S8orFC4rv52Rb/sOqTxT4/Wjh+2Wh0aYCm+G/BFlicS//znP9WnTx998sknioyMlCQdPXpU/fr1U6NGjSyODu6YMeQdq0MAvBLfjbuYl05LmMHyNRJvvfWWLl68qKioKJUqVUqlSpVSdHS0Ll68qMmTJ1sdHgAAt8/hMOflhSwfkShWrJiSkpK0fPly7dq1S5IUExOj2NhYiyMDAAB/x/JEQpJsNpsefvhhPfzww1aHAgCA+Xx4asMrEokVK1ZoxYoVOnXqVLoFlh988IFFUQEAYBJ2bXhOQkKCRo4cqWrVqikiIiLDJ4ECAADvZHkiMXXqVM2cOVP/+te/rA4FAADPYGrDc65fv65atWpZHQYAAB7DEdke1L17d3388cdWhwEAANxg+YjEtWvXNG3aNC1fvlwVK1ZMd8rl+PHjLYoMAACTMLXhOVu3blXlypUlST///LPLNRZeAgB8AomE56xatcrqEAAA8Cwf3v5p+RoJAACQc1k+IgEAgM9jagMAALjL8OFEgqkNAADgNkYkAADwNB8ekbAkkfjiiy+yXPexxx7zYCQAANwBPnyypSWJRKtWrVze22w2GYbh8v6WtLS0OxUWAADIJkvWSDgcDufr22+/VeXKlfX111/r/PnzOn/+vL766is9+OCDWrp0qRXhAQBgLodhzssLWb5Gom/fvpo6darq1KnjLGvSpIny5MmjZ599Vjt37rQwOgAATOClSYAZLN+1sX//fuXLly9deVhYmA4ePHjH4wEAAFlneSLx0EMPqX///jp58qSz7OTJkxo0aJCqV69uYWQAAJjDMAxTXtmRlpamYcOGKTo6WkFBQSpVqpRGjRqV7Xb+juVTGx988IFat26t4sWLq1ixYpKkI0eO6L777tPChQutDQ4AADNYMLXx+uuv65133tGsWbP0wAMPaOPGjerSpYvCwsLUu3dv0/qxPJEoXbq0tm7dqmXLlmnXrl2SpJiYGMXGxvL0TwCAb7AgkVi7dq1atmypRx99VJIUFRWlTz75RD/99JOp/VieSEi/b/ds3LixGjdubHUoAAB4rdTUVKWmprqU2e122e32dHVr1aqladOmac+ePbr//vu1ZcsW/fDDDxo/frypMVmSSEyaNEnPPvusAgMDNWnSpL+sa+bwCwAAVjDrWRuJiYlKSEhwKRsxYoTi4+PT1X3xxRd18eJFlS1bVrly5VJaWppeeeUVderUyZRYbrEZZq+6yILo6Ght3LhRBQoUUHR0dKb1bDabDhw4kO32u0Y9cTvhAQDuIh8c/MzjfVyIa2RKO4HTvsryiMTcuXM1aNAgjR07Vg888ICSk5PVt29fjR8/XnFxcabEI1k0IpGSkpLhzwAAIHOZJQ0ZGTRokF588UW1b99eklShQgUdOnRIiYmJOT+RAADgrmLBozauXLkiPz/XUx5y5colh8nP/bAkkejfv3+W65q9KAQAgDvNrDUS2dGiRQu98sorKl68uB544AFt3rxZ48ePV9euXU3tx5JEYvPmzS7vk5KSdPPmTZUpU0aStGfPHuXKlUtVq1a1IjwAAHK8yZMna9iwYerRo4dOnTqlyMhI/fvf/9bw4cNN7ceSRGLVqlXOn8ePH6+QkBDNmjVL4eHhkqRz586pS5cuqlu3rhXhAQBgLgtGJEJCQjRx4kRNnDjRo/1YfkT2uHHjlJiY6EwiJCk8PFyjR4/WuHHjLIwMAACTOEx6eSHLE4mLFy/q9OnT6cpPnz6t3377zYKIAABAVlm+a6N169bq0qWLxo0b53xI1/r16zVo0CC1adPG4ugAALh9Viy2vFMsTySmTp2qgQMHqmPHjrpx44Ykyd/fX926ddPYsWMtjg4AABN46bSEGSxPJPLkyaMpU6Zo7Nix2r9/vySpVKlSCg4OtjgyAADMwYjEHRAcHKyKFStaHQYAAMgGr0kkAADwWUxtAAAAdxk+nEhYvv0TAADkXIxIAADgaT48IkEiAQCAhzG1AQAAkAFGJAAA8DQfHpEgkQAAwMN8eWqDRAIAAA/z5USCNRIAAMBtjEgAAOBhvjwiQSIBAICnGTarI/AYpjYAAIDbTBmROH/+vPLly2dGUwAA+BxfntrI9ojE66+/rk8//dT5vm3btipQoIDuuecebdmyxdTgAADwBYbDZsrLG2U7kZg6daqKFSsmSVq2bJmWLVumr7/+Ws2aNdOgQYNMDxAAAHivbE9tnDhxwplILFmyRG3btlXjxo0VFRWlGjVqmB4gAAA5HVMbfxAeHq4jR45IkpYuXarY2FhJkmEYSktLMzc6AAB8gGHYTHl5o2yPSLRp00YdO3bUfffdpzNnzqhZs2aSpM2bN6t06dKmBwgAALxXthOJCRMmKCoqSkeOHNGYMWOUN29eSdLx48fVo0cP0wMEACCn8+WpjWwnErlz59bAgQPTlffr18+UgAAA8DXeuuPCDFlKJL744ossN/jYY4+5HQwAAL7IMKyOwHOylEi0atUqS43ZbDYWXAIAcBfJUiLhcPjw5A4AAB52109tZObatWsKDAw0KxYAAHySLycS2T5HIi0tTaNGjdI999yjvHnz6sCBA5KkYcOG6f333zc9QAAA4L2ynUi88sormjlzpsaMGaOAgABnefny5fXee++ZGhwAAL7AMMx5eaNsJxIffvihpk2bpk6dOilXrlzO8kqVKmnXrl2mBgcAgC/goV1/cPTo0QxPsHQ4HLpx44YpQQEAgJwh24lEuXLl9P3336cr/+yzz1SlShVTggIAwJfwrI0/GD58uOLi4nT06FE5HA59/vnn2r17tz788EMtWbLEEzECAJCj+fIR2dkekWjZsqUWL16s5cuXKzg4WMOHD9fOnTu1ePFiPfzww56IEQAAeCm3zpGoW7euli1bZnYsAAD4JIeXTkuYwe0DqTZu3KidO3dK+n3dRNWqVU0LCgAAX+Kt6xvMkO1E4pdfflGHDh30v//9T/ny5ZMknT9/XrVq1dLcuXN17733mh0jAAA5mrdu3TRDttdIdO/eXTdu3NDOnTt19uxZnT17Vjt37pTD4VD37t09ESMAAPBS2R6R+O6777R27VqVKVPGWVamTBlNnjxZdevWNTU4AAB8gbeeSmmGbCcSxYoVy/DgqbS0NEVGRpoSFAAAvoSpjT8YO3asXnjhBW3cuNFZtnHjRvXp00dvvPGGqcEBAADvlqURifDwcNls/5dNXb58WTVq1JC//+8fv3nzpvz9/dW1a1e1atXKI4ECAJBT3fXbPydOnOjhMAAA8F13/fbPuLg4T8cBAAByILcPpJKka9eu6fr16y5loaGhtxUQAAC+hl0bf3D58mUNGTJE8+bN05kzZ9JdT0tLMyUwAAB8hS+vkcj2ro3Bgwdr5cqVeuedd2S32/Xee+8pISFBkZGR+vDDDz0RIwAA8FLZTiQWL16sKVOm6PHHH5e/v7/q1q2rl19+Wa+++qrmzJnjiRgBAMjRDMNmyiu7jh49qqeeekoFChRQUFCQKlSo4HJ8gxmyPbVx9uxZlSxZUtLv6yHOnj0rSapTp46ef/55U4MDAMAXWLFG4ty5c6pdu7YaNmyor7/+WoUKFdLevXsVHh5uaj/ZTiRKliyplJQUFS9eXGXLltW8efNUvXp1LV682PkQLwAA8H+sWCPx+uuvq1ixYpoxY4azLDo62vR+sj210aVLF23ZskWS9OKLL+rtt99WYGCg+vXrp0GDBpkeIAAA+F1qaqouXrzo8kpNTc2w7hdffKFq1arpySefVOHChVWlShVNnz7d9JhshnF7Ay6HDh3Spk2bVLp0aVWsWNGsuG6Lf8A9VocAeKWrx763OgTA6+QuWNLjfWy4p7Up7Xz5TCUlJCS4lI0YMULx8fHp6gYGBkqS+vfvryeffFIbNmxQnz59NHXqVFPPh7rtRMIbkUgAGSORANK7E4nE+sg2prRTOeWTdCMQdrtddrs9Xd2AgABVq1ZNa9eudZb17t1bGzZs0Lp160yJR8riGolJkyZlucHevXu7HQwAAMhcZklDRiIiIlSuXDmXspiYGM2fP9/UmLKUSEyYMCFLjdlsNhIJAAD+xIqh/9q1a2v37t0uZXv27FGJEiVM7SdLiURKSoqpnQIAcDexYtdGv379VKtWLb366qtq27atfvrpJ02bNk3Tpk0ztZ9s79oAAADe76GHHtKCBQv0ySefqHz58ho1apQmTpyoTp06mdrPbT20CwAA/D2rHiPevHlzNW/e3KN9kEgAAOBhDqsD8CCmNgAAgNsYkQAAwMMM8RhxF99//72eeuop1axZU0ePHpUkzZ49Wz/88IOpwQEA4Aschjkvb5TtRGL+/Plq0qSJgoKCtHnzZucJWxcuXNCrr75qeoAAAOR0DtlMeXmjbCcSo0eP1tSpUzV9+nTlzp3bWV67dm0lJSWZGhwAAPBu2V4jsXv3btWrVy9deVhYmM6fP29GTAAA+BTWSPxB0aJFtW/fvnTlP/zwg0qW9PyDTwAAyGkcJr28UbYTiWeeeUZ9+vTR+vXrZbPZdOzYMc2ZM0cDBw7U888/74kYAQCAl8r21MaLL74oh8OhRo0a6cqVK6pXr57sdrsGDhyoF154wRMxAgCQo/ny1IbNMAy3NpRcv35d+/bt06VLl1SuXDnlzZvX7Njc5h9wj9UhAF7p6rHvrQ4B8Dq5C3p+Wn5pkfamtNP05FxT2jGT2wdSBQQEpHvOOQAAuLtkO5Fo2LChbLbMh2hWrlx5WwEBAOBrvHWhpBmynUhUrlzZ5f2NGzeUnJysn3/+WXFxcWbFBQCAz/DlNRLZTiQmTJiQYXl8fLwuXbp02wEBAICcw7Snfz711FP64IMPzGoOAACf4bCZ8/JGpj39c926dQoMDDSrOQAAfIa3PifDDNlOJNq0aePy3jAMHT9+XBs3btSwYcNMCwwAAF/hpQ/uNEW2E4mwsDCX935+fipTpoxGjhypxo0bmxYYAADwftlKJNLS0tSlSxdVqFBB4eHhnooJAACf4svbP7O12DJXrlxq3LgxT/kEACAbHDabKS9vlO1dG+XLl9eBAwc8EQsAAMhhsp1IjB49WgMHDtSSJUt0/PhxXbx40eUFAABcGSa9vFGW10iMHDlSAwYM0COPPCJJeuyxx1yOyjYMQzabTWlpaeZHCQBADubLaySynEgkJCToueee06pVqzwZDwAAyEGynEjcetp4/fr1PRYMAAC+yFtPpTRDtrZ//tVTPwEAQMY42fL/u//++/82mTh79uxtBQQAAHKObCUSCQkJ6U62BAAAf81bd1yYIVuJRPv27VW4cGFPxQIAgE9ijYRYHwEAgLt8eftnlg+kurVrAwAA4JYsj0g4HL6cTwEA4Dm+/E/xbD9GHAAAZI8vr5HI9rM2AAAAbmFEAgAAD/PlxQEkEgAAeJgvJxJMbQAAALcxIgEAgIcZPrzYkkQCAAAPY2oDAAAgA4xIAADgYb48IkEiAQCAh3GyJQAAcBsnWwIAAGSAEQkAADyMNRIAAMBtvpxIMLUBAADcxogEAAAexq4NAADgNnZtAACAHO21116TzWZT3759TW2XEQkAADzM6sWWGzZs0LvvvquKFSua3jYjEgAAeJhh0ssdly5dUqdOnTR9+nSFh4ffzm1kiEQCAAAf1rNnTz366KOKjY31SPtMbQAA4GEOk/ZtpKamKjU11aXMbrfLbrdnWH/u3LlKSkrShg0bTOk/I4xIAADgYQ6TXomJiQoLC3N5JSYmZtjnkSNH1KdPH82ZM0eBgYEeuzebYRg+t73VP+Aeq0MAvNLVY99bHQLgdXIXLOnxPkaW6GRKO0P2fJDlEYmFCxeqdevWypUrl7MsLS1NNptNfn5+Sk1NdbnmLqY2AADIIf5qGuPPGjVqpG3btrmUdenSRWXLltWQIUNMSSIkEgkAADzOiu2fISEhKl++vEtZcHCwChQokK78dpBIAADgYb58siWJBAAAd4nVq1eb3iaJBAAAHmbW9k9vRCIBAICH+W4awTkSAADgNjAiAQCAh1n90C5PsiyR2Lp1a5breuJpZQAA3CmskfCAypUry2azyTAM2Wx/vS8mLS3tDkUFAACyw7I1EikpKTpw4IBSUlI0f/58RUdHa8qUKdq8ebM2b96sKVOmqFSpUpo/f75VIQIAYAorHyPuaZaNSJQoUcL585NPPqlJkybpkUcecZZVrFhRxYoV07Bhw9SqVSsLIgQAwByskfCwbdu2KTo6Ol15dHS0duzYYUFEAACYx5fXSHjF9s+YmBglJibq+vXrzrLr168rMTFRMTExFkYGAAD+ileMSEydOlUtWrTQvffe69yhsXXrVtlsNi1evNji6AAAuD2+Ox7hJYlE9erVdeDAAc2ZM0e7du2SJLVr104dO3ZUcHCwxdEBAHB7WCNxBwQHB+vZZ5+1OgwAAJANXrFGQpJmz56tOnXqKDIyUocOHZIkTZgwQYsWLbI4MgAAbo9h0n/eyCsSiXfeeUf9+/dXs2bNdO7cOecBVOHh4Zo4caK1wQEAcJscJr28kVckEpMnT9b06dP10ksvyd///2ZbqlWrpm3btlkYGQAA+CtesUYiJSVFVapUSVdut9t1+fJlCyICAMA8nCPhYdHR0UpOTk5XvnTpUs6RAADkeByR7WH9+/dXz549de3aNRmGoZ9++kmffPKJEhMT9d5771kdHgAAyIRXJBLdu3dXUFCQXn75ZV25ckUdO3ZUZGSk3nzzTbVv397q8OCm55+L04D+z6to0ULaunWH+vQdpg0bk60OC7hjNiZv04yPP9OOXft0+sxZvZk4TI3q1ZIk3bh5U5OnzdL36zbql2PHlTc4WP94qIr6PddFhQsVsDhymI2pjTugU6dO2rt3ry5duqQTJ07ol19+Ubdu3awOC2568snH9MbYERo1erweqtFUW7bu0FdfzlEh/oDEXeTq1WsqU7qkXhrQI921a9dStWP3fv27cwfN++AtTXz1ZR08/It6DUmwIFJ4mi/v2rAZhuFzaZJ/wD1Wh3DXW/vDYm3YuEV9+r4sSbLZbDp4YIPenjJDY8a+bXF0d6+rx763OoS7VvnazVxGJDKybedudejeV8vmz1JE0cJ3MLq7W+6CJT3eR/eoJ0xp572Dn5nSjpksm9qoUqWKbDZbluomJSV5OBqYKXfu3HrwwYp6bcxbzjLDMLRi5Q/6xz+qWhgZ4N0uXboim82mkBAeDYCcw7JEolWrVs6fr127pilTpqhcuXKqWbOmJOnHH3/U9u3b1aNH+iHBP0pNTVVqaqpLmWEYWU5SYL6CBfPL399fp07+6lJ+6tRplS1TyqKoAO+WmnpdE975QI/E1ldenjHkc7x1WsIMliUSI0aMcP7cvXt39e7dW6NGjUpX58iRI3/ZTmJiohISXOcUbX55ZcsVal6wAOBBN27e1IBhr8owDA0b1MvqcOAB3nq8tRm8YrHlf//7Xz399NPpyp966inNnz//Lz87dOhQXbhwweVl8wvxVKjIgl9/PaubN2+qcJGCLuWFCxfSiZOnLYoK8E63kohjJ09p+sRXGY1AjuMViURQUJD+97//pSv/3//+p8DAwL/8rN1uV2hoqMuLaQ1r3bhxQ0lJW/XPhnWcZTabTf9sWEc//rjJwsgA73IriTh85Jjem/iq8oUxkuqrfHnXhlecI9G3b189//zzSkpKUvXq1SVJ69ev1wcffKBhw4ZZHB3cMeHN6Zrx/gRtStqqDRs2q/cLzyg4OEgzZ31qdWjAHXPlylUd/uWY8/3RYye1a89+hYWGqGDB/Or/0ivasWef3h6TIIfDoV/PnJUkhYWGKHfu3FaFDQ9w+N4GSSev2f45b948vfnmm9q5c6ckKSYmRn369FHbtm2z3RbbP71Dj+c7Ow+k2rJlu/r2G66fNmy2Oqy7Gts/76yfkraq6wtD0pW3bBarHt2eUpMnOmf4uQ8mv67qD1b0cHS45U5s//xXiTamtDP70OemtGMmr0kkzEQiAWSMRAJI704kEk+ZlEh85IWJhFdMbdxy/fp1nTp1Sg6H60xQ8eLFLYoIAIDb58tHZHtFIrF371517dpVa9eudSm/dR5EWlqaRZEBAIC/4hWJROfOneXv768lS5YoIiKCXRcAAJ/iy+dIeEUikZycrE2bNqls2bJWhwIAgOm8deumGbwikShXrpx+/fXXv68IAEAO5MtrJLziQKrXX39dgwcP1urVq3XmzBldvHjR5QUAALyTV4xIxMbGSpIaNWrkUs5iSwCAL2CNhIetWrXK6hAAAPAY1kh4WP369a0OAQAAuMErEolbrly5osOHD+v69esu5RUrclQsACDn8sFDpJ28IpE4ffq0unTpoq+//jrD66yRAADkZOza8LC+ffvq/PnzWr9+vYKCgrR06VLNmjVL9913n7744gurwwMAAJnwihGJlStXatGiRapWrZr8/PxUokQJPfzwwwoNDVViYqIeffRRq0MEAMBtvrzY0itGJC5fvqzChQtLksLDw3X69GlJUoUKFZSUlGRlaAAA3DbDpP+8kVckEmXKlNHu3bslSZUqVdK7776ro0ePaurUqYqIiLA4OgAAkBmvmNro06ePjh8/LkkaMWKEmjZtqo8++kgBAQGaNWuWxdEBAHB7fHmxpVckEk899ZTz56pVq+rQoUPatWuXihcvroIFC1oYGQAAt4/tnx7Qv3//LNcdP368ByMBAMCzfHmxpWWJxObNm7NUz2azeTgSAADgLssSCZ6vAQC4W3jrjgszeMUaCQAAfJkvL7b0iu2fAADAXImJiXrooYcUEhKiwoULq1WrVs6jFsxEIgEAgIcZhmHKKzu+++479ezZUz/++KOWLVumGzduqHHjxrp8+bKp98bUBgAAHmbF1MbSpUtd3s+cOVOFCxfWpk2bVK9ePdP6IZEAACCHSE1NVWpqqkuZ3W6X3W7/289euHBBkpQ/f35TY2JqAwAADzPrWRuJiYkKCwtzeSUmJv5t/w6HQ3379lXt2rVVvnx5U++NEQkAADzMYdLJlkOHDk13oGNWRiN69uypn3/+WT/88IMpcfwRiQQAADlEVqcx/qhXr15asmSJ1qxZo3vvvdf0mEgkAADwMCtOkTAMQy+88IIWLFig1atXKzo62iP9kEgAAOBhVuza6Nmzpz7++GMtWrRIISEhOnHihCQpLCxMQUFBpvVjM3zwkWT+AfdYHQLgla4e+97qEACvk7tgSY/3UfOehqa0s+5o1h8vkdmzqmbMmKHOnTubEo/EiAQAAD7pTo0TkEgAAOBhPjj470QiAQCAh/HQLgAAgAwwIgEAgIcZPjwiQSIBAICH+fIaCaY2AACA2xiRAADAw3x5sSWJBAAAHsbUBgAAQAYYkQAAwMOY2gAAAG5j+ycAAHCbgzUSAAAA6TEiAQCAhzG1AQAA3MbUBgAAQAYYkQAAwMOY2gAAAG5jagMAACADjEgAAOBhTG0AAAC3MbUBAACQAUYkAADwMKY2AACA2wzDYXUIHkMiAQCAh/nyY8RZIwEAANzGiAQAAB5m+PCuDRIJAAA8jKkNAACADDAiAQCAhzG1AQAA3MbJlgAAABlgRAIAAA/jZEsAAOA2X14jwdQGAABwGyMSAAB4mC+fI0EiAQCAh/ny1AaJBAAAHsb2TwAAgAwwIgEAgIcxtQEAANzmy4stmdoAAABuY0QCAAAPY2oDAAC4jV0bAAAAGWBEAgAAD+OhXQAAwG1MbQAAAGSAEQkAADyMXRsAAMBtvrxGgqkNAAA8zDAMU17uePvttxUVFaXAwEDVqFFDP/30k6n3RiIBAICP+vTTT9W/f3+NGDFCSUlJqlSpkpo0aaJTp06Z1ofN8MGJG/+Ae6wOAfBKV499b3UIgNfJXbCk5/sw6e+lG9ePZqt+jRo19NBDD+mtt96SJDkcDhUrVkwvvPCCXnzxRVNiYkQCAAAPM0x6Zcf169e1adMmxcbGOsv8/PwUGxurdevW3db9/BGLLQEAyCFSU1OVmprqUma322W329PV/fXXX5WWlqYiRYq4lBcpUkS7du0yLSafTCRuZnPoB56RmpqqxMREDR06NMP/yYG7Fd+Nu49Zfy/Fx8crISHBpWzEiBGKj483pX13+OQaCXiHixcvKiwsTBcuXFBoaKjV4QBeg+8G3JWdEYnr168rT548+uyzz9SqVStneVxcnM6fP69FixaZEhNrJAAAyCHsdrtCQ0NdXpmNagUEBKhq1apasWKFs8zhcGjFihWqWbOmaTH55NQGAACQ+vfvr7i4OFWrVk3Vq1fXxIkTdfnyZXXp0sW0PkgkAADwUe3atdPp06c1fPhwnThxQpUrV9bSpUvTLcC8HSQS8Bi73a4RI0awmAz4E74buJN69eqlXr16eax9FlsCAAC3sdgSAAC4jUQCAAC4jUQCAAC4jUTCBzVo0EB9+/Z1vo+KitLEiRP/8jM2m00LFy70aFxZ1blzZ5fDUzxl5syZypcvn8f7gW/78/fNE+Lj41W5cmWP9gG4i10bd4ENGzYoODjY6jAUHx+vhQsXKjk52epQAAAmIZG4CxQqVMjqEAAAPoqpjRzu8uXLevrpp5U3b15FRERo3Lhx6er8eWpj7969qlevngIDA1WuXDktW7bsb/tp0KCBevfurcGDByt//vwqWrRouofEHD58WC1btlTevHkVGhqqtm3b6uTJk5J+n0ZISEjQli1bZLPZZLPZNHPmzCzd49KlS1WnTh3ly5dPBQoUUPPmzbV//37n9YMHD8pms+nzzz9Xw4YNlSdPHlWqVCndY3Jnzpyp4sWLK0+ePGrdurXOnDmTpf6BrJo9e7aqVaumkJAQFS1aVB07dtSpU6ec11evXi2bzaYVK1aoWrVqypMnj2rVqqXdu3e7tPPaa6+pSJEiCgkJUbdu3XTt2rU7fStAlpFI5HCDBg3Sd999p0WLFunbb7/V6tWrlZSUlGl9h8OhNm3aKCAgQOvXr9fUqVM1ZMiQLPU1a9YsBQcHa/369RozZoxGjhzpTEIcDodatmyps2fP6rvvvtOyZct04MABtWvXTtLvp6sNGDBADzzwgI4fP67jx487r/2dy5cvq3///tq4caNWrFghPz8/tW7dWg6Hw6XeSy+9pIEDByo5OVn333+/OnTooJs3b0qS1q9fr27duqlXr15KTk5Ww4YNNXr06Cz1D2TVjRs3NGrUKG3ZskULFy7UwYMH1blz53T1XnrpJY0bN04bN26Uv7+/unbt6rw2b948xcfH69VXX9XGjRsVERGhKVOm3MG7ALLJQI7122+/GQEBAca8efOcZWfOnDGCgoKMPn36OMtKlChhTJgwwTAMw/jmm28Mf39/4+jRo87rX3/9tSHJWLBgQaZ91a9f36hTp45L2UMPPWQMGTLEMAzD+Pbbb41cuXIZhw8fdl7fvn27Icn46aefDMMwjBEjRhiVKlX62/uKi4szWrZsmen106dPG5KMbdu2GYZhGCkpKYYk47333kvX986dOw3DMIwOHToYjzzyiEs77dq1M8LCwv42HuCv1K9f3+X79kcbNmwwJBm//fabYRiGsWrVKkOSsXz5cmedL7/80pBkXL161TAMw6hZs6bRo0cPl3Zq1KiRpe8OYAVGJHKw/fv36/r166pRo4azLH/+/CpTpkymn9m5c6eKFSumyMhIZ1lWnwJXsWJFl/cRERHOYdtb7RYrVsx5vVy5csqXL5927tyZpfYzs3fvXnXo0EElS5ZUaGiooqKiJP0+lZJZfBEREZLkEt8ff09S1u8byKpNmzapRYsWKl68uEJCQlS/fn1J/L8K30YigSzLnTu3y3ubzZZuesETWrRoobNnz2r69Olav3691q9fL0m6fv16pvHZbDZJuiPxAdLvU3BNmjRRaGio5syZow0bNmjBggWS+H8Vvo1EIgcrVaqUcufO7fyLVZLOnTunPXv2ZPqZmJgYHTlyRMePH3eW/fjjj7cdy612jxw54izbsWOHzp8/r3LlykmSAgIClJaWlq12z5w5o927d+vll19Wo0aNFBMTo3PnzrkV3x9/T5I59w3csmvXLp05c0avvfaa6tatq7Jly7ostMwq/l9FTsP2zxwsb9686tatmwYNGqQCBQqocOHCeumll+Tnl3l+GBsbq/vvv19xcXEaO3asLl68qJdeeum2Y4mNjVWFChXUqVMnTZw4UTdv3lSPHj1Uv359VatWTdLvu0dSUlKUnJyse++9VyEhIX/79MPw8HAVKFBA06ZNU0REhA4fPqwXX3wx2/H17t1btWvX1htvvKGWLVvqm2++0dKlS926VyAjxYsXV0BAgCZPnqznnntOP//8s0aNGpXtdvr06aPOnTurWrVqql27tubMmaPt27erZMmSHogauH2MSORwY8eOVd26ddWiRQvFxsaqTp06qlq1aqb1/fz8tGDBAl29elXVq1dX9+7d9corr9x2HDabTYsWLVJ4eLjq1aun2NhYlSxZUp9++qmzzuOPP66mTZuqYcOGKlSokD755JO/bdfPz09z587Vpk2bVL58efXr109jx47Ndnz/+Mc/NH36dL355puqVKmSvv32W7388svZbgfITKFChTRz5kz997//Vbly5fTaa6/pjTfeyHY77dq107BhwzR48GBVrVpVhw4d0vPPP++BiAFz8BhxAADgNkYkAACA20gkAACA20gkAACA20gkAACA20gkAACA20gkAACA20gkAACA20gkAAt17txZrVq1cr5v0KCB+vbte8fjWL16tWw2m86fP59pHZvNpoULF2a5zfj4eFWuXPm24jp48KBsNpuSk5Nvqx0AnkMiAfxJ586dZbPZZLPZFBAQoNKlS2vkyJG6efOmx/v+/PPPs3ysclb+8gcAT+NZG0AGmjZtqhkzZig1NVVfffWVevbsqdy5c2vo0KHp6l6/fl0BAQGm9Js/f35T2gGAO4URCSADdrtdRYsWVYkSJfT8888rNjZWX3zxhaT/m4545ZVXFBkZqTJlykiSjhw5orZt2ypfvnzKnz+/WrZsqYMHDzrbTEtLU//+/ZUvXz4VKFBAgwcP1p9PqP/z1EZqaqqGDBmiYsWKyW63q3Tp0nr//fd18OBBNWzYUNLvDzaz2Wzq3LmzpN8fR52YmKjo6GgFBQWpUqVK+uyzz1z6+eqrr3T//fcrKChIDRs2dIkzq4YMGaL7779fefLkUcmSJTVs2DDduHEjXb13331XxYoVU548edS2bVtduHDB5fp7772nmJgYBQYGqmzZspoyZUqmfZ47d06dOnVSoUKFFBQUpPvuu08zZszIduwAzMOIBJAFQUFBOnPmjPP9ihUrFBoaqmXLlkmSbty4oSZNmqhmzZr6/vvv5e/vr9GjR6tp06baunWrAgICNG7cOM2cOVMffPCBYmJiNG7cOC1YsED//Oc/M+336aef1rp16zRp0iRVqlRJKSkp+vXXX1WsWDHNnz9fjz/+uHbv3q3Q0FAFBQVJkhITE/XRRx9p6tSpuu+++7RmzRo99dRTKlSokOrXr68jR46oTZs26tmzp5599llt3LhRAwYMyPbvJCQkRDNnzlRkZKS2bdumZ555RiEhIRo8eLCzzr59+zRv3jwtXrxYFy9eVLdu3dSjRw/NmTNHkjRnzhwNHz5cb731lqpUqaLNmzfrmWeeUXBwsOLi4tL1OWzYMO3YsUNff/21ChYsqH379unq1avZjh2AiQwALuLi4oyWLVsahmEYDofDWLZsmWG3242BAwc6rxcpUsRITU11fmb27NlGmTJlDIfD4SxLTU01goKCjG+++cYwDMOIiIgwxowZ47x+48YN495773X2ZRiGUb9+faNPnz6GYRjG7t27DUnGsmXLMoxz1apVhiTj3LlzzrJr164ZefLkMdauXetSt1u3bkaHDh0MwzCMoUOHGuXKlXO5PmTIkHRt/ZkkY8GCBZleHzt2rFG1alXn+xEjRhi5cuUyfvnlF2fZ119/bfj5+RnHjx83DMMwSpUqZXz88ccu7YwaNcqoWbOmYRiGkZKSYkgyNm/ebBiGYbRo0cLo0qVLpjEAuPMYkQAysGTJEuXNm1c3btyQw+FQx44dFR8f77xeoUIFl3URW7Zs0b59+xQSEuLSzrVr17R//35duHBBx48fV40aNZzX/P39Va1atXTTG7ckJycrV65cql+/fpbj3rdvn65cuaKHH37Ypfz69euqUqWKJGnnzp0ucUhSzZo1s9zHLZ9++qkmTZqk/fv369KlS7p586ZCQ0Nd6hQvXlz33HOPSz8Oh0O7d+9WSEiI9u/fr27duumZZ55x1rl586bCwsIy7PP555/X448/rqSkJDVu3FitWrVSrVq1sh07APOQSAAZaNiwod555x0FBAQoMjJS/v6uX5Xg4GCX95cuXVLVqlWdQ/Z/VKhQIbdiuDVVkR2XLl2SJH355Zcuf4FLv6/7MMu6devUqVMnJSQkqEmTJgoLC9PcuXM1bty4bMc6ffr0dIlNrly5MvxMs2bNdOjQIX311VdatmyZGjVqpJ49e+qNN95w/2YA3BYSCSADwcHBKl26dJbrP/jgg/r0009VuHDhdP8qvyUiIkLr169XvXr1JP3+L+9NmzbpwQcfzLB+hQoV5HA49N133yk2Njbd9VsjImlpac6ycuXKyW636/Dhw5mOZMTExDgXjt7y448//v1N/sHatWtVokQJvfTSS86yQ4cOpat3+PBhHTt2TJGRkc5+/Pz8VKZMGRUpUkSRkZE6cOCAOnXqlOW+CxUqpLi4OMXFxalu3boaNGgQiQRgIXZtACbo1KmTChYsqJYtW+r7779XSkqKVq9erd69e+uXX36RJPXp00evvfaaFi5cqF27dqlHjx5/eQZEVFSU4uLi1LVrVy1cuNDZ5rx58yRJJUqUkM1m05IlS3T69GldunRJISEhGjhwoPr166dZs2Zp//79SkpK0uTJkzVr1ixJ0nPPPae9e/dq0KBB2r17tz7++GPNnDkzW/d733336fDhw5o7d67279+vSZMmacGCBenqBQYGKi4uTlu2bNH333+v3r17q23btipatKgkKSEhQYmJiZo0aZL27Nmjbdu2acaMGRo/fnyG/Q4fPlyLFi3Svn37tH37di1ZskQxMTHZih2AuUgkABPkyZNHa9asUfHixdWmTRvFxMSoW7duunbtmnOEYsCAAfrXv/6luLg41axZUyEhIWrduvVftvvOO+/oiSeeUI8ePVS2bFk988wzunz5siTpnnvuUUJCgl588UUVKVJEvXr1kiSNGjVKw4YNU2JiomJiYtS0aVN9+eWXio6OlvT7uoX58+dr4cKFqlSpkqZOnapXX301W/f72GOPqV+/furVq5cqV66stWvXatiwYenqlS5dWm3atNEjjzyixo0bq2LFii7bO7t376733ntPM2bMUIUKFVS/fn3NnDnTGeufBQQEaOjQoapYsaLq1aunXLlyae7cudmKHYC5bEZmK70AAAD+BiMSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbf8PkAtlXISM2hwAAAAASUVORK5CYII=","text/plain":["<Figure size 640x480 with 2 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":["yhat=logreg_cv.predict(X_test)\n","plot_confusion_matrix(Y_test,yhat)"]},{"cell_type":"markdown","id":"1bb0162d-4577-4dd2-9773-137ae970d460","metadata":{},"source":["Examining the confusion matrix, we see that logistic regression can distinguish between the different classes.  We see that the major problem is false positives.\n"]},{"cell_type":"markdown","id":"aa59ebcf-a60d-4227-b51b-61e537d65874","metadata":{},"source":["## TASK  6\n"]},{"cell_type":"markdown","id":"1a6d6733-e05a-447d-ba11-4dc93f3a74ee","metadata":{},"source":["Create a support vector machine object then  create a  <code>GridSearchCV</code> object  <code>svm_cv</code> with cv = 10.  Fit the object to find the best parameters from the dictionary <code>parameters</code>.\n"]},{"cell_type":"code","execution_count":37,"id":"16a5ddeb-e22c-4221-8d93-240139faa2ab","metadata":{},"outputs":[],"source":["parameters = {'kernel':('linear', 'rbf','poly','rbf', 'sigmoid'),\n","              'C': np.logspace(-3, 3, 5),\n","              'gamma':np.logspace(-3, 3, 5)}\n","svm = SVC()\n","svm_cv = GridSearchCV(svm, parameters, cv=10)"]},{"cell_type":"code","execution_count":39,"metadata":{},"outputs":[{"data":{"text/html":["<style>#sk-container-id-5 {\n","  /* Definition of color scheme common for light and dark mode */\n","  --sklearn-color-text: black;\n","  --sklearn-color-line: gray;\n","  /* Definition of color scheme for unfitted estimators */\n","  --sklearn-color-unfitted-level-0: #fff5e6;\n","  --sklearn-color-unfitted-level-1: #f6e4d2;\n","  --sklearn-color-unfitted-level-2: #ffe0b3;\n","  --sklearn-color-unfitted-level-3: chocolate;\n","  /* Definition of color scheme for fitted estimators */\n","  --sklearn-color-fitted-level-0: #f0f8ff;\n","  --sklearn-color-fitted-level-1: #d4ebff;\n","  --sklearn-color-fitted-level-2: #b3dbfd;\n","  --sklearn-color-fitted-level-3: cornflowerblue;\n","\n","  /* Specific color for light theme */\n","  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n","  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n","  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n","  --sklearn-color-icon: #696969;\n","\n","  @media (prefers-color-scheme: dark) {\n","    /* Redefinition of color scheme for dark theme */\n","    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n","    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n","    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n","    --sklearn-color-icon: #878787;\n","  }\n","}\n","\n","#sk-container-id-5 {\n","  color: var(--sklearn-color-text);\n","}\n","\n","#sk-container-id-5 pre {\n","  padding: 0;\n","}\n","\n","#sk-container-id-5 input.sk-hidden--visually {\n","  border: 0;\n","  clip: rect(1px 1px 1px 1px);\n","  clip: rect(1px, 1px, 1px, 1px);\n","  height: 1px;\n","  margin: -1px;\n","  overflow: hidden;\n","  padding: 0;\n","  position: absolute;\n","  width: 1px;\n","}\n","\n","#sk-container-id-5 div.sk-dashed-wrapped {\n","  border: 1px dashed var(--sklearn-color-line);\n","  margin: 0 0.4em 0.5em 0.4em;\n","  box-sizing: border-box;\n","  padding-bottom: 0.4em;\n","  background-color: var(--sklearn-color-background);\n","}\n","\n","#sk-container-id-5 div.sk-container {\n","  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n","     but bootstrap.min.css set `[hidden] { display: none !important; }`\n","     so we also need the `!important` here to be able to override the\n","     default hidden behavior on the sphinx rendered scikit-learn.org.\n","     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n","  display: inline-block !important;\n","  position: relative;\n","}\n","\n","#sk-container-id-5 div.sk-text-repr-fallback {\n","  display: none;\n","}\n","\n","div.sk-parallel-item,\n","div.sk-serial,\n","div.sk-item {\n","  /* draw centered vertical line to link estimators */\n","  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n","  background-size: 2px 100%;\n","  background-repeat: no-repeat;\n","  background-position: center center;\n","}\n","\n","/* Parallel-specific style estimator block */\n","\n","#sk-container-id-5 div.sk-parallel-item::after {\n","  content: \"\";\n","  width: 100%;\n","  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n","  flex-grow: 1;\n","}\n","\n","#sk-container-id-5 div.sk-parallel {\n","  display: flex;\n","  align-items: stretch;\n","  justify-content: center;\n","  background-color: var(--sklearn-color-background);\n","  position: relative;\n","}\n","\n","#sk-container-id-5 div.sk-parallel-item {\n","  display: flex;\n","  flex-direction: column;\n","}\n","\n","#sk-container-id-5 div.sk-parallel-item:first-child::after {\n","  align-self: flex-end;\n","  width: 50%;\n","}\n","\n","#sk-container-id-5 div.sk-parallel-item:last-child::after {\n","  align-self: flex-start;\n","  width: 50%;\n","}\n","\n","#sk-container-id-5 div.sk-parallel-item:only-child::after {\n","  width: 0;\n","}\n","\n","/* Serial-specific style estimator block */\n","\n","#sk-container-id-5 div.sk-serial {\n","  display: flex;\n","  flex-direction: column;\n","  align-items: center;\n","  background-color: var(--sklearn-color-background);\n","  padding-right: 1em;\n","  padding-left: 1em;\n","}\n","\n","\n","/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n","clickable and can be expanded/collapsed.\n","- Pipeline and ColumnTransformer use this feature and define the default style\n","- Estimators will overwrite some part of the style using the `sk-estimator` class\n","*/\n","\n","/* Pipeline and ColumnTransformer style (default) */\n","\n","#sk-container-id-5 div.sk-toggleable {\n","  /* Default theme specific background. It is overwritten whether we have a\n","  specific estimator or a Pipeline/ColumnTransformer */\n","  background-color: var(--sklearn-color-background);\n","}\n","\n","/* Toggleable label */\n","#sk-container-id-5 label.sk-toggleable__label {\n","  cursor: pointer;\n","  display: block;\n","  width: 100%;\n","  margin-bottom: 0;\n","  padding: 0.5em;\n","  box-sizing: border-box;\n","  text-align: center;\n","}\n","\n","#sk-container-id-5 label.sk-toggleable__label-arrow:before {\n","  /* Arrow on the left of the label */\n","  content: \"▸\";\n","  float: left;\n","  margin-right: 0.25em;\n","  color: var(--sklearn-color-icon);\n","}\n","\n","#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {\n","  color: var(--sklearn-color-text);\n","}\n","\n","/* Toggleable content - dropdown */\n","\n","#sk-container-id-5 div.sk-toggleable__content {\n","  max-height: 0;\n","  max-width: 0;\n","  overflow: hidden;\n","  text-align: left;\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-5 div.sk-toggleable__content.fitted {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","#sk-container-id-5 div.sk-toggleable__content pre {\n","  margin: 0.2em;\n","  border-radius: 0.25em;\n","  color: var(--sklearn-color-text);\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-5 div.sk-toggleable__content.fitted pre {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n","  /* Expand drop-down */\n","  max-height: 200px;\n","  max-width: 100%;\n","  overflow: auto;\n","}\n","\n","#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n","  content: \"▾\";\n","}\n","\n","/* Pipeline/ColumnTransformer-specific style */\n","\n","#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-5 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Estimator-specific style */\n","\n","/* Colorize estimator box */\n","#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-5 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","#sk-container-id-5 div.sk-label label.sk-toggleable__label,\n","#sk-container-id-5 div.sk-label label {\n","  /* The background is the default theme color */\n","  color: var(--sklearn-color-text-on-default-background);\n","}\n","\n","/* On hover, darken the color of the background */\n","#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","/* Label box, darken color on hover, fitted */\n","#sk-container-id-5 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Estimator label */\n","\n","#sk-container-id-5 div.sk-label label {\n","  font-family: monospace;\n","  font-weight: bold;\n","  display: inline-block;\n","  line-height: 1.2em;\n","}\n","\n","#sk-container-id-5 div.sk-label-container {\n","  text-align: center;\n","}\n","\n","/* Estimator-specific */\n","#sk-container-id-5 div.sk-estimator {\n","  font-family: monospace;\n","  border: 1px dotted var(--sklearn-color-border-box);\n","  border-radius: 0.25em;\n","  box-sizing: border-box;\n","  margin-bottom: 0.5em;\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-5 div.sk-estimator.fitted {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","/* on hover */\n","#sk-container-id-5 div.sk-estimator:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-5 div.sk-estimator.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Specification for estimator info (e.g. \"i\" and \"?\") */\n","\n","/* Common style for \"i\" and \"?\" */\n","\n",".sk-estimator-doc-link,\n","a:link.sk-estimator-doc-link,\n","a:visited.sk-estimator-doc-link {\n","  float: right;\n","  font-size: smaller;\n","  line-height: 1em;\n","  font-family: monospace;\n","  background-color: var(--sklearn-color-background);\n","  border-radius: 1em;\n","  height: 1em;\n","  width: 1em;\n","  text-decoration: none !important;\n","  margin-left: 1ex;\n","  /* unfitted */\n","  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-unfitted-level-1);\n","}\n","\n",".sk-estimator-doc-link.fitted,\n","a:link.sk-estimator-doc-link.fitted,\n","a:visited.sk-estimator-doc-link.fitted {\n","  /* fitted */\n","  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-fitted-level-1);\n","}\n","\n","/* On hover */\n","div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",".sk-estimator-doc-link:hover,\n","div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",".sk-estimator-doc-link:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",".sk-estimator-doc-link.fitted:hover,\n","div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",".sk-estimator-doc-link.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","/* Span, style for the box shown on hovering the info icon */\n",".sk-estimator-doc-link span {\n","  display: none;\n","  z-index: 9999;\n","  position: relative;\n","  font-weight: normal;\n","  right: .2ex;\n","  padding: .5ex;\n","  margin: .5ex;\n","  width: min-content;\n","  min-width: 20ex;\n","  max-width: 50ex;\n","  color: var(--sklearn-color-text);\n","  box-shadow: 2pt 2pt 4pt #999;\n","  /* unfitted */\n","  background: var(--sklearn-color-unfitted-level-0);\n","  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n","}\n","\n",".sk-estimator-doc-link.fitted span {\n","  /* fitted */\n","  background: var(--sklearn-color-fitted-level-0);\n","  border: var(--sklearn-color-fitted-level-3);\n","}\n","\n",".sk-estimator-doc-link:hover span {\n","  display: block;\n","}\n","\n","/* \"?\"-specific style due to the `<a>` HTML tag */\n","\n","#sk-container-id-5 a.estimator_doc_link {\n","  float: right;\n","  font-size: 1rem;\n","  line-height: 1em;\n","  font-family: monospace;\n","  background-color: var(--sklearn-color-background);\n","  border-radius: 1rem;\n","  height: 1rem;\n","  width: 1rem;\n","  text-decoration: none;\n","  /* unfitted */\n","  color: var(--sklearn-color-unfitted-level-1);\n","  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n","}\n","\n","#sk-container-id-5 a.estimator_doc_link.fitted {\n","  /* fitted */\n","  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-fitted-level-1);\n","}\n","\n","/* On hover */\n","#sk-container-id-5 a.estimator_doc_link:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","#sk-container-id-5 a.estimator_doc_link.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-3);\n","}\n","</style><div id=\"sk-container-id-5\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=10, estimator=SVC(),\n","             param_grid={&#x27;C&#x27;: array([1.00000000e-03, 3.16227766e-02, 1.00000000e+00, 3.16227766e+01,\n","       1.00000000e+03]),\n","                         &#x27;gamma&#x27;: array([1.00000000e-03, 3.16227766e-02, 1.00000000e+00, 3.16227766e+01,\n","       1.00000000e+03]),\n","                         &#x27;kernel&#x27;: (&#x27;linear&#x27;, &#x27;rbf&#x27;, &#x27;poly&#x27;, &#x27;rbf&#x27;, &#x27;sigmoid&#x27;)})</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-13\" type=\"checkbox\" ><label for=\"sk-estimator-id-13\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;GridSearchCV<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.model_selection.GridSearchCV.html\">?<span>Documentation for GridSearchCV</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>GridSearchCV(cv=10, estimator=SVC(),\n","             param_grid={&#x27;C&#x27;: array([1.00000000e-03, 3.16227766e-02, 1.00000000e+00, 3.16227766e+01,\n","       1.00000000e+03]),\n","                         &#x27;gamma&#x27;: array([1.00000000e-03, 3.16227766e-02, 1.00000000e+00, 3.16227766e+01,\n","       1.00000000e+03]),\n","                         &#x27;kernel&#x27;: (&#x27;linear&#x27;, &#x27;rbf&#x27;, &#x27;poly&#x27;, &#x27;rbf&#x27;, &#x27;sigmoid&#x27;)})</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-14\" type=\"checkbox\" ><label for=\"sk-estimator-id-14\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">best_estimator_: SVC</label><div class=\"sk-toggleable__content fitted\"><pre>SVC(C=np.float64(1.0), gamma=np.float64(0.03162277660168379), kernel=&#x27;sigmoid&#x27;)</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-15\" type=\"checkbox\" ><label for=\"sk-estimator-id-15\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;SVC<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.svm.SVC.html\">?<span>Documentation for SVC</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>SVC(C=np.float64(1.0), gamma=np.float64(0.03162277660168379), kernel=&#x27;sigmoid&#x27;)</pre></div> </div></div></div></div></div></div></div></div></div>"],"text/plain":["GridSearchCV(cv=10, estimator=SVC(),\n","             param_grid={'C': array([1.00000000e-03, 3.16227766e-02, 1.00000000e+00, 3.16227766e+01,\n","       1.00000000e+03]),\n","                         'gamma': array([1.00000000e-03, 3.16227766e-02, 1.00000000e+00, 3.16227766e+01,\n","       1.00000000e+03]),\n","                         'kernel': ('linear', 'rbf', 'poly', 'rbf', 'sigmoid')})"]},"execution_count":39,"metadata":{},"output_type":"execute_result"}],"source":["svm_cv.fit(X_train, Y_train)"]},{"cell_type":"code","execution_count":40,"id":"14736c90-b8e5-4108-b4d6-0cea3212971c","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["tuned hpyerparameters :(best parameters)  {'C': np.float64(1.0), 'gamma': np.float64(0.03162277660168379), 'kernel': 'sigmoid'}\n","accuracy : 0.8482142857142856\n"]}],"source":["print(\"tuned hpyerparameters :(best parameters) \",svm_cv.best_params_)\n","print(\"accuracy :\",svm_cv.best_score_)"]},{"cell_type":"markdown","id":"d59ecf47-409f-40dc-aa2d-e1f943b9f4ea","metadata":{},"source":["## TASK  7\n"]},{"cell_type":"markdown","id":"33a150aa-1ff4-4f22-89e5-62177a125059","metadata":{},"source":["Calculate the accuracy on the test data using the method <code>score</code>:\n"]},{"cell_type":"code","execution_count":71,"id":"470a0fa8-2dde-49e9-87f0-c847e3d18273","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["MSE =  0.16666666666666666\n","Variance score: 0.83\n"]}],"source":["y_hat = svm_cv.predict(X_test)\n","\n","print(\"MSE = \", np.mean((y_hat-Y_test)**2))\n","print('Variance score: %.2f' % svm_cv.score(X_test, Y_test))"]},{"cell_type":"markdown","id":"6ca3d0ac-ad5b-49ca-aa69-025c7aadce56","metadata":{},"source":["We can plot the confusion matrix\n"]},{"cell_type":"code","execution_count":44,"id":"12d3c1a4-663f-453b-bf6d-589c501afefb","metadata":{},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAhIAAAHHCAYAAADqJrG+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAABD30lEQVR4nO3dd3RU1fr/8c+EkEkISQg9USABFIJUQbh0uESKghSV6jU0vQpIL3IVSACNghRBEUEFRBS5IiCoKFX0ggiEANJLKNKlSguQOb8//DFfxySaDGc4k+H9cs1amX327P2cLAcedjs2wzAMAQAAuMHP6gAAAEDORSIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBeNDevXvVuHFjhYWFyWazaeHChaa2f/DgQdlsNs2cOdPUdnOyBg0aqEGDBlaHAdw1SCTg8/bv369///vfKlmypAIDAxUaGqratWvrzTff1NWrVz3ad1xcnLZt26ZXXnlFs2fPVrVq1Tza353UuXNn2Ww2hYaGZvh73Lt3r2w2m2w2m954441st3/s2DHFx8crOTnZhGgBeIq/1QEAnvTll1/qySeflN1u19NPP63y5cvr+vXr+uGHHzRo0CBt375d06ZN80jfV69e1bp16/TSSy+pV69eHumjRIkSunr1qnLnzu2R9v+Ov7+/rly5osWLF6tt27Yu1+bMmaPAwEBdu3bNrbaPHTumhIQERUVFqXLlyln+3LfffutWfwDcQyIBn5WSkqL27durRIkSWrlypSIiIpzXevbsqX379unLL7/0WP+nT5+WJOXLl89jfdhsNgUGBnqs/b9jt9tVu3ZtffLJJ+kSiY8//liPPvqo5s+ff0diuXLlivLkyaOAgIA70h+A3zG1AZ81ZswYXbp0Se+//75LEnFL6dKl1adPH+f7mzdvatSoUSpVqpTsdruioqL0n//8R6mpqS6fi4qKUvPmzfXDDz+oevXqCgwMVMmSJfXhhx8668THx6tEiRKSpEGDBslmsykqKkrS71MCt37+o/j4eNlsNpeyZcuWqU6dOsqXL5/y5s2rMmXK6D//+Y/zemZrJFauXKm6desqODhY+fLlU8uWLbVz584M+9u3b586d+6sfPnyKSwsTF26dNGVK1cy/8X+SceOHfX111/r/PnzzrINGzZo79696tixY7r6Z8+e1cCBA1WhQgXlzZtXoaGhatasmbZs2eKss3r1aj300EOSpC5dujinSG7dZ4MGDVS+fHlt2rRJ9erVU548eZy/lz+vkYiLi1NgYGC6+2/SpInCw8N17NixLN8rgPRIJOCzFi9erJIlS6pWrVpZqt+9e3cNHz5cDz74oCZMmKD69esrMTFR7du3T1d33759euKJJ/Twww9r3LhxCg8PV+fOnbV9+3ZJUps2bTRhwgRJUocOHTR79mxNnDgxW/Fv375dzZs3V2pqqkaOHKlx48bpscce0//+97+//Nzy5cvVpEkTnTp1SvHx8erfv7/Wrl2r2rVr6+DBg+nqt23bVr/99psSExPVtm1bzZw5UwkJCVmOs02bNrLZbPr888+dZR9//LHKli2rBx98MF39AwcOaOHChWrevLnGjx+vQYMGadu2bapfv77zL/WYmBiNHDlSkvTss89q9uzZmj17turVq+ds58yZM2rWrJkqV66siRMnqmHDhhnG9+abb6pQoUKKi4tTWlqaJOndd9/Vt99+q8mTJysyMjLL9wogAwbggy5cuGBIMlq2bJml+snJyYYko3v37i7lAwcONCQZK1eudJaVKFHCkGSsWbPGWXbq1CnDbrcbAwYMcJalpKQYkoyxY8e6tBkXF2eUKFEiXQwjRoww/viVnDBhgiHJOH36dKZx3+pjxowZzrLKlSsbhQsXNs6cOeMs27Jli+Hn52c8/fTT6frr2rWrS5utW7c2ChQokGmff7yP4OBgwzAM44knnjAaNWpkGIZhpKWlGUWLFjUSEhIy/B1cu3bNSEtLS3cfdrvdGDlypLNsw4YN6e7tlvr16xuSjKlTp2Z4rX79+i5l33zzjSHJGD16tHHgwAEjb968RqtWrf72HgH8PUYk4JMuXrwoSQoJCclS/a+++kqS1L9/f5fyAQMGSFK6tRTlypVT3bp1ne8LFSqkMmXK6MCBA27H/Ge31lYsWrRIDocjS585fvy4kpOT1blzZ+XPn99ZXrFiRT388MPO+/yj5557zuV93bp1debMGefvMCs6duyo1atX68SJE1q5cqVOnDiR4bSG9Pu6Cj+/3//oSUtL05kzZ5zTNklJSVnu0263q0uXLlmq27hxY/373//WyJEj1aZNGwUGBurdd9/Ncl8AMkciAZ8UGhoqSfrtt9+yVP/QoUPy8/NT6dKlXcqLFi2qfPny6dChQy7lxYsXT9dGeHi4zp0752bE6bVr1061a9dW9+7dVaRIEbVv317z5s37y6TiVpxlypRJdy0mJka//vqrLl++7FL+53sJDw+XpGzdyyOPPKKQkBB9+umnmjNnjh566KF0v8tbHA6HJkyYoPvuu092u10FCxZUoUKFtHXrVl24cCHLfd5zzz3ZWlj5xhtvKH/+/EpOTtakSZNUuHDhLH8WQOZIJOCTQkNDFRkZqZ9//jlbn/vzYsfM5MqVK8NywzDc7uPW/P0tQUFBWrNmjZYvX65//etf2rp1q9q1a6eHH344Xd3bcTv3covdblebNm00a9YsLViwINPRCEl69dVX1b9/f9WrV08fffSRvvnmGy1btkwPPPBAlkdepN9/P9mxefNmnTp1SpK0bdu2bH0WQOZIJOCzmjdvrv3792vdunV/W7dEiRJyOBzau3evS/nJkyd1/vx55w4MM4SHh7vscLjlz6MekuTn56dGjRpp/Pjx2rFjh1555RWtXLlSq1atyrDtW3Hu3r073bVdu3apYMGCCg4Ovr0byETHjh21efNm/fbbbxkuUL3ls88+U8OGDfX++++rffv2aty4sWJjY9P9TrKa1GXF5cuX1aVLF5UrV07PPvusxowZow0bNpjWPnA3I5GAzxo8eLCCg4PVvXt3nTx5Mt31/fv3680335T0+9C8pHQ7K8aPHy9JevTRR02Lq1SpUrpw4YK2bt3qLDt+/LgWLFjgUu/s2bPpPnvrYKY/b0m9JSIiQpUrV9asWbNc/mL++eef9e233zrv0xMaNmyoUaNG6a233lLRokUzrZcrV650ox3//e9/dfToUZeyWwlPRklXdg0ZMkSHDx/WrFmzNH78eEVFRSkuLi7T3yOArONAKvisUqVK6eOPP1a7du0UExPjcrLl2rVr9d///ledO3eWJFWqVElxcXGaNm2azp8/r/r16+unn37SrFmz1KpVq0y3Frqjffv2GjJkiFq3bq3evXvrypUreuedd3T//fe7LDYcOXKk1qxZo0cffVQlSpTQqVOnNGXKFN17772qU6dOpu2PHTtWzZo1U82aNdWtWzddvXpVkydPVlhYmOLj4027jz/z8/PTyy+//Lf1mjdvrpEjR6pLly6qVauWtm3bpjlz5qhkyZIu9UqVKqV8+fJp6tSpCgkJUXBwsGrUqKHo6OhsxbVy5UpNmTJFI0aMcG5HnTFjhho0aKBhw4ZpzJgx2WoPwJ9YvGsE8Lg9e/YYzzzzjBEVFWUEBAQYISEhRu3atY3Jkycb165dc9a7ceOGkZCQYERHRxu5c+c2ihUrZgwdOtSljmH8vv3z0UcfTdfPn7cdZrb90zAM49tvvzXKly9vBAQEGGXKlDE++uijdNs/V6xYYbRs2dKIjIw0AgICjMjISKNDhw7Gnj170vXx5y2Sy5cvN2rXrm0EBQUZoaGhRosWLYwdO3a41LnV35+3l86YMcOQZKSkpGT6OzUM1+2fmcls++eAAQOMiIgIIygoyKhdu7axbt26DLdtLlq0yChXrpzh7+/vcp/169c3HnjggQz7/GM7Fy9eNEqUKGE8+OCDxo0bN1zq9evXz/Dz8zPWrVv3l/cA4K/ZDCMbK6oAAAD+gDUSAADAbSQSAADAbSQSAADAbSQSAAD4qDVr1qhFixaKjIyUzWbTwoULnddu3LihIUOGqEKFCgoODlZkZKSefvrpbD8Rl0QCAAAfdfnyZVWqVElvv/12umtXrlxRUlKShg0bpqSkJH3++efavXu3HnvssWz1wa4NAADuAjabTQsWLFCrVq0yrbNhwwZVr15dhw4dyvCZQhnhQCoAAHKI1NTUdCey2u122e12U9q/cOGCbDab8+nDWeGTiUTXqCesDgEAkEN8cPAzj/dx49cDprST+NaHSkhIcCkbMWKEKafWXrt2TUOGDFGHDh2cT1DOCp9MJAAA8EVDhw5V//79XcrMGI24ceOG2rZtK8Mw9M4772TrsyQSAAB4miPNlGbMnMa45VYScejQIa1cuTJboxESiQQAAJ5nOKyOIEO3koi9e/dq1apVKlCgQLbbIJEAAMDTHNYkEpcuXdK+ffuc71NSUpScnKz8+fMrIiJCTzzxhJKSkrRkyRKlpaXpxIkTkqT8+fMrICAgS3345PZPFlsCALLqjiy2PL7TlHZyR8Rkq/7q1avVsGHDdOVxcXGKj49XdHR0hp9btWqVGjRokKU+GJEAAMDDDIumNho0aKC/Gi8wYyyBRAIAAE+zaGrjTuCIbAAA4DZGJAAA8DQv3bVhBhIJAAA8zaRzJLwRUxsAAMBtjEgAAOBpTG0AAAC3sWsDAAAgPUYkAADwMKsOpLoTSCQAAPA0H57aIJEAAMDTfHhEgjUSAADAbYxIAADgaT58IBWJBAAAnsbUBgAAQHqMSAAA4Gns2gAAAG5jagMAACA9RiQAAPA0pjYAAIC7DMN3t38ytQEAANzGiAQAAJ7mw4stSSQAAPA01kgAAAC3+fCIBGskAACA2xiRAADA03hoFwAAcBtTGwAAAOkxIgEAgKexawMAALiNqQ0AAID0GJEAAMDTmNoAAABu8+FEgqkNAADgNkYkAADwMF9+jDiJBAAAnubDUxskEgAAeBrbPwEAANJjRAIAAE9jagMAALiNqQ0AAID0GJEAAMDTmNoAAABuY2oDAAAgPUYkAADwNKY2AACA20gkzBUeHi6bzZalumfPnvVwNAAAwF2WJBITJ050/nzmzBmNHj1aTZo0Uc2aNSVJ69at0zfffKNhw4ZZER4AAOby4cWWNsMwDCsDePzxx9WwYUP16tXLpfytt97S8uXLtXDhwmy32TXqCZOiAwD4ug8OfubxPq5+8YYp7QQ9NtCUdsxk+a6Nb775Rk2bNk1X3rRpUy1fvtyCiAAAMJnhMOflhSxPJAoUKKBFixalK1+0aJEKFChgQUQAAPiGNWvWqEWLFoqMjJTNZks3ym8YhoYPH66IiAgFBQUpNjZWe/fuzVYflu/aSEhIUPfu3bV69WrVqFFDkrR+/XotXbpU06dPtzg6AABMYNGujcuXL6tSpUrq2rWr2rRpk+76mDFjNGnSJM2aNUvR0dEaNmyYmjRpoh07digwMDBLfVieSHTu3FkxMTGaNGmSPv/8c0lSTEyMfvjhB2diAQBAjmbRtESzZs3UrFmzDK8ZhqGJEyfq5ZdfVsuWLSVJH374oYoUKaKFCxeqffv2WerD8kRCkmrUqKE5c+ZYHQYAAHeNlJQUnThxQrGxsc6ysLAw1ahRQ+vWrctZiYTD4dC+fft06tQpOf40/FOvXj2LogIAwCQmTW2kpqYqNTXVpcxut8tut2e7rRMnTkiSihQp4lJepEgR57WssDyR+PHHH9WxY0cdOnRIf96JarPZlJaWZlFkAACYxKREIjExUQkJCS5lI0aMUHx8vCntu8PyROK5555TtWrV9OWXXyoiIiLLJ14CAHC3GTp0qPr37+9S5s5ohCQVLVpUknTy5ElFREQ4y0+ePKnKlStnuR3LE4m9e/fqs88+U+nSpa0OBQAAzzDp7Ed3pzEyEh0draJFi2rFihXOxOHixYtav369nn/++Sy3Y3kiUaNGDe3bt49EAgDguyza/nnp0iXt27fP+T4lJUXJycnKnz+/ihcvrr59+2r06NG67777nNs/IyMj1apVqyz3YXki8cILL2jAgAE6ceKEKlSooNy5c7tcr1ixokWRAQCQs23cuFENGzZ0vr81LRIXF6eZM2dq8ODBunz5sp599lmdP39ederU0dKlS7N8hoTkBc/a8PNLf7imzWaTYRhuL7bkWRsAgKy6I8/amGPOQyiDOo0ypR0zWT4ikZKSYnUIAAB4lpc+J8MMlicSJUqUsDoEAAA8y6I1EneC5YnELTt27NDhw4d1/fp1l/LHHnvMoogAAMDfsTyROHDggFq3bq1t27Y510ZIcp4nwYFUAIAcz9rliB5l+WPE+/Tpo+joaJ06dUp58uTR9u3btWbNGlWrVk2rV6+2OjwAAG6fw2HOywtZPiKxbt06rVy5UgULFpSfn5/8/PxUp04dJSYmqnfv3tq8ebPVIQIAgExYPiKRlpamkJAQSVLBggV17NgxSb8vwty9e7eVoQEAYA5GJDynfPny2rJli6Kjo1WjRg2NGTNGAQEBmjZtmkqWLGl1eAAA3D62f3rOyy+/rMuXL0uSRo4cqebNm6tu3boqUKCAPv30U4ujAwAAf8XyRKJJkybOn0uXLq1du3bp7NmzCg8P50mgAACfYDh8d9eG5YlERvLnz291CAAAmMdL1zeYwZJEok2bNlmu+/nnn3swEgAAcDssSSTCwsKs6BYAAGuw2NJcM2bMsKJbAACswRoJAADgNh9eI2H5gVQAACDnYkQCAABP8+ERCRIJAAA8jad/es6HH36o1NTUdOXXr1/Xhx9+aEFEAAAgq2yGYW2alCtXLh0/flyFCxd2KT9z5owKFy6stLS0bLfZNeoJs8KDGxo81VgNOzVRwXsLSZKO7j2ixZM+07bVPMkVdze+G97pg4OfebyPK+OfMaWdPP2nm9KOmSyf2jAMI8OjsH/55RfOm8ihzh0/o89e/0gnDx6XzWZT7ccb6IVpgxX/6CAd2/uL1eEBluG7cRdj+6f5qlSpIpvNJpvNpkaNGsnf//9CSUtLU0pKipo2bWpVeLgNW1Zscnn/+RufqMFTjVWqyv38YYm7Gt8N+CLLEolWrVpJkpKTk9WkSRPlzZvXeS0gIEBRUVF6/PHHLYoOZrH5+emhR2vKHhSo/Ul7rA4H8Bp8N+4ynGxpvhEjRkiSoqKi1K5dOwUGBloVCjzgnjLF9dLnryi3PUCpV67prX+P0bF9/IsL4Ltxl/LhqQ3LF1vesmnTJu3cuVOS9MADD6hKlSpZ+lxqamq6XR8vVIhTLlsu02NE1uXK7a8CkQUVFJJH1R75h+q1a6TX243gD0zc9fhueJ87stjy9S6mtJNniPc9YsLyxZanTp1S+/bttXr1auXLl0+SdP78eTVs2FBz585VoUKF/vLziYmJSkhIcCmrHBajKvnKeSpkZEHajZs6deiEJOnQzwcUXbG0Yrs+og//M83iyABr8d24Oxk+fCCV5edIvPDCC/rtt9+0fft2nT17VmfPntXPP/+sixcvqnfv3n/7+aFDh+rChQsur4phZe5A5MgOm59N/gG5rQ4D8Dp8N+4SDsOclxeyfERi6dKlWr58uWJiYpxl5cqV09tvv63GjRv/7eftdrvsdrtLGdMa1np8cEdtW71ZZ479qsDgIP2jZR2V+ccDGv/0aKtDAyzFd+MuxmJLz3E4HMqdO302njt3bjl8eCjIl4UWCFP38S8orFC4rv52Rb/sOqTxT4/Wjh+2Wh0aYCm+G/BFlicS//znP9WnTx998sknioyMlCQdPXpU/fr1U6NGjSyODu6YMeQdq0MAvBLfjbuYl05LmMHyNRJvvfWWLl68qKioKJUqVUqlSpVSdHS0Ll68qMmTJ1sdHgAAt8/hMOflhSwfkShWrJiSkpK0fPly7dq1S5IUExOj2NhYiyMDAAB/x/JEQpJsNpsefvhhPfzww1aHAgCA+Xx4asMrEokVK1ZoxYoVOnXqVLoFlh988IFFUQEAYBJ2bXhOQkKCRo4cqWrVqikiIiLDJ4ECAADvZHkiMXXqVM2cOVP/+te/rA4FAADPYGrDc65fv65atWpZHQYAAB7DEdke1L17d3388cdWhwEAANxg+YjEtWvXNG3aNC1fvlwVK1ZMd8rl+PHjLYoMAACTMLXhOVu3blXlypUlST///LPLNRZeAgB8AomE56xatcrqEAAA8Cwf3v5p+RoJAACQc1k+IgEAgM9jagMAALjL8OFEgqkNAADgNkYkAADwNB8ekbAkkfjiiy+yXPexxx7zYCQAANwBPnyypSWJRKtWrVze22w2GYbh8v6WtLS0OxUWAADIJkvWSDgcDufr22+/VeXKlfX111/r/PnzOn/+vL766is9+OCDWrp0qRXhAQBgLodhzssLWb5Gom/fvpo6darq1KnjLGvSpIny5MmjZ599Vjt37rQwOgAATOClSYAZLN+1sX//fuXLly9deVhYmA4ePHjH4wEAAFlneSLx0EMPqX///jp58qSz7OTJkxo0aJCqV69uYWQAAJjDMAxTXtmRlpamYcOGKTo6WkFBQSpVqpRGjRqV7Xb+juVTGx988IFat26t4sWLq1ixYpKkI0eO6L777tPChQutDQ4AADNYMLXx+uuv65133tGsWbP0wAMPaOPGjerSpYvCwsLUu3dv0/qxPJEoXbq0tm7dqmXLlmnXrl2SpJiYGMXGxvL0TwCAb7AgkVi7dq1atmypRx99VJIUFRWlTz75RD/99JOp/VieSEi/b/ds3LixGjdubHUoAAB4rdTUVKWmprqU2e122e32dHVr1aqladOmac+ePbr//vu1ZcsW/fDDDxo/frypMVmSSEyaNEnPPvusAgMDNWnSpL+sa+bwCwAAVjDrWRuJiYlKSEhwKRsxYoTi4+PT1X3xxRd18eJFlS1bVrly5VJaWppeeeUVderUyZRYbrEZZq+6yILo6Ght3LhRBQoUUHR0dKb1bDabDhw4kO32u0Y9cTvhAQDuIh8c/MzjfVyIa2RKO4HTvsryiMTcuXM1aNAgjR07Vg888ICSk5PVt29fjR8/XnFxcabEI1k0IpGSkpLhzwAAIHOZJQ0ZGTRokF588UW1b99eklShQgUdOnRIiYmJOT+RAADgrmLBozauXLkiPz/XUx5y5colh8nP/bAkkejfv3+W65q9KAQAgDvNrDUS2dGiRQu98sorKl68uB544AFt3rxZ48ePV9euXU3tx5JEYvPmzS7vk5KSdPPmTZUpU0aStGfPHuXKlUtVq1a1IjwAAHK8yZMna9iwYerRo4dOnTqlyMhI/fvf/9bw4cNN7ceSRGLVqlXOn8ePH6+QkBDNmjVL4eHhkqRz586pS5cuqlu3rhXhAQBgLgtGJEJCQjRx4kRNnDjRo/1YfkT2uHHjlJiY6EwiJCk8PFyjR4/WuHHjLIwMAACTOEx6eSHLE4mLFy/q9OnT6cpPnz6t3377zYKIAABAVlm+a6N169bq0qWLxo0b53xI1/r16zVo0CC1adPG4ugAALh9Viy2vFMsTySmTp2qgQMHqmPHjrpx44Ykyd/fX926ddPYsWMtjg4AABN46bSEGSxPJPLkyaMpU6Zo7Nix2r9/vySpVKlSCg4OtjgyAADMwYjEHRAcHKyKFStaHQYAAMgGr0kkAADwWUxtAAAAdxk+nEhYvv0TAADkXIxIAADgaT48IkEiAQCAhzG1AQAAkAFGJAAA8DQfHpEgkQAAwMN8eWqDRAIAAA/z5USCNRIAAMBtjEgAAOBhvjwiQSIBAICnGTarI/AYpjYAAIDbTBmROH/+vPLly2dGUwAA+BxfntrI9ojE66+/rk8//dT5vm3btipQoIDuuecebdmyxdTgAADwBYbDZsrLG2U7kZg6daqKFSsmSVq2bJmWLVumr7/+Ws2aNdOgQYNMDxAAAHivbE9tnDhxwplILFmyRG3btlXjxo0VFRWlGjVqmB4gAAA5HVMbfxAeHq4jR45IkpYuXarY2FhJkmEYSktLMzc6AAB8gGHYTHl5o2yPSLRp00YdO3bUfffdpzNnzqhZs2aSpM2bN6t06dKmBwgAALxXthOJCRMmKCoqSkeOHNGYMWOUN29eSdLx48fVo0cP0wMEACCn8+WpjWwnErlz59bAgQPTlffr18+UgAAA8DXeuuPCDFlKJL744ossN/jYY4+5HQwAAL7IMKyOwHOylEi0atUqS43ZbDYWXAIAcBfJUiLhcPjw5A4AAB52109tZObatWsKDAw0KxYAAHySLycS2T5HIi0tTaNGjdI999yjvHnz6sCBA5KkYcOG6f333zc9QAAA4L2ynUi88sormjlzpsaMGaOAgABnefny5fXee++ZGhwAAL7AMMx5eaNsJxIffvihpk2bpk6dOilXrlzO8kqVKmnXrl2mBgcAgC/goV1/cPTo0QxPsHQ4HLpx44YpQQEAgJwh24lEuXLl9P3336cr/+yzz1SlShVTggIAwJfwrI0/GD58uOLi4nT06FE5HA59/vnn2r17tz788EMtWbLEEzECAJCj+fIR2dkekWjZsqUWL16s5cuXKzg4WMOHD9fOnTu1ePFiPfzww56IEQAAeCm3zpGoW7euli1bZnYsAAD4JIeXTkuYwe0DqTZu3KidO3dK+n3dRNWqVU0LCgAAX+Kt6xvMkO1E4pdfflGHDh30v//9T/ny5ZMknT9/XrVq1dLcuXN17733mh0jAAA5mrdu3TRDttdIdO/eXTdu3NDOnTt19uxZnT17Vjt37pTD4VD37t09ESMAAPBS2R6R+O6777R27VqVKVPGWVamTBlNnjxZdevWNTU4AAB8gbeeSmmGbCcSxYoVy/DgqbS0NEVGRpoSFAAAvoSpjT8YO3asXnjhBW3cuNFZtnHjRvXp00dvvPGGqcEBAADvlqURifDwcNls/5dNXb58WTVq1JC//+8fv3nzpvz9/dW1a1e1atXKI4ECAJBT3fXbPydOnOjhMAAA8F13/fbPuLg4T8cBAAByILcPpJKka9eu6fr16y5loaGhtxUQAAC+hl0bf3D58mUNGTJE8+bN05kzZ9JdT0tLMyUwAAB8hS+vkcj2ro3Bgwdr5cqVeuedd2S32/Xee+8pISFBkZGR+vDDDz0RIwAA8FLZTiQWL16sKVOm6PHHH5e/v7/q1q2rl19+Wa+++qrmzJnjiRgBAMjRDMNmyiu7jh49qqeeekoFChRQUFCQKlSo4HJ8gxmyPbVx9uxZlSxZUtLv6yHOnj0rSapTp46ef/55U4MDAMAXWLFG4ty5c6pdu7YaNmyor7/+WoUKFdLevXsVHh5uaj/ZTiRKliyplJQUFS9eXGXLltW8efNUvXp1LV682PkQLwAA8H+sWCPx+uuvq1ixYpoxY4azLDo62vR+sj210aVLF23ZskWS9OKLL+rtt99WYGCg+vXrp0GDBpkeIAAA+F1qaqouXrzo8kpNTc2w7hdffKFq1arpySefVOHChVWlShVNnz7d9JhshnF7Ay6HDh3Spk2bVLp0aVWsWNGsuG6Lf8A9VocAeKWrx763OgTA6+QuWNLjfWy4p7Up7Xz5TCUlJCS4lI0YMULx8fHp6gYGBkqS+vfvryeffFIbNmxQnz59NHXqVFPPh7rtRMIbkUgAGSORANK7E4nE+sg2prRTOeWTdCMQdrtddrs9Xd2AgABVq1ZNa9eudZb17t1bGzZs0Lp160yJR8riGolJkyZlucHevXu7HQwAAMhcZklDRiIiIlSuXDmXspiYGM2fP9/UmLKUSEyYMCFLjdlsNhIJAAD+xIqh/9q1a2v37t0uZXv27FGJEiVM7SdLiURKSoqpnQIAcDexYtdGv379VKtWLb366qtq27atfvrpJ02bNk3Tpk0ztZ9s79oAAADe76GHHtKCBQv0ySefqHz58ho1apQmTpyoTp06mdrPbT20CwAA/D2rHiPevHlzNW/e3KN9kEgAAOBhDqsD8CCmNgAAgNsYkQAAwMMM8RhxF99//72eeuop1axZU0ePHpUkzZ49Wz/88IOpwQEA4Aschjkvb5TtRGL+/Plq0qSJgoKCtHnzZucJWxcuXNCrr75qeoAAAOR0DtlMeXmjbCcSo0eP1tSpUzV9+nTlzp3bWV67dm0lJSWZGhwAAPBu2V4jsXv3btWrVy9deVhYmM6fP29GTAAA+BTWSPxB0aJFtW/fvnTlP/zwg0qW9PyDTwAAyGkcJr28UbYTiWeeeUZ9+vTR+vXrZbPZdOzYMc2ZM0cDBw7U888/74kYAQCAl8r21MaLL74oh8OhRo0a6cqVK6pXr57sdrsGDhyoF154wRMxAgCQo/ny1IbNMAy3NpRcv35d+/bt06VLl1SuXDnlzZvX7Njc5h9wj9UhAF7p6rHvrQ4B8Dq5C3p+Wn5pkfamtNP05FxT2jGT2wdSBQQEpHvOOQAAuLtkO5Fo2LChbLbMh2hWrlx5WwEBAOBrvHWhpBmynUhUrlzZ5f2NGzeUnJysn3/+WXFxcWbFBQCAz/DlNRLZTiQmTJiQYXl8fLwuXbp02wEBAICcw7Snfz711FP64IMPzGoOAACf4bCZ8/JGpj39c926dQoMDDSrOQAAfIa3PifDDNlOJNq0aePy3jAMHT9+XBs3btSwYcNMCwwAAF/hpQ/uNEW2E4mwsDCX935+fipTpoxGjhypxo0bmxYYAADwftlKJNLS0tSlSxdVqFBB4eHhnooJAACf4svbP7O12DJXrlxq3LgxT/kEACAbHDabKS9vlO1dG+XLl9eBAwc8EQsAAMhhsp1IjB49WgMHDtSSJUt0/PhxXbx40eUFAABcGSa9vFGW10iMHDlSAwYM0COPPCJJeuyxx1yOyjYMQzabTWlpaeZHCQBADubLaySynEgkJCToueee06pVqzwZDwAAyEGynEjcetp4/fr1PRYMAAC+yFtPpTRDtrZ//tVTPwEAQMY42fL/u//++/82mTh79uxtBQQAAHKObCUSCQkJ6U62BAAAf81bd1yYIVuJRPv27VW4cGFPxQIAgE9ijYRYHwEAgLt8eftnlg+kurVrAwAA4JYsj0g4HL6cTwEA4Dm+/E/xbD9GHAAAZI8vr5HI9rM2AAAAbmFEAgAAD/PlxQEkEgAAeJgvJxJMbQAAALcxIgEAgIcZPrzYkkQCAAAPY2oDAAAgA4xIAADgYb48IkEiAQCAh3GyJQAAcBsnWwIAAGSAEQkAADyMNRIAAMBtvpxIMLUBAADcxogEAAAexq4NAADgNnZtAACAHO21116TzWZT3759TW2XEQkAADzM6sWWGzZs0LvvvquKFSua3jYjEgAAeJhh0ssdly5dUqdOnTR9+nSFh4ffzm1kiEQCAAAf1rNnTz366KOKjY31SPtMbQAA4GEOk/ZtpKamKjU11aXMbrfLbrdnWH/u3LlKSkrShg0bTOk/I4xIAADgYQ6TXomJiQoLC3N5JSYmZtjnkSNH1KdPH82ZM0eBgYEeuzebYRg+t73VP+Aeq0MAvNLVY99bHQLgdXIXLOnxPkaW6GRKO0P2fJDlEYmFCxeqdevWypUrl7MsLS1NNptNfn5+Sk1NdbnmLqY2AADIIf5qGuPPGjVqpG3btrmUdenSRWXLltWQIUNMSSIkEgkAADzOiu2fISEhKl++vEtZcHCwChQokK78dpBIAADgYb58siWJBAAAd4nVq1eb3iaJBAAAHmbW9k9vRCIBAICH+W4awTkSAADgNjAiAQCAh1n90C5PsiyR2Lp1a5breuJpZQAA3CmskfCAypUry2azyTAM2Wx/vS8mLS3tDkUFAACyw7I1EikpKTpw4IBSUlI0f/58RUdHa8qUKdq8ebM2b96sKVOmqFSpUpo/f75VIQIAYAorHyPuaZaNSJQoUcL585NPPqlJkybpkUcecZZVrFhRxYoV07Bhw9SqVSsLIgQAwByskfCwbdu2KTo6Ol15dHS0duzYYUFEAACYx5fXSHjF9s+YmBglJibq+vXrzrLr168rMTFRMTExFkYGAAD+ileMSEydOlUtWrTQvffe69yhsXXrVtlsNi1evNji6AAAuD2+Ox7hJYlE9erVdeDAAc2ZM0e7du2SJLVr104dO3ZUcHCwxdEBAHB7WCNxBwQHB+vZZ5+1OgwAAJANXrFGQpJmz56tOnXqKDIyUocOHZIkTZgwQYsWLbI4MgAAbo9h0n/eyCsSiXfeeUf9+/dXs2bNdO7cOecBVOHh4Zo4caK1wQEAcJscJr28kVckEpMnT9b06dP10ksvyd///2ZbqlWrpm3btlkYGQAA+CtesUYiJSVFVapUSVdut9t1+fJlCyICAMA8nCPhYdHR0UpOTk5XvnTpUs6RAADkeByR7WH9+/dXz549de3aNRmGoZ9++kmffPKJEhMT9d5771kdHgAAyIRXJBLdu3dXUFCQXn75ZV25ckUdO3ZUZGSk3nzzTbVv397q8OCm55+L04D+z6to0ULaunWH+vQdpg0bk60OC7hjNiZv04yPP9OOXft0+sxZvZk4TI3q1ZIk3bh5U5OnzdL36zbql2PHlTc4WP94qIr6PddFhQsVsDhymI2pjTugU6dO2rt3ry5duqQTJ07ol19+Ubdu3awOC2568snH9MbYERo1erweqtFUW7bu0FdfzlEh/oDEXeTq1WsqU7qkXhrQI921a9dStWP3fv27cwfN++AtTXz1ZR08/It6DUmwIFJ4mi/v2rAZhuFzaZJ/wD1Wh3DXW/vDYm3YuEV9+r4sSbLZbDp4YIPenjJDY8a+bXF0d6+rx763OoS7VvnazVxGJDKybedudejeV8vmz1JE0cJ3MLq7W+6CJT3eR/eoJ0xp572Dn5nSjpksm9qoUqWKbDZbluomJSV5OBqYKXfu3HrwwYp6bcxbzjLDMLRi5Q/6xz+qWhgZ4N0uXboim82mkBAeDYCcw7JEolWrVs6fr127pilTpqhcuXKqWbOmJOnHH3/U9u3b1aNH+iHBP0pNTVVqaqpLmWEYWU5SYL6CBfPL399fp07+6lJ+6tRplS1TyqKoAO+WmnpdE975QI/E1ldenjHkc7x1WsIMliUSI0aMcP7cvXt39e7dW6NGjUpX58iRI3/ZTmJiohISXOcUbX55ZcsVal6wAOBBN27e1IBhr8owDA0b1MvqcOAB3nq8tRm8YrHlf//7Xz399NPpyp966inNnz//Lz87dOhQXbhwweVl8wvxVKjIgl9/PaubN2+qcJGCLuWFCxfSiZOnLYoK8E63kohjJ09p+sRXGY1AjuMViURQUJD+97//pSv/3//+p8DAwL/8rN1uV2hoqMuLaQ1r3bhxQ0lJW/XPhnWcZTabTf9sWEc//rjJwsgA73IriTh85Jjem/iq8oUxkuqrfHnXhlecI9G3b189//zzSkpKUvXq1SVJ69ev1wcffKBhw4ZZHB3cMeHN6Zrx/gRtStqqDRs2q/cLzyg4OEgzZ31qdWjAHXPlylUd/uWY8/3RYye1a89+hYWGqGDB/Or/0ivasWef3h6TIIfDoV/PnJUkhYWGKHfu3FaFDQ9w+N4GSSev2f45b948vfnmm9q5c6ckKSYmRn369FHbtm2z3RbbP71Dj+c7Ow+k2rJlu/r2G66fNmy2Oqy7Gts/76yfkraq6wtD0pW3bBarHt2eUpMnOmf4uQ8mv67qD1b0cHS45U5s//xXiTamtDP70OemtGMmr0kkzEQiAWSMRAJI704kEk+ZlEh85IWJhFdMbdxy/fp1nTp1Sg6H60xQ8eLFLYoIAIDb58tHZHtFIrF371517dpVa9eudSm/dR5EWlqaRZEBAIC/4hWJROfOneXv768lS5YoIiKCXRcAAJ/iy+dIeEUikZycrE2bNqls2bJWhwIAgOm8deumGbwikShXrpx+/fXXv68IAEAO5MtrJLziQKrXX39dgwcP1urVq3XmzBldvHjR5QUAALyTV4xIxMbGSpIaNWrkUs5iSwCAL2CNhIetWrXK6hAAAPAY1kh4WP369a0OAQAAuMErEolbrly5osOHD+v69esu5RUrclQsACDn8sFDpJ28IpE4ffq0unTpoq+//jrD66yRAADkZOza8LC+ffvq/PnzWr9+vYKCgrR06VLNmjVL9913n7744gurwwMAAJnwihGJlStXatGiRapWrZr8/PxUokQJPfzwwwoNDVViYqIeffRRq0MEAMBtvrzY0itGJC5fvqzChQtLksLDw3X69GlJUoUKFZSUlGRlaAAA3DbDpP+8kVckEmXKlNHu3bslSZUqVdK7776ro0ePaurUqYqIiLA4OgAAkBmvmNro06ePjh8/LkkaMWKEmjZtqo8++kgBAQGaNWuWxdEBAHB7fHmxpVckEk899ZTz56pVq+rQoUPatWuXihcvroIFC1oYGQAAt4/tnx7Qv3//LNcdP368ByMBAMCzfHmxpWWJxObNm7NUz2azeTgSAADgLssSCZ6vAQC4W3jrjgszeMUaCQAAfJkvL7b0iu2fAADAXImJiXrooYcUEhKiwoULq1WrVs6jFsxEIgEAgIcZhmHKKzu+++479ezZUz/++KOWLVumGzduqHHjxrp8+bKp98bUBgAAHmbF1MbSpUtd3s+cOVOFCxfWpk2bVK9ePdP6IZEAACCHSE1NVWpqqkuZ3W6X3W7/289euHBBkpQ/f35TY2JqAwAADzPrWRuJiYkKCwtzeSUmJv5t/w6HQ3379lXt2rVVvnx5U++NEQkAADzMYdLJlkOHDk13oGNWRiN69uypn3/+WT/88IMpcfwRiQQAADlEVqcx/qhXr15asmSJ1qxZo3vvvdf0mEgkAADwMCtOkTAMQy+88IIWLFig1atXKzo62iP9kEgAAOBhVuza6Nmzpz7++GMtWrRIISEhOnHihCQpLCxMQUFBpvVjM3zwkWT+AfdYHQLgla4e+97qEACvk7tgSY/3UfOehqa0s+5o1h8vkdmzqmbMmKHOnTubEo/EiAQAAD7pTo0TkEgAAOBhPjj470QiAQCAh/HQLgAAgAwwIgEAgIcZPjwiQSIBAICH+fIaCaY2AACA2xiRAADAw3x5sSWJBAAAHsbUBgAAQAYYkQAAwMOY2gAAAG5j+ycAAHCbgzUSAAAA6TEiAQCAhzG1AQAA3MbUBgAAQAYYkQAAwMOY2gAAAG5jagMAACADjEgAAOBhTG0AAAC3MbUBAACQAUYkAADwMKY2AACA2wzDYXUIHkMiAQCAh/nyY8RZIwEAANzGiAQAAB5m+PCuDRIJAAA8jKkNAACADDAiAQCAhzG1AQAA3MbJlgAAABlgRAIAAA/jZEsAAOA2X14jwdQGAABwGyMSAAB4mC+fI0EiAQCAh/ny1AaJBAAAHsb2TwAAgAwwIgEAgIcxtQEAANzmy4stmdoAAABuY0QCAAAPY2oDAAC4jV0bAAAAGWBEAgAAD+OhXQAAwG1MbQAAAGSAEQkAADyMXRsAAMBtvrxGgqkNAAA8zDAMU17uePvttxUVFaXAwEDVqFFDP/30k6n3RiIBAICP+vTTT9W/f3+NGDFCSUlJqlSpkpo0aaJTp06Z1ofN8MGJG/+Ae6wOAfBKV499b3UIgNfJXbCk5/sw6e+lG9ePZqt+jRo19NBDD+mtt96SJDkcDhUrVkwvvPCCXnzxRVNiYkQCAAAPM0x6Zcf169e1adMmxcbGOsv8/PwUGxurdevW3db9/BGLLQEAyCFSU1OVmprqUma322W329PV/fXXX5WWlqYiRYq4lBcpUkS7du0yLSafTCRuZnPoB56RmpqqxMREDR06NMP/yYG7Fd+Nu49Zfy/Fx8crISHBpWzEiBGKj483pX13+OQaCXiHixcvKiwsTBcuXFBoaKjV4QBeg+8G3JWdEYnr168rT548+uyzz9SqVStneVxcnM6fP69FixaZEhNrJAAAyCHsdrtCQ0NdXpmNagUEBKhq1apasWKFs8zhcGjFihWqWbOmaTH55NQGAACQ+vfvr7i4OFWrVk3Vq1fXxIkTdfnyZXXp0sW0PkgkAADwUe3atdPp06c1fPhwnThxQpUrV9bSpUvTLcC8HSQS8Bi73a4RI0awmAz4E74buJN69eqlXr16eax9FlsCAAC3sdgSAAC4jUQCAAC4jUQCAAC4jUTCBzVo0EB9+/Z1vo+KitLEiRP/8jM2m00LFy70aFxZ1blzZ5fDUzxl5syZypcvn8f7gW/78/fNE+Lj41W5cmWP9gG4i10bd4ENGzYoODjY6jAUHx+vhQsXKjk52epQAAAmIZG4CxQqVMjqEAAAPoqpjRzu8uXLevrpp5U3b15FRERo3Lhx6er8eWpj7969qlevngIDA1WuXDktW7bsb/tp0KCBevfurcGDByt//vwqWrRouofEHD58WC1btlTevHkVGhqqtm3b6uTJk5J+n0ZISEjQli1bZLPZZLPZNHPmzCzd49KlS1WnTh3ly5dPBQoUUPPmzbV//37n9YMHD8pms+nzzz9Xw4YNlSdPHlWqVCndY3Jnzpyp4sWLK0+ePGrdurXOnDmTpf6BrJo9e7aqVaumkJAQFS1aVB07dtSpU6ec11evXi2bzaYVK1aoWrVqypMnj2rVqqXdu3e7tPPaa6+pSJEiCgkJUbdu3XTt2rU7fStAlpFI5HCDBg3Sd999p0WLFunbb7/V6tWrlZSUlGl9h8OhNm3aKCAgQOvXr9fUqVM1ZMiQLPU1a9YsBQcHa/369RozZoxGjhzpTEIcDodatmyps2fP6rvvvtOyZct04MABtWvXTtLvp6sNGDBADzzwgI4fP67jx487r/2dy5cvq3///tq4caNWrFghPz8/tW7dWg6Hw6XeSy+9pIEDByo5OVn333+/OnTooJs3b0qS1q9fr27duqlXr15KTk5Ww4YNNXr06Cz1D2TVjRs3NGrUKG3ZskULFy7UwYMH1blz53T1XnrpJY0bN04bN26Uv7+/unbt6rw2b948xcfH69VXX9XGjRsVERGhKVOm3MG7ALLJQI7122+/GQEBAca8efOcZWfOnDGCgoKMPn36OMtKlChhTJgwwTAMw/jmm28Mf39/4+jRo87rX3/9tSHJWLBgQaZ91a9f36hTp45L2UMPPWQMGTLEMAzD+Pbbb41cuXIZhw8fdl7fvn27Icn46aefDMMwjBEjRhiVKlX62/uKi4szWrZsmen106dPG5KMbdu2GYZhGCkpKYYk47333kvX986dOw3DMIwOHToYjzzyiEs77dq1M8LCwv42HuCv1K9f3+X79kcbNmwwJBm//fabYRiGsWrVKkOSsXz5cmedL7/80pBkXL161TAMw6hZs6bRo0cPl3Zq1KiRpe8OYAVGJHKw/fv36/r166pRo4azLH/+/CpTpkymn9m5c6eKFSumyMhIZ1lWnwJXsWJFl/cRERHOYdtb7RYrVsx5vVy5csqXL5927tyZpfYzs3fvXnXo0EElS5ZUaGiooqKiJP0+lZJZfBEREZLkEt8ff09S1u8byKpNmzapRYsWKl68uEJCQlS/fn1J/L8K30YigSzLnTu3y3ubzZZuesETWrRoobNnz2r69Olav3691q9fL0m6fv16pvHZbDZJuiPxAdLvU3BNmjRRaGio5syZow0bNmjBggWS+H8Vvo1EIgcrVaqUcufO7fyLVZLOnTunPXv2ZPqZmJgYHTlyRMePH3eW/fjjj7cdy612jxw54izbsWOHzp8/r3LlykmSAgIClJaWlq12z5w5o927d+vll19Wo0aNFBMTo3PnzrkV3x9/T5I59w3csmvXLp05c0avvfaa6tatq7Jly7ostMwq/l9FTsP2zxwsb9686tatmwYNGqQCBQqocOHCeumll+Tnl3l+GBsbq/vvv19xcXEaO3asLl68qJdeeum2Y4mNjVWFChXUqVMnTZw4UTdv3lSPHj1Uv359VatWTdLvu0dSUlKUnJyse++9VyEhIX/79MPw8HAVKFBA06ZNU0REhA4fPqwXX3wx2/H17t1btWvX1htvvKGWLVvqm2++0dKlS926VyAjxYsXV0BAgCZPnqznnntOP//8s0aNGpXtdvr06aPOnTurWrVqql27tubMmaPt27erZMmSHogauH2MSORwY8eOVd26ddWiRQvFxsaqTp06qlq1aqb1/fz8tGDBAl29elXVq1dX9+7d9corr9x2HDabTYsWLVJ4eLjq1aun2NhYlSxZUp9++qmzzuOPP66mTZuqYcOGKlSokD755JO/bdfPz09z587Vpk2bVL58efXr109jx47Ndnz/+Mc/NH36dL355puqVKmSvv32W7388svZbgfITKFChTRz5kz997//Vbly5fTaa6/pjTfeyHY77dq107BhwzR48GBVrVpVhw4d0vPPP++BiAFz8BhxAADgNkYkAACA20gkAACA20gkAACA20gkAACA20gkAACA20gkAACA20gkAACA20gkAAt17txZrVq1cr5v0KCB+vbte8fjWL16tWw2m86fP59pHZvNpoULF2a5zfj4eFWuXPm24jp48KBsNpuSk5Nvqx0AnkMiAfxJ586dZbPZZLPZFBAQoNKlS2vkyJG6efOmx/v+/PPPs3ysclb+8gcAT+NZG0AGmjZtqhkzZig1NVVfffWVevbsqdy5c2vo0KHp6l6/fl0BAQGm9Js/f35T2gGAO4URCSADdrtdRYsWVYkSJfT8888rNjZWX3zxhaT/m4545ZVXFBkZqTJlykiSjhw5orZt2ypfvnzKnz+/WrZsqYMHDzrbTEtLU//+/ZUvXz4VKFBAgwcP1p9PqP/z1EZqaqqGDBmiYsWKyW63q3Tp0nr//fd18OBBNWzYUNLvDzaz2Wzq3LmzpN8fR52YmKjo6GgFBQWpUqVK+uyzz1z6+eqrr3T//fcrKChIDRs2dIkzq4YMGaL7779fefLkUcmSJTVs2DDduHEjXb13331XxYoVU548edS2bVtduHDB5fp7772nmJgYBQYGqmzZspoyZUqmfZ47d06dOnVSoUKFFBQUpPvuu08zZszIduwAzMOIBJAFQUFBOnPmjPP9ihUrFBoaqmXLlkmSbty4oSZNmqhmzZr6/vvv5e/vr9GjR6tp06baunWrAgICNG7cOM2cOVMffPCBYmJiNG7cOC1YsED//Oc/M+336aef1rp16zRp0iRVqlRJKSkp+vXXX1WsWDHNnz9fjz/+uHbv3q3Q0FAFBQVJkhITE/XRRx9p6tSpuu+++7RmzRo99dRTKlSokOrXr68jR46oTZs26tmzp5599llt3LhRAwYMyPbvJCQkRDNnzlRkZKS2bdumZ555RiEhIRo8eLCzzr59+zRv3jwtXrxYFy9eVLdu3dSjRw/NmTNHkjRnzhwNHz5cb731lqpUqaLNmzfrmWeeUXBwsOLi4tL1OWzYMO3YsUNff/21ChYsqH379unq1avZjh2AiQwALuLi4oyWLVsahmEYDofDWLZsmWG3242BAwc6rxcpUsRITU11fmb27NlGmTJlDIfD4SxLTU01goKCjG+++cYwDMOIiIgwxowZ47x+48YN495773X2ZRiGUb9+faNPnz6GYRjG7t27DUnGsmXLMoxz1apVhiTj3LlzzrJr164ZefLkMdauXetSt1u3bkaHDh0MwzCMoUOHGuXKlXO5PmTIkHRt/ZkkY8GCBZleHzt2rFG1alXn+xEjRhi5cuUyfvnlF2fZ119/bfj5+RnHjx83DMMwSpUqZXz88ccu7YwaNcqoWbOmYRiGkZKSYkgyNm/ebBiGYbRo0cLo0qVLpjEAuPMYkQAysGTJEuXNm1c3btyQw+FQx44dFR8f77xeoUIFl3URW7Zs0b59+xQSEuLSzrVr17R//35duHBBx48fV40aNZzX/P39Va1atXTTG7ckJycrV65cql+/fpbj3rdvn65cuaKHH37Ypfz69euqUqWKJGnnzp0ucUhSzZo1s9zHLZ9++qkmTZqk/fv369KlS7p586ZCQ0Nd6hQvXlz33HOPSz8Oh0O7d+9WSEiI9u/fr27duumZZ55x1rl586bCwsIy7PP555/X448/rqSkJDVu3FitWrVSrVq1sh07APOQSAAZaNiwod555x0FBAQoMjJS/v6uX5Xg4GCX95cuXVLVqlWdQ/Z/VKhQIbdiuDVVkR2XLl2SJH355Zcuf4FLv6/7MMu6devUqVMnJSQkqEmTJgoLC9PcuXM1bty4bMc6ffr0dIlNrly5MvxMs2bNdOjQIX311VdatmyZGjVqpJ49e+qNN95w/2YA3BYSCSADwcHBKl26dJbrP/jgg/r0009VuHDhdP8qvyUiIkLr169XvXr1JP3+L+9NmzbpwQcfzLB+hQoV5HA49N133yk2Njbd9VsjImlpac6ycuXKyW636/Dhw5mOZMTExDgXjt7y448//v1N/sHatWtVokQJvfTSS86yQ4cOpat3+PBhHTt2TJGRkc5+/Pz8VKZMGRUpUkSRkZE6cOCAOnXqlOW+CxUqpLi4OMXFxalu3boaNGgQiQRgIXZtACbo1KmTChYsqJYtW+r7779XSkqKVq9erd69e+uXX36RJPXp00evvfaaFi5cqF27dqlHjx5/eQZEVFSU4uLi1LVrVy1cuNDZ5rx58yRJJUqUkM1m05IlS3T69GldunRJISEhGjhwoPr166dZs2Zp//79SkpK0uTJkzVr1ixJ0nPPPae9e/dq0KBB2r17tz7++GPNnDkzW/d733336fDhw5o7d67279+vSZMmacGCBenqBQYGKi4uTlu2bNH333+v3r17q23btipatKgkKSEhQYmJiZo0aZL27Nmjbdu2acaMGRo/fnyG/Q4fPlyLFi3Svn37tH37di1ZskQxMTHZih2AuUgkABPkyZNHa9asUfHixdWmTRvFxMSoW7duunbtmnOEYsCAAfrXv/6luLg41axZUyEhIWrduvVftvvOO+/oiSeeUI8ePVS2bFk988wzunz5siTpnnvuUUJCgl588UUVKVJEvXr1kiSNGjVKw4YNU2JiomJiYtS0aVN9+eWXio6OlvT7uoX58+dr4cKFqlSpkqZOnapXX301W/f72GOPqV+/furVq5cqV66stWvXatiwYenqlS5dWm3atNEjjzyixo0bq2LFii7bO7t376733ntPM2bMUIUKFVS/fn3NnDnTGeufBQQEaOjQoapYsaLq1aunXLlyae7cudmKHYC5bEZmK70AAAD+BiMSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbf8PkAtlXISM2hwAAAAASUVORK5CYII=","text/plain":["<Figure size 640x480 with 2 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":["yhat=svm_cv.predict(X_test)\n","plot_confusion_matrix(Y_test,yhat)"]},{"cell_type":"markdown","id":"6e555a5a-a5d6-4556-9bf9-484ca164fc1a","metadata":{},"source":["## TASK  8\n"]},{"cell_type":"markdown","id":"c3582efe-6691-4736-8f93-ae37c99f521e","metadata":{},"source":["Create a decision tree classifier object then  create a  <code>GridSearchCV</code> object  <code>tree_cv</code> with cv = 10.  Fit the object to find the best parameters from the dictionary <code>parameters</code>.\n"]},{"cell_type":"code","execution_count":74,"id":"ab02dfc6-99f9-4b33-a06b-18eccbe9e365","metadata":{},"outputs":[],"source":["parameters = {'criterion': ['gini', 'entropy'],\n","     'splitter': ['best', 'random'],\n","     'max_depth': [2*n for n in range(1,10)],\n","     'max_features': ['auto', 'sqrt'],\n","     'min_samples_leaf': [1, 2, 4],\n","     'min_samples_split': [2, 5, 10]}\n","\n","tree = DecisionTreeClassifier()"]},{"cell_type":"code","execution_count":75,"id":"e85e077d-fd11-455e-a9e9-a5a460f22371","metadata":{},"outputs":[{"name":"stderr","output_type":"stream","text":["C:\\Users\\marco\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\sklearn\\model_selection\\_validation.py:540: FitFailedWarning: \n","3240 fits failed out of a total of 6480.\n","The score on these train-test partitions for these parameters will be set to nan.\n","If these failures are not expected, you can try to debug them by setting error_score='raise'.\n","\n","Below are more details about the failures:\n","--------------------------------------------------------------------------------\n","3240 fits failed with the following error:\n","Traceback (most recent call last):\n","  File \"C:\\Users\\marco\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\sklearn\\model_selection\\_validation.py\", line 888, in _fit_and_score\n","    estimator.fit(X_train, y_train, **fit_params)\n","  File \"C:\\Users\\marco\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\sklearn\\base.py\", line 1466, in wrapper\n","    estimator._validate_params()\n","  File \"C:\\Users\\marco\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\sklearn\\base.py\", line 666, in _validate_params\n","    validate_parameter_constraints(\n","  File \"C:\\Users\\marco\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\sklearn\\utils\\_param_validation.py\", line 95, in validate_parameter_constraints\n","    raise InvalidParameterError(\n","sklearn.utils._param_validation.InvalidParameterError: The 'max_features' parameter of DecisionTreeClassifier must be an int in the range [1, inf), a float in the range (0.0, 1.0], a str among {'log2', 'sqrt'} or None. Got 'auto' instead.\n","\n","  warnings.warn(some_fits_failed_message, FitFailedWarning)\n","C:\\Users\\marco\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\sklearn\\model_selection\\_search.py:1052: UserWarning: One or more of the test scores are non-finite: [       nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.73392857 0.76785714 0.77678571 0.78928571 0.77678571 0.77857143\n"," 0.73392857 0.77857143 0.79107143 0.725      0.81785714 0.81785714\n"," 0.74821429 0.76428571 0.78035714 0.78928571 0.76071429 0.70535714\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.79464286 0.81785714 0.775      0.79285714 0.81785714 0.83392857\n"," 0.85892857 0.80357143 0.77857143 0.81964286 0.80535714 0.84642857\n"," 0.79107143 0.77678571 0.75178571 0.76607143 0.73214286 0.83214286\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.80714286 0.79285714 0.82142857 0.81964286 0.81964286 0.83214286\n"," 0.75       0.80535714 0.83392857 0.80357143 0.84821429 0.7625\n"," 0.80535714 0.78928571 0.77321429 0.73392857 0.74821429 0.7625\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.75       0.81785714 0.81607143 0.775      0.78928571 0.76071429\n"," 0.78928571 0.83035714 0.7375     0.79285714 0.87321429 0.79285714\n"," 0.7625     0.84642857 0.72142857 0.79285714 0.84642857 0.80357143\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.78928571 0.80714286 0.78035714 0.84642857 0.81964286 0.79285714\n"," 0.75       0.77857143 0.70357143 0.77678571 0.71964286 0.80357143\n"," 0.77857143 0.80535714 0.7625     0.83392857 0.81785714 0.79107143\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.80178571 0.83214286 0.77857143 0.7625     0.81785714 0.78035714\n"," 0.69464286 0.77857143 0.69821429 0.7625     0.775      0.81964286\n"," 0.80535714 0.81964286 0.80714286 0.77678571 0.77678571 0.76785714\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.83214286 0.76607143 0.75357143 0.73571429 0.76607143 0.775\n"," 0.77678571 0.76071429 0.69107143 0.81964286 0.73214286 0.82321429\n"," 0.78928571 0.83392857 0.79464286 0.73392857 0.72142857 0.77678571\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.77678571 0.75178571 0.775      0.86071429 0.76428571 0.7625\n"," 0.7625     0.78928571 0.70535714 0.7625     0.79285714 0.74642857\n"," 0.81964286 0.86071429 0.725      0.71785714 0.79107143 0.80535714\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.775      0.71071429 0.80714286 0.83392857 0.84642857 0.76428571\n"," 0.73392857 0.77678571 0.77678571 0.78035714 0.81607143 0.72321429\n"," 0.79107143 0.79107143 0.74642857 0.7625     0.78928571 0.82321429\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.77678571 0.78035714 0.76428571 0.74642857 0.76607143 0.75178571\n"," 0.73392857 0.71964286 0.79107143 0.77678571 0.72321429 0.79107143\n"," 0.73392857 0.81785714 0.69107143 0.7375     0.7625     0.71071429\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.79285714 0.84642857 0.69642857 0.75178571 0.77321429 0.7875\n"," 0.73392857 0.80357143 0.87678571 0.71785714 0.81785714 0.84821429\n"," 0.725      0.77857143 0.7625     0.82142857 0.80535714 0.81964286\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.77857143 0.78928571 0.81785714 0.82142857 0.80357143 0.80357143\n"," 0.83214286 0.76607143 0.775      0.76428571 0.7875     0.775\n"," 0.77678571 0.7875     0.6625     0.81964286 0.77678571 0.84821429\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.81428571 0.75       0.75535714 0.76428571 0.77678571 0.78928571\n"," 0.75       0.79107143 0.875      0.74642857 0.83214286 0.81785714\n"," 0.79107143 0.79642857 0.725      0.85892857 0.78928571 0.80535714\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.73392857 0.80535714 0.77321429 0.80892857 0.79285714 0.83214286\n"," 0.80535714 0.79107143 0.79285714 0.73571429 0.78928571 0.80357143\n"," 0.78035714 0.75178571 0.69285714 0.81071429 0.79107143 0.75178571\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.81964286 0.77678571 0.72142857 0.74821429 0.81964286 0.79285714\n"," 0.7875     0.80535714 0.75       0.74821429 0.775      0.80535714\n"," 0.80714286 0.79107143 0.81785714 0.8875     0.76071429 0.9\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.75357143 0.80357143 0.79107143 0.77678571 0.73392857 0.83571429\n"," 0.7625     0.73392857 0.67857143 0.775      0.81785714 0.80535714\n"," 0.79107143 0.76071429 0.75       0.86071429 0.80357143 0.81428571\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.73571429 0.81785714 0.75178571 0.76428571 0.72321429 0.83214286\n"," 0.81785714 0.76607143 0.78928571 0.80357143 0.81607143 0.78928571\n"," 0.74464286 0.80535714 0.79285714 0.83392857 0.73571429 0.79107143\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n","        nan        nan        nan        nan        nan        nan\n"," 0.84642857 0.775      0.79285714 0.79107143 0.75       0.76607143\n"," 0.7625     0.63928571 0.80535714 0.84464286 0.80357143 0.80535714\n"," 0.83392857 0.85       0.79285714 0.84642857 0.83035714 0.73392857]\n","  warnings.warn(\n"]},{"data":{"text/html":["<style>#sk-container-id-8 {\n","  /* Definition of color scheme common for light and dark mode */\n","  --sklearn-color-text: black;\n","  --sklearn-color-line: gray;\n","  /* Definition of color scheme for unfitted estimators */\n","  --sklearn-color-unfitted-level-0: #fff5e6;\n","  --sklearn-color-unfitted-level-1: #f6e4d2;\n","  --sklearn-color-unfitted-level-2: #ffe0b3;\n","  --sklearn-color-unfitted-level-3: chocolate;\n","  /* Definition of color scheme for fitted estimators */\n","  --sklearn-color-fitted-level-0: #f0f8ff;\n","  --sklearn-color-fitted-level-1: #d4ebff;\n","  --sklearn-color-fitted-level-2: #b3dbfd;\n","  --sklearn-color-fitted-level-3: cornflowerblue;\n","\n","  /* Specific color for light theme */\n","  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n","  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n","  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n","  --sklearn-color-icon: #696969;\n","\n","  @media (prefers-color-scheme: dark) {\n","    /* Redefinition of color scheme for dark theme */\n","    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n","    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n","    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n","    --sklearn-color-icon: #878787;\n","  }\n","}\n","\n","#sk-container-id-8 {\n","  color: var(--sklearn-color-text);\n","}\n","\n","#sk-container-id-8 pre {\n","  padding: 0;\n","}\n","\n","#sk-container-id-8 input.sk-hidden--visually {\n","  border: 0;\n","  clip: rect(1px 1px 1px 1px);\n","  clip: rect(1px, 1px, 1px, 1px);\n","  height: 1px;\n","  margin: -1px;\n","  overflow: hidden;\n","  padding: 0;\n","  position: absolute;\n","  width: 1px;\n","}\n","\n","#sk-container-id-8 div.sk-dashed-wrapped {\n","  border: 1px dashed var(--sklearn-color-line);\n","  margin: 0 0.4em 0.5em 0.4em;\n","  box-sizing: border-box;\n","  padding-bottom: 0.4em;\n","  background-color: var(--sklearn-color-background);\n","}\n","\n","#sk-container-id-8 div.sk-container {\n","  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n","     but bootstrap.min.css set `[hidden] { display: none !important; }`\n","     so we also need the `!important` here to be able to override the\n","     default hidden behavior on the sphinx rendered scikit-learn.org.\n","     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n","  display: inline-block !important;\n","  position: relative;\n","}\n","\n","#sk-container-id-8 div.sk-text-repr-fallback {\n","  display: none;\n","}\n","\n","div.sk-parallel-item,\n","div.sk-serial,\n","div.sk-item {\n","  /* draw centered vertical line to link estimators */\n","  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n","  background-size: 2px 100%;\n","  background-repeat: no-repeat;\n","  background-position: center center;\n","}\n","\n","/* Parallel-specific style estimator block */\n","\n","#sk-container-id-8 div.sk-parallel-item::after {\n","  content: \"\";\n","  width: 100%;\n","  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n","  flex-grow: 1;\n","}\n","\n","#sk-container-id-8 div.sk-parallel {\n","  display: flex;\n","  align-items: stretch;\n","  justify-content: center;\n","  background-color: var(--sklearn-color-background);\n","  position: relative;\n","}\n","\n","#sk-container-id-8 div.sk-parallel-item {\n","  display: flex;\n","  flex-direction: column;\n","}\n","\n","#sk-container-id-8 div.sk-parallel-item:first-child::after {\n","  align-self: flex-end;\n","  width: 50%;\n","}\n","\n","#sk-container-id-8 div.sk-parallel-item:last-child::after {\n","  align-self: flex-start;\n","  width: 50%;\n","}\n","\n","#sk-container-id-8 div.sk-parallel-item:only-child::after {\n","  width: 0;\n","}\n","\n","/* Serial-specific style estimator block */\n","\n","#sk-container-id-8 div.sk-serial {\n","  display: flex;\n","  flex-direction: column;\n","  align-items: center;\n","  background-color: var(--sklearn-color-background);\n","  padding-right: 1em;\n","  padding-left: 1em;\n","}\n","\n","\n","/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n","clickable and can be expanded/collapsed.\n","- Pipeline and ColumnTransformer use this feature and define the default style\n","- Estimators will overwrite some part of the style using the `sk-estimator` class\n","*/\n","\n","/* Pipeline and ColumnTransformer style (default) */\n","\n","#sk-container-id-8 div.sk-toggleable {\n","  /* Default theme specific background. It is overwritten whether we have a\n","  specific estimator or a Pipeline/ColumnTransformer */\n","  background-color: var(--sklearn-color-background);\n","}\n","\n","/* Toggleable label */\n","#sk-container-id-8 label.sk-toggleable__label {\n","  cursor: pointer;\n","  display: block;\n","  width: 100%;\n","  margin-bottom: 0;\n","  padding: 0.5em;\n","  box-sizing: border-box;\n","  text-align: center;\n","}\n","\n","#sk-container-id-8 label.sk-toggleable__label-arrow:before {\n","  /* Arrow on the left of the label */\n","  content: \"▸\";\n","  float: left;\n","  margin-right: 0.25em;\n","  color: var(--sklearn-color-icon);\n","}\n","\n","#sk-container-id-8 label.sk-toggleable__label-arrow:hover:before {\n","  color: var(--sklearn-color-text);\n","}\n","\n","/* Toggleable content - dropdown */\n","\n","#sk-container-id-8 div.sk-toggleable__content {\n","  max-height: 0;\n","  max-width: 0;\n","  overflow: hidden;\n","  text-align: left;\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-8 div.sk-toggleable__content.fitted {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","#sk-container-id-8 div.sk-toggleable__content pre {\n","  margin: 0.2em;\n","  border-radius: 0.25em;\n","  color: var(--sklearn-color-text);\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-8 div.sk-toggleable__content.fitted pre {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","#sk-container-id-8 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n","  /* Expand drop-down */\n","  max-height: 200px;\n","  max-width: 100%;\n","  overflow: auto;\n","}\n","\n","#sk-container-id-8 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n","  content: \"▾\";\n","}\n","\n","/* Pipeline/ColumnTransformer-specific style */\n","\n","#sk-container-id-8 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-8 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Estimator-specific style */\n","\n","/* Colorize estimator box */\n","#sk-container-id-8 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-8 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","#sk-container-id-8 div.sk-label label.sk-toggleable__label,\n","#sk-container-id-8 div.sk-label label {\n","  /* The background is the default theme color */\n","  color: var(--sklearn-color-text-on-default-background);\n","}\n","\n","/* On hover, darken the color of the background */\n","#sk-container-id-8 div.sk-label:hover label.sk-toggleable__label {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","/* Label box, darken color on hover, fitted */\n","#sk-container-id-8 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Estimator label */\n","\n","#sk-container-id-8 div.sk-label label {\n","  font-family: monospace;\n","  font-weight: bold;\n","  display: inline-block;\n","  line-height: 1.2em;\n","}\n","\n","#sk-container-id-8 div.sk-label-container {\n","  text-align: center;\n","}\n","\n","/* Estimator-specific */\n","#sk-container-id-8 div.sk-estimator {\n","  font-family: monospace;\n","  border: 1px dotted var(--sklearn-color-border-box);\n","  border-radius: 0.25em;\n","  box-sizing: border-box;\n","  margin-bottom: 0.5em;\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-8 div.sk-estimator.fitted {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","/* on hover */\n","#sk-container-id-8 div.sk-estimator:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-8 div.sk-estimator.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Specification for estimator info (e.g. \"i\" and \"?\") */\n","\n","/* Common style for \"i\" and \"?\" */\n","\n",".sk-estimator-doc-link,\n","a:link.sk-estimator-doc-link,\n","a:visited.sk-estimator-doc-link {\n","  float: right;\n","  font-size: smaller;\n","  line-height: 1em;\n","  font-family: monospace;\n","  background-color: var(--sklearn-color-background);\n","  border-radius: 1em;\n","  height: 1em;\n","  width: 1em;\n","  text-decoration: none !important;\n","  margin-left: 1ex;\n","  /* unfitted */\n","  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-unfitted-level-1);\n","}\n","\n",".sk-estimator-doc-link.fitted,\n","a:link.sk-estimator-doc-link.fitted,\n","a:visited.sk-estimator-doc-link.fitted {\n","  /* fitted */\n","  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-fitted-level-1);\n","}\n","\n","/* On hover */\n","div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",".sk-estimator-doc-link:hover,\n","div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",".sk-estimator-doc-link:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",".sk-estimator-doc-link.fitted:hover,\n","div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",".sk-estimator-doc-link.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","/* Span, style for the box shown on hovering the info icon */\n",".sk-estimator-doc-link span {\n","  display: none;\n","  z-index: 9999;\n","  position: relative;\n","  font-weight: normal;\n","  right: .2ex;\n","  padding: .5ex;\n","  margin: .5ex;\n","  width: min-content;\n","  min-width: 20ex;\n","  max-width: 50ex;\n","  color: var(--sklearn-color-text);\n","  box-shadow: 2pt 2pt 4pt #999;\n","  /* unfitted */\n","  background: var(--sklearn-color-unfitted-level-0);\n","  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n","}\n","\n",".sk-estimator-doc-link.fitted span {\n","  /* fitted */\n","  background: var(--sklearn-color-fitted-level-0);\n","  border: var(--sklearn-color-fitted-level-3);\n","}\n","\n",".sk-estimator-doc-link:hover span {\n","  display: block;\n","}\n","\n","/* \"?\"-specific style due to the `<a>` HTML tag */\n","\n","#sk-container-id-8 a.estimator_doc_link {\n","  float: right;\n","  font-size: 1rem;\n","  line-height: 1em;\n","  font-family: monospace;\n","  background-color: var(--sklearn-color-background);\n","  border-radius: 1rem;\n","  height: 1rem;\n","  width: 1rem;\n","  text-decoration: none;\n","  /* unfitted */\n","  color: var(--sklearn-color-unfitted-level-1);\n","  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n","}\n","\n","#sk-container-id-8 a.estimator_doc_link.fitted {\n","  /* fitted */\n","  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-fitted-level-1);\n","}\n","\n","/* On hover */\n","#sk-container-id-8 a.estimator_doc_link:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","#sk-container-id-8 a.estimator_doc_link.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-3);\n","}\n","</style><div id=\"sk-container-id-8\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=10, estimator=DecisionTreeClassifier(),\n","             param_grid={&#x27;criterion&#x27;: [&#x27;gini&#x27;, &#x27;entropy&#x27;],\n","                         &#x27;max_depth&#x27;: [2, 4, 6, 8, 10, 12, 14, 16, 18],\n","                         &#x27;max_features&#x27;: [&#x27;auto&#x27;, &#x27;sqrt&#x27;],\n","                         &#x27;min_samples_leaf&#x27;: [1, 2, 4],\n","                         &#x27;min_samples_split&#x27;: [2, 5, 10],\n","                         &#x27;splitter&#x27;: [&#x27;best&#x27;, &#x27;random&#x27;]})</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-22\" type=\"checkbox\" ><label for=\"sk-estimator-id-22\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;GridSearchCV<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.model_selection.GridSearchCV.html\">?<span>Documentation for GridSearchCV</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>GridSearchCV(cv=10, estimator=DecisionTreeClassifier(),\n","             param_grid={&#x27;criterion&#x27;: [&#x27;gini&#x27;, &#x27;entropy&#x27;],\n","                         &#x27;max_depth&#x27;: [2, 4, 6, 8, 10, 12, 14, 16, 18],\n","                         &#x27;max_features&#x27;: [&#x27;auto&#x27;, &#x27;sqrt&#x27;],\n","                         &#x27;min_samples_leaf&#x27;: [1, 2, 4],\n","                         &#x27;min_samples_split&#x27;: [2, 5, 10],\n","                         &#x27;splitter&#x27;: [&#x27;best&#x27;, &#x27;random&#x27;]})</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-23\" type=\"checkbox\" ><label for=\"sk-estimator-id-23\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">best_estimator_: DecisionTreeClassifier</label><div class=\"sk-toggleable__content fitted\"><pre>DecisionTreeClassifier(criterion=&#x27;entropy&#x27;, max_depth=12, max_features=&#x27;sqrt&#x27;,\n","                       min_samples_leaf=4, min_samples_split=10,\n","                       splitter=&#x27;random&#x27;)</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-24\" type=\"checkbox\" ><label for=\"sk-estimator-id-24\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;DecisionTreeClassifier<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.tree.DecisionTreeClassifier.html\">?<span>Documentation for DecisionTreeClassifier</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>DecisionTreeClassifier(criterion=&#x27;entropy&#x27;, max_depth=12, max_features=&#x27;sqrt&#x27;,\n","                       min_samples_leaf=4, min_samples_split=10,\n","                       splitter=&#x27;random&#x27;)</pre></div> </div></div></div></div></div></div></div></div></div>"],"text/plain":["GridSearchCV(cv=10, estimator=DecisionTreeClassifier(),\n","             param_grid={'criterion': ['gini', 'entropy'],\n","                         'max_depth': [2, 4, 6, 8, 10, 12, 14, 16, 18],\n","                         'max_features': ['auto', 'sqrt'],\n","                         'min_samples_leaf': [1, 2, 4],\n","                         'min_samples_split': [2, 5, 10],\n","                         'splitter': ['best', 'random']})"]},"execution_count":75,"metadata":{},"output_type":"execute_result"}],"source":["tree_cv = GridSearchCV(tree, parameters, cv =10)\n","tree_cv.fit(X_train, Y_train)"]},{"cell_type":"code","execution_count":76,"id":"ac572f94-5a92-4893-97fc-274b3192ca1a","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["tuned hpyerparameters :(best parameters)  {'criterion': 'entropy', 'max_depth': 12, 'max_features': 'sqrt', 'min_samples_leaf': 4, 'min_samples_split': 10, 'splitter': 'random'}\n","accuracy : 0.9\n"]}],"source":["print(\"tuned hpyerparameters :(best parameters) \",tree_cv.best_params_)\n","print(\"accuracy :\",tree_cv.best_score_)"]},{"cell_type":"markdown","id":"e2c429a1-02a5-4ac7-ba4d-848a7fcf9f75","metadata":{},"source":["## TASK  9\n"]},{"cell_type":"markdown","metadata":{},"source":["Calculate the accuracy of tree_cv on the test data using the method <code>score</code>:\n"]},{"cell_type":"code","execution_count":77,"id":"7d3f2880-8c11-4fc6-b97d-2070ae8b389f","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["MSE =  0.16666666666666666\n","Variance score: 0.83\n"]}],"source":["y_hat = tree_cv.predict(X_test)\n","\n","print(\"MSE = \", np.mean((y_hat-Y_test)**2))\n","print('Variance score: %.2f' % tree_cv.score(X_test, Y_test))"]},{"cell_type":"markdown","id":"26b85dcf-e2ff-428b-9305-9f5680eabe8c","metadata":{},"source":["We can plot the confusion matrix\n"]},{"cell_type":"code","execution_count":51,"id":"b4bef500-aa32-471e-8367-368275ae9188","metadata":{},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAhIAAAHHCAYAAADqJrG+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAABGBElEQVR4nO3dd3xUVf7/8feEkEkISQg9QQgdQldRvtTIEpqAtJXqGpquigJSRFaRhGJckCK6gKACIogoAjZAEBERRFoAaVJCkSIsvQZI7u8Pfsw6JIFkuJc7jK/nPu7jkTlz5pzPzRLzyWnXYRiGIQAAAA/42R0AAAC4d5FIAAAAj5FIAAAAj5FIAAAAj5FIAAAAj5FIAAAAj5FIAAAAj5FIAAAAj5FIAAAAj5FIABbatWuXGjVqpLCwMDkcDs2fP9/U9vft2yeHw6Fp06aZ2u697JFHHtEjjzxidxjAXwaJBHzenj179M9//lMlS5ZUYGCgQkNDVbt2bb311lu6dOmSpX3HxcVpy5YtGjFihGbMmKHq1atb2t/d1KVLFzkcDoWGhmb4fdy1a5ccDoccDofefPPNbLd/+PBhxcfHKykpyYRoAVjF3+4AACt9/fXXevzxx+V0OvXkk0+qUqVKunLlilauXKkBAwZo69atmjx5siV9X7p0SatXr9Yrr7yi559/3pI+oqKidOnSJeXMmdOS9m/H399fFy9e1Jdffql27dq5vTdz5kwFBgbq8uXLHrV9+PBhJSQkqHjx4qpWrVqWP/ftt9961B8Az5BIwGclJyerQ4cOioqK0rJlyxQREeF6r2fPntq9e7e+/vpry/o/fvy4JClPnjyW9eFwOBQYGGhZ+7fjdDpVu3Ztffzxx+kSiVmzZqlZs2aaO3fuXYnl4sWLypUrlwICAu5KfwCuY2oDPmvkyJE6f/683n//fbck4obSpUurd+/ertfXrl3TsGHDVKpUKTmdThUvXlz/+te/lJKS4va54sWLq3nz5lq5cqUefvhhBQYGqmTJkvrwww9ddeLj4xUVFSVJGjBggBwOh4oXLy7p+pTAja//LD4+Xg6Hw61syZIlqlOnjvLkyaPcuXOrXLly+te//uV6P7M1EsuWLVPdunUVHBysPHnyqGXLltq+fXuG/e3evVtdunRRnjx5FBYWpq5du+rixYuZf2Nv0qlTJy1cuFCnT592la1du1a7du1Sp06d0tU/efKk+vfvr8qVKyt37twKDQ1V06ZNtWnTJled5cuX66GHHpIkde3a1TVFcuM+H3nkEVWqVEnr169XvXr1lCtXLtf35eY1EnFxcQoMDEx3/40bN1Z4eLgOHz6c5XsFkB6JBHzWl19+qZIlS6pWrVpZqt+jRw+99tpreuCBBzR27FjFxMQoMTFRHTp0SFd39+7d+vvf/66GDRtq9OjRCg8PV5cuXbR161ZJUps2bTR27FhJUseOHTVjxgyNGzcuW/Fv3bpVzZs3V0pKioYOHarRo0frscce008//XTLzy1dulSNGzfWsWPHFB8fr759+2rVqlWqXbu29u3bl65+u3btdO7cOSUmJqpdu3aaNm2aEhISshxnmzZt5HA49Pnnn7vKZs2apfLly+uBBx5IV3/v3r2aP3++mjdvrjFjxmjAgAHasmWLYmJiXL/Uo6OjNXToUEnS008/rRkzZmjGjBmqV6+eq50TJ06oadOmqlatmsaNG6f69etnGN9bb72lAgUKKC4uTqmpqZKkd999V99++63efvttRUZGZvleAWTAAHzQmTNnDElGy5Yts1Q/KSnJkGT06NHDrbx///6GJGPZsmWusqioKEOSsWLFClfZsWPHDKfTafTr189VlpycbEgyRo0a5dZmXFycERUVlS6GIUOGGH/+kRw7dqwhyTh+/Himcd/oY+rUqa6yatWqGQULFjROnDjhKtu0aZPh5+dnPPnkk+n669atm1ubrVu3NvLly5dpn3++j+DgYMMwDOPvf/+70aBBA8MwDCM1NdUoXLiwkZCQkOH34PLly0Zqamq6+3A6ncbQoUNdZWvXrk13bzfExMQYkoxJkyZl+F5MTIxb2eLFiw1JxvDhw429e/cauXPnNlq1anXbewRwe4xIwCedPXtWkhQSEpKl+t98840kqW/fvm7l/fr1k6R0aykqVKigunXrul4XKFBA5cqV0969ez2O+WY31lYsWLBAaWlpWfrMkSNHlJSUpC5duihv3ryu8ipVqqhhw4au+/yzZ555xu113bp1deLECdf3MCs6deqk5cuX6+jRo1q2bJmOHj2a4bSGdH1dhZ/f9f/0pKam6sSJE65pmw0bNmS5T6fTqa5du2apbqNGjfTPf/5TQ4cOVZs2bRQYGKh33303y30ByByJBHxSaGioJOncuXNZqr9//375+fmpdOnSbuWFCxdWnjx5tH//frfyYsWKpWsjPDxcp06d8jDi9Nq3b6/atWurR48eKlSokDp06KA5c+bcMqm4EWe5cuXSvRcdHa3//ve/unDhglv5zfcSHh4uSdm6l0cffVQhISH65JNPNHPmTD300EPpvpc3pKWlaezYsSpTpoycTqfy58+vAgUKaPPmzTpz5kyW+yxSpEi2Fla++eabyps3r5KSkjR+/HgVLFgwy58FkDkSCfik0NBQRUZG6tdff83W525e7JiZHDlyZFhuGIbHfdyYv78hKChIK1as0NKlS/WPf/xDmzdvVvv27dWwYcN0de/EndzLDU6nU23atNH06dM1b968TEcjJOn1119X3759Va9ePX300UdavHixlixZoooVK2Z55EW6/v3Jjo0bN+rYsWOSpC1btmTrswAyRyIBn9W8eXPt2bNHq1evvm3dqKgopaWladeuXW7lf/zxh06fPu3agWGG8PBwtx0ON9w86iFJfn5+atCggcaMGaNt27ZpxIgRWrZsmb7//vsM274R586dO9O9t2PHDuXPn1/BwcF3dgOZ6NSpkzZu3Khz585luED1hs8++0z169fX+++/rw4dOqhRo0aKjY1N9z3JalKXFRcuXFDXrl1VoUIFPf300xo5cqTWrl1rWvvAXxmJBHzWSy+9pODgYPXo0UN//PFHuvf37Nmjt956S9L1oXlJ6XZWjBkzRpLUrFkz0+IqVaqUzpw5o82bN7vKjhw5onnz5rnVO3nyZLrP3jiY6eYtqTdERESoWrVqmj59utsv5l9//VXffvut6z6tUL9+fQ0bNkzvvPOOChcunGm9HDlypBvt+PTTT3Xo0CG3shsJT0ZJV3YNHDhQBw4c0PTp0zVmzBgVL15ccXFxmX4fAWQdB1LBZ5UqVUqzZs1S+/btFR0d7Xay5apVq/Tpp5+qS5cukqSqVasqLi5OkydP1unTpxUTE6NffvlF06dPV6tWrTLdWuiJDh06aODAgWrdurV69eqlixcvauLEiSpbtqzbYsOhQ4dqxYoVatasmaKionTs2DFNmDBB9913n+rUqZNp+6NGjVLTpk1Vs2ZNde/eXZcuXdLbb7+tsLAwxcfHm3YfN/Pz89Orr75623rNmzfX0KFD1bVrV9WqVUtbtmzRzJkzVbJkSbd6pUqVUp48eTRp0iSFhIQoODhYNWrUUIkSJbIV17JlyzRhwgQNGTLEtR116tSpeuSRRzR48GCNHDkyW+0BuInNu0YAy/3222/GU089ZRQvXtwICAgwQkJCjNq1axtvv/22cfnyZVe9q1evGgkJCUaJEiWMnDlzGkWLFjUGDRrkVscwrm//bNasWbp+bt52mNn2T8MwjG+//daoVKmSERAQYJQrV8746KOP0m3//O6774yWLVsakZGRRkBAgBEZGWl07NjR+O2339L1cfMWyaVLlxq1a9c2goKCjNDQUKNFixbGtm3b3Orc6O/m7aVTp041JBnJycmZfk8Nw337Z2Yy2/7Zr18/IyIiwggKCjJq165trF69OsNtmwsWLDAqVKhg+Pv7u91nTEyMUbFixQz7/HM7Z8+eNaKioowHHnjAuHr1qlu9F1980fDz8zNWr159y3sAcGsOw8jGiioAAIA/YY0EAADwGIkEAADwGIkEAADwGIkEAAA+asWKFWrRooUiIyPlcDg0f/58t/cNw9Brr72miIgIBQUFKTY2Nt15OrdDIgEAgI+6cOGCqlatqv/85z8Zvj9y5EiNHz9ekyZN0po1axQcHKzGjRvr8uXLWe6DXRsAAPwFOBwOzZs3T61atZJ0fTQiMjJS/fr1U//+/SVJZ86cUaFChTRt2rRbnlD7Z4xIAABwj0hJSdHZs2fdLk9PaE1OTtbRo0cVGxvrKgsLC1ONGjWy9GiBG3zyZEv/gCJ2hwB4pScja9odAuB1Ptj3meV9XP3vXlPaSXznQyUkJLiVDRkyxKNTa48ePSpJKlSokFt5oUKFXO9lhU8mEgAA+KJBgwapb9++bmVOp9OmaK4jkQAAwGppqaY043Q6TUscbjxc748//lBERISr/I8//nA9IDArWCMBAIDVjDRzLhOVKFFChQsX1nfffecqO3v2rNasWaOaNbM+DcqIBAAAVkszNwnIqvPnz2v37t2u18nJyUpKSlLevHlVrFgx9enTR8OHD1eZMmVUokQJDR48WJGRka6dHVlBIgEAgI9at26d6tev73p9Y31FXFycpk2bppdeekkXLlzQ008/rdOnT6tOnTpatGiRAgMDs9yHT54jwa4NIGPs2gDSuxu7Nq4c3mpKOwGRFU1px0yMSAAAYDWbpjbuBhZbAgAAjzEiAQCA1UzeceFNSCQAALCaSedIeCOmNgAAgMcYkQAAwGpMbQAAAI+xawMAACA9RiQAALCYwdQGAADwmA9PbZBIAABgNR8ekWCNBAAA8BgjEgAAWM2HD6QikQAAwGpMbQAAAKTHiAQAAFZj1wYAAPAYUxsAAADpMSIBAIDVmNoAAACeMgzf3f7J1AYAAPAYIxIAAFjNhxdbkkgAAGA11kgAAACP+fCIBGskAACAxxiRAADAajy0CwAAeIypDQAAgPQYkQAAwGrs2gAAAB5jagMAACA9RiQAALAaUxsAAMBjPpxIMLUBAAA8xogEAAAW4zHiAADAc2lp5lzZdO7cOfXp00dRUVEKCgpSrVq1tHbtWlNvjUQCAACrGWnmXNnUo0cPLVmyRDNmzNCWLVvUqFEjxcbG6tChQ6bdGokEAAA+6NKlS5o7d65GjhypevXqqXTp0oqPj1fp0qU1ceJE0/phjQQAAFYzaddGSkqKUlJS3MqcTqecTme6uteuXVNqaqoCAwPdyoOCgrRy5UpT4pEYkQAAwHomTW0kJiYqLCzM7UpMTMywy5CQENWsWVPDhg3T4cOHlZqaqo8++kirV6/WkSNHTLs1EgkAAO4RgwYN0pkzZ9yuQYMGZVp/xowZMgxDRYoUkdPp1Pjx49WxY0f5+Zn365+pDQAArGbS1EZm0xiZKVWqlH744QdduHBBZ8+eVUREhNq3b6+SJUuaEo/EiAQAANazadfGDcHBwYqIiNCpU6e0ePFitWzZ0rRbY0QCAAAftXjxYhmGoXLlymn37t0aMGCAypcvr65du5rWB4kEAABWs+lZGzfWUPz+++/Kmzev2rZtqxEjRihnzpym9UEiAQCA1WxKJNq1a6d27dpZ2octiUR4eLgcDkeW6p48edLiaAAAgKdsSSTGjRvn+vrEiRMaPny4GjdurJo1a0qSVq9ercWLF2vw4MF2hAcAgLnuYKGkt3MYhmHYGUDbtm1Vv359Pf/8827l77zzjpYuXar58+dnu03/gCImRQf4licja9odAuB1Ptj3meV9XPriTVPaCXqsvyntmMn27Z+LFy9WkyZN0pU3adJES5cutSEiAABMZvP2TyvZnkjky5dPCxYsSFe+YMEC5cuXz4aIAABAVtm+ayMhIUE9evTQ8uXLVaNGDUnSmjVrtGjRIk2ZMsXm6AAAMIFNuzbuBtsTiS5duig6Olrjx4/X559/LkmKjo7WypUrXYkFAAD3NC+dljCD7YmEJNWoUUMzZ860OwwAAJBNXpFIpKWlaffu3Tp27JjSbhr+qVevnk1RAQBgEqY2rPPzzz+rU6dO2r9/v27eiepwOJSammpTZAAAmIREwjrPPPOMqlevrq+//loRERFZPvESAADYz/ZEYteuXfrss89UunRpu0MBAMAa9p79aCnbz5GoUaOGdu/ebXcYAABYJy3NnMsL2T4i8cILL6hfv346evSoKleunO7RplWqVLEpMgAAcDu2JxJt27aVJHXr1s1V5nA4ZBgGiy0BAL7BS0cTzGB7IpGcnGx3CAAAWIsDqawTFRVldwgAAFiLEQnrbdu2TQcOHNCVK1fcyh977DGbIgIAALdjeyKxd+9etW7dWlu2bHGtjZDkOk+CNRIAgHse2z+t07t3b5UoUULHjh1Trly5tHXrVq1YsULVq1fX8uXL7Q4PAIA7x/ZP66xevVrLli1T/vz55efnJz8/P9WpU0eJiYnq1auXNm7caHeIAAAgE7aPSKSmpiokJESSlD9/fh0+fFjS9UWYO3futDM0AADMwYiEdSpVqqRNmzapRIkSqlGjhkaOHKmAgABNnjxZJUuWtDs8AADuHNs/rfPqq6/qwoULkqShQ4eqefPmqlu3rvLly6dPPvnE5ugAAMCt2J5ING7c2PV16dKltWPHDp08eVLh4eE8CRQA4BOMNN/dtWF7IpGRvHnz2h0CAADm8dL1DWawJZFo06ZNlut+/vnnFkYCAADuhC2JRFhYmB3dAgBgDxZbmmvq1Kl2dAsAgD1YIwEAADzmw2skbD+QCgAA3LsYkQAAwGo+PCJBIgEAgNV4+qd1PvzwQ6WkpKQrv3Llij788EMbIgIAAFlleyLRtWtXnTlzJl35uXPn1LVrVxsiwp0a+NLzWr3qa506sVOHf9+kuZ+9r7JlS9kdFuBVHn22lT7Y95k6vtbF7lBwN/jwQ7tsTyQMw8jwKOzff/+d8ybuUfXq/p8mTpyu2nVbqMmjHZXTP6cWfj1LuXIF2R0a4BWKVymlmE4NdXD7PrtDwd2SZphzZUNqaqoGDx6sEiVKKCgoSKVKldKwYcNkmDzNYtsaifvvv18Oh0MOh0MNGjSQv///QklNTVVycrKaNGliV3i4A81aPOH2uluPPjp6eIsefKCKfly5xqaoAO/gzBWop8f11vSXJ6n5C3+3Oxz4sH//+9+aOHGipk+frooVK2rdunXq2rWrwsLC1KtXL9P6sS2RaNWqlSQpKSlJjRs3Vu7cuV3vBQQEqHjx4mrbtq1N0cFMYWGhkqSTp07bGwjgBZ4Y1kObv9+gbT9tIZH4K7HhZMtVq1apZcuWatasmSSpePHi+vjjj/XLL7+Y2o9ticSQIUMkXb+x9u3bKzAw0K5QYCGHw6Exbybop59+0datO+0OB7DVwy1qK6piCQ1t+bLdoeBus+Fky1q1amny5Mn67bffVLZsWW3atEkrV67UmDFjTO3H9u2fcXFxkqT169dr+/btkqSKFSvq/vvvz9LnU1JS0u36yGzdBe6+t8e/rooVyymmfmu7QwFsFR6RTx1f66rR/ximaylX7Q4H96iMfuc5nU45nc50dV9++WWdPXtW5cuXV44cOZSamqoRI0aoc+fOpsZkeyJx7NgxdejQQcuXL1eePHkkSadPn1b9+vU1e/ZsFShQ4JafT0xMVEJCgluZwy+3HDlCrQoZWfTWuOFq9mis6jdoo0OHjtgdDmCr4pVLKqxAHg35aqSrLId/DpV9OFp/e7Kpni7bUYaXrsrHnTPr/9uMfucNGTJE8fHx6erOmTNHM2fO1KxZs1SxYkUlJSWpT58+ioyMdP0RbwaHYfbyzWxq37699u7dqw8//FDR0dGSpG3btikuLk6lS5fWxx9/fMvPZ5Sdhecrz4iEzd4aN1ytWjZRg4aPa/fuZLvDwf/3ZGRNu0P4ywoMDlS+Iu5/GHUb1VNH9hzSwknzdei3gzZFhg/2fWZ5HxdGPGlKO/79p2R5RKJo0aJ6+eWX1bNnT1fZ8OHD9dFHH2nHjh2mxCN5wYjEokWLtHTpUlcSIUkVKlTQf/7zHzVq1Oi2n8/oG0gSYa+3x7+ujh1aqU3bbjp37rwKFbr+H88zZ87p8uXLNkcH2OPyhcvpkoWUSym6cPocScRfgUmLLTNLGjJy8eJF+fm5n/KQI0cOpZk88mV7IpGWlqacOXOmK8+ZM6fpN4u749lnrg+ZLfturlt5t+4v6sMZc+wICQD+clq0aKERI0aoWLFiqlixojZu3KgxY8aoW7dupvZj+9RGy5Ytdfr0aX388ceKjIyUJB06dEidO3dWeHi45s2bl+02/QOKmB0m4BOY2gDSuytTG0PNWeAY/NrMLNc9d+6cBg8erHnz5unYsWOKjIxUx44d9dprrykgIMCUeCQvGJF455139Nhjj6l48eIqWrSoJOngwYOqVKmSPvroI5ujAwDABDaMsIeEhGjcuHEaN26cpf3YnkgULVpUGzZs0NKlS12LP6KjoxUbG2tzZAAA4HZsTySk64sjGzZsqIYNG9odCgAA5rPhQKq7xSsSie+++07fffedjh07lm6B5QcffGBTVAAAmMSGI7LvFtsTiYSEBA0dOlTVq1dXREQEWzcBALiH2J5ITJo0SdOmTdM//vEPu0MBAMAaTG1Y58qVK6pVq5bdYQAAYBlfPv7c7/ZVrNWjRw/NmjXL7jAAAIAHbB+RuHz5siZPnqylS5eqSpUq6U65NPtxpwAA3HVMbVhn8+bNqlatmiTp119/dXuPhZcAAJ9AImGd77//3u4QAACwlg9v/7R9jQQAALh32T4iAQCAz2NqAwAAeMrw4USCqQ0AAOAxRiQAALCaD49I2JJIfPHFF1mu+9hjj1kYCQAAd4EPn2xpSyLRqlUrt9cOh0OGYbi9viE1NfVuhQUAALLJljUSaWlpruvbb79VtWrVtHDhQp0+fVqnT5/WN998owceeECLFi2yIzwAAMyVZphzeSHb10j06dNHkyZNUp06dVxljRs3Vq5cufT0009r+/btNkYHAIAJvDQJMIPtuzb27NmjPHnypCsPCwvTvn377no8AAAg62xPJB566CH17dtXf/zxh6vsjz/+0IABA/Twww/bGBkAAOYwDMOUyxvZPrXxwQcfqHXr1ipWrJiKFi0qSTp48KDKlCmj+fPn2xscAABm8OGpDdsTidKlS2vz5s1asmSJduzYIUmKjo5WbGwsT/8EAPgGEglrORwONWrUSI0aNbI7FAAAkA22JBLjx4/X008/rcDAQI0fP/6WdXv16nWXogIAwBq+/KwNh2HD6o0SJUpo3bp1ypcvn0qUKJFpPYfDob1792a7ff+AIncSHuCznoysaXcIgNf5YN9nlvdxJq6BKe2ETf/OlHbMZMuIRHJycoZfAwCAe4tXrJEAAMCn+e6jNuxJJPr27ZvlumPGjLEwEgAArOfLayRsSSQ2btzo9nrDhg26du2aypUrJ0n67bfflCNHDj344IN2hAcAALLIlkTi+++/d309ZswYhYSEaPr06QoPD5cknTp1Sl27dlXdunXtCA8AAHP58IiE7Udkjx49WomJia4kQpLCw8M1fPhwjR492sbIAAAwSZpJlxeyPZE4e/asjh8/nq78+PHjOnfunA0RAQCArLJ910br1q3VtWtXjR492vWQrjVr1mjAgAFq06aNzdEBAHDnWGxpoUmTJql///7q1KmTrl69Kkny9/dX9+7dNWrUKJujAwDABF46LWEG26c2cuXKpQkTJujEiRPauHGjNm7cqJMnT2rChAkKDg62OzwAAO6YkWaYcmVH8eLF5XA40l09e/Y09d5sH5G4ITg4WFWqVLE7DAAAfMLatWuVmprqev3rr7+qYcOGevzxx03tx2sSCQAAfJYNUxsFChRwe/3GG2+oVKlSiomJMbUfEgkAACxm2LxG4sqVK/roo4/Ut29fORwOU9smkQAA4B6RkpKilJQUtzKn0ymn03nLz82fP1+nT59Wly5dTI/J9sWWAAD4PJMOpEpMTFRYWJjblZiYeNvu33//fTVt2lSRkZGm3xojEgAAWMysqY1Bgwale/Dl7UYj9u/fr6VLl+rzzz83J4ibkEgAAHCPyMo0xs2mTp2qggULqlmzZpbERCIBAIDVbFpsmZaWpqlTpyouLk7+/tb8yieRAADAYnbt2li6dKkOHDigbt26WdYHiQQAABazK5Fo1KiRDMPa53ywawMAAHiMEQkAACxm94FUViKRAADAaoa5p0l6E6Y2AACAx0wZkTh9+rTy5MljRlMAAPgcX57ayPaIxL///W998sknrtft2rVTvnz5VKRIEW3atMnU4AAA8AVGmsOUyxtlO5GYNGmSihYtKklasmSJlixZooULF6pp06YaMGCA6QECAADvle2pjaNHj7oSia+++krt2rVTo0aNVLx4cdWoUcP0AAEAuNcxtfEn4eHhOnjwoCRp0aJFio2NlSQZhqHU1FRzowMAwAcYhsOUyxtle0SiTZs26tSpk8qUKaMTJ06oadOmkqSNGzeqdOnSpgcIAAC8V7YTibFjx6p48eI6ePCgRo4cqdy5c0uSjhw5oueee870AAEAuNf58tRGthOJnDlzqn///unKX3zxRVMCAgDA13jrjgszZCmR+OKLL7Lc4GOPPeZxMAAA+CKLn5tlqywlEq1atcpSYw6HgwWXAAD8hWQpkUhL8+HJHQAALPaXn9rIzOXLlxUYGGhWLAAA+CRfTiSyfY5Eamqqhg0bpiJFiih37tzau3evJGnw4MF6//33TQ8QAAB4r2wnEiNGjNC0adM0cuRIBQQEuMorVaqk9957z9TgAADwBYZhzuWNsp1IfPjhh5o8ebI6d+6sHDlyuMqrVq2qHTt2mBocAAC+gId2/cmhQ4cyPMEyLS1NV69eNSUoAABwb8h2IlGhQgX9+OOP6co/++wz3X///aYEBQCAL+FZG3/y2muvKS4uTocOHVJaWpo+//xz7dy5Ux9++KG++uorK2IEAOCe5stHZGd7RKJly5b68ssvtXTpUgUHB+u1117T9u3b9eWXX6phw4ZWxAgAALyUR+dI1K1bV0uWLDE7FgAAfFKal05LmMHjA6nWrVun7du3S7q+buLBBx80LSgAAHyJt65vMEO2E4nff/9dHTt21E8//aQ8efJIkk6fPq1atWpp9uzZuu+++8yOEQCAe5q3bt00Q7bXSPTo0UNXr17V9u3bdfLkSZ08eVLbt29XWlqaevToYUWMAADAS2V7ROKHH37QqlWrVK5cOVdZuXLl9Pbbb6tu3bqmBgcAgC/w1lMpzZDtRKJo0aIZHjyVmpqqyMhIU4ICAMCXMLXxJ6NGjdILL7ygdevWucrWrVun3r1768033zQ1OAAA4N2yNCIRHh4uh+N/2dSFCxdUo0YN+ftf//i1a9fk7++vbt26qVWrVpYECgDAveovv/1z3LhxFocBAIDv+stv/4yLi7M6DgAAcA/y+EAqSbp8+bKuXLniVhYaGnpHAQEA4GvYtfEnFy5c0MCBAzVnzhydOHEi3fupqammBAYAgK/w5TUS2d618dJLL2nZsmWaOHGinE6n3nvvPSUkJCgyMlIffvihFTECAAAvle1E4ssvv9SECRPUtm1b+fv7q27dunr11Vf1+uuva+bMmVbECADAPc0wHKZc2XXo0CE98cQTypcvn4KCglS5cmW34xvMkO2pjZMnT6pkyZKSrq+HOHnypCSpTp06evbZZ00NDgAAX2DHGolTp06pdu3aql+/vhYuXKgCBQpo165dCg8PN7WfbCcSJUuWVHJysooVK6by5ctrzpw5evjhh/Xll1+6HuIFAAD+x441Ev/+979VtGhRTZ061VVWokQJ0/vJ9tRG165dtWnTJknSyy+/rP/85z8KDAzUiy++qAEDBpgeIAAAuC4lJUVnz551u1JSUjKs+8UXX6h69ep6/PHHVbBgQd1///2aMmWK6TE5DOPOBlz279+v9evXq3Tp0qpSpYpZcd0R/4AidocAeKVLh3+0OwTA6+TMX9LyPtYWaW1KO18/VVUJCQluZUOGDFF8fHy6uoGBgZKkvn376vHHH9fatWvVu3dvTZo0ydTzoe44kfBGJBJAxkgkgPTuRiKxJrKNKe1US/443QiE0+mU0+lMVzcgIEDVq1fXqlWrXGW9evXS2rVrtXr1alPikbK4RmL8+PFZbrBXr14eBwMAADKXWdKQkYiICFWoUMGtLDo6WnPnzjU1piwlEmPHjs1SYw6Hg0QCAICb2DH0X7t2be3cudOt7LffflNUVJSp/WQpkUhOTja1UwAA/krs2LXx4osvqlatWnr99dfVrl07/fLLL5o8ebImT55saj/Z3rUBAAC830MPPaR58+bp448/VqVKlTRs2DCNGzdOnTt3NrWfO3poFwAAuD27HiPevHlzNW/e3NI+SCQAALBYmt0BWIipDQAA4DFGJAAAsJghHiPu5scff9QTTzyhmjVr6tChQ5KkGTNmaOXKlaYGBwCAL0gzzLm8UbYTiblz56px48YKCgrSxo0bXSdsnTlzRq+//rrpAQIAcK9Lk8OUyxtlO5EYPny4Jk2apClTpihnzpyu8tq1a2vDhg2mBgcAALxbttdI7Ny5U/Xq1UtXHhYWptOnT5sREwAAPoU1En9SuHBh7d69O135ypUrVbKk9Q8+AQDgXpNm0uWNsp1IPPXUU+rdu7fWrFkjh8Ohw4cPa+bMmerfv7+effZZK2IEAABeKttTGy+//LLS0tLUoEEDXbx4UfXq1ZPT6VT//v31wgsvWBEjAAD3NF+e2nAYhuHRhpIrV65o9+7dOn/+vCpUqKDcuXObHZvH/AOK2B0C4JUuHf7R7hAAr5Mzv/XT8osKdTClnSZ/zDalHTN5fCBVQEBAuuecAwCAv5ZsJxL169eXw5H5EM2yZcvuKCAAAHyNty6UNEO2E4lq1aq5vb569aqSkpL066+/Ki4uzqy4AADwGb68RiLbicTYsWMzLI+Pj9f58+fvOCAAAHDvMO3pn0888YQ++OADs5oDAMBnpDnMubyRaU//XL16tQIDA81qDgAAn+Gtz8kwQ7YTiTZt2ri9NgxDR44c0bp16zR48GDTAgMAwFd46YM7TZHtRCIsLMzttZ+fn8qVK6ehQ4eqUaNGpgUGAAC8X7YSidTUVHXt2lWVK1dWeHi4VTEBAOBTfHn7Z7YWW+bIkUONGjXiKZ8AAGRDmsNhyuWNsr1ro1KlStq7d68VsQAAgHtMthOJ4cOHq3///vrqq6905MgRnT171u0CAADuDJMub5TlNRJDhw5Vv3799Oijj0qSHnvsMbejsg3DkMPhUGpqqvlRAgBwD/PlNRJZTiQSEhL0zDPP6Pvvv7cyHgAAcA/JciJx42njMTExlgUDAIAv8tZTKc2Qre2ft3rqJwAAyBgnW/5/ZcuWvW0ycfLkyTsKCAAA3DuylUgkJCSkO9kSAADcmrfuuDBDthKJDh06qGDBglbFAgCAT2KNhFgfAQCAp3x5+2eWD6S6sWsDAADghiyPSKSl+XI+BQCAdXz5T/FsP0YcAABkjy+vkcj2szYAAABuIJEAAMBiaSZd2REfHy+Hw+F2lS9f3ozbccPUBgAAFrNrlWHFihW1dOlS12t/f/N/7ZNIAADgo/z9/VW4cGFL+2BqAwAAixkOc66UlBSdPXvW7UpJScm03127dikyMlIlS5ZU586ddeDAAdPvjUQCAACLmbVGIjExUWFhYW5XYmJihn3WqFFD06ZN06JFizRx4kQlJyerbt26OnfunKn35jB88KQp/4AidocAeKVLh3+0OwTA6+TMX9LyPiYUfcKUdrrvfj/dCITT6ZTT6bztZ0+fPq2oqCiNGTNG3bt3NyUeiTUSAABYzqzFlllNGjKSJ08elS1bVrt37zYpmuuY2gAAwGKGSdedOH/+vPbs2aOIiIg7bMkdiQQAABZLc5hzZUf//v31ww8/aN++fVq1apVat26tHDlyqGPHjqbeG1MbAAD4oN9//10dO3bUiRMnVKBAAdWpU0c///yzChQoYGo/JBIAAFjMjgOpZs+efVf6IZEAAMBivvz8bNZIAAAAjzEiAQCAxXzuwKY/IZEAAMBi2d1xcS9hagMAAHiMEQkAACzmy4stSSQAALCYL6+RYGoDAAB4jBEJAAAslubDYxIkEgAAWIw1EgAAwGO+Ox7BGgkAAHAHGJEAAMBiTG0AAACPcbIlAABABhiRAADAYmz/BAAAHvPdNIKpDQAAcAcYkQAAwGLs2rDA5s2bs1y3SpUqFkYCAIC1WCNhgWrVqsnhcMgwDDkct94Xk5qaepeiAgAA2WHbGonk5GTt3btXycnJmjt3rkqUKKEJEyZo48aN2rhxoyZMmKBSpUpp7ty5doUIAIApDJMub2TbiERUVJTr68cff1zjx4/Xo48+6iqrUqWKihYtqsGDB6tVq1Y2RAgAgDlYI2GxLVu2qESJEunKS5QooW3bttkQEQAA5vHlNRJesf0zOjpaiYmJunLliqvsypUrSkxMVHR0tI2RAQCAW/GKEYlJkyapRYsWuu+++1w7NDZv3iyHw6Evv/zS5ugAALgzvjse4SWJxMMPP6y9e/dq5syZ2rFjhySpffv26tSpk4KDg22ODgCAO8MaibsgODhYTz/9tN1hAACAbPCKNRKSNGPGDNWpU0eRkZHav3+/JGns2LFasGCBzZEBAHBnDJP+5428IpGYOHGi+vbtq6ZNm+rUqVOuA6jCw8M1btw4e4MDAOAOpZl0eSOvSCTefvttTZkyRa+88or8/f8321K9enVt2bLFxsgAAMCteMUaieTkZN1///3pyp1Opy5cuGBDRAAAmIdzJCxWokQJJSUlpStftGgR50gAAO55HJFtsb59+6pnz566fPmyDMPQL7/8oo8//liJiYl677337A4PAABkwisSiR49eigoKEivvvqqLl68qE6dOikyMlJvvfWWOnToYHd4yKaBLz2vVq2aqny50rp06bJW/7xOg/71un77bY/doQF31bqkLZo66zNt27Fbx0+c1FuJg9WgXi3X+4Zh6D/vzdBnXy7SuXMXdH+VChrc/3lFFS1iY9SwAlMbd0Hnzp21a9cunT9/XkePHtXvv/+u7t272x0WPFCv7v9p4sTpql23hZo82lE5/XNq4dezlCtXkN2hAXfVpUuXVa50Sb3S77kM3/9g5qea+dkXem3AC5o1ZZyCAgP1z76vKiXlSob1ce/yhl0bb7zxhhwOh/r06XOHLbnzihGJP8uVK5dy5cpldxi4A81aPOH2uluPPjp6eIsefKCKfly5xqaogLuvbs2HVLfmQxm+ZxiGZsyZr6fjOuhvdWtKkl4f3F8xLTrqux9X6dHYR+5ipLCa3WdArF27Vu+++67rMRRmsi2RuP/+++VwOLJUd8OGDRZHAyuFhYVKkk6eOm1vIIAX+f3wUf33xCnVrP6/HWshuYNVpUI5bfp1B4kETHP+/Hl17txZU6ZM0fDhw01v37ZEolWrVq6vL1++rAkTJqhChQqqWfN6Zv7zzz9r69ateu65jIcEb0hJSVFKSopbmWEYWU5SYC2Hw6Exbybop59+0datO+0OB/Aa/z15SpKUL2+4W3m+vOH674lTdoQEC5l1mFRGv/OcTqecTmemn+nZs6eaNWum2NhY30okhgwZ4vq6R48e6tWrl4YNG5auzsGDB2/ZTmJiohISEtzKHH655cgRal6w8Njb419XxYrlFFO/td2hAIBtzJrayOh33pAhQxQfH59h/dmzZ2vDhg1au3atKf1nxCsWW3766ad68skn05U/8cQTmjt37i0/O2jQIJ05c8btcviFWBUqsuGtccPV7NFYxTZ6XIcOHbE7HMCr5P//IxEnTrqPPpw4eUr584Vn9BEgw995gwYNyrDuwYMH1bt3b82cOVOBgYGWxeQViy2DgoL0008/qUyZMm7lP/30021vPqMhHaY17PfWuOFq1bKJGjR8XPv23XpUCfgrui+ysPLnC9fP65NUvmwpSdL5Cxe0edtOtWvdzOboYDazpjZuN43xZ+vXr9exY8f0wAMPuMpSU1O1YsUKvfPOO0pJSVGOHDnuOCavSCT69OmjZ599Vhs2bNDDDz8sSVqzZo0++OADDR482ObokF1vj39dHTu0Upu23XTu3HkVKlRAknTmzDldvnzZ5uiAu+fixUs68Pth1+tDh//Qjt/2KCw0RBGFC+of7Vpp8vTZirqviIpEFtI7U2aoYP58alC31i1axb0ozbj7uzYaNGiQ7nlVXbt2Vfny5TVw4EBTkghJchiGDXeXgTlz5uitt97S9u3bJUnR0dHq3bu32rVrl+22/AM4zMVO164cyrC8W/cX9eGMOXc5GvzZpcM/2h3CX8ovGzar2wsD05W3bBqrEa/2cx1I9ekXi3Tu/Hk9UKWiXu3XU8WL3WdDtH9dOfOXtLyPf0S1MaWdGfs/v6PPP/LII6pWrZqpT9b2mkTCTCQSQMZIJID07kYi8YRJicRHXphIeMXUxg1XrlzRsWPHlJbmPptUrFgxmyICAODOecsR2cuXLze9Ta9IJHbt2qVu3bpp1apVbuU3zoNITU21KTIAAHArXpFIdOnSRf7+/vrqq68UERHBrgsAgE+x+4hsK3lFIpGUlKT169erfPnydocCAIDpzNr+6Y28IpGoUKGC/vvf/9odBgAAlvCWNRJW8IqTLf/973/rpZde0vLly3XixAmdPXvW7QIAAN7JK0YkYmNjJV0/POPPWGwJAPAFrJGw2Pfff293CAAAWIY1EhaLiYmxOwQAAOABr0gkbrh48aIOHDigK1euuJVXqVLFpogAALhzPniItItXJBLHjx9X165dtXDhwgzfZ40EAOBexq4Ni/Xp00enT5/WmjVrFBQUpEWLFmn69OkqU6aMvvjiC7vDAwAAmfCKEYlly5ZpwYIFql69uvz8/BQVFaWGDRsqNDRUiYmJatasmd0hAgDgMV9ebOkVIxIXLlxQwYIFJUnh4eE6fvy4JKly5crasGGDnaEBAHDHDJP+5428IpEoV66cdu7cKUmqWrWq3n33XR06dEiTJk1SRESEzdEBAIDMeMXURu/evXXkyBFJ0pAhQ9SkSRN99NFHCggI0PTp022ODgCAO+PLiy29IpF44oknXF8/+OCD2r9/v3bs2KFixYopf/78NkYGAMCdY/unBfr27ZvlumPGjLEwEgAArOXLiy1tSyQ2btyYpXoOh8PiSAAAgKdsSyR4vgYA4K/CW3dcmMEr1kgAAODLfHmxpVds/wQAAPcmRiQAALAYuzYAAIDHmNoAAADIACMSAABYjF0bAADAY2k+vEaCqQ0AAOAxRiQAALCY745HkEgAAGA5X961QSIBAIDFfDmRYI0EAADwGCMSAABYjJMtAQCAx5jaAAAAyAAjEgAAWMyXT7ZkRAIAAIsZhmHKlR0TJ05UlSpVFBoaqtDQUNWsWVMLFy40/d5IJAAA8EH33Xef3njjDa1fv17r1q3T3/72N7Vs2VJbt241tR+H4YNLSf0DitgdAuCVLh3+0e4QAK+TM39Jy/t4IKKOKe1sOLLyjj6fN29ejRo1St27dzclHok1EgAAWM7uv9lTU1P16aef6sKFC6pZs6apbZNIAABwj0hJSVFKSopbmdPplNPpzLD+li1bVLNmTV2+fFm5c+fWvHnzVKFCBVNjYo0EAAAWS5NhypWYmKiwsDC3KzExMdN+y5Urp6SkJK1Zs0bPPvus4uLitG3bNlPvjTUSwF8IaySA9O7GGokqhc2ZTli7f3m2RiRuFhsbq1KlSundd981JR6JqQ0AACyXZtLf7NlJGjKMIy0tXSJyp0gkAADwQYMGDVLTpk1VrFgxnTt3TrNmzdLy5cu1ePFiU/shkQAAwGJ2nGx57NgxPfnkkzpy5IjCwsJUpUoVLV68WA0bNjS1HxIJAAAsZtbURna8//77d6Ufdm0AAACPMSIBAIDFfPmhXSQSAABYzI6pjbuFqQ0AAOAxRiQAALAYUxsAAMBjTG0AAABkgBEJAAAsxtQGAADwmGGk2R2CZUgkAACwWJoPj0iwRgIAAHiMEQkAACxm+PCuDRIJAAAsxtQGAABABhiRAADAYkxtAAAAj3GyJQAAQAYYkQAAwGKcbAkAADzmy2skmNoAAAAeY0QCAACL+fI5EiQSAABYzJenNkgkAACwGNs/AQAAMsCIBAAAFmNqAwAAeMyXF1sytQEAADzGiAQAABZjagMAAHiMXRsAAAAZYEQCAACL8dAuAADgMaY2AAAAMsCIBAAAFmPXBgAA8Jgvr5FgagMAAIsZhmHKlR2JiYl66KGHFBISooIFC6pVq1bauXOn6fdGIgEAgA/64Ycf1LNnT/38889asmSJrl69qkaNGunChQum9uMwfHDixj+giN0hAF7p0uEf7Q4B8Do585e0vg+Tfi9dvXLI488eP35cBQsW1A8//KB69eqZEo/EGgkAACznDX+xnzlzRpKUN29eU9slkQAA4B6RkpKilJQUtzKn0ymn03nLz6WlpalPnz6qXbu2KlWqZGpMPplIXLuDoR+YJyUlRYmJiRo0aNBt/5EDfyX8bPz1mPV7KT4+XgkJCW5lQ4YMUXx8/C0/17NnT/36669auXKlKXH8mU+ukYB3OHv2rMLCwnTmzBmFhobaHQ7gNfjZgKc8GZF4/vnntWDBAq1YsUIlSpQwPSafHJEAAMAXZWUa4wbDMPTCCy9o3rx5Wr58uSVJhEQiAQCAT+rZs6dmzZqlBQsWKCQkREePHpUkhYWFKSgoyLR+mNqAZRi+BTLGzwbuBofDkWH51KlT1aVLF9P6YUQClnE6nRoyZAiLyYCb8LOBu+FujRMwIgEAADzGEdkAAMBjJBIAAMBjJBIAAMBjJBI+6JFHHlGfPn1cr4sXL65x48bd8jMOh0Pz58+3NK6s6tKli1q1amV5P9OmTVOePHks7we+7eafNyvEx8erWrVqlvYBeIpdG38Ba9euVXBwsN1hKD4+XvPnz1dSUpLdoQAATEIi8RdQoEABu0MAAPgopjbucRcuXNCTTz6p3LlzKyIiQqNHj05X5+apjV27dqlevXoKDAxUhQoVtGTJktv288gjj6hXr1566aWXlDdvXhUuXDjdQ2IOHDigli1bKnfu3AoNDVW7du30xx9/SLo+jZCQkKBNmzbJ4XDI4XBo2rRpWbrHRYsWqU6dOsqTJ4/y5cun5s2ba8+ePa739+3bJ4fDoc8//1z169dXrly5VLVqVa1evdqtnWnTpqlYsWLKlSuXWrdurRMnTmSpfyCrZsyYoerVqyskJESFCxdWp06ddOzYMdf7y5cvl8Ph0Hfffafq1asrV65cqlWrlnbu3OnWzhtvvKFChQopJCRE3bt31+XLl+/2rQBZRiJxjxswYIB++OEHLViwQN9++62WL1+uDRs2ZFo/LS1Nbdq0UUBAgNasWaNJkyZp4MCBWepr+vTpCg4O1po1azRy5EgNHTrUlYSkpaWpZcuWOnnypH744QctWbJEe/fuVfv27SVJ7du3V79+/VSxYkUdOXJER44ccb13OxcuXFDfvn21bt06fffdd/Lz81Pr1q2VlpbmVu+VV15R//79lZSUpLJly6pjx466du2aJGnNmjXq3r27nn/+eSUlJal+/foaPnx4lvoHsurq1asaNmyYNm3apPnz52vfvn0ZniD4yiuvaPTo0Vq3bp38/f3VrVs313tz5sxRfHy8Xn/9da1bt04RERGaMGHCXbwLIJsM3LPOnTtnBAQEGHPmzHGVnThxwggKCjJ69+7tKouKijLGjh1rGIZhLF682PD39zcOHTrken/hwoWGJGPevHmZ9hUTE2PUqVPHreyhhx4yBg4caBiGYXz77bdGjhw5jAMHDrje37p1qyHJ+OWXXwzDMIwhQ4YYVatWve19xcXFGS1btsz0/ePHjxuSjC1bthiGYRjJycmGJOO9995L1/f27dsNwzCMjh07Go8++qhbO+3btzfCwsJuGw9wKzExMW4/b3+2du1aQ5Jx7tw5wzAM4/vvvzckGUuXLnXV+frrrw1JxqVLlwzDMIyaNWsazz33nFs7NWrUyNLPDmAHRiTuYXv27NGVK1dUo0YNV1nevHlVrly5TD+zfft2FS1aVJGRka6ymjVrZqm/KlWquL2OiIhwDdveaLdo0aKu9ytUqKA8efJo+/btWWo/M7t27VLHjh1VsmRJhYaGqnjx4pKuT6VkFl9ERIQkucX35++TlPX7BrJq/fr1atGihYoVK6aQkBDFxMRI4t8qfBuJBLIsZ86cbq8dDke66QUrtGjRQidPntSUKVO0Zs0arVmzRpJ05cqVTOO78bCauxEfIF2fgmvcuLFCQ0M1c+ZMrV27VvPmzZPEv1X4NhKJe1ipUqWUM2dO1y9WSTp16pR+++23TD8THR2tgwcP6siRI66yn3/++Y5judHuwYMHXWXbtm3T6dOnVaFCBUlSQECAUlNTs9XuiRMntHPnTr366qtq0KCBoqOjderUKY/i+/P3STLnvoEbduzYoRMnTuiNN95Q3bp1Vb58ebeFllnFv1Xca9j+eQ/LnTu3unfvrgEDBihfvnwqWLCgXnnlFfn5ZZ4fxsbGqmzZsoqLi9OoUaN09uxZvfLKK3ccS2xsrCpXrqzOnTtr3Lhxunbtmp577jnFxMSoevXqkq7vHklOTlZSUpLuu+8+hYSE3Pbph+Hh4cqXL58mT56siIgIHThwQC+//HK24+vVq5dq166tN998Uy1bttTixYu1aNEij+4VyEixYsUUEBCgt99+W88884x+/fVXDRs2LNvt9O7dW126dFH16tVVu3ZtzZw5U1u3blXJkiUtiBq4c4xI3ONGjRqlunXrqkWLFoqNjVWdOnX04IMPZlrfz89P8+bN06VLl/Twww+rR48eGjFixB3H4XA4tGDBAoWHh6tevXqKjY1VyZIl9cknn7jqtG3bVk2aNFH9+vVVoEABffzxx7dt18/PT7Nnz9b69etVqVIlvfjiixo1alS24/u///s/TZkyRW+99ZaqVq2qb7/9Vq+++mq22wEyU6BAAU2bNk2ffvqpKlSooDfeeENvvvlmtttp3769Bg8erJdeekkPPvig9u/fr2effdaCiAFz8BhxAADgMUYkAACAx0gkAACAx0gkAACAx0gkAACAx0gkAACAx0gkAACAx0gkAACAx0gkABt16dJFrVq1cr1+5JFH1KdPn7sex/Lly+VwOHT69OlM6zgcDs2fPz/LbcbHx6tatWp3FNe+ffvkcDiUlJR0R+0AsA6JBHCTLl26yOFwyOFwKCAgQKVLl9bQoUN17do1y/v+/PPPs3ysclZ++QOA1XjWBpCBJk2aaOrUqUpJSdE333yjnj17KmfOnBo0aFC6uleuXFFAQIAp/ebNm9eUdgDgbmFEAsiA0+lU4cKFFRUVpWeffVaxsbH64osvJP1vOmLEiBGKjIxUuXLlJEkHDx5Uu3btlCdPHuXNm1ctW7bUvn37XG2mpqaqb9++ypMnj/Lly6eXXnpJN59Qf/PURkpKigYOHKiiRYvK6XSqdOnSev/997Vv3z7Vr19f0vUHmzkcDnXp0kXS9cdRJyYmqkSJEgoKClLVqlX12WefufXzzTffqGzZsgoKClL9+vXd4syqgQMHqmzZssqVK5dKliypwYMH6+rVq+nqvfvuuypatKhy5cqldu3a6cyZM27vv/fee4qOjlZgYKDKly+vCRMmZNrnqVOn1LlzZxUoUEBBQUEqU6aMpk6dmu3YAZiHEQkgC4KCgnTixAnX6++++06hoaFasmSJJOnq1atq3LixatasqR9//FH+/v4aPny4mjRpos2bNysgIECjR4/WtGnT9MEHHyg6OlqjR4/WvHnz9Le//S3Tfp988kmtXr1a48ePV9WqVZWcnKz//ve/Klq0qObOnau2bdtq586dCg0NVVBQkCQpMTFRH330kSZNmqQyZcpoxYoVeuKJJ1SgQAHFxMTo4MGDatOmjXr27Kmnn35a69atU79+/bL9PQkJCdG0adMUGRmpLVu26KmnnlJISIheeuklV53du3drzpw5+vLLL3X27Fl1795dzz33nGbOnClJmjlzpl577TW98847uv/++7Vx40Y99dRTCg4OVlxcXLo+Bw8erG3btmnhwoXKnz+/du/erUuXLmU7dgAmMgC4iYuLM1q2bGkYhmGkpaUZS5YsMZxOp9G/f3/X+4UKFTJSUlJcn5kxY4ZRrlw5Iy0tzVWWkpJiBAUFGYsXLzYMwzAiIiKMkSNHut6/evWqcd9997n6MgzDiImJMXr37m0YhmHs3LnTkGQsWbIkwzi///57Q5Jx6tQpV9nly5eNXLlyGatWrXKr2717d6Njx46GYRjGoEGDjAoVKri9P3DgwHRt3UySMW/evEzfHzVqlPHggw+6Xg8ZMsTIkSOH8fvvv7vKFi5caPj5+RlHjhwxDMMwSpUqZcyaNcutnWHDhhk1a9Y0DMMwkpOTDUnGxo0bDcMwjBYtWhhdu3bNNAYAdx8jEkAGvvrqK+XOnVtXr15VWlqaOnXqpPj4eNf7lStXdlsXsWnTJu3evVshISFu7Vy+fFl79uzRmTNndOTIEdWoUcP1nr+/v6pXr55ueuOGpKQk5ciRQzExMVmOe/fu3bp48aIaNmzoVn7lyhXdf//9kqTt27e7xSFJNWvWzHIfN3zyyScaP3689uzZo/Pnz+vatWsKDQ11q1OsWDEVKVLErZ+0tDTt3LlTISEh2rNnj7p3766nnnrKVefatWsKCwvLsM9nn31Wbdu21YYNG9SoUSO1atVKtWrVynbsAMxDIgFkoH79+po4caICAgIUGRkpf3/3H5Xg4GC31+fPn9eDDz7oGrL/swIFCngUw42piuw4f/68JOnrr792+wUuXV/3YZbVq1erc+fOSkhIUOPGjRUWFqbZs2dr9OjR2Y51ypQp6RKbHDlyZPiZpk2bav/+/frmm2+0ZMkSNWjQQD179tSbb77p+c0AuCMkEkAGgoODVbp06SzXf+CBB/TJJ5+oYMGC6f4qvyEiIkJr1qxRvXr1JF3/y3v9+vV64IEHMqxfuXJlpaWl6YcfflBsbGy692+MiKSmprrKKlSoIKfTqQMHDmQ6khEdHe1aOHrDzz//fPub/JNVq1YpKipKr7zyiqts//796eodOHBAhw8fVmRkpKsfPz8/lStXToUKFVJkZKT27t2rzp07Z7nvAgUKKC4uTnFxcapbt64GDBhAIgHYiF0bgAk6d+6s/Pnzq2XLlvrxxx+VnJys5cuXq1evXvr9998lSb1799Ybb7yh+fPna8eOHXruuedueQZE8eLFFRcXp27dumn+/PmuNufMmSNJioqKksPh0FdffaXjx4/r/PnzCgkJUf/+/fXiiy9q+vTp2rNnjzZs2KC3335b06dPlyQ988wz2rVrlwYMGKCdO3dq1qxZmjZtWrbut0yZMjpw4IBmz56tPXv2aPz48Zo3b166eoGBgYqLi9OmTZv0448/qlevXmrXrp0KFy4sSUpISFBiYqLGjx+v3377TVu2bNHUqVM1ZsyYDPt97bXXtGDBAu3evVtbt27VV199pejo6GzFDsBcJBKACXLlyqUVK1aoWLFiatOmjaKjo9W9e3ddvnzZNULRr18//eMf/1BcXJxq1qypkJAQtW7d+pbtTpw4UX//+9/13HPPqXz58nrqqad04cIFSVKRIkWUkJCgl19+WYUKFdLzzz8vSRo2bJgGDx6sxMRERUdHq0mTJvr6669VokQJSdfXLcydO1fz589X1apVNWnSJL3++uvZut/HHntML774op5//nlVq1ZNq1at0uDBg9PVK126tNq0aaNHH31UjRo1UpUqVdy2d/bo0UPvvfeepk6dqsqVKysmJkbTpk1zxXqzgIAADRo0SFWqVFG9evWUI0cOzZ49O1uxAzCXw8hspRcAAMBtMCIBAAA8RiIBAAA8RiIBAAA8RiIBAAA8RiIBAAA8RiIBAAA8RiIBAAA8RiIBAAA8RiIBAAA8RiIBAAA8RiIBAAA8RiIBAAA89v8AgrAGKG5FB3YAAAAASUVORK5CYII=","text/plain":["<Figure size 640x480 with 2 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":["yhat = tree_cv.predict(X_test)\n","plot_confusion_matrix(Y_test,yhat)"]},{"cell_type":"markdown","id":"fd954881-0671-4a83-ae57-0b4977ebf3bb","metadata":{},"source":["## TASK  10\n"]},{"cell_type":"markdown","id":"93f46ed0-52d1-4ef3-934b-8ac8d7ec0289","metadata":{},"source":["Create a k nearest neighbors object then  create a  <code>GridSearchCV</code> object  <code>knn_cv</code> with cv = 10.  Fit the object to find the best parameters from the dictionary <code>parameters</code>.\n"]},{"cell_type":"code","execution_count":52,"id":"9e04256a-1a09-4b5b-9c2b-6473eb3eaad5","metadata":{},"outputs":[],"source":["parameters = {'n_neighbors': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n","              'algorithm': ['auto', 'ball_tree', 'kd_tree', 'brute'],\n","              'p': [1,2]}\n","\n","KNN = KNeighborsClassifier()"]},{"cell_type":"code","execution_count":54,"id":"88bfb36b-bcda-4382-a501-86a27fc1a19c","metadata":{},"outputs":[{"data":{"text/html":["<style>#sk-container-id-7 {\n","  /* Definition of color scheme common for light and dark mode */\n","  --sklearn-color-text: black;\n","  --sklearn-color-line: gray;\n","  /* Definition of color scheme for unfitted estimators */\n","  --sklearn-color-unfitted-level-0: #fff5e6;\n","  --sklearn-color-unfitted-level-1: #f6e4d2;\n","  --sklearn-color-unfitted-level-2: #ffe0b3;\n","  --sklearn-color-unfitted-level-3: chocolate;\n","  /* Definition of color scheme for fitted estimators */\n","  --sklearn-color-fitted-level-0: #f0f8ff;\n","  --sklearn-color-fitted-level-1: #d4ebff;\n","  --sklearn-color-fitted-level-2: #b3dbfd;\n","  --sklearn-color-fitted-level-3: cornflowerblue;\n","\n","  /* Specific color for light theme */\n","  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n","  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n","  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n","  --sklearn-color-icon: #696969;\n","\n","  @media (prefers-color-scheme: dark) {\n","    /* Redefinition of color scheme for dark theme */\n","    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n","    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n","    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n","    --sklearn-color-icon: #878787;\n","  }\n","}\n","\n","#sk-container-id-7 {\n","  color: var(--sklearn-color-text);\n","}\n","\n","#sk-container-id-7 pre {\n","  padding: 0;\n","}\n","\n","#sk-container-id-7 input.sk-hidden--visually {\n","  border: 0;\n","  clip: rect(1px 1px 1px 1px);\n","  clip: rect(1px, 1px, 1px, 1px);\n","  height: 1px;\n","  margin: -1px;\n","  overflow: hidden;\n","  padding: 0;\n","  position: absolute;\n","  width: 1px;\n","}\n","\n","#sk-container-id-7 div.sk-dashed-wrapped {\n","  border: 1px dashed var(--sklearn-color-line);\n","  margin: 0 0.4em 0.5em 0.4em;\n","  box-sizing: border-box;\n","  padding-bottom: 0.4em;\n","  background-color: var(--sklearn-color-background);\n","}\n","\n","#sk-container-id-7 div.sk-container {\n","  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n","     but bootstrap.min.css set `[hidden] { display: none !important; }`\n","     so we also need the `!important` here to be able to override the\n","     default hidden behavior on the sphinx rendered scikit-learn.org.\n","     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n","  display: inline-block !important;\n","  position: relative;\n","}\n","\n","#sk-container-id-7 div.sk-text-repr-fallback {\n","  display: none;\n","}\n","\n","div.sk-parallel-item,\n","div.sk-serial,\n","div.sk-item {\n","  /* draw centered vertical line to link estimators */\n","  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n","  background-size: 2px 100%;\n","  background-repeat: no-repeat;\n","  background-position: center center;\n","}\n","\n","/* Parallel-specific style estimator block */\n","\n","#sk-container-id-7 div.sk-parallel-item::after {\n","  content: \"\";\n","  width: 100%;\n","  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n","  flex-grow: 1;\n","}\n","\n","#sk-container-id-7 div.sk-parallel {\n","  display: flex;\n","  align-items: stretch;\n","  justify-content: center;\n","  background-color: var(--sklearn-color-background);\n","  position: relative;\n","}\n","\n","#sk-container-id-7 div.sk-parallel-item {\n","  display: flex;\n","  flex-direction: column;\n","}\n","\n","#sk-container-id-7 div.sk-parallel-item:first-child::after {\n","  align-self: flex-end;\n","  width: 50%;\n","}\n","\n","#sk-container-id-7 div.sk-parallel-item:last-child::after {\n","  align-self: flex-start;\n","  width: 50%;\n","}\n","\n","#sk-container-id-7 div.sk-parallel-item:only-child::after {\n","  width: 0;\n","}\n","\n","/* Serial-specific style estimator block */\n","\n","#sk-container-id-7 div.sk-serial {\n","  display: flex;\n","  flex-direction: column;\n","  align-items: center;\n","  background-color: var(--sklearn-color-background);\n","  padding-right: 1em;\n","  padding-left: 1em;\n","}\n","\n","\n","/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n","clickable and can be expanded/collapsed.\n","- Pipeline and ColumnTransformer use this feature and define the default style\n","- Estimators will overwrite some part of the style using the `sk-estimator` class\n","*/\n","\n","/* Pipeline and ColumnTransformer style (default) */\n","\n","#sk-container-id-7 div.sk-toggleable {\n","  /* Default theme specific background. It is overwritten whether we have a\n","  specific estimator or a Pipeline/ColumnTransformer */\n","  background-color: var(--sklearn-color-background);\n","}\n","\n","/* Toggleable label */\n","#sk-container-id-7 label.sk-toggleable__label {\n","  cursor: pointer;\n","  display: block;\n","  width: 100%;\n","  margin-bottom: 0;\n","  padding: 0.5em;\n","  box-sizing: border-box;\n","  text-align: center;\n","}\n","\n","#sk-container-id-7 label.sk-toggleable__label-arrow:before {\n","  /* Arrow on the left of the label */\n","  content: \"▸\";\n","  float: left;\n","  margin-right: 0.25em;\n","  color: var(--sklearn-color-icon);\n","}\n","\n","#sk-container-id-7 label.sk-toggleable__label-arrow:hover:before {\n","  color: var(--sklearn-color-text);\n","}\n","\n","/* Toggleable content - dropdown */\n","\n","#sk-container-id-7 div.sk-toggleable__content {\n","  max-height: 0;\n","  max-width: 0;\n","  overflow: hidden;\n","  text-align: left;\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-7 div.sk-toggleable__content.fitted {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","#sk-container-id-7 div.sk-toggleable__content pre {\n","  margin: 0.2em;\n","  border-radius: 0.25em;\n","  color: var(--sklearn-color-text);\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-7 div.sk-toggleable__content.fitted pre {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","#sk-container-id-7 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n","  /* Expand drop-down */\n","  max-height: 200px;\n","  max-width: 100%;\n","  overflow: auto;\n","}\n","\n","#sk-container-id-7 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n","  content: \"▾\";\n","}\n","\n","/* Pipeline/ColumnTransformer-specific style */\n","\n","#sk-container-id-7 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-7 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Estimator-specific style */\n","\n","/* Colorize estimator box */\n","#sk-container-id-7 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-7 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","#sk-container-id-7 div.sk-label label.sk-toggleable__label,\n","#sk-container-id-7 div.sk-label label {\n","  /* The background is the default theme color */\n","  color: var(--sklearn-color-text-on-default-background);\n","}\n","\n","/* On hover, darken the color of the background */\n","#sk-container-id-7 div.sk-label:hover label.sk-toggleable__label {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","/* Label box, darken color on hover, fitted */\n","#sk-container-id-7 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n","  color: var(--sklearn-color-text);\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Estimator label */\n","\n","#sk-container-id-7 div.sk-label label {\n","  font-family: monospace;\n","  font-weight: bold;\n","  display: inline-block;\n","  line-height: 1.2em;\n","}\n","\n","#sk-container-id-7 div.sk-label-container {\n","  text-align: center;\n","}\n","\n","/* Estimator-specific */\n","#sk-container-id-7 div.sk-estimator {\n","  font-family: monospace;\n","  border: 1px dotted var(--sklearn-color-border-box);\n","  border-radius: 0.25em;\n","  box-sizing: border-box;\n","  margin-bottom: 0.5em;\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-0);\n","}\n","\n","#sk-container-id-7 div.sk-estimator.fitted {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-0);\n","}\n","\n","/* on hover */\n","#sk-container-id-7 div.sk-estimator:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-2);\n","}\n","\n","#sk-container-id-7 div.sk-estimator.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-2);\n","}\n","\n","/* Specification for estimator info (e.g. \"i\" and \"?\") */\n","\n","/* Common style for \"i\" and \"?\" */\n","\n",".sk-estimator-doc-link,\n","a:link.sk-estimator-doc-link,\n","a:visited.sk-estimator-doc-link {\n","  float: right;\n","  font-size: smaller;\n","  line-height: 1em;\n","  font-family: monospace;\n","  background-color: var(--sklearn-color-background);\n","  border-radius: 1em;\n","  height: 1em;\n","  width: 1em;\n","  text-decoration: none !important;\n","  margin-left: 1ex;\n","  /* unfitted */\n","  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-unfitted-level-1);\n","}\n","\n",".sk-estimator-doc-link.fitted,\n","a:link.sk-estimator-doc-link.fitted,\n","a:visited.sk-estimator-doc-link.fitted {\n","  /* fitted */\n","  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-fitted-level-1);\n","}\n","\n","/* On hover */\n","div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",".sk-estimator-doc-link:hover,\n","div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",".sk-estimator-doc-link:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",".sk-estimator-doc-link.fitted:hover,\n","div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",".sk-estimator-doc-link.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","/* Span, style for the box shown on hovering the info icon */\n",".sk-estimator-doc-link span {\n","  display: none;\n","  z-index: 9999;\n","  position: relative;\n","  font-weight: normal;\n","  right: .2ex;\n","  padding: .5ex;\n","  margin: .5ex;\n","  width: min-content;\n","  min-width: 20ex;\n","  max-width: 50ex;\n","  color: var(--sklearn-color-text);\n","  box-shadow: 2pt 2pt 4pt #999;\n","  /* unfitted */\n","  background: var(--sklearn-color-unfitted-level-0);\n","  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n","}\n","\n",".sk-estimator-doc-link.fitted span {\n","  /* fitted */\n","  background: var(--sklearn-color-fitted-level-0);\n","  border: var(--sklearn-color-fitted-level-3);\n","}\n","\n",".sk-estimator-doc-link:hover span {\n","  display: block;\n","}\n","\n","/* \"?\"-specific style due to the `<a>` HTML tag */\n","\n","#sk-container-id-7 a.estimator_doc_link {\n","  float: right;\n","  font-size: 1rem;\n","  line-height: 1em;\n","  font-family: monospace;\n","  background-color: var(--sklearn-color-background);\n","  border-radius: 1rem;\n","  height: 1rem;\n","  width: 1rem;\n","  text-decoration: none;\n","  /* unfitted */\n","  color: var(--sklearn-color-unfitted-level-1);\n","  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n","}\n","\n","#sk-container-id-7 a.estimator_doc_link.fitted {\n","  /* fitted */\n","  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n","  color: var(--sklearn-color-fitted-level-1);\n","}\n","\n","/* On hover */\n","#sk-container-id-7 a.estimator_doc_link:hover {\n","  /* unfitted */\n","  background-color: var(--sklearn-color-unfitted-level-3);\n","  color: var(--sklearn-color-background);\n","  text-decoration: none;\n","}\n","\n","#sk-container-id-7 a.estimator_doc_link.fitted:hover {\n","  /* fitted */\n","  background-color: var(--sklearn-color-fitted-level-3);\n","}\n","</style><div id=\"sk-container-id-7\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=10, estimator=KNeighborsClassifier(),\n","             param_grid={&#x27;algorithm&#x27;: [&#x27;auto&#x27;, &#x27;ball_tree&#x27;, &#x27;kd_tree&#x27;, &#x27;brute&#x27;],\n","                         &#x27;n_neighbors&#x27;: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n","                         &#x27;p&#x27;: [1, 2]})</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-19\" type=\"checkbox\" ><label for=\"sk-estimator-id-19\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;GridSearchCV<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.model_selection.GridSearchCV.html\">?<span>Documentation for GridSearchCV</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>GridSearchCV(cv=10, estimator=KNeighborsClassifier(),\n","             param_grid={&#x27;algorithm&#x27;: [&#x27;auto&#x27;, &#x27;ball_tree&#x27;, &#x27;kd_tree&#x27;, &#x27;brute&#x27;],\n","                         &#x27;n_neighbors&#x27;: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n","                         &#x27;p&#x27;: [1, 2]})</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-20\" type=\"checkbox\" ><label for=\"sk-estimator-id-20\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">best_estimator_: KNeighborsClassifier</label><div class=\"sk-toggleable__content fitted\"><pre>KNeighborsClassifier(n_neighbors=10, p=1)</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-21\" type=\"checkbox\" ><label for=\"sk-estimator-id-21\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;KNeighborsClassifier<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.neighbors.KNeighborsClassifier.html\">?<span>Documentation for KNeighborsClassifier</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>KNeighborsClassifier(n_neighbors=10, p=1)</pre></div> </div></div></div></div></div></div></div></div></div>"],"text/plain":["GridSearchCV(cv=10, estimator=KNeighborsClassifier(),\n","             param_grid={'algorithm': ['auto', 'ball_tree', 'kd_tree', 'brute'],\n","                         'n_neighbors': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n","                         'p': [1, 2]})"]},"execution_count":54,"metadata":{},"output_type":"execute_result"}],"source":["KNN_cv = GridSearchCV(KNN, parameters, cv = 10)\n","KNN_cv.fit(X_train, Y_train)"]},{"cell_type":"code","execution_count":55,"id":"91be642d-515b-4c73-8248-1567ee62c68e","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["tuned hpyerparameters :(best parameters)  {'algorithm': 'auto', 'n_neighbors': 10, 'p': 1}\n","accuracy : 0.8482142857142858\n"]}],"source":["print(\"tuned hpyerparameters :(best parameters) \",KNN_cv.best_params_)\n","print(\"accuracy :\",KNN_cv.best_score_)"]},{"cell_type":"markdown","id":"de4713a0-3659-4f98-ac8c-fac54c98d09d","metadata":{},"source":["## TASK  11\n"]},{"cell_type":"markdown","id":"21914e8a-3d97-455f-9ea3-e8b0fd9c970a","metadata":{},"source":["Calculate the accuracy of knn_cv on the test data using the method <code>score</code>:\n"]},{"cell_type":"code","execution_count":57,"id":"82398f2f-c8c7-4e47-8b24-7114263f57e6","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["MSE =  0.16666666666666666\n","Variance score: 0.86\n"]}],"source":["y_hat = KNN_cv.predict(X_test)\n","\n","print(\"MSE = \", np.mean((y_hat-Y_test)**2))\n","print('Variance score: %.2f' % KNN_cv.score(X, Y))"]},{"cell_type":"markdown","id":"0e8f127e-3d12-45a5-8936-c40540d16d82","metadata":{},"source":["We can plot the confusion matrix\n"]},{"cell_type":"code","execution_count":58,"id":"12e77ba8-7721-4979-84ea-41a18111645b","metadata":{},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAhIAAAHHCAYAAADqJrG+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAABD30lEQVR4nO3dd3RU1fr/8c+EkEkISQg9USABFIJUQbh0uESKghSV6jU0vQpIL3IVSACNghRBEUEFRBS5IiCoKFX0ggiEANJLKNKlSguQOb8//DFfxySaDGc4k+H9cs1amX327P2cLAcedjs2wzAMAQAAuMHP6gAAAEDORSIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBAADcRiIBeNDevXvVuHFjhYWFyWazaeHChaa2f/DgQdlsNs2cOdPUdnOyBg0aqEGDBlaHAdw1SCTg8/bv369///vfKlmypAIDAxUaGqratWvrzTff1NWrVz3ad1xcnLZt26ZXXnlFs2fPVrVq1Tza353UuXNn2Ww2hYaGZvh73Lt3r2w2m2w2m954441st3/s2DHFx8crOTnZhGgBeIq/1QEAnvTll1/qySeflN1u19NPP63y5cvr+vXr+uGHHzRo0CBt375d06ZN80jfV69e1bp16/TSSy+pV69eHumjRIkSunr1qnLnzu2R9v+Ov7+/rly5osWLF6tt27Yu1+bMmaPAwEBdu3bNrbaPHTumhIQERUVFqXLlyln+3LfffutWfwDcQyIBn5WSkqL27durRIkSWrlypSIiIpzXevbsqX379unLL7/0WP+nT5+WJOXLl89jfdhsNgUGBnqs/b9jt9tVu3ZtffLJJ+kSiY8//liPPvqo5s+ff0diuXLlivLkyaOAgIA70h+A3zG1AZ81ZswYXbp0Se+//75LEnFL6dKl1adPH+f7mzdvatSoUSpVqpTsdruioqL0n//8R6mpqS6fi4qKUvPmzfXDDz+oevXqCgwMVMmSJfXhhx8668THx6tEiRKSpEGDBslmsykqKkrS71MCt37+o/j4eNlsNpeyZcuWqU6dOsqXL5/y5s2rMmXK6D//+Y/zemZrJFauXKm6desqODhY+fLlU8uWLbVz584M+9u3b586d+6sfPnyKSwsTF26dNGVK1cy/8X+SceOHfX111/r/PnzzrINGzZo79696tixY7r6Z8+e1cCBA1WhQgXlzZtXoaGhatasmbZs2eKss3r1aj300EOSpC5dujinSG7dZ4MGDVS+fHlt2rRJ9erVU548eZy/lz+vkYiLi1NgYGC6+2/SpInCw8N17NixLN8rgPRIJOCzFi9erJIlS6pWrVpZqt+9e3cNHz5cDz74oCZMmKD69esrMTFR7du3T1d33759euKJJ/Twww9r3LhxCg8PV+fOnbV9+3ZJUps2bTRhwgRJUocOHTR79mxNnDgxW/Fv375dzZs3V2pqqkaOHKlx48bpscce0//+97+//Nzy5cvVpEkTnTp1SvHx8erfv7/Wrl2r2rVr6+DBg+nqt23bVr/99psSExPVtm1bzZw5UwkJCVmOs02bNrLZbPr888+dZR9//LHKli2rBx98MF39AwcOaOHChWrevLnGjx+vQYMGadu2bapfv77zL/WYmBiNHDlSkvTss89q9uzZmj17turVq+ds58yZM2rWrJkqV66siRMnqmHDhhnG9+abb6pQoUKKi4tTWlqaJOndd9/Vt99+q8mTJysyMjLL9wogAwbggy5cuGBIMlq2bJml+snJyYYko3v37i7lAwcONCQZK1eudJaVKFHCkGSsWbPGWXbq1CnDbrcbAwYMcJalpKQYkoyxY8e6tBkXF2eUKFEiXQwjRoww/viVnDBhgiHJOH36dKZx3+pjxowZzrLKlSsbhQsXNs6cOeMs27Jli+Hn52c8/fTT6frr2rWrS5utW7c2ChQokGmff7yP4OBgwzAM44knnjAaNWpkGIZhpKWlGUWLFjUSEhIy/B1cu3bNSEtLS3cfdrvdGDlypLNsw4YN6e7tlvr16xuSjKlTp2Z4rX79+i5l33zzjSHJGD16tHHgwAEjb968RqtWrf72HgH8PUYk4JMuXrwoSQoJCclS/a+++kqS1L9/f5fyAQMGSFK6tRTlypVT3bp1ne8LFSqkMmXK6MCBA27H/Ge31lYsWrRIDocjS585fvy4kpOT1blzZ+XPn99ZXrFiRT388MPO+/yj5557zuV93bp1debMGefvMCs6duyo1atX68SJE1q5cqVOnDiR4bSG9Pu6Cj+/3//oSUtL05kzZ5zTNklJSVnu0263q0uXLlmq27hxY/373//WyJEj1aZNGwUGBurdd9/Ncl8AMkciAZ8UGhoqSfrtt9+yVP/QoUPy8/NT6dKlXcqLFi2qfPny6dChQy7lxYsXT9dGeHi4zp0752bE6bVr1061a9dW9+7dVaRIEbVv317z5s37y6TiVpxlypRJdy0mJka//vqrLl++7FL+53sJDw+XpGzdyyOPPKKQkBB9+umnmjNnjh566KF0v8tbHA6HJkyYoPvuu092u10FCxZUoUKFtHXrVl24cCHLfd5zzz3ZWlj5xhtvKH/+/EpOTtakSZNUuHDhLH8WQOZIJOCTQkNDFRkZqZ9//jlbn/vzYsfM5MqVK8NywzDc7uPW/P0tQUFBWrNmjZYvX65//etf2rp1q9q1a6eHH344Xd3bcTv3covdblebNm00a9YsLViwINPRCEl69dVX1b9/f9WrV08fffSRvvnmGy1btkwPPPBAlkdepN9/P9mxefNmnTp1SpK0bdu2bH0WQOZIJOCzmjdvrv3792vdunV/W7dEiRJyOBzau3evS/nJkyd1/vx55w4MM4SHh7vscLjlz6MekuTn56dGjRpp/Pjx2rFjh1555RWtXLlSq1atyrDtW3Hu3r073bVdu3apYMGCCg4Ovr0byETHjh21efNm/fbbbxkuUL3ls88+U8OGDfX++++rffv2aty4sWJjY9P9TrKa1GXF5cuX1aVLF5UrV07PPvusxowZow0bNpjWPnA3I5GAzxo8eLCCg4PVvXt3nTx5Mt31/fv3680335T0+9C8pHQ7K8aPHy9JevTRR02Lq1SpUrpw4YK2bt3qLDt+/LgWLFjgUu/s2bPpPnvrYKY/b0m9JSIiQpUrV9asWbNc/mL++eef9e233zrv0xMaNmyoUaNG6a233lLRokUzrZcrV650ox3//e9/dfToUZeyWwlPRklXdg0ZMkSHDx/WrFmzNH78eEVFRSkuLi7T3yOArONAKvisUqVK6eOPP1a7du0UExPjcrLl2rVr9d///ledO3eWJFWqVElxcXGaNm2azp8/r/r16+unn37SrFmz1KpVq0y3Frqjffv2GjJkiFq3bq3evXvrypUreuedd3T//fe7LDYcOXKk1qxZo0cffVQlSpTQqVOnNGXKFN17772qU6dOpu2PHTtWzZo1U82aNdWtWzddvXpVkydPVlhYmOLj4027jz/z8/PTyy+//Lf1mjdvrpEjR6pLly6qVauWtm3bpjlz5qhkyZIu9UqVKqV8+fJp6tSpCgkJUXBwsGrUqKHo6OhsxbVy5UpNmTJFI0aMcG5HnTFjhho0aKBhw4ZpzJgx2WoPwJ9YvGsE8Lg9e/YYzzzzjBEVFWUEBAQYISEhRu3atY3Jkycb165dc9a7ceOGkZCQYERHRxu5c+c2ihUrZgwdOtSljmH8vv3z0UcfTdfPn7cdZrb90zAM49tvvzXKly9vBAQEGGXKlDE++uijdNs/V6xYYbRs2dKIjIw0AgICjMjISKNDhw7Gnj170vXx5y2Sy5cvN2rXrm0EBQUZoaGhRosWLYwdO3a41LnV35+3l86YMcOQZKSkpGT6OzUM1+2fmcls++eAAQOMiIgIIygoyKhdu7axbt26DLdtLlq0yChXrpzh7+/vcp/169c3HnjggQz7/GM7Fy9eNEqUKGE8+OCDxo0bN1zq9evXz/Dz8zPWrVv3l/cA4K/ZDCMbK6oAAAD+gDUSAADAbSQSAADAbSQSAADAbSQSAAD4qDVr1qhFixaKjIyUzWbTwoULnddu3LihIUOGqEKFCgoODlZkZKSefvrpbD8Rl0QCAAAfdfnyZVWqVElvv/12umtXrlxRUlKShg0bpqSkJH3++efavXu3HnvssWz1wa4NAADuAjabTQsWLFCrVq0yrbNhwwZVr15dhw4dyvCZQhnhQCoAAHKI1NTUdCey2u122e12U9q/cOGCbDab8+nDWeGTiUTXqCesDgEAkEN8cPAzj/dx49cDprST+NaHSkhIcCkbMWKEKafWXrt2TUOGDFGHDh2cT1DOCp9MJAAA8EVDhw5V//79XcrMGI24ceOG2rZtK8Mw9M4772TrsyQSAAB4miPNlGbMnMa45VYScejQIa1cuTJboxESiQQAAJ5nOKyOIEO3koi9e/dq1apVKlCgQLbbIJEAAMDTHNYkEpcuXdK+ffuc71NSUpScnKz8+fMrIiJCTzzxhJKSkrRkyRKlpaXpxIkTkqT8+fMrICAgS3345PZPFlsCALLqjiy2PL7TlHZyR8Rkq/7q1avVsGHDdOVxcXGKj49XdHR0hp9btWqVGjRokKU+GJEAAMDDDIumNho0aKC/Gi8wYyyBRAIAAE+zaGrjTuCIbAAA4DZGJAAA8DQv3bVhBhIJAAA8zaRzJLwRUxsAAMBtjEgAAOBpTG0AAAC3sWsDAAAgPUYkAADwMKsOpLoTSCQAAPA0H57aIJEAAMDTfHhEgjUSAADAbYxIAADgaT58IBWJBAAAnsbUBgAAQHqMSAAA4Gns2gAAAG5jagMAACA9RiQAAPA0pjYAAIC7DMN3t38ytQEAANzGiAQAAJ7mw4stSSQAAPA01kgAAAC3+fCIBGskAACA2xiRAADA03hoFwAAcBtTGwAAAOkxIgEAgKexawMAALiNqQ0AAID0GJEAAMDTmNoAAABu8+FEgqkNAADgNkYkAADwMF9+jDiJBAAAnubDUxskEgAAeBrbPwEAANJjRAIAAE9jagMAALiNqQ0AAID0GJEAAMDTmNoAAABuY2oDAAAgPUYkAADwNKY2AACA20gkzBUeHi6bzZalumfPnvVwNAAAwF2WJBITJ050/nzmzBmNHj1aTZo0Uc2aNSVJ69at0zfffKNhw4ZZER4AAOby4cWWNsMwDCsDePzxx9WwYUP16tXLpfytt97S8uXLtXDhwmy32TXqCZOiAwD4ug8OfubxPq5+8YYp7QQ9NtCUdsxk+a6Nb775Rk2bNk1X3rRpUy1fvtyCiAAAMJnhMOflhSxPJAoUKKBFixalK1+0aJEKFChgQUQAAPiGNWvWqEWLFoqMjJTNZks3ym8YhoYPH66IiAgFBQUpNjZWe/fuzVYflu/aSEhIUPfu3bV69WrVqFFDkrR+/XotXbpU06dPtzg6AABMYNGujcuXL6tSpUrq2rWr2rRpk+76mDFjNGnSJM2aNUvR0dEaNmyYmjRpoh07digwMDBLfVieSHTu3FkxMTGaNGmSPv/8c0lSTEyMfvjhB2diAQBAjmbRtESzZs3UrFmzDK8ZhqGJEyfq5ZdfVsuWLSVJH374oYoUKaKFCxeqffv2WerD8kRCkmrUqKE5c+ZYHQYAAHeNlJQUnThxQrGxsc6ysLAw1ahRQ+vWrctZiYTD4dC+fft06tQpOf40/FOvXj2LogIAwCQmTW2kpqYqNTXVpcxut8tut2e7rRMnTkiSihQp4lJepEgR57WssDyR+PHHH9WxY0cdOnRIf96JarPZlJaWZlFkAACYxKREIjExUQkJCS5lI0aMUHx8vCntu8PyROK5555TtWrV9OWXXyoiIiLLJ14CAHC3GTp0qPr37+9S5s5ohCQVLVpUknTy5ElFREQ4y0+ePKnKlStnuR3LE4m9e/fqs88+U+nSpa0OBQAAzzDp7Ed3pzEyEh0draJFi2rFihXOxOHixYtav369nn/++Sy3Y3kiUaNGDe3bt49EAgDguyza/nnp0iXt27fP+T4lJUXJycnKnz+/ihcvrr59+2r06NG67777nNs/IyMj1apVqyz3YXki8cILL2jAgAE6ceKEKlSooNy5c7tcr1ixokWRAQCQs23cuFENGzZ0vr81LRIXF6eZM2dq8ODBunz5sp599lmdP39ederU0dKlS7N8hoTkBc/a8PNLf7imzWaTYRhuL7bkWRsAgKy6I8/amGPOQyiDOo0ypR0zWT4ikZKSYnUIAAB4lpc+J8MMlicSJUqUsDoEAAA8y6I1EneC5YnELTt27NDhw4d1/fp1l/LHHnvMoogAAMDfsTyROHDggFq3bq1t27Y510ZIcp4nwYFUAIAcz9rliB5l+WPE+/Tpo+joaJ06dUp58uTR9u3btWbNGlWrVk2rV6+2OjwAAG6fw2HOywtZPiKxbt06rVy5UgULFpSfn5/8/PxUp04dJSYmqnfv3tq8ebPVIQIAgExYPiKRlpamkJAQSVLBggV17NgxSb8vwty9e7eVoQEAYA5GJDynfPny2rJli6Kjo1WjRg2NGTNGAQEBmjZtmkqWLGl1eAAA3D62f3rOyy+/rMuXL0uSRo4cqebNm6tu3boqUKCAPv30U4ujAwAAf8XyRKJJkybOn0uXLq1du3bp7NmzCg8P50mgAACfYDh8d9eG5YlERvLnz291CAAAmMdL1zeYwZJEok2bNlmu+/nnn3swEgAAcDssSSTCwsKs6BYAAGuw2NJcM2bMsKJbAACswRoJAADgNh9eI2H5gVQAACDnYkQCAABP8+ERCRIJAAA8jad/es6HH36o1NTUdOXXr1/Xhx9+aEFEAAAgq2yGYW2alCtXLh0/flyFCxd2KT9z5owKFy6stLS0bLfZNeoJs8KDGxo81VgNOzVRwXsLSZKO7j2ixZM+07bVPMkVdze+G97pg4OfebyPK+OfMaWdPP2nm9KOmSyf2jAMI8OjsH/55RfOm8ihzh0/o89e/0gnDx6XzWZT7ccb6IVpgxX/6CAd2/uL1eEBluG7cRdj+6f5qlSpIpvNJpvNpkaNGsnf//9CSUtLU0pKipo2bWpVeLgNW1Zscnn/+RufqMFTjVWqyv38YYm7Gt8N+CLLEolWrVpJkpKTk9WkSRPlzZvXeS0gIEBRUVF6/PHHLYoOZrH5+emhR2vKHhSo/Ul7rA4H8Bp8N+4ynGxpvhEjRkiSoqKi1K5dOwUGBloVCjzgnjLF9dLnryi3PUCpV67prX+P0bF9/IsL4Ltxl/LhqQ3LF1vesmnTJu3cuVOS9MADD6hKlSpZ+lxqamq6XR8vVIhTLlsu02NE1uXK7a8CkQUVFJJH1R75h+q1a6TX243gD0zc9fhueJ87stjy9S6mtJNniPc9YsLyxZanTp1S+/bttXr1auXLl0+SdP78eTVs2FBz585VoUKF/vLziYmJSkhIcCmrHBajKvnKeSpkZEHajZs6deiEJOnQzwcUXbG0Yrs+og//M83iyABr8d24Oxk+fCCV5edIvPDCC/rtt9+0fft2nT17VmfPntXPP/+sixcvqnfv3n/7+aFDh+rChQsur4phZe5A5MgOm59N/gG5rQ4D8Dp8N+4SDsOclxeyfERi6dKlWr58uWJiYpxl5cqV09tvv63GjRv/7eftdrvsdrtLGdMa1np8cEdtW71ZZ479qsDgIP2jZR2V+ccDGv/0aKtDAyzFd+MuxmJLz3E4HMqdO302njt3bjl8eCjIl4UWCFP38S8orFC4rv52Rb/sOqTxT4/Wjh+2Wh0aYCm+G/BFlicS//znP9WnTx998sknioyMlCQdPXpU/fr1U6NGjSyODu6YMeQdq0MAvBLfjbuYl05LmMHyNRJvvfWWLl68qKioKJUqVUqlSpVSdHS0Ll68qMmTJ1sdHgAAt8/hMOflhSwfkShWrJiSkpK0fPly7dq1S5IUExOj2NhYiyMDAAB/x/JEQpJsNpsefvhhPfzww1aHAgCA+Xx4asMrEokVK1ZoxYoVOnXqVLoFlh988IFFUQEAYBJ2bXhOQkKCRo4cqWrVqikiIiLDJ4ECAADvZHkiMXXqVM2cOVP/+te/rA4FAADPYGrDc65fv65atWpZHQYAAB7DEdke1L17d3388cdWhwEAANxg+YjEtWvXNG3aNC1fvlwVK1ZMd8rl+PHjLYoMAACTMLXhOVu3blXlypUlST///LPLNRZeAgB8AomE56xatcrqEAAA8Cwf3v5p+RoJAACQc1k+IgEAgM9jagMAALjL8OFEgqkNAADgNkYkAADwNB8ekbAkkfjiiy+yXPexxx7zYCQAANwBPnyypSWJRKtWrVze22w2GYbh8v6WtLS0OxUWAADIJkvWSDgcDufr22+/VeXKlfX111/r/PnzOn/+vL766is9+OCDWrp0qRXhAQBgLodhzssLWb5Gom/fvpo6darq1KnjLGvSpIny5MmjZ599Vjt37rQwOgAATOClSYAZLN+1sX//fuXLly9deVhYmA4ePHjH4wEAAFlneSLx0EMPqX///jp58qSz7OTJkxo0aJCqV69uYWQAAJjDMAxTXtmRlpamYcOGKTo6WkFBQSpVqpRGjRqV7Xb+juVTGx988IFat26t4sWLq1ixYpKkI0eO6L777tPChQutDQ4AADNYMLXx+uuv65133tGsWbP0wAMPaOPGjerSpYvCwsLUu3dv0/qxPJEoXbq0tm7dqmXLlmnXrl2SpJiYGMXGxvL0TwCAb7AgkVi7dq1atmypRx99VJIUFRWlTz75RD/99JOp/VieSEi/b/ds3LixGjdubHUoAAB4rdTUVKWmprqU2e122e32dHVr1aqladOmac+ePbr//vu1ZcsW/fDDDxo/frypMVmSSEyaNEnPPvusAgMDNWnSpL+sa+bwCwAAVjDrWRuJiYlKSEhwKRsxYoTi4+PT1X3xxRd18eJFlS1bVrly5VJaWppeeeUVderUyZRYbrEZZq+6yILo6Ght3LhRBQoUUHR0dKb1bDabDhw4kO32u0Y9cTvhAQDuIh8c/MzjfVyIa2RKO4HTvsryiMTcuXM1aNAgjR07Vg888ICSk5PVt29fjR8/XnFxcabEI1k0IpGSkpLhzwAAIHOZJQ0ZGTRokF588UW1b99eklShQgUdOnRIiYmJOT+RAADgrmLBozauXLkiPz/XUx5y5colh8nP/bAkkejfv3+W65q9KAQAgDvNrDUS2dGiRQu98sorKl68uB544AFt3rxZ48ePV9euXU3tx5JEYvPmzS7vk5KSdPPmTZUpU0aStGfPHuXKlUtVq1a1IjwAAHK8yZMna9iwYerRo4dOnTqlyMhI/fvf/9bw4cNN7ceSRGLVqlXOn8ePH6+QkBDNmjVL4eHhkqRz586pS5cuqlu3rhXhAQBgLgtGJEJCQjRx4kRNnDjRo/1YfkT2uHHjlJiY6EwiJCk8PFyjR4/WuHHjLIwMAACTOEx6eSHLE4mLFy/q9OnT6cpPnz6t3377zYKIAABAVlm+a6N169bq0qWLxo0b53xI1/r16zVo0CC1adPG4ugAALh9Viy2vFMsTySmTp2qgQMHqmPHjrpx44Ykyd/fX926ddPYsWMtjg4AABN46bSEGSxPJPLkyaMpU6Zo7Nix2r9/vySpVKlSCg4OtjgyAADMwYjEHRAcHKyKFStaHQYAAMgGr0kkAADwWUxtAAAAdxk+nEhYvv0TAADkXIxIAADgaT48IkEiAQCAhzG1AQAAkAFGJAAA8DQfHpEgkQAAwMN8eWqDRAIAAA/z5USCNRIAAMBtjEgAAOBhvjwiQSIBAICnGTarI/AYpjYAAIDbTBmROH/+vPLly2dGUwAA+BxfntrI9ojE66+/rk8//dT5vm3btipQoIDuuecebdmyxdTgAADwBYbDZsrLG2U7kZg6daqKFSsmSVq2bJmWLVumr7/+Ws2aNdOgQYNMDxAAAHivbE9tnDhxwplILFmyRG3btlXjxo0VFRWlGjVqmB4gAAA5HVMbfxAeHq4jR45IkpYuXarY2FhJkmEYSktLMzc6AAB8gGHYTHl5o2yPSLRp00YdO3bUfffdpzNnzqhZs2aSpM2bN6t06dKmBwgAALxXthOJCRMmKCoqSkeOHNGYMWOUN29eSdLx48fVo0cP0wMEACCn8+WpjWwnErlz59bAgQPTlffr18+UgAAA8DXeuuPCDFlKJL744ossN/jYY4+5HQwAAL7IMKyOwHOylEi0atUqS43ZbDYWXAIAcBfJUiLhcPjw5A4AAB52109tZObatWsKDAw0KxYAAHySLycS2T5HIi0tTaNGjdI999yjvHnz6sCBA5KkYcOG6f333zc9QAAA4L2ynUi88sormjlzpsaMGaOAgABnefny5fXee++ZGhwAAL7AMMx5eaNsJxIffvihpk2bpk6dOilXrlzO8kqVKmnXrl2mBgcAgC/goV1/cPTo0QxPsHQ4HLpx44YpQQEAgJwh24lEuXLl9P3336cr/+yzz1SlShVTggIAwJfwrI0/GD58uOLi4nT06FE5HA59/vnn2r17tz788EMtWbLEEzECAJCj+fIR2dkekWjZsqUWL16s5cuXKzg4WMOHD9fOnTu1ePFiPfzww56IEQAAeCm3zpGoW7euli1bZnYsAAD4JIeXTkuYwe0DqTZu3KidO3dK+n3dRNWqVU0LCgAAX+Kt6xvMkO1E4pdfflGHDh30v//9T/ny5ZMknT9/XrVq1dLcuXN17733mh0jAAA5mrdu3TRDttdIdO/eXTdu3NDOnTt19uxZnT17Vjt37pTD4VD37t09ESMAAPBS2R6R+O6777R27VqVKVPGWVamTBlNnjxZdevWNTU4AAB8gbeeSmmGbCcSxYoVy/DgqbS0NEVGRpoSFAAAvoSpjT8YO3asXnjhBW3cuNFZtnHjRvXp00dvvPGGqcEBAADvlqURifDwcNls/5dNXb58WTVq1JC//+8fv3nzpvz9/dW1a1e1atXKI4ECAJBT3fXbPydOnOjhMAAA8F13/fbPuLg4T8cBAAByILcPpJKka9eu6fr16y5loaGhtxUQAAC+hl0bf3D58mUNGTJE8+bN05kzZ9JdT0tLMyUwAAB8hS+vkcj2ro3Bgwdr5cqVeuedd2S32/Xee+8pISFBkZGR+vDDDz0RIwAA8FLZTiQWL16sKVOm6PHHH5e/v7/q1q2rl19+Wa+++qrmzJnjiRgBAMjRDMNmyiu7jh49qqeeekoFChRQUFCQKlSo4HJ8gxmyPbVx9uxZlSxZUtLv6yHOnj0rSapTp46ef/55U4MDAMAXWLFG4ty5c6pdu7YaNmyor7/+WoUKFdLevXsVHh5uaj/ZTiRKliyplJQUFS9eXGXLltW8efNUvXp1LV682PkQLwAA8H+sWCPx+uuvq1ixYpoxY4azLDo62vR+sj210aVLF23ZskWS9OKLL+rtt99WYGCg+vXrp0GDBpkeIAAA+F1qaqouXrzo8kpNTc2w7hdffKFq1arpySefVOHChVWlShVNnz7d9JhshnF7Ay6HDh3Spk2bVLp0aVWsWNGsuG6Lf8A9VocAeKWrx763OgTA6+QuWNLjfWy4p7Up7Xz5TCUlJCS4lI0YMULx8fHp6gYGBkqS+vfvryeffFIbNmxQnz59NHXqVFPPh7rtRMIbkUgAGSORANK7E4nE+sg2prRTOeWTdCMQdrtddrs9Xd2AgABVq1ZNa9eudZb17t1bGzZs0Lp160yJR8riGolJkyZlucHevXu7HQwAAMhcZklDRiIiIlSuXDmXspiYGM2fP9/UmLKUSEyYMCFLjdlsNhIJAAD+xIqh/9q1a2v37t0uZXv27FGJEiVM7SdLiURKSoqpnQIAcDexYtdGv379VKtWLb366qtq27atfvrpJ02bNk3Tpk0ztZ9s79oAAADe76GHHtKCBQv0ySefqHz58ho1apQmTpyoTp06mdrPbT20CwAA/D2rHiPevHlzNW/e3KN9kEgAAOBhDqsD8CCmNgAAgNsYkQAAwMMM8RhxF99//72eeuop1axZU0ePHpUkzZ49Wz/88IOpwQEA4Aschjkvb5TtRGL+/Plq0qSJgoKCtHnzZucJWxcuXNCrr75qeoAAAOR0DtlMeXmjbCcSo0eP1tSpUzV9+nTlzp3bWV67dm0lJSWZGhwAAPBu2V4jsXv3btWrVy9deVhYmM6fP29GTAAA+BTWSPxB0aJFtW/fvnTlP/zwg0qW9PyDTwAAyGkcJr28UbYTiWeeeUZ9+vTR+vXrZbPZdOzYMc2ZM0cDBw7U888/74kYAQCAl8r21MaLL74oh8OhRo0a6cqVK6pXr57sdrsGDhyoF154wRMxAgCQo/ny1IbNMAy3NpRcv35d+/bt06VLl1SuXDnlzZvX7Njc5h9wj9UhAF7p6rHvrQ4B8Dq5C3p+Wn5pkfamtNP05FxT2jGT2wdSBQQEpHvOOQAAuLtkO5Fo2LChbLbMh2hWrlx5WwEBAOBrvHWhpBmynUhUrlzZ5f2NGzeUnJysn3/+WXFxcWbFBQCAz/DlNRLZTiQmTJiQYXl8fLwuXbp02wEBAICcw7Snfz711FP64IMPzGoOAACf4bCZ8/JGpj39c926dQoMDDSrOQAAfIa3PifDDNlOJNq0aePy3jAMHT9+XBs3btSwYcNMCwwAAF/hpQ/uNEW2E4mwsDCX935+fipTpoxGjhypxo0bmxYYAADwftlKJNLS0tSlSxdVqFBB4eHhnooJAACf4svbP7O12DJXrlxq3LgxT/kEACAbHDabKS9vlO1dG+XLl9eBAwc8EQsAAMhhsp1IjB49WgMHDtSSJUt0/PhxXbx40eUFAABcGSa9vFGW10iMHDlSAwYM0COPPCJJeuyxx1yOyjYMQzabTWlpaeZHCQBADubLaySynEgkJCToueee06pVqzwZDwAAyEGynEjcetp4/fr1PRYMAAC+yFtPpTRDtrZ//tVTPwEAQMY42fL/u//++/82mTh79uxtBQQAAHKObCUSCQkJ6U62BAAAf81bd1yYIVuJRPv27VW4cGFPxQIAgE9ijYRYHwEAgLt8eftnlg+kurVrAwAA4JYsj0g4HL6cTwEA4Dm+/E/xbD9GHAAAZI8vr5HI9rM2AAAAbmFEAgAAD/PlxQEkEgAAeJgvJxJMbQAAALcxIgEAgIcZPrzYkkQCAAAPY2oDAAAgA4xIAADgYb48IkEiAQCAh3GyJQAAcBsnWwIAAGSAEQkAADyMNRIAAMBtvpxIMLUBAADcxogEAAAexq4NAADgNnZtAACAHO21116TzWZT3759TW2XEQkAADzM6sWWGzZs0LvvvquKFSua3jYjEgAAeJhh0ssdly5dUqdOnTR9+nSFh4ffzm1kiEQCAAAf1rNnTz366KOKjY31SPtMbQAA4GEOk/ZtpKamKjU11aXMbrfLbrdnWH/u3LlKSkrShg0bTOk/I4xIAADgYQ6TXomJiQoLC3N5JSYmZtjnkSNH1KdPH82ZM0eBgYEeuzebYRg+t73VP+Aeq0MAvNLVY99bHQLgdXIXLOnxPkaW6GRKO0P2fJDlEYmFCxeqdevWypUrl7MsLS1NNptNfn5+Sk1NdbnmLqY2AADIIf5qGuPPGjVqpG3btrmUdenSRWXLltWQIUNMSSIkEgkAADzOiu2fISEhKl++vEtZcHCwChQokK78dpBIAADgYb58siWJBAAAd4nVq1eb3iaJBAAAHmbW9k9vRCIBAICH+W4awTkSAADgNjAiAQCAh1n90C5PsiyR2Lp1a5breuJpZQAA3CmskfCAypUry2azyTAM2Wx/vS8mLS3tDkUFAACyw7I1EikpKTpw4IBSUlI0f/58RUdHa8qUKdq8ebM2b96sKVOmqFSpUpo/f75VIQIAYAorHyPuaZaNSJQoUcL585NPPqlJkybpkUcecZZVrFhRxYoV07Bhw9SqVSsLIgQAwByskfCwbdu2KTo6Ol15dHS0duzYYUFEAACYx5fXSHjF9s+YmBglJibq+vXrzrLr168rMTFRMTExFkYGAAD+ileMSEydOlUtWrTQvffe69yhsXXrVtlsNi1evNji6AAAuD2+Ox7hJYlE9erVdeDAAc2ZM0e7du2SJLVr104dO3ZUcHCwxdEBAHB7WCNxBwQHB+vZZ5+1OgwAAJANXrFGQpJmz56tOnXqKDIyUocOHZIkTZgwQYsWLbI4MgAAbo9h0n/eyCsSiXfeeUf9+/dXs2bNdO7cOecBVOHh4Zo4caK1wQEAcJscJr28kVckEpMnT9b06dP10ksvyd///2ZbqlWrpm3btlkYGQAA+CtesUYiJSVFVapUSVdut9t1+fJlCyICAMA8nCPhYdHR0UpOTk5XvnTpUs6RAADkeByR7WH9+/dXz549de3aNRmGoZ9++kmffPKJEhMT9d5771kdHgAAyIRXJBLdu3dXUFCQXn75ZV25ckUdO3ZUZGSk3nzzTbVv397q8OCm55+L04D+z6to0ULaunWH+vQdpg0bk60OC7hjNiZv04yPP9OOXft0+sxZvZk4TI3q1ZIk3bh5U5OnzdL36zbql2PHlTc4WP94qIr6PddFhQsVsDhymI2pjTugU6dO2rt3ry5duqQTJ07ol19+Ubdu3awOC2568snH9MbYERo1erweqtFUW7bu0FdfzlEh/oDEXeTq1WsqU7qkXhrQI921a9dStWP3fv27cwfN++AtTXz1ZR08/It6DUmwIFJ4mi/v2rAZhuFzaZJ/wD1Wh3DXW/vDYm3YuEV9+r4sSbLZbDp4YIPenjJDY8a+bXF0d6+rx763OoS7VvnazVxGJDKybedudejeV8vmz1JE0cJ3MLq7W+6CJT3eR/eoJ0xp572Dn5nSjpksm9qoUqWKbDZbluomJSV5OBqYKXfu3HrwwYp6bcxbzjLDMLRi5Q/6xz+qWhgZ4N0uXboim82mkBAeDYCcw7JEolWrVs6fr127pilTpqhcuXKqWbOmJOnHH3/U9u3b1aNH+iHBP0pNTVVqaqpLmWEYWU5SYL6CBfPL399fp07+6lJ+6tRplS1TyqKoAO+WmnpdE975QI/E1ldenjHkc7x1WsIMliUSI0aMcP7cvXt39e7dW6NGjUpX58iRI3/ZTmJiohISXOcUbX55ZcsVal6wAOBBN27e1IBhr8owDA0b1MvqcOAB3nq8tRm8YrHlf//7Xz399NPpyp966inNnz//Lz87dOhQXbhwweVl8wvxVKjIgl9/PaubN2+qcJGCLuWFCxfSiZOnLYoK8E63kohjJ09p+sRXGY1AjuMViURQUJD+97//pSv/3//+p8DAwL/8rN1uV2hoqMuLaQ1r3bhxQ0lJW/XPhnWcZTabTf9sWEc//rjJwsgA73IriTh85Jjem/iq8oUxkuqrfHnXhlecI9G3b189//zzSkpKUvXq1SVJ69ev1wcffKBhw4ZZHB3cMeHN6Zrx/gRtStqqDRs2q/cLzyg4OEgzZ31qdWjAHXPlylUd/uWY8/3RYye1a89+hYWGqGDB/Or/0ivasWef3h6TIIfDoV/PnJUkhYWGKHfu3FaFDQ9w+N4GSSev2f45b948vfnmm9q5c6ckKSYmRn369FHbtm2z3RbbP71Dj+c7Ow+k2rJlu/r2G66fNmy2Oqy7Gts/76yfkraq6wtD0pW3bBarHt2eUpMnOmf4uQ8mv67qD1b0cHS45U5s//xXiTamtDP70OemtGMmr0kkzEQiAWSMRAJI704kEk+ZlEh85IWJhFdMbdxy/fp1nTp1Sg6H60xQ8eLFLYoIAIDb58tHZHtFIrF371517dpVa9eudSm/dR5EWlqaRZEBAIC/4hWJROfOneXv768lS5YoIiKCXRcAAJ/iy+dIeEUikZycrE2bNqls2bJWhwIAgOm8deumGbwikShXrpx+/fXXv68IAEAO5MtrJLziQKrXX39dgwcP1urVq3XmzBldvHjR5QUAALyTV4xIxMbGSpIaNWrkUs5iSwCAL2CNhIetWrXK6hAAAPAY1kh4WP369a0OAQAAuMErEolbrly5osOHD+v69esu5RUrclQsACDn8sFDpJ28IpE4ffq0unTpoq+//jrD66yRAADkZOza8LC+ffvq/PnzWr9+vYKCgrR06VLNmjVL9913n7744gurwwMAAJnwihGJlStXatGiRapWrZr8/PxUokQJPfzwwwoNDVViYqIeffRRq0MEAMBtvrzY0itGJC5fvqzChQtLksLDw3X69GlJUoUKFZSUlGRlaAAA3DbDpP+8kVckEmXKlNHu3bslSZUqVdK7776ro0ePaurUqYqIiLA4OgAAkBmvmNro06ePjh8/LkkaMWKEmjZtqo8++kgBAQGaNWuWxdEBAHB7fHmxpVckEk899ZTz56pVq+rQoUPatWuXihcvroIFC1oYGQAAt4/tnx7Qv3//LNcdP368ByMBAMCzfHmxpWWJxObNm7NUz2azeTgSAADgLssSCZ6vAQC4W3jrjgszeMUaCQAAfJkvL7b0iu2fAADAXImJiXrooYcUEhKiwoULq1WrVs6jFsxEIgEAgIcZhmHKKzu+++479ezZUz/++KOWLVumGzduqHHjxrp8+bKp98bUBgAAHmbF1MbSpUtd3s+cOVOFCxfWpk2bVK9ePdP6IZEAACCHSE1NVWpqqkuZ3W6X3W7/289euHBBkpQ/f35TY2JqAwAADzPrWRuJiYkKCwtzeSUmJv5t/w6HQ3379lXt2rVVvnx5U++NEQkAADzMYdLJlkOHDk13oGNWRiN69uypn3/+WT/88IMpcfwRiQQAADlEVqcx/qhXr15asmSJ1qxZo3vvvdf0mEgkAADwMCtOkTAMQy+88IIWLFig1atXKzo62iP9kEgAAOBhVuza6Nmzpz7++GMtWrRIISEhOnHihCQpLCxMQUFBpvVjM3zwkWT+AfdYHQLgla4e+97qEACvk7tgSY/3UfOehqa0s+5o1h8vkdmzqmbMmKHOnTubEo/EiAQAAD7pTo0TkEgAAOBhPjj470QiAQCAh/HQLgAAgAwwIgEAgIcZPjwiQSIBAICH+fIaCaY2AACA2xiRAADAw3x5sSWJBAAAHsbUBgAAQAYYkQAAwMOY2gAAAG5j+ycAAHCbgzUSAAAA6TEiAQCAhzG1AQAA3MbUBgAAQAYYkQAAwMOY2gAAAG5jagMAACADjEgAAOBhTG0AAAC3MbUBAACQAUYkAADwMKY2AACA2wzDYXUIHkMiAQCAh/nyY8RZIwEAANzGiAQAAB5m+PCuDRIJAAA8jKkNAACADDAiAQCAhzG1AQAA3MbJlgAAABlgRAIAAA/jZEsAAOA2X14jwdQGAABwGyMSAAB4mC+fI0EiAQCAh/ny1AaJBAAAHsb2TwAAgAwwIgEAgIcxtQEAANzmy4stmdoAAABuY0QCAAAPY2oDAAC4jV0bAAAAGWBEAgAAD+OhXQAAwG1MbQAAAGSAEQkAADyMXRsAAMBtvrxGgqkNAAA8zDAMU17uePvttxUVFaXAwEDVqFFDP/30k6n3RiIBAICP+vTTT9W/f3+NGDFCSUlJqlSpkpo0aaJTp06Z1ofN8MGJG/+Ae6wOAfBKV499b3UIgNfJXbCk5/sw6e+lG9ePZqt+jRo19NBDD+mtt96SJDkcDhUrVkwvvPCCXnzxRVNiYkQCAAAPM0x6Zcf169e1adMmxcbGOsv8/PwUGxurdevW3db9/BGLLQEAyCFSU1OVmprqUma322W329PV/fXXX5WWlqYiRYq4lBcpUkS7du0yLSafTCRuZnPoB56RmpqqxMREDR06NMP/yYG7Fd+Nu49Zfy/Fx8crISHBpWzEiBGKj483pX13+OQaCXiHixcvKiwsTBcuXFBoaKjV4QBeg+8G3JWdEYnr168rT548+uyzz9SqVStneVxcnM6fP69FixaZEhNrJAAAyCHsdrtCQ0NdXpmNagUEBKhq1apasWKFs8zhcGjFihWqWbOmaTH55NQGAACQ+vfvr7i4OFWrVk3Vq1fXxIkTdfnyZXXp0sW0PkgkAADwUe3atdPp06c1fPhwnThxQpUrV9bSpUvTLcC8HSQS8Bi73a4RI0awmAz4E74buJN69eqlXr16eax9FlsCAAC3sdgSAAC4jUQCAAC4jUQCAAC4jUTCBzVo0EB9+/Z1vo+KitLEiRP/8jM2m00LFy70aFxZ1blzZ5fDUzxl5syZypcvn8f7gW/78/fNE+Lj41W5cmWP9gG4i10bd4ENGzYoODjY6jAUHx+vhQsXKjk52epQAAAmIZG4CxQqVMjqEAAAPoqpjRzu8uXLevrpp5U3b15FRERo3Lhx6er8eWpj7969qlevngIDA1WuXDktW7bsb/tp0KCBevfurcGDByt//vwqWrRouofEHD58WC1btlTevHkVGhqqtm3b6uTJk5J+n0ZISEjQli1bZLPZZLPZNHPmzCzd49KlS1WnTh3ly5dPBQoUUPPmzbV//37n9YMHD8pms+nzzz9Xw4YNlSdPHlWqVCndY3Jnzpyp4sWLK0+ePGrdurXOnDmTpf6BrJo9e7aqVaumkJAQFS1aVB07dtSpU6ec11evXi2bzaYVK1aoWrVqypMnj2rVqqXdu3e7tPPaa6+pSJEiCgkJUbdu3XTt2rU7fStAlpFI5HCDBg3Sd999p0WLFunbb7/V6tWrlZSUlGl9h8OhNm3aKCAgQOvXr9fUqVM1ZMiQLPU1a9YsBQcHa/369RozZoxGjhzpTEIcDodatmyps2fP6rvvvtOyZct04MABtWvXTtLvp6sNGDBADzzwgI4fP67jx487r/2dy5cvq3///tq4caNWrFghPz8/tW7dWg6Hw6XeSy+9pIEDByo5OVn333+/OnTooJs3b0qS1q9fr27duqlXr15KTk5Ww4YNNXr06Cz1D2TVjRs3NGrUKG3ZskULFy7UwYMH1blz53T1XnrpJY0bN04bN26Uv7+/unbt6rw2b948xcfH69VXX9XGjRsVERGhKVOm3MG7ALLJQI7122+/GQEBAca8efOcZWfOnDGCgoKMPn36OMtKlChhTJgwwTAMw/jmm28Mf39/4+jRo87rX3/9tSHJWLBgQaZ91a9f36hTp45L2UMPPWQMGTLEMAzD+Pbbb41cuXIZhw8fdl7fvn27Icn46aefDMMwjBEjRhiVKlX62/uKi4szWrZsmen106dPG5KMbdu2GYZhGCkpKYYk47333kvX986dOw3DMIwOHToYjzzyiEs77dq1M8LCwv42HuCv1K9f3+X79kcbNmwwJBm//fabYRiGsWrVKkOSsXz5cmedL7/80pBkXL161TAMw6hZs6bRo0cPl3Zq1KiRpe8OYAVGJHKw/fv36/r166pRo4azLH/+/CpTpkymn9m5c6eKFSumyMhIZ1lWnwJXsWJFl/cRERHOYdtb7RYrVsx5vVy5csqXL5927tyZpfYzs3fvXnXo0EElS5ZUaGiooqKiJP0+lZJZfBEREZLkEt8ff09S1u8byKpNmzapRYsWKl68uEJCQlS/fn1J/L8K30YigSzLnTu3y3ubzZZuesETWrRoobNnz2r69Olav3691q9fL0m6fv16pvHZbDZJuiPxAdLvU3BNmjRRaGio5syZow0bNmjBggWS+H8Vvo1EIgcrVaqUcufO7fyLVZLOnTunPXv2ZPqZmJgYHTlyRMePH3eW/fjjj7cdy612jxw54izbsWOHzp8/r3LlykmSAgIClJaWlq12z5w5o927d+vll19Wo0aNFBMTo3PnzrkV3x9/T5I59w3csmvXLp05c0avvfaa6tatq7Jly7ostMwq/l9FTsP2zxwsb9686tatmwYNGqQCBQqocOHCeumll+Tnl3l+GBsbq/vvv19xcXEaO3asLl68qJdeeum2Y4mNjVWFChXUqVMnTZw4UTdv3lSPHj1Uv359VatWTdLvu0dSUlKUnJyse++9VyEhIX/79MPw8HAVKFBA06ZNU0REhA4fPqwXX3wx2/H17t1btWvX1htvvKGWLVvqm2++0dKlS926VyAjxYsXV0BAgCZPnqznnntOP//8s0aNGpXtdvr06aPOnTurWrVqql27tubMmaPt27erZMmSHogauH2MSORwY8eOVd26ddWiRQvFxsaqTp06qlq1aqb1/fz8tGDBAl29elXVq1dX9+7d9corr9x2HDabTYsWLVJ4eLjq1aun2NhYlSxZUp9++qmzzuOPP66mTZuqYcOGKlSokD755JO/bdfPz09z587Vpk2bVL58efXr109jx47Ndnz/+Mc/NH36dL355puqVKmSvv32W7388svZbgfITKFChTRz5kz997//Vbly5fTaa6/pjTfeyHY77dq107BhwzR48GBVrVpVhw4d0vPPP++BiAFz8BhxAADgNkYkAACA20gkAACA20gkAACA20gkAACA20gkAACA20gkAACA20gkAACA20gkAAt17txZrVq1cr5v0KCB+vbte8fjWL16tWw2m86fP59pHZvNpoULF2a5zfj4eFWuXPm24jp48KBsNpuSk5Nvqx0AnkMiAfxJ586dZbPZZLPZFBAQoNKlS2vkyJG6efOmx/v+/PPPs3ysclb+8gcAT+NZG0AGmjZtqhkzZig1NVVfffWVevbsqdy5c2vo0KHp6l6/fl0BAQGm9Js/f35T2gGAO4URCSADdrtdRYsWVYkSJfT8888rNjZWX3zxhaT/m4545ZVXFBkZqTJlykiSjhw5orZt2ypfvnzKnz+/WrZsqYMHDzrbTEtLU//+/ZUvXz4VKFBAgwcP1p9PqP/z1EZqaqqGDBmiYsWKyW63q3Tp0nr//fd18OBBNWzYUNLvDzaz2Wzq3LmzpN8fR52YmKjo6GgFBQWpUqVK+uyzz1z6+eqrr3T//fcrKChIDRs2dIkzq4YMGaL7779fefLkUcmSJTVs2DDduHEjXb13331XxYoVU548edS2bVtduHDB5fp7772nmJgYBQYGqmzZspoyZUqmfZ47d06dOnVSoUKFFBQUpPvuu08zZszIduwAzMOIBJAFQUFBOnPmjPP9ihUrFBoaqmXLlkmSbty4oSZNmqhmzZr6/vvv5e/vr9GjR6tp06baunWrAgICNG7cOM2cOVMffPCBYmJiNG7cOC1YsED//Oc/M+336aef1rp16zRp0iRVqlRJKSkp+vXXX1WsWDHNnz9fjz/+uHbv3q3Q0FAFBQVJkhITE/XRRx9p6tSpuu+++7RmzRo99dRTKlSokOrXr68jR46oTZs26tmzp5599llt3LhRAwYMyPbvJCQkRDNnzlRkZKS2bdumZ555RiEhIRo8eLCzzr59+zRv3jwtXrxYFy9eVLdu3dSjRw/NmTNHkjRnzhwNHz5cb731lqpUqaLNmzfrmWeeUXBwsOLi4tL1OWzYMO3YsUNff/21ChYsqH379unq1avZjh2AiQwALuLi4oyWLVsahmEYDofDWLZsmWG3242BAwc6rxcpUsRITU11fmb27NlGmTJlDIfD4SxLTU01goKCjG+++cYwDMOIiIgwxowZ47x+48YN495773X2ZRiGUb9+faNPnz6GYRjG7t27DUnGsmXLMoxz1apVhiTj3LlzzrJr164ZefLkMdauXetSt1u3bkaHDh0MwzCMoUOHGuXKlXO5PmTIkHRt/ZkkY8GCBZleHzt2rFG1alXn+xEjRhi5cuUyfvnlF2fZ119/bfj5+RnHjx83DMMwSpUqZXz88ccu7YwaNcqoWbOmYRiGkZKSYkgyNm/ebBiGYbRo0cLo0qVLpjEAuPMYkQAysGTJEuXNm1c3btyQw+FQx44dFR8f77xeoUIFl3URW7Zs0b59+xQSEuLSzrVr17R//35duHBBx48fV40aNZzX/P39Va1atXTTG7ckJycrV65cql+/fpbj3rdvn65cuaKHH37Ypfz69euqUqWKJGnnzp0ucUhSzZo1s9zHLZ9++qkmTZqk/fv369KlS7p586ZCQ0Nd6hQvXlz33HOPSz8Oh0O7d+9WSEiI9u/fr27duumZZ55x1rl586bCwsIy7PP555/X448/rqSkJDVu3FitWrVSrVq1sh07APOQSAAZaNiwod555x0FBAQoMjJS/v6uX5Xg4GCX95cuXVLVqlWdQ/Z/VKhQIbdiuDVVkR2XLl2SJH355Zcuf4FLv6/7MMu6devUqVMnJSQkqEmTJgoLC9PcuXM1bty4bMc6ffr0dIlNrly5MvxMs2bNdOjQIX311VdatmyZGjVqpJ49e+qNN95w/2YA3BYSCSADwcHBKl26dJbrP/jgg/r0009VuHDhdP8qvyUiIkLr169XvXr1JP3+L+9NmzbpwQcfzLB+hQoV5HA49N133yk2Njbd9VsjImlpac6ycuXKyW636/Dhw5mOZMTExDgXjt7y448//v1N/sHatWtVokQJvfTSS86yQ4cOpat3+PBhHTt2TJGRkc5+/Pz8VKZMGRUpUkSRkZE6cOCAOnXqlOW+CxUqpLi4OMXFxalu3boaNGgQiQRgIXZtACbo1KmTChYsqJYtW+r7779XSkqKVq9erd69e+uXX36RJPXp00evvfaaFi5cqF27dqlHjx5/eQZEVFSU4uLi1LVrVy1cuNDZ5rx58yRJJUqUkM1m05IlS3T69GldunRJISEhGjhwoPr166dZs2Zp//79SkpK0uTJkzVr1ixJ0nPPPae9e/dq0KBB2r17tz7++GPNnDkzW/d733336fDhw5o7d67279+vSZMmacGCBenqBQYGKi4uTlu2bNH333+v3r17q23btipatKgkKSEhQYmJiZo0aZL27Nmjbdu2acaMGRo/fnyG/Q4fPlyLFi3Svn37tH37di1ZskQxMTHZih2AuUgkABPkyZNHa9asUfHixdWmTRvFxMSoW7duunbtmnOEYsCAAfrXv/6luLg41axZUyEhIWrduvVftvvOO+/oiSeeUI8ePVS2bFk988wzunz5siTpnnvuUUJCgl588UUVKVJEvXr1kiSNGjVKw4YNU2JiomJiYtS0aVN9+eWXio6OlvT7uoX58+dr4cKFqlSpkqZOnapXX301W/f72GOPqV+/furVq5cqV66stWvXatiwYenqlS5dWm3atNEjjzyixo0bq2LFii7bO7t376733ntPM2bMUIUKFVS/fn3NnDnTGeufBQQEaOjQoapYsaLq1aunXLlyae7cudmKHYC5bEZmK70AAAD+BiMSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbSQSAADAbf8PkAtlXISM2hwAAAAASUVORK5CYII=","text/plain":["<Figure size 640x480 with 2 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":["yhat = KNN_cv.predict(X_test)\n","plot_confusion_matrix(Y_test,yhat)"]},{"cell_type":"markdown","id":"bd1e29e0-fe8d-4f8a-afca-6cf8b9cbb02c","metadata":{},"source":["## TASK  12\n"]},{"cell_type":"markdown","id":"d871ff3f-a83c-43a6-8cab-235e45deabc3","metadata":{},"source":["Find the method performs best:\n"]},{"cell_type":"code","execution_count":65,"id":"48c42491-34de-4f53-a233-24aebf1bd484","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["The best method is: SVM with a score of: 0.8777777777777778\n"]}],"source":["logreg_score = logreg_cv.score(X, Y)\n","svm_score = svm_cv.score(X, Y)\n","tree_score = tree_cv.score(X, Y)\n","KNN_score = KNN_cv.score(X, Y)\n","\n","methods = ['Logistic Regression', 'SVM', 'Decision Tree', 'K-Nearest Neighbors']\n","scores = [logreg_score, svm_score, tree_score, KNN_score]\n","\n","best_method_index = scores.index(max(scores))\n","best_method = methods[best_method_index]\n","best_score = max(scores)\n","\n","print(f\"The best method is: {best_method} with a score of: {best_score}\")\n"]},{"cell_type":"markdown","id":"5b5851ae-571b-4b11-9893-8baa1923a09f","metadata":{},"source":["## Authors\n"]},{"cell_type":"markdown","id":"ece60c33-4d9a-4b5b-95cc-49db1b9aa6c4","metadata":{},"source":["[Pratiksha Verma](https://www.linkedin.com/in/pratiksha-verma-6487561b1/)\n"]},{"cell_type":"markdown","id":"beefd61a-ed54-4d42-9b12-7592c98484bc","metadata":{},"source":["## Change Log\n"]},{"cell_type":"markdown","id":"e5e25016-a703-46f4-92b0-944d9162998b","metadata":{},"source":["| Date (YYYY-MM-DD) | Version | Changed By      | Change Description      |\n","| ----------------- | ------- | -------------   | ----------------------- |\n","| 2022-11-09        | 1.0     | Pratiksha Verma | Converted initial version to Jupyterlite|\n"]},{"cell_type":"markdown","id":"357e8091-5f6c-4838-a127-bb2c4340332f","metadata":{},"source":["### <h3 align=\"center\"> IBM Corporation 2022. All rights reserved. <h3/>\n"]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.9"},"prev_pub_hash":"d01cd38c8e03632c08cdf905ff282a58063434d964488aa1bb4965ef6563e17d"},"nbformat":4,"nbformat_minor":4}