19bcd7243-lab3-seattle-weather-forecast.ipynb

{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Name: **SIMRAN ANAND**\n# Registration Number: **19BCD7243**\n# Weather Forecast in Seattle using Decision Trees \n\n## Lab 3 assignment of CSE4029\n\n### Importing important libraries:","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0"}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport seaborn as sb\nimport os\n\nfrom pandas import Series, DataFrame\nfrom pylab import rcParams\nfrom sklearn import preprocessing\n\nfrom sklearn import metrics\nfrom sklearn.model_selection import train_test_split \nfrom sklearn.metrics import classification_report\n\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.ensemble import AdaBoostClassifier, RandomForestClassifier\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.naive_bayes import GaussianNB\nfrom sklearn.svm import SVC","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-03-23T15:25:31.829489Z","iopub.execute_input":"2022-03-23T15:25:31.830169Z","iopub.status.idle":"2022-03-23T15:25:33.715394Z","shell.execute_reply.started":"2022-03-23T15:25:31.830126Z","shell.execute_reply":"2022-03-23T15:25:33.714555Z"},"trusted":true},"execution_count":1,"outputs":[]},{"cell_type":"markdown","source":"### Reading the data:","metadata":{}},{"cell_type":"code","source":"met_df = pd.read_csv('../input/seattleweather-19482017/seattleWeather_1948-2017.csv')\nprint(met_df.head()); print(); print()\nmet_df.info()","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:33.717856Z","iopub.execute_input":"2022-03-23T15:25:33.718196Z","iopub.status.idle":"2022-03-23T15:25:33.803985Z","shell.execute_reply.started":"2022-03-23T15:25:33.718149Z","shell.execute_reply":"2022-03-23T15:25:33.802719Z"},"trusted":true},"execution_count":2,"outputs":[]},{"cell_type":"markdown","source":"The description and unit of each variable:\n- DATE = the date of the observation\n- PRCP = the amount of precipitation, in inches\n- TMAX = the maximum temperature for that day, in degrees Fahrenheit\n- TMIN = the minimum temperature for that day, in degrees Fahrenheit\n- RAIN = TRUE if rain was observed on that day, FALSE if it was not","metadata":{}},{"cell_type":"markdown","source":"## Data Cleaing:\n\n### Step 1: Correcting wrong values or outliers:","metadata":{}},{"cell_type":"code","source":"met_df.describe(include = 'all')","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:33.806993Z","iopub.execute_input":"2022-03-23T15:25:33.807710Z","iopub.status.idle":"2022-03-23T15:25:33.890429Z","shell.execute_reply.started":"2022-03-23T15:25:33.807658Z","shell.execute_reply":"2022-03-23T15:25:33.889470Z"},"trusted":true},"execution_count":3,"outputs":[]},{"cell_type":"markdown","source":"The data description makes sense, and the mean, min, and max values of each variable is reasonable meaning there should not be a mistake in the data (such as a very large temperature of 200 F).","metadata":{}},{"cell_type":"markdown","source":"### Step 2: Imputing missing values:\n\nThere are only three missing data points for each PRCP and RAIN. So, we use median for PRCP and mode for RAIN to fill in the gaps.","metadata":{}},{"cell_type":"code","source":"met_df.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:33.891925Z","iopub.execute_input":"2022-03-23T15:25:33.892434Z","iopub.status.idle":"2022-03-23T15:25:33.907404Z","shell.execute_reply.started":"2022-03-23T15:25:33.892388Z","shell.execute_reply":"2022-03-23T15:25:33.906387Z"},"trusted":true},"execution_count":4,"outputs":[]},{"cell_type":"code","source":"P_median = met_df.PRCP.median()\nR_mode   = met_df.RAIN.mode()[0]\n\nmet_df.PRCP.fillna(P_median, inplace = True)\nmet_df.RAIN.fillna(R_mode, inplace = True)\n\nmet_df.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:33.911569Z","iopub.execute_input":"2022-03-23T15:25:33.911840Z","iopub.status.idle":"2022-03-23T15:25:33.937341Z","shell.execute_reply.started":"2022-03-23T15:25:33.911812Z","shell.execute_reply":"2022-03-23T15:25:33.936567Z"},"trusted":true},"execution_count":5,"outputs":[]},{"cell_type":"markdown","source":"### Step 3: Converting boolean variable to dummy variable:\n- We should change RAIN from True/False to 1/0.\n- We then replace the new variable with the original one.","metadata":{}},{"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nRAIN_encode = LabelEncoder().fit_transform(met_df.RAIN)\nRAIN_encode","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:33.938746Z","iopub.execute_input":"2022-03-23T15:25:33.939142Z","iopub.status.idle":"2022-03-23T15:25:33.950548Z","shell.execute_reply.started":"2022-03-23T15:25:33.939112Z","shell.execute_reply":"2022-03-23T15:25:33.949755Z"},"trusted":true},"execution_count":6,"outputs":[]},{"cell_type":"code","source":"met_df['RAIN'] = RAIN_encode\n\nmet_df.describe(include = 'all')","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:33.952291Z","iopub.execute_input":"2022-03-23T15:25:33.952893Z","iopub.status.idle":"2022-03-23T15:25:34.013468Z","shell.execute_reply.started":"2022-03-23T15:25:33.952843Z","shell.execute_reply":"2022-03-23T15:25:34.012856Z"},"trusted":true},"execution_count":7,"outputs":[]},{"cell_type":"markdown","source":"### Making sure all predictors are independent\n\n### Insightful plots","metadata":{}},{"cell_type":"code","source":"%matplotlib inline\nrcParams['figure.figsize'] = 6, 5\nsb.set_style('whitegrid')\n\nsb.pairplot(met_df, palette = 'husl', hue = 'RAIN')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:34.014525Z","iopub.execute_input":"2022-03-23T15:25:34.014948Z","iopub.status.idle":"2022-03-23T15:25:39.266011Z","shell.execute_reply.started":"2022-03-23T15:25:34.014903Z","shell.execute_reply":"2022-03-23T15:25:39.264965Z"},"trusted":true},"execution_count":8,"outputs":[]},{"cell_type":"code","source":"sb.heatmap(met_df.corr(), vmin=-1, vmax=1, annot=True, cmap = 'RdBu_r')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:39.267453Z","iopub.execute_input":"2022-03-23T15:25:39.267987Z","iopub.status.idle":"2022-03-23T15:25:39.721409Z","shell.execute_reply.started":"2022-03-23T15:25:39.267942Z","shell.execute_reply":"2022-03-23T15:25:39.719809Z"},"trusted":true},"execution_count":9,"outputs":[]},{"cell_type":"code","source":"sb.scatterplot(x = 'TMIN', y ='TMAX', data = met_df, hue = 'RAIN')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:39.723272Z","iopub.execute_input":"2022-03-23T15:25:39.723626Z","iopub.status.idle":"2022-03-23T15:25:40.975948Z","shell.execute_reply.started":"2022-03-23T15:25:39.723569Z","shell.execute_reply":"2022-03-23T15:25:40.974787Z"},"trusted":true},"execution_count":10,"outputs":[]},{"cell_type":"markdown","source":"- Ax expected, TMIN and TMAX are highliy correlated, so we drop TMIN that has lower correlation with RAIN.","metadata":{}},{"cell_type":"code","source":"fig, axis = plt.subplots(1, 2,figsize=(10,4))\nsb.boxplot(x = 'RAIN', y ='TMAX', data = met_df, ax = axis[0], showfliers = False)\nsb.boxplot(x = 'RAIN', y ='TMIN', data = met_df, ax = axis[1], showfliers = False)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:40.977915Z","iopub.execute_input":"2022-03-23T15:25:40.978293Z","iopub.status.idle":"2022-03-23T15:25:41.332683Z","shell.execute_reply.started":"2022-03-23T15:25:40.978245Z","shell.execute_reply":"2022-03-23T15:25:41.331821Z"},"trusted":true},"execution_count":11,"outputs":[]},{"cell_type":"markdown","source":"- Here, each TMAX and TMIN is grouped based on RAIN.\n- Again, we see that TMAX is a better predictor of RAIN.\n\n- We should also drop 'PRCP' variable: if we know the amount of precipitation on each day, we can certainly say whether it rains or not on that day.","metadata":{}},{"cell_type":"code","source":"met_df.drop(['TMIN', 'PRCP','DATE'], inplace = True, axis=1)\nmet_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:41.334237Z","iopub.execute_input":"2022-03-23T15:25:41.334497Z","iopub.status.idle":"2022-03-23T15:25:41.348142Z","shell.execute_reply.started":"2022-03-23T15:25:41.334468Z","shell.execute_reply":"2022-03-23T15:25:41.347216Z"},"trusted":true},"execution_count":12,"outputs":[]},{"cell_type":"markdown","source":"## Implementing MLAs:\n\n### Spliting the data into test and train sets:\n","metadata":{}},{"cell_type":"code","source":"X_train, X_test, Y_train, Y_test = train_test_split(met_df.drop('RAIN', axis=1),\n                                                   met_df['RAIN'], test_size=0.2, random_state=10)                             \n","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:41.350011Z","iopub.execute_input":"2022-03-23T15:25:41.350545Z","iopub.status.idle":"2022-03-23T15:25:41.362821Z","shell.execute_reply.started":"2022-03-23T15:25:41.350494Z","shell.execute_reply":"2022-03-23T15:25:41.361872Z"},"trusted":true},"execution_count":13,"outputs":[]},{"cell_type":"markdown","source":"### Using a variety of MLAs to get the best results\n\n- The outcome is binary, so we can use Logistic Regression, Decision Tree, or Naive Bayes. \n- We also use ensemble algorithms (such as Random Forest) to see if the score can be improved.","metadata":{}},{"cell_type":"code","source":"all_classifiers = {'Ada Boost': AdaBoostClassifier(),\n                 'Random Forest': RandomForestClassifier(n_estimators=50, min_samples_leaf=1, min_samples_split=2, max_depth=4),\n                 'Gaussian NB': GaussianNB(),\n                 'Logistic Regression': LogisticRegression(solver='liblinear'),#fit_intercept=True,\n                 'Decision Tree' : DecisionTreeClassifier(),\n                  'SVC': SVC()} #probability = False ","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:41.364546Z","iopub.execute_input":"2022-03-23T15:25:41.364959Z","iopub.status.idle":"2022-03-23T15:25:41.377181Z","shell.execute_reply.started":"2022-03-23T15:25:41.364916Z","shell.execute_reply":"2022-03-23T15:25:41.376226Z"},"trusted":true},"execution_count":14,"outputs":[]},{"cell_type":"code","source":"ML_name = []\nML_accuracy = []\nfor Name,classifier in all_classifiers.items():\n    classifier.fit(X_train,Y_train)\n    Y_pred = classifier.predict(X_test)\n    ML_accuracy.append(metrics.accuracy_score(Y_test,Y_pred)) \n    ML_name.append(Name) ","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:41.379659Z","iopub.execute_input":"2022-03-23T15:25:41.379996Z","iopub.status.idle":"2022-03-23T15:25:59.036402Z","shell.execute_reply.started":"2022-03-23T15:25:41.379962Z","shell.execute_reply":"2022-03-23T15:25:59.035502Z"},"trusted":true},"execution_count":15,"outputs":[]},{"cell_type":"code","source":"rcParams['figure.figsize'] = 8, 4\nplt.barh(ML_name, ML_accuracy, color = 'brown')\nplt.xlabel('Accuracy Score', fontsize = '14')\nplt.ylabel('Machine Learning Algorithms', fontsize = '14')\nplt.xlim([0.65, 0.685])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:59.037963Z","iopub.execute_input":"2022-03-23T15:25:59.038376Z","iopub.status.idle":"2022-03-23T15:25:59.345838Z","shell.execute_reply.started":"2022-03-23T15:25:59.038330Z","shell.execute_reply":"2022-03-23T15:25:59.344848Z"},"trusted":true},"execution_count":16,"outputs":[]},{"cell_type":"markdown","source":"### Tuning models with hyper parameters:\n\n\n### **Decision Tree**:\n","metadata":{"trusted":true}},{"cell_type":"code","source":"criteri       = ['gini', 'entropy']\nmin_samp_lf   = [1, 2, 5, 10]\nmin_samp_splt = [2, 4, 8, 12]\nmaxim_depth   = [2, 4, 8, 12, None]\n\nmax_score = 0\n\nfor c in criteri:\n    for ml in min_samp_lf:\n        for ms in min_samp_splt:\n            for md in maxim_depth:\n                MLA = DecisionTreeClassifier(criterion=c, min_samples_leaf=ml, min_samples_split=ms, max_depth=md)\n                MLA.fit(X_train,Y_train)\n                Y_pred = MLA.predict(X_test)\n                if metrics.accuracy_score(Y_test,Y_pred) > max_score:\n                    max_score, c_best, l_best, s_best, d_best = metrics.accuracy_score(Y_test,Y_pred), c, ml, ms, md\n\nprint('maximum accuracy score, criterion, min_samples_leaf, min_samples_split, max_depth:')\nprint(max_score, c_best, l_best, s_best, d_best)","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:25:59.347300Z","iopub.execute_input":"2022-03-23T15:25:59.347653Z","iopub.status.idle":"2022-03-23T15:26:00.574683Z","shell.execute_reply.started":"2022-03-23T15:25:59.347611Z","shell.execute_reply":"2022-03-23T15:26:00.573675Z"},"trusted":true},"execution_count":17,"outputs":[]},{"cell_type":"markdown","source":"### Ada Boost Classifier:","metadata":{}},{"cell_type":"code","source":"learning_R    = [1, 2, 3]\nrandom_st     = [None, 20]\nn_estimat     = [50, 100]\n\nmax_score = 0\n\nfor lr in learning_R:\n    for rs in random_st:\n        for ne in n_estimat:\n            MLA = AdaBoostClassifier(random_state=rs, learning_rate=lr, n_estimators=ne)\n            MLA.fit(X_train,Y_train)\n            Y_pred = MLA.predict(X_test)\n            if metrics.accuracy_score(Y_test,Y_pred) > max_score:\n                max_score, r_best, l_best, n_best = metrics.accuracy_score(Y_test,Y_pred), rs, lr, ne\n\nprint('maximum accuracy score, random_state, learning_rate, n_estimators:')\nprint(max_score, r_best, l_best, n_best)","metadata":{"execution":{"iopub.status.busy":"2022-03-23T15:26:00.576276Z","iopub.execute_input":"2022-03-23T15:26:00.576626Z","iopub.status.idle":"2022-03-23T15:26:07.786341Z","shell.execute_reply.started":"2022-03-23T15:26:00.576581Z","shell.execute_reply":"2022-03-23T15:26:07.785557Z"},"trusted":true},"execution_count":18,"outputs":[]},{"cell_type":"markdown","source":"## Observations:\n\n- It seems that tuning hyper-parameters for various MLAs would give us an accuracy score of ~ 68.1%, meaning that our MLAs can predict the rain correctly in 68% of times for test datasets.\n\n- This is better than the baseline: we should be able to predict rain by 50% accuracy only by tossing a coin. Moreover, there are 57% of instances of not rain and 43% instances of rain. So, if we always select not rain, we would get a score of 57%. So far, we improved the accuracy score by 11.1%.\n","metadata":{}},{"cell_type":"markdown","source":"# Thank you\n# - SIMRAN ANAND \n# - 19BCD7243","metadata":{}}]}