3_training_dataset_and_modeling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7eb83ff8",
   "metadata": {
    "tags": []
   },
   "source": [
    "# <span style=\"font-width:bold; font-size: 3rem; color:#1EB182;\"><img src=\"../../images/icon102.png\" width=\"38px\"></img> **Hopsworks Feature Store** </span><span style=\"font-width:bold; font-size: 3rem; color:#333;\">- Part 03: Training Dataset & Modeling</span>\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/logicalclocks/hopsworks-tutorials/blob/master/advanced_tutorials/air_quality/3_training_dataset_and_modeling.ipynb)\n",
    "\n",
    "<span style=\"font-width:bold; font-size: 1.4rem;\">This notebook explains how to read from a feature group and create training dataset within the feature store</span>\n",
    "\n",
    "## 🗒️ This notebook is divided into the following sections:\n",
    "\n",
    "1. Fetch Feature Groups\n",
    "2. Define Transformation functions\n",
    "4. Create Feature Views\n",
    "5. Create Training Dataset with training, validation and test splits\n",
    "\n",
    "![part2](../../images/02_training-dataset.png) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3b5f602-a575-49a8-bce9-a997cca936e0",
   "metadata": {},
   "source": [
    "### <span style='color:#ff5f27'> 📝 Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad609eec-0b46-445f-a0f5-5657e5f69866",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install xgboost joblib matplotlib seaborn scikit-learn --q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b3f2ac81-423a-4380-8fd6-b70aa55eb864",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import datetime\n",
    "import time\n",
    "import json\n",
    "import pickle\n",
    "import joblib\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "from xgboost import XGBRegressor\n",
    "from sklearn.metrics import mean_squared_error, r2_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0b3bcd1",
   "metadata": {},
   "source": [
    "## <span style=\"color:#ff5f27;\"> 📡 Connecting to Hopsworks Feature Store </span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "89ad779f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import hopsworks\n",
    "\n",
    "project = hopsworks.login()\n",
    "\n",
    "fs = project.get_feature_store() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "735a083e",
   "metadata": {},
   "outputs": [],
   "source": [
    "air_quality_fg = fs.get_feature_group(\n",
    "    name = 'air_quality',\n",
    "    version = 1\n",
    ")\n",
    "weather_fg = fs.get_feature_group(\n",
    "    name = 'weather',\n",
    "    version = 1\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be427dca",
   "metadata": {},
   "source": [
    "--- \n",
    "\n",
    "## <span style=\"color:#ff5f27;\"> 🖍 Feature View Creation and Retrieving </span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc3192d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "query = air_quality_fg.select(['pm2_5']).join(weather_fg.select_except(['unix_time']), on=['city_name', 'date'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d83a1681",
   "metadata": {},
   "source": [
    "`Feature Views` stands between **Feature Groups** and **Training Dataset**. Сombining **Feature Groups** we can create **Feature Views** which store a metadata of our data. Having **Feature Views** we can create **Training Dataset**.\n",
    "\n",
    "The Feature Views allows schema in form of a query with filters, define a model target feature/label and additional transformation functions.\n",
    "\n",
    "In order to create Feature View we can use `FeatureStore.create_feature_view()` method.\n",
    "\n",
    "You can specify next parameters:\n",
    "\n",
    "- `name` - name of a feature group.\n",
    "\n",
    "- `version` - version of a feature group.\n",
    "\n",
    "- `labels`- our target variable.\n",
    "\n",
    "- `transformation_functions` - functions to transform our features.\n",
    "\n",
    "- `query` - query object with data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "403df0b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "version=3\n",
    "try:\n",
    "    feature_view = fs.create_feature_view(\n",
    "        name='air_quality_fv',\n",
    "        version=version,\n",
    "        labels=['pm2_5'],\n",
    "        query=query\n",
    "    )\n",
    "except:\n",
    "    feature_view = fs.get_feature_view(name='air_quality_fv', version=version)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c723c54",
   "metadata": {},
   "source": [
    "For now `Feature View` is saved in Hopsworks and you can retrieve it using `FeatureStore.get_feature_view()`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e1187a2",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## <span style=\"color:#ff5f27;\"> 🏋️ Training Dataset Creation</span>\n",
    "\n",
    "In Hopsworks training data is a query where the projection (set of features) is determined by the parent FeatureView with an optional snapshot on disk of the data returned by the query.\n",
    "\n",
    "**Training Dataset  may contain splits such as:** \n",
    "* Training set - the subset of training data used to train a model.\n",
    "* Validation set - the subset of training data used to evaluate hparams when training a model\n",
    "* Test set - the holdout subset of training data used to evaluate a mode\n",
    "\n",
    "To create training dataset you use `FeatureView.create_training_data()` method.\n",
    "\n",
    "Here are some importand things:\n",
    "\n",
    "- It will inherit the name of FeatureView.\n",
    "\n",
    "- The feature store currently supports the following data formats for\n",
    "training datasets: **tfrecord, csv, tsv, parquet, avro, orc**.\n",
    "\n",
    "- You can choose necessary format using **data_format** parameter.\n",
    "\n",
    "- **start_time** and **end_time** in order to filter dataset in specific time range."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4310a35-191e-44cc-b35c-7ce20c9695dd",
   "metadata": {},
   "source": [
    "### <span style=\"color:#ff5f27;\">🪝 Training Dataset Retrieval</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b04e30a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = feature_view.train_test_split(test_size=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3707fc1c",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f63effb0",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c995b340-5ba6-4116-b8b6-86ca34f0a0ab",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95783124-8303-47c5-bd15-2804efa15611",
   "metadata": {},
   "source": [
    "## <span style=\"color:#ff5f27;\">🧬 Modeling</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5b937dec",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.preprocessing import StandardScaler, OneHotEncoder, LabelEncoder\n",
    "from sklearn.feature_selection import SelectPercentile, chi2\n",
    "from sklearn.compose import make_column_transformer\n",
    "\n",
    "categorical_transformer = Pipeline(\n",
    "    steps=[\n",
    "        (\"encoder\", OneHotEncoder(handle_unknown=\"ignore\")),\n",
    "        (\"selector\", SelectPercentile(chi2, percentile=50)),\n",
    "    ]\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0299506-195f-4ebc-b43e-347fe59db31c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "categorical_feature_ids = []\n",
    "idx=0\n",
    "for col in X_train.columns:\n",
    "    if X_train[col].dtype == object:\n",
    "        categorical_feature_ids.append(idx)\n",
    "    idx += 1\n",
    "\n",
    "categorical_features = []\n",
    "numeric_features = []\n",
    "for col in X_train.columns:\n",
    "    if X_train[col].dtype == object:\n",
    "        categorical_features.append(col)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "abdb1397",
   "metadata": {},
   "outputs": [],
   "source": [
    "preprocessor = ColumnTransformer(\n",
    "    transformers=[\n",
    "        (\"cat\", categorical_transformer, categorical_features),\n",
    "      ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fd4e24e-7f02-4944-a309-6475b65e7846",
   "metadata": {},
   "source": [
    "### <span style='color:#ff5f27'> ⚖️ Model Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4e5be9f8-0f88-4a7e-8fc1-65ec8b02920d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "start_of_cell = time.time()\n",
    "xgb_regressor = XGBRegressor()\n",
    "xgb_regressor.feature_names = X_train.columns\n",
    "clf = Pipeline(\n",
    "    steps=[(\"preprocessor\", preprocessor), (\"regressor\", xgb_regressor)]\n",
    ")\n",
    "\n",
    "clf.fit(X_train, y_train['pm2_5'].astype('int'))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ad2979f",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "y_pred = clf.predict(X_test)\n",
    "\n",
    "# calculate MSE using sklearn\n",
    "mse = mean_squared_error(y_test, y_pred)\n",
    "print(\"MSE:\", mse)\n",
    "\n",
    "# calculate RMSE using sklearn\n",
    "rmse = mean_squared_error(y_test, y_pred, squared=False)\n",
    "print(\"RMSE:\", rmse)\n",
    "\n",
    "# calculate R squared using sklearn\n",
    "r2 = r2_score(y_test, y_pred)\n",
    "print(\"R squared:\", r2)\n",
    "\n",
    "end_of_cell = time.time()\n",
    "print(f\"Took {round(end_of_cell - start_of_cell, 2)} sec.\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac31f9fb-7904-416a-9938-e85320340412",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_ = pd.DataFrame({\n",
    "    \"y_true\": y_test['pm2_5'].astype('int'),\n",
    "    \"y_pred\": y_pred\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2fc8448-2150-4cfd-803d-afbd4845b59e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "residplot = sns.residplot(data=df_, x=\"y_true\", y=\"y_pred\", color='orange')\n",
    "plt.title('Model Residuals')\n",
    "plt.xlabel('Obsevation #')\n",
    "plt.ylabel('Error')\n",
    "\n",
    "plt.show()\n",
    "fig = residplot.get_figure()\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ae4e226-7a93-4e4d-8131-c1de62a7b6f9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from xgboost import plot_importance\n",
    "\n",
    "plot_importance(xgb_regressor, max_num_features=25)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dcea831-6c21-4396-a0ce-0631d21d1875",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c066fe79-315e-4b85-b2ab-32d503679dc7",
   "metadata": {
    "tags": []
   },
   "source": [
    "## <span style='color:#ff5f27'>🗄 Model Registry</span>\n",
    "\n",
    "One of the features in Hopsworks is the model registry. This is where you can store different versions of models and compare their performance. Models from the registry can then be served as API endpoints."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a787ec40-6bd7-4950-aa5d-bf004e1e5ade",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "mr = project.get_model_registry()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d240dc7-8a02-47b2-9667-7483508b2d24",
   "metadata": {},
   "source": [
    "### <span style=\"color:#ff5f27;\">⚙️ Model Schema</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c658df3-56a4-450b-90ee-127d0afe5b74",
   "metadata": {},
   "source": [
    "The model needs to be set up with a [Model Schema](https://docs.hopsworks.ai/machine-learning-api/latest/generated/model_schema/), which describes the inputs and outputs for a model.\n",
    "\n",
    "A Model Schema can be automatically generated from training examples, as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd3f3751",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from hsml.schema import Schema\n",
    "from hsml.model_schema import ModelSchema\n",
    "\n",
    "input_schema = Schema(X_train)\n",
    "output_schema = Schema(y_train)\n",
    "model_schema = ModelSchema(input_schema=input_schema, output_schema=output_schema)\n",
    "\n",
    "model_schema.to_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d2777f5e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# The 'nyc_taxi_fares_model' directory will be saved to the model registry\n",
    "model_dir=\"air_quality_model\"\n",
    "if os.path.isdir(model_dir) == False:\n",
    "    os.mkdir(model_dir)\n",
    "\n",
    "joblib.dump(clf, model_dir + '/xgboost_pipeline.pkl')\n",
    "\n",
    "fig.savefig(model_dir + \"/residplot.png\") "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41f6811e",
   "metadata": {},
   "outputs": [],
   "source": [
    "aq_model = mr.python.create_model(\n",
    "    name=\"air_quality_xgboost_model\", \n",
    "    metrics={\n",
    "        \"RMSE\": rmse,\n",
    "        \"MSE\": mse,\n",
    "        \"R squared\": r2\n",
    "    },\n",
    "    model_schema=model_schema,\n",
    "    input_example=X_test.sample().values, \n",
    "    description=\"Air Quality (PM2.5) predictor.\")\n",
    "\n",
    "aq_model.save(model_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ce57d64-99f1-436c-8eb8-e43e8fb57e19",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bf025f1-eef3-427c-8537-1242874716ca",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "### <span style=\"color:#ff5f27;\">🥳 <b> Next Steps </b> </span>\n",
    "\n",
    "Check out our other tutorials on ➡ https://github.com/logicalclocks/hopsworks-tutorials\n",
    "\n",
    "Or documentation at ➡ https://docs.hopsworks.ai"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "38",
   "language": "python",
   "name": "38"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  },
  "vscode": {
   "interpreter": {
    "hash": "63265f9757e7c73c149a91832e3b2b12ced37a5390b9151ad08a04f276cd5846"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}