Bring Your Own Locally Trained ML&DL Model in SageMaker (XGBoost).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bring Your Own Locally Trained ML/DL Model in SageMaker (XGBoost)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook shows how to train an Xgboost model in scikit-learn and then inject it into Amazon SageMaker's first party XGboost container for scoring. This addresses the usecase where a customer has already trained their model outside of Amazon SageMaker, but wishes to host it for predictions within Amazon SageMaker."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Amazon SageMaker includes functionality to support a hosted notebook environment, distributed, serverless training, and real-time hosting. We think it works best when all three of these services are used together, but they can also be used independently. Some use cases may only require hosting. Maybe the model was trained prior to Amazon SageMaker existing, in a different service.\n",
    "\n",
    "Please note that scikit-learn XGBoost model is compatible with SageMaker XGBoost container, whereas other gradient boosted tree models (such as one trained in SparkML) are not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np                                # For matrix operations and numerical processing\n",
    "import pandas as pd                               # For munging tabular data\n",
    "import matplotlib.pyplot as plt                   # For charts and visualizations\n",
    "import matplotlib.ticker as ticker\n",
    "%matplotlib inline\n",
    "\n",
    "from IPython.display import Image                 # For displaying images in the notebook\n",
    "from IPython.display import display               # For displaying outputs in the notebook\n",
    "from time import gmtime, strftime                 # For labeling SageMaker models, endpoints, etc.\n",
    "import time\n",
    "import tqdm\n",
    "import sys                                        # For writing outputs to notebook\n",
    "import math                                       # For ceiling function\n",
    "import json                                       # For parsing hosting outputs\n",
    "import os                                         # For manipulating filepath names\n",
    "import boto3\n",
    "import botocore\n",
    "from botocore.exceptions import ClientError\n",
    "\n",
    "import struct\n",
    "import csv\n",
    "import io\n",
    "import re\n",
    "import s3fs\n",
    "import mxnet as mx\n",
    "import seaborn as sns\n",
    "import pickle\n",
    "import gzip\n",
    "import urllib\n",
    "import csv\n",
    "from datetime import datetime\n",
    "import pytz\n",
    "\n",
    "import cv2\n",
    "\n",
    "import sagemaker                                 \n",
    "from sagemaker.predictor import csv_serializer \n",
    "from sagemaker.predictor import json_deserializer\n",
    "import sagemaker.amazon.common as smac\n",
    "from sagemaker import get_execution_role"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Preparation (Specifying Sagemaker roles)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess   = sagemaker.Session()\n",
    "bucket = sess.default_bucket()                     \n",
    "prefix = 'xgboost-Bring-Your-Own-Model'\n",
    "region = boto3.Session().region_name\n",
    "role = 'arn:aws:iam::570447867175:role/SageMakerNotebookRole' # pass your IAM role name\n",
    "bucket_path = 'https://s3-{}.amazonaws.com/{}'.format(region, bucket)\n",
    "\n",
    "print('Sagemaker session :', sess)\n",
    "print('S3 bucket :', bucket)\n",
    "print('Bucket Path :', bucket_path)\n",
    "print('Prefix :', prefix)\n",
    "print('Region selected :', region)\n",
    "print('IAM role :', role)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.amazon.amazon_estimator import get_image_uri\n",
    "container = get_image_uri(boto3.Session().region_name, 'xgboost', '0.90-2')\n",
    "print(container)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we use MNIST dataset for a sample load"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the dataset\n",
    "urllib.request.urlretrieve(\"http://deeplearning.net/data/mnist/mnist.pkl.gz\", \"mnist.pkl.gz\")\n",
    "f = gzip.open('mnist.pkl.gz', 'rb')\n",
    "train_set, valid_set, test_set = pickle.load(f, encoding='latin1')\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "def get_dataset():\n",
    "    with gzip.open('mnist.pkl.gz', 'rb') as f:\n",
    "        u = pickle._Unpickler(f)\n",
    "        u.encoding = 'latin1'\n",
    "        return u.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_set, valid_set, test_set = get_dataset()\n",
    "\n",
    "train_X = train_set[0]\n",
    "train_y = train_set[1]\n",
    "\n",
    "valid_X = valid_set[0]\n",
    "valid_y = valid_set[1]\n",
    "\n",
    "test_X = test_set[0]\n",
    "test_y = test_set[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Train Model Locally"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xgboost as xgb\n",
    "import sklearn as sk \n",
    "\n",
    "bt = xgb.XGBClassifier(max_depth=5,\n",
    "                       learning_rate=0.2,\n",
    "                       n_estimators=10,\n",
    "                       objective='multi:softmax')   # Setup xgboost model\n",
    "bt.fit(train_X, train_y, # Train it to our data\n",
    "       eval_set=[(valid_X, valid_y)], \n",
    "       verbose=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Save the locally-trained model "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Note that the model file name must satisfy the regular expression pattern: ^[a-zA-Z0-9](-*[a-zA-Z0-9])*;. The model file also need to tar-zipped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_file_name = \"local-xgboost\"\n",
    "bt._Booster.save_model(model_file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!tar czvf model.tar.gz $model_file_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Upload the locally-trained model to S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fObj = open(\"model.tar.gz\", 'rb')\n",
    "key= os.path.join(prefix, model_file_name, 'model.tar.gz')\n",
    "boto3.Session().resource('s3').Bucket(bucket).Object(key).upload_fileobj(fObj)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Create a Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "from time import gmtime, strftime\n",
    "\n",
    "model_name = model_file_name + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "model_url = 'https://s3-{}.amazonaws.com/{}/{}'.format(region,bucket,key)\n",
    "sm_client = boto3.client('sagemaker')\n",
    "\n",
    "print (model_url)\n",
    "\n",
    "primary_container = {\n",
    "    'Image': container,\n",
    "    'ModelDataUrl': model_url,\n",
    "}\n",
    "\n",
    "create_model_response2 = sm_client.create_model(\n",
    "    ModelName = model_name,\n",
    "    ExecutionRoleArn = role,\n",
    "    PrimaryContainer = primary_container)\n",
    "\n",
    "print(create_model_response2['ModelArn'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7. Create endpoint configuration\n",
    "SageMaker supports configuring REST endpoints in hosting with multiple models, e.g. for A/B testing purposes. In order to support this, you can create an endpoint configuration, that describes the distribution of traffic across the models, whether split, shadowed, or sampled in some way. In addition, the endpoint configuration describes the instance type required for model deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint_config_name = 'DEMO-XGBoostEndpointConfig-' + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "print(endpoint_config_name)\n",
    "create_endpoint_config_response = sm_client.create_endpoint_config(\n",
    "    EndpointConfigName = endpoint_config_name,\n",
    "    ProductionVariants=[{\n",
    "        'InstanceType':'ml.m4.xlarge',\n",
    "        'InitialInstanceCount':1,\n",
    "        'InitialVariantWeight':1,\n",
    "        'ModelName':model_name,\n",
    "        'VariantName':'AllTraffic'}])\n",
    "\n",
    "print(\"Endpoint Config Arn: \" + create_endpoint_config_response['EndpointConfigArn'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8. Create endpoint\n",
    "Lastly, you create the endpoint that serves up the model, through specifying the name and configuration defined above. The end result is an endpoint that can be validated and incorporated into production applications. This takes 9-11 minutes to complete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint_name = 'DEMO-XGBoostEndpoint-' + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "print(endpoint_name)\n",
    "create_endpoint_response = sm_client.create_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    EndpointConfigName=endpoint_config_name)\n",
    "print(create_endpoint_response['EndpointArn'])\n",
    "\n",
    "resp = sm_client.describe_endpoint(EndpointName=endpoint_name)\n",
    "status = resp['EndpointStatus']\n",
    "print(\"Status: \" + status)\n",
    "\n",
    "while status=='Creating':\n",
    "    time.sleep(60)\n",
    "    resp = sm_client.describe_endpoint(EndpointName=endpoint_name)\n",
    "    status = resp['EndpointStatus']\n",
    "    print(\"Status: \" + status)\n",
    "\n",
    "print(\"Arn: \" + resp['EndpointArn'])\n",
    "print(\"Status: \" + status)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9. Evaluating with the test set (Inference)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "runtime_client = boto3.client('runtime.sagemaker')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "point_X = test_X[0]\n",
    "point_X = np.expand_dims(point_X, axis=0)\n",
    "point_y = test_y[0]\n",
    "np.savetxt(\"test_point.csv\", point_X, delimiter=\",\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_name = 'test_point.csv' #customize to your test file, will be 'mnist.single.test' if use data above\n",
    "\n",
    "with open(file_name, 'r') as f:\n",
    "    payload = f.read().strip()\n",
    "\n",
    "response = runtime_client.invoke_endpoint(EndpointName=endpoint_name, \n",
    "                                   ContentType='text/csv', \n",
    "                                   Body=payload)\n",
    "result = response['Body'].read().decode('ascii')\n",
    "print('Predicted Class Probabilities: {}.'.format(result))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Determine the the output class label\n",
    "floatArr = np.array(json.loads(result))\n",
    "predictedLabel = np.argmax(floatArr)\n",
    "print('Predicted Class Label: {}.'.format(predictedLabel))\n",
    "print('Actual Class Label: {}.'.format(point_y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 10. Close the SageMaker Instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sm_client.delete_endpoint(EndpointName=endpoint_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}