Serverless ML Workshop

This workshop focusses on deployment of ML models with Serverless APIs (AWS Lambda) and Docker. We'll learn how to:

• Train and serialize a model inside a container
• Package the service in an image
• Serve the model with serverless API
• Build, Test, and Deploy, with CI/CD workflow

Technologies

• Docker + ECR: Container & Registy
• AWS Lambda: Serving API
• SAM: Serverless Framework (optional)
• GitHub Actions: CI/CD

Project Structure

|-- service
     |-- app.py: source code lambda handler
     |-- train.py: to train the model
     |-- Dockerfile: to build the Docker image
     |-- requirements.txt: dependencies
|-- tests
     |-- unit
          |--test_handler.py: unit test/s for lambda handler
|-- samconfig.toml: configured by SAM
|-- template.yaml: A template that defines the application's AWS resources.

Pre-requisites

• python3.8 (used by this service) or python3 Installed Python 3
• Docker - Install Docker community edition
• awscli - Install AWS CLI
• aws-sam-cli - Install SAM CLI

Setup

• AWS account with IAM user & required permissions
• ECR Repository (Here's how you create one:)

$ aws ecr create-repository --repository-name <repo-name> [--image-scanning-configuration scanOnPush=true]

Steps

Build

1. Using Docker

$ docker build -t serverless-ml ./service

2. Using SAM

$ sam build

The processed template file is saved in the .aws-sam/build folder.

Testing locally

1. Using Docker

$ docker run -p 8080:8080 serverless-ml

$ curl -XPOST "http://localhost:8080/2015-03-31/functions/function/invocations" -d '{"body": {"data": ".10"}}'

2. Using SAM

$ sam local start-api

$ curl -XPOST http://127.0.0.1:3000/classify -H 'Content-Type: application/json' -d '{"data":".10"}'

Unit tests

Tests are defined in the tests folder in this project. Use PIP to install the pytest and run unit tests from your local machine.

$ pip install pytest pytest-mock --user
$ python -m pytest tests/ -v

Deployment

1. Using Docker

$ aws ecr get-login-password | docker login --username AWS --password-stdin ${AWS_ACCOUNT}.dkr.ecr.eu-central-1.amazonaws.com

$ docker push ${AWS_ACCOUNT}.dkr.ecr.eu-central-1.amazonaws.com/serverless-ml:latest

2. Using SAM

sam deploy --guided

Cleanup

To delete the sample application that you created, use the AWS CLI. Assuming you used your project name for the stack name, you can run the following:

aws cloudformation delete-stack --stack-name <stack-name>

Resources

Workshops built on new AWS features:

• AWS Lambda – Container Image tutorial
• Deploying a DL Model on AWS by Alexey Grigorev

The Docker-based workflow (multi-stage build) is based on:

• Docker for Machine Learning series by Luigi Patruno (MLinProduction)
• A Simple Docker-based Workflow For Deploying A Machine Learning Model