Multi Model Server Documentation

Basic Features

Serving Quick Start - Basic server usage tutorial
Model Archive Quick Start - Tutorial that shows you how to package a model archive file.
Installation - Installation procedures and troubleshooting
Serving Models - Explains how to use multi-model-server.
- REST API - Specification on the API endpoint for MMS
- Model Zoo - A collection of MMS model archive (.mar) files that you can use with MMS.
Packaging Model Archive - Explains how to package model archive file, use model-archiver.
Docker - How to use MMS with Docker and cloud services
Logging - How to configure logging
Metrics - How to configure metrics

Advanced Features

Advanced settings - Describes advanced MMS configurations.
Custom Model Service - Describes how to develop custom inference services.
Unit Tests - Housekeeping unit tests for MMS.
Benchmark - Use JMeter to run MMS through the paces and collect benchmark data.
Model Serving with Amazon Elastic Inference - Run Model server on Elastic Inference enabled EC2 instances.

Example Projects

MMS on Fargate, Serverless Inference - The project which illustrates the step-by-step process to launch MMS as a managed inference production service, on ECS Fargate.
MXNet Vision Service - An example MMS project for a MXNet Image Classification model. The project takes JPEG image as input for inference.
LSTM - An example MMS project for a recurrent neural network (RNN) using long short-term memory (LSTM). The project takes JSON inputs for inference against a model trained with a specific vocabulary.
Object Detection - An example MMS project that uses a pretrained Single Shot Multi Object Detection (SSD) model that takes image inputs and infers the types and locations of several classes of objects.