Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 29 additions & 5 deletions about.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,47 @@ description: Deploy open source AI models to AWS, GCP, and Azure in minutes

Magemaker is a Python tool that simplifies the process of deploying open source AI models to your preferred cloud provider. Instead of spending hours digging through documentation, Magemaker lets you deploy Hugging Face models directly to AWS SageMaker, Google Cloud Vertex AI, or Azure Machine Learning.

## Current Features

- **Multi-Cloud Deployment**: Deploy to AWS SageMaker, GCP Vertex AI, and Azure ML
- **Hugging Face Integration**: Deploy any public or gated Hugging Face model
- **Custom Model Support**: Deploy fine-tuned models from S3 or local paths
- **Fine-Tuning**: Train AWS SageMaker JumpStart models on your data
- **Interactive CLI**: User-friendly dropdown menus for AWS deployments
- **YAML Configuration**: Infrastructure as Code for reproducible deployments

## What we're working on next

- Interactive deployment support for GCP and Azure
- Hugging Face model fine-tuning
- GCP and Azure fine-tuning support
- More robust error handling for various edge cases
- Verbose logging
- Enabling / disabling autoscaling
- Enhanced multi-cloud support features

Do submit your feature requests at https://magemaker.featurebase.app/

## Known issues
## Known Issues and Limitations

### Platform Support
- Python 3.13 is not supported due to an Azure SDK compatibility issue: https://github.com/Azure/azure-sdk-for-python/issues/37600
- Python 3.11 or 3.12 is required

### Deployment Limitations
- Interactive model deployment from dropdown menu is only available for AWS (GCP and Azure require YAML files)
- Querying within Magemaker currently only works with text-based models
- Deleting a model is not instant, it may show up briefly after deletion
- Deploying the same model within the same minute will break
- Hugging-face models on Azure have different Ids than their Hugging-face counterparts. Follow the steps specified in the quick-start guide to find the relevant models
- For Azure deploying models other than Hugging-face is not supported yet.
- Python3.13 is not supported because of an open-issue by Azure. https://github.com/Azure/azure-sdk-for-python/issues/37600
- Deploying the same model within the same minute will cause conflicts

### Azure-Specific Issues
- Hugging Face models on Azure have different IDs than their Hugging Face counterparts (follow the steps in the quick-start guide to find model IDs)
- Only Hugging Face models are supported for Azure deployments

### Fine-Tuning Limitations
- Fine-tuning is only supported for AWS SageMaker JumpStart models
- Hugging Face model fine-tuning is not yet implemented
- GCP and Azure fine-tuning are not yet supported


If there is anything we missed, do point them out at https://magemaker.featurebase.app/
Expand Down
32 changes: 27 additions & 5 deletions concepts/deployment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,15 @@ When you run the `magemaker --cloud [aws|gcp|azure|all]` command, you'll get an
magemaker --cloud [aws|gcp|azure|all]
```

<Note>
**Important:** Interactive model deployment from the dropdown menu is currently only supported for AWS. For GCP and Azure deployments, you must use YAML configuration files.
</Note>

This method is great for:

- First-time users
- Exploring available models
- Testing different configurations
- First-time users (AWS only)
- Exploring available models (AWS only)
- Testing different configurations (AWS only)

### YAML-based Deployment

Expand Down Expand Up @@ -62,7 +66,7 @@ deployment: !Deployment
destination: gcp
endpoint_name: opt-125m-gcp
instance_count: 1
machine_type: n1-standard-4
instance_type: n1-standard-4
accelerator_type: NVIDIA_TESLA_T4
accelerator_count: 1

Expand Down Expand Up @@ -104,7 +108,7 @@ models:
source: huggingface
```

### Advanced Configuration
### Advanced Configuration with Prediction Parameters

```yaml
deployment: !Deployment
Expand All @@ -125,6 +129,24 @@ models:
max_new_tokens: 250
```

### Custom Model Deployment

Deploy a fine-tuned or custom model from S3 or local path:

```yaml
deployment: !Deployment
destination: aws
endpoint_name: my-custom-model
instance_count: 1
instance_type: ml.m5.xlarge

models:
- !Model
id: google-bert/bert-base-uncased
source: custom
location: s3://my-bucket/my-model/ # or local path
```

## Cloud-Specific Instance Types

### AWS SageMaker Types
Expand Down
45 changes: 30 additions & 15 deletions concepts/fine-tuning.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@ description: Guide to fine-tuning models with Magemaker

Fine-tuning allows you to adapt pre-trained models to your specific use case. Magemaker simplifies this process through YAML configuration.

<Warning>
**Important Limitations:**
- Fine-tuning is currently **only supported for AWS SageMaker JumpStart models**
- Hugging Face model fine-tuning is NOT yet implemented
- GCP and Azure fine-tuning are NOT yet supported
- Training data must be uploaded to S3
</Warning>

### Basic Command

```sh
Expand All @@ -17,6 +25,8 @@ magemaker --train .magemaker_config/train-config.yaml

### Basic Training Configuration

Example for AWS SageMaker JumpStart model:

```yaml
training: !Training
destination: aws
Expand All @@ -26,27 +36,30 @@ training: !Training

models:
- !Model
id: your-model-id
source: huggingface
id: tensorflow-tc-bert-en-uncased-L-12-H-768-A-12-2
version: 1.0.0
source: sagemaker
```

### Advanced Configuration
### Advanced Configuration with Hyperparameters

```yaml
training: !Training
destination: aws
instance_type: ml.p3.2xlarge
instance_count: 1
training_input_path: s3://your-bucket/data.csv
output_path: s3://your-bucket/model-output/
hyperparameters: !Hyperparameters
epochs: 3
per_device_train_batch_size: 32
learning_rate: 2e-5
weight_decay: 0.01
warmup_steps: 500
evaluation_strategy: "steps"
eval_steps: 500
save_steps: 1000

models:
- !Model
id: tensorflow-tc-bert-en-uncased-L-12-H-768-A-12-2
version: 1.0.0
source: sagemaker
```

## Data Preparation
Expand Down Expand Up @@ -107,19 +120,21 @@ hyperparameters: !Hyperparameters
batch_size: 32
```

### Advanced Tuning
### Supported Hyperparameters

The following hyperparameters are supported and will be merged with the model's default hyperparameters:

```yaml
hyperparameters: !Hyperparameters
epochs: 3
learning_rate:
min: 1e-5
max: 1e-4
scaling: log
batch_size:
values: [16, 32, 64]
per_device_train_batch_size: 32
learning_rate: 2e-5
```

<Note>
Any hyperparameters not specified will use the SageMaker JumpStart model's default values. Additional hyperparameters beyond those defined in the schema may not be supported.
</Note>

## Monitoring Training

### CloudWatch Metrics
Expand Down
32 changes: 25 additions & 7 deletions concepts/models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,13 @@ description: Guide to supported models and their requirements

## Supported Models

Magemaker currently supports two model sources for deployment:

1. **Hugging Face Models** - Deploy any public or gated model from Hugging Face Hub
2. **Custom Models** - Deploy your own fine-tuned models from S3 or local paths

<Note>
Currently, Magemaker supports deployment of Hugging Face models only. Support for cloud provider marketplace models is coming soon!
AWS SageMaker JumpStart models are supported for **fine-tuning only**, not for direct deployment via Magemaker. Cloud provider marketplace models (GCP Vertex AI Model Garden, Azure ML Model Catalog) are not yet supported for deployment.
</Note>

### Hugging Face Models
Expand Down Expand Up @@ -65,17 +70,19 @@ We plan to add support for the following model sources:
#### GCP Vertex AI
1. **Small Models** (n1-standard-4)
```yaml
machine_type: n1-standard-4
instance_type: n1-standard-4
```
2. **Medium Models** (n1-standard-8 + GPU)
```yaml
machine_type: n1-standard-8
instance_type: n1-standard-8
accelerator_type: NVIDIA_TESLA_T4
accelerator_count: 1
```
3. **Large Models** (a2-highgpu-1g)
3. **Large Models** (g2-standard-12 + GPU)
```yaml
machine_type: a2-highgpu-1g
instance_type: g2-standard-12
accelerator_type: NVIDIA_L4
accelerator_count: 1
```

#### Azure ML
Expand Down Expand Up @@ -155,8 +162,19 @@ deployment: !Deployment
models:
- !Model
id: your-model-id
source: huggingface|sagemaker # we don't support vertex and azure specific models yet
revision: latest # Optional: specify model version
source: huggingface # or 'custom' for fine-tuned models
```

### Custom Model Parameters

For deploying custom or fine-tuned models:

```yaml
models:
- !Model
id: google-bert/bert-base-uncased # base model that was fine-tuned
source: custom
location: s3://my-bucket/my-model/ # S3 URI or local path
```

### Advanced Parameters
Expand Down
17 changes: 15 additions & 2 deletions installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,24 @@ title: Installation
description: Configure Magemaker for your cloud provider
---

## Prerequisites

- Python 3.11 or 3.12 (Python 3.13 is NOT supported due to Azure SDK compatibility issues)
- Cloud provider account(s):
- AWS for SageMaker
- GCP for Vertex AI
- Azure for Azure ML
- Cloud CLI tools (recommended):
- AWS CLI (optional for AWS)
- Google Cloud SDK for GCP
- Azure CLI for Azure
- Hugging Face account (for gated models like Llama)

<Note>
For Macs, maxOS >= 13.6.6 is required. Apply Silicon devices (M1) must use Rosetta terminal. You can verify, your terminals architecture by running `arch`. It should print `i386` for Rosetta terminal.
For Macs, maxOS >= 13.6.6 is required. Apple Silicon devices (M1) must use Rosetta terminal. You can verify your terminal's architecture by running `arch`. It should print `i386` for Rosetta terminal.
</Note>

## Installation

Install via pip:

Expand Down Expand Up @@ -47,7 +60,7 @@ magemaker --cloud gcp
### Azure Configuration

- Follow this detailed guide for setting up Azure credentials:
[GCP Setup Guide](/configuration/Azure)
[Azure Setup Guide](/configuration/Azure)


Once you have your Azure credentials, you can configure Magemaker by running:
Expand Down
1 change: 1 addition & 0 deletions mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
"pages": [
"concepts/deployment",
"concepts/models",
"concepts/fine-tuning",
"concepts/contributing"
]
}
Expand Down
39 changes: 24 additions & 15 deletions quick-start.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,22 @@ title: Quick Start

## Interactive View

1. Run Magemaker with your desired cloud provider:
Run Magemaker with your desired cloud provider (the `--cloud` flag is required):

```sh
magemaker --cloud [aws|gcp|azure|all]
```

Supported providers:

- `--cloud aws` AWS SageMaker deployment
- `--cloud gcp` Google Cloud Vertex AI deployment
- `--cloud azure` Azure Machine Learning deployment
- `--cloud all` Configure all three providers at the same time
- `--cloud aws` - AWS SageMaker deployment
- `--cloud gcp` - Google Cloud Vertex AI deployment
- `--cloud azure` - Azure Machine Learning deployment
- `--cloud all` - Configure all three providers at the same time

<Note>
**Important:** Interactive model deployment from the dropdown menu is currently only supported for AWS. For GCP and Azure deployments, you must use YAML configuration files with the `--deploy` flag.
</Note>


### List Models
Expand Down Expand Up @@ -76,20 +80,15 @@ For GCP Vertex AI:
deployment: !Deployment
destination: gcp
endpoint_name: facebook-opt-test
accelerator_count: 1
instance_count: 1
instance_type: g2-standard-12
accelerator_type: NVIDIA_L4
num_gpus: null
quantization: null
accelerator_count: 1

models:
- !Model
id: facebook/opt-125m
location: null
predict: null
source: huggingface
task: null
version: null
```

For Azure ML:
Expand Down Expand Up @@ -137,18 +136,28 @@ Fine-tune models using the `train` command:
magemaker --train .magemaker_config/train-config.yaml
```

Example training configuration:
<Warning>
**Training Limitations:** Fine-tuning is currently only supported for AWS SageMaker JumpStart models. Hugging Face model fine-tuning and GCP/Azure training are not yet implemented.
</Warning>

Example training configuration for AWS SageMaker JumpStart models:

```yaml
training: !Training
destination: aws # or gcp, azure
instance_type: ml.p3.2xlarge # varies by cloud provider
destination: aws
instance_type: ml.p3.2xlarge
instance_count: 1
training_input_path: s3://your-bucket/data.csv
hyperparameters: !Hyperparameters
epochs: 3
per_device_train_batch_size: 32
learning_rate: 2e-5

models:
- !Model
id: tensorflow-tc-bert-en-uncased-L-12-H-768-A-12-2
version: 1.0.0
source: sagemaker
```
{/*
### Recommended Models
Expand Down
Loading