From 7d8b951ab78f030dde8e84f2d475a219096a8545 Mon Sep 17 00:00:00 2001 From: "pr-test1[bot]" <226697212+pr-test1[bot]@users.noreply.github.com> Date: Sat, 18 Oct 2025 22:36:18 +0000 Subject: [PATCH 1/9] docs: update about.mdx for changes #1760826977284 --- about.mdx | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/about.mdx b/about.mdx index d9c04a4..c401f3e 100644 --- a/about.mdx +++ b/about.mdx @@ -8,8 +8,20 @@ description: Deploy open source AI models to AWS, GCP, and Azure in minutes Magemaker is a Python tool that simplifies the process of deploying open source AI models to your preferred cloud provider. Instead of spending hours digging through documentation, Magemaker lets you deploy Hugging Face models directly to AWS SageMaker, Google Cloud Vertex AI, or Azure Machine Learning. +## Current Features + +- **Multi-Cloud Deployment**: Deploy to AWS SageMaker, GCP Vertex AI, and Azure ML +- **Hugging Face Integration**: Deploy any public or gated Hugging Face model +- **Custom Model Support**: Deploy fine-tuned models from S3 or local paths +- **Fine-Tuning**: Train AWS SageMaker JumpStart models on your data +- **Interactive CLI**: User-friendly dropdown menus for AWS deployments +- **YAML Configuration**: Infrastructure as Code for reproducible deployments + ## What we're working on next +- Interactive deployment support for GCP and Azure +- Hugging Face model fine-tuning +- GCP and Azure fine-tuning support - More robust error handling for various edge cases - Verbose logging - Enabling / disabling autoscaling @@ -17,14 +29,26 @@ Magemaker is a Python tool that simplifies the process of deploying open source Do submit your feature requests at https://magemaker.featurebase.app/ -## Known issues +## Known Issues and Limitations + +### Platform Support +- Python 3.13 is not supported due to an Azure SDK compatibility issue: https://github.com/Azure/azure-sdk-for-python/issues/37600 +- Python 3.11 or 3.12 is required +### Deployment Limitations +- Interactive model deployment from dropdown menu is only available for AWS (GCP and Azure require YAML files) - Querying within Magemaker currently only works with text-based models - Deleting a model is not instant, it may show up briefly after deletion -- Deploying the same model within the same minute will break -- Hugging-face models on Azure have different Ids than their Hugging-face counterparts. Follow the steps specified in the quick-start guide to find the relevant models -- For Azure deploying models other than Hugging-face is not supported yet. -- Python3.13 is not supported because of an open-issue by Azure. https://github.com/Azure/azure-sdk-for-python/issues/37600 +- Deploying the same model within the same minute will cause conflicts + +### Azure-Specific Issues +- Hugging Face models on Azure have different IDs than their Hugging Face counterparts (follow the steps in the quick-start guide to find model IDs) +- Only Hugging Face models are supported for Azure deployments + +### Fine-Tuning Limitations +- Fine-tuning is only supported for AWS SageMaker JumpStart models +- Hugging Face model fine-tuning is not yet implemented +- GCP and Azure fine-tuning are not yet supported If there is anything we missed, do point them out at https://magemaker.featurebase.app/ From 7561e40e00699481f973c753d2c2f3807b114a78 Mon Sep 17 00:00:00 2001 From: "pr-test1[bot]" <226697212+pr-test1[bot]@users.noreply.github.com> Date: Sat, 18 Oct 2025 22:36:19 +0000 Subject: [PATCH 2/9] docs: update installation.mdx for changes #1760826977284 --- installation.mdx | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/installation.mdx b/installation.mdx index 1d843eb..c091ea1 100644 --- a/installation.mdx +++ b/installation.mdx @@ -3,11 +3,24 @@ title: Installation description: Configure Magemaker for your cloud provider --- +## Prerequisites + +- Python 3.11 or 3.12 (Python 3.13 is NOT supported due to Azure SDK compatibility issues) +- Cloud provider account(s): + - AWS for SageMaker + - GCP for Vertex AI + - Azure for Azure ML +- Cloud CLI tools (recommended): + - AWS CLI (optional for AWS) + - Google Cloud SDK for GCP + - Azure CLI for Azure +- Hugging Face account (for gated models like Llama) - For Macs, maxOS >= 13.6.6 is required. Apply Silicon devices (M1) must use Rosetta terminal. You can verify, your terminals architecture by running `arch`. It should print `i386` for Rosetta terminal. + For Macs, maxOS >= 13.6.6 is required. Apple Silicon devices (M1) must use Rosetta terminal. You can verify your terminal's architecture by running `arch`. It should print `i386` for Rosetta terminal. +## Installation Install via pip: @@ -47,7 +60,7 @@ magemaker --cloud gcp ### Azure Configuration - Follow this detailed guide for setting up Azure credentials: - [GCP Setup Guide](/configuration/Azure) + [Azure Setup Guide](/configuration/Azure) Once you have your Azure credentials, you can configure Magemaker by running: From 13fffa2b4caee67703365ab1ad1570f7f3c819ef Mon Sep 17 00:00:00 2001 From: "pr-test1[bot]" <226697212+pr-test1[bot]@users.noreply.github.com> Date: Sat, 18 Oct 2025 22:36:19 +0000 Subject: [PATCH 3/9] docs: update mint.json for changes #1760826977284 --- mint.json | 1 + 1 file changed, 1 insertion(+) diff --git a/mint.json b/mint.json index ccb1843..2044552 100644 --- a/mint.json +++ b/mint.json @@ -64,6 +64,7 @@ "pages": [ "concepts/deployment", "concepts/models", + "concepts/fine-tuning", "concepts/contributing" ] } From 8db8bc6ec872f6de042fccd170b328f1dea96f0a Mon Sep 17 00:00:00 2001 From: "pr-test1[bot]" <226697212+pr-test1[bot]@users.noreply.github.com> Date: Sat, 18 Oct 2025 22:36:20 +0000 Subject: [PATCH 4/9] docs: update quick-start.mdx for changes #1760826977284 --- quick-start.mdx | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-) diff --git a/quick-start.mdx b/quick-start.mdx index 5853ef8..53542d5 100644 --- a/quick-start.mdx +++ b/quick-start.mdx @@ -8,7 +8,7 @@ title: Quick Start ## Interactive View -1. Run Magemaker with your desired cloud provider: +Run Magemaker with your desired cloud provider (the `--cloud` flag is required): ```sh magemaker --cloud [aws|gcp|azure|all] @@ -16,10 +16,14 @@ magemaker --cloud [aws|gcp|azure|all] Supported providers: -- `--cloud aws` AWS SageMaker deployment -- `--cloud gcp` Google Cloud Vertex AI deployment -- `--cloud azure` Azure Machine Learning deployment -- `--cloud all` Configure all three providers at the same time +- `--cloud aws` - AWS SageMaker deployment +- `--cloud gcp` - Google Cloud Vertex AI deployment +- `--cloud azure` - Azure Machine Learning deployment +- `--cloud all` - Configure all three providers at the same time + + + **Important:** Interactive model deployment from the dropdown menu is currently only supported for AWS. For GCP and Azure deployments, you must use YAML configuration files with the `--deploy` flag. + ### List Models @@ -76,20 +80,15 @@ For GCP Vertex AI: deployment: !Deployment destination: gcp endpoint_name: facebook-opt-test - accelerator_count: 1 + instance_count: 1 instance_type: g2-standard-12 accelerator_type: NVIDIA_L4 - num_gpus: null - quantization: null + accelerator_count: 1 models: - !Model id: facebook/opt-125m - location: null - predict: null source: huggingface - task: null - version: null ``` For Azure ML: @@ -137,18 +136,28 @@ Fine-tune models using the `train` command: magemaker --train .magemaker_config/train-config.yaml ``` -Example training configuration: + + **Training Limitations:** Fine-tuning is currently only supported for AWS SageMaker JumpStart models. Hugging Face model fine-tuning and GCP/Azure training are not yet implemented. + + +Example training configuration for AWS SageMaker JumpStart models: ```yaml training: !Training - destination: aws # or gcp, azure - instance_type: ml.p3.2xlarge # varies by cloud provider + destination: aws + instance_type: ml.p3.2xlarge instance_count: 1 training_input_path: s3://your-bucket/data.csv hyperparameters: !Hyperparameters epochs: 3 per_device_train_batch_size: 32 learning_rate: 2e-5 + +models: +- !Model + id: tensorflow-tc-bert-en-uncased-L-12-H-768-A-12-2 + version: 1.0.0 + source: sagemaker ``` {/* ### Recommended Models From 5bacb780e5f3cee66e09315c5753f17593d51412 Mon Sep 17 00:00:00 2001 From: "pr-test1[bot]" <226697212+pr-test1[bot]@users.noreply.github.com> Date: Sat, 18 Oct 2025 22:36:21 +0000 Subject: [PATCH 5/9] docs: update tutorials/deploying-llama-3-to-azure.mdx for changes #1760826977284 --- tutorials/deploying-llama-3-to-azure.mdx | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/tutorials/deploying-llama-3-to-azure.mdx b/tutorials/deploying-llama-3-to-azure.mdx index 679ba23..9561d00 100644 --- a/tutorials/deploying-llama-3-to-azure.mdx +++ b/tutorials/deploying-llama-3-to-azure.mdx @@ -3,7 +3,11 @@ title: Deploying Llama 3 to Azure --- ## Introduction -This tutorial guides you through deploying Llama 3 to Azure ML platform using Magemaker and querying it using the interactive dropdown menu. Ensure you have followed the [installation](installation) steps before proceeding. +This tutorial guides you through deploying Llama 3 to Azure ML platform using Magemaker. Ensure you have followed the [installation](installation) steps before proceeding. + + + **Important:** Azure deployments currently require YAML configuration files. Interactive dropdown deployment is only available for AWS. + You may need to request a quota increase for specific machine types and GPUs in the region where you plan to deploy the model. Check your Azure quotas before proceeding. @@ -40,16 +44,12 @@ deployment: !Deployment destination: azure endpoint_name: llama3-endpoint instance_count: 1 - instance_type: Standard_NC24ads_A100_v4 + instance_type: Standard_NC24ads_A100_v4 models: - !Model - id: meta-llama-meta-llama-3-8b-instruct - location: null - predict: null + id: meta-llama-Meta-Llama-3-8B-Instruct source: huggingface - task: text-generation - version: null ``` @@ -139,5 +139,5 @@ print(resp) ``` ## Conclusion -You have successfully deployed and queried Llama 3 on Azure using Magemaker's interactive dropdown menu. For any questions or feedback, feel free to contact us at [support@slashml.com](mailto:support@slashml.com). +You have successfully deployed and queried Llama 3 on Azure using Magemaker. For any questions or feedback, feel free to contact us at [support@slashml.com](mailto:support@slashml.com). From 33d9706ad0b3c718100f742342bf70d789768f77 Mon Sep 17 00:00:00 2001 From: "pr-test1[bot]" <226697212+pr-test1[bot]@users.noreply.github.com> Date: Sat, 18 Oct 2025 22:36:21 +0000 Subject: [PATCH 6/9] docs: update tutorials/deploying-llama-3-to-gcp.mdx for changes #1760826977284 --- tutorials/deploying-llama-3-to-gcp.mdx | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/tutorials/deploying-llama-3-to-gcp.mdx b/tutorials/deploying-llama-3-to-gcp.mdx index a94d616..7677c7e 100644 --- a/tutorials/deploying-llama-3-to-gcp.mdx +++ b/tutorials/deploying-llama-3-to-gcp.mdx @@ -3,7 +3,11 @@ title: Deploying Llama 3 to GCP --- ## Introduction -This tutorial guides you through deploying Llama 3 to Google Cloud Platform (GCP) Vertex AI using Magemaker and querying it using the interactive dropdown menu. Ensure you have followed the [installation](installation) steps before proceeding. +This tutorial guides you through deploying Llama 3 to Google Cloud Platform (GCP) Vertex AI using Magemaker. Ensure you have followed the [installation](installation) steps before proceeding. + + + **Important:** GCP deployments currently require YAML configuration files. Interactive dropdown deployment is only available for AWS. + You may need to request a quota increase for specific machine types and GPUs in the region where you plan to deploy the model. Check your GCP quotas before proceeding. @@ -33,20 +37,15 @@ Example YAML for GCP deployment: deployment: !Deployment destination: gcp endpoint_name: llama3-endpoint - accelerator_count: 1 + instance_count: 1 instance_type: n1-standard-8 - accelerator_type: NVIDIA_T4 - num_gpus: 1 - quantization: null + accelerator_type: NVIDIA_TESLA_T4 + accelerator_count: 1 models: - !Model id: meta-llama/Meta-Llama-3-8B-Instruct - location: null - predict: null source: huggingface - task: text-generation - version: null ``` For gated models like llama from Meta, you have to accept terms of use for model on hugging face and adding Hugging face token to the environment are necessary for deployment to go through. @@ -54,7 +53,7 @@ models: ### Selecting an Appropriate Instance -For Llama 3, a machine type such as `n1-standard-8` with an attached NVIDIA T4 GPU (`NVIDIA_T4`) is a suitable configuration for most use cases. Adjust the instance type and GPU based on your workload requirements. +For Llama 3, a machine type such as `n1-standard-8` with an attached NVIDIA T4 GPU (`NVIDIA_TESLA_T4`) is a suitable configuration for most use cases. For larger models or better performance, consider using `g2-standard-12` with `NVIDIA_L4` GPUs. Adjust the instance type and GPU based on your workload requirements. If you encounter quota issues, submit a quota increase request in the GCP console under "IAM & Admin > Quotas" for the specific GPU type in your deployment region. @@ -142,5 +141,5 @@ print(resp) ``` ## Conclusion -You have successfully deployed and queried Llama 3 on GCP Vertex AI using Magemaker's interactive dropdown menu. For any questions or feedback, feel free to contact us at [support@slashml.com](mailto:support@slashml.com). +You have successfully deployed and queried Llama 3 on GCP Vertex AI using Magemaker. For any questions or feedback, feel free to contact us at [support@slashml.com](mailto:support@slashml.com). From 43112ae92b8a7c03471976b9c604f891feab83da Mon Sep 17 00:00:00 2001 From: "pr-test1[bot]" <226697212+pr-test1[bot]@users.noreply.github.com> Date: Sat, 18 Oct 2025 22:36:22 +0000 Subject: [PATCH 7/9] docs: update concepts/deployment.mdx for changes #1760826977284 --- concepts/deployment.mdx | 32 +++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-) diff --git a/concepts/deployment.mdx b/concepts/deployment.mdx index 66ca7a9..1c06e90 100644 --- a/concepts/deployment.mdx +++ b/concepts/deployment.mdx @@ -15,11 +15,15 @@ When you run the `magemaker --cloud [aws|gcp|azure|all]` command, you'll get an magemaker --cloud [aws|gcp|azure|all] ``` + + **Important:** Interactive model deployment from the dropdown menu is currently only supported for AWS. For GCP and Azure deployments, you must use YAML configuration files. + + This method is great for: -- First-time users -- Exploring available models -- Testing different configurations +- First-time users (AWS only) +- Exploring available models (AWS only) +- Testing different configurations (AWS only) ### YAML-based Deployment @@ -62,7 +66,7 @@ deployment: !Deployment destination: gcp endpoint_name: opt-125m-gcp instance_count: 1 - machine_type: n1-standard-4 + instance_type: n1-standard-4 accelerator_type: NVIDIA_TESLA_T4 accelerator_count: 1 @@ -104,7 +108,7 @@ models: source: huggingface ``` -### Advanced Configuration +### Advanced Configuration with Prediction Parameters ```yaml deployment: !Deployment @@ -125,6 +129,24 @@ models: max_new_tokens: 250 ``` +### Custom Model Deployment + +Deploy a fine-tuned or custom model from S3 or local path: + +```yaml +deployment: !Deployment + destination: aws + endpoint_name: my-custom-model + instance_count: 1 + instance_type: ml.m5.xlarge + +models: + - !Model + id: google-bert/bert-base-uncased + source: custom + location: s3://my-bucket/my-model/ # or local path +``` + ## Cloud-Specific Instance Types ### AWS SageMaker Types From bb15c8c7c712d345ac2f86a28ff14ed2e9dc3c88 Mon Sep 17 00:00:00 2001 From: "pr-test1[bot]" <226697212+pr-test1[bot]@users.noreply.github.com> Date: Sat, 18 Oct 2025 22:36:22 +0000 Subject: [PATCH 8/9] docs: update concepts/fine-tuning.mdx for changes #1760826977284 --- concepts/fine-tuning.mdx | 45 ++++++++++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 15 deletions(-) diff --git a/concepts/fine-tuning.mdx b/concepts/fine-tuning.mdx index 88835aa..eb4e7c8 100644 --- a/concepts/fine-tuning.mdx +++ b/concepts/fine-tuning.mdx @@ -7,6 +7,14 @@ description: Guide to fine-tuning models with Magemaker Fine-tuning allows you to adapt pre-trained models to your specific use case. Magemaker simplifies this process through YAML configuration. + + **Important Limitations:** + - Fine-tuning is currently **only supported for AWS SageMaker JumpStart models** + - Hugging Face model fine-tuning is NOT yet implemented + - GCP and Azure fine-tuning are NOT yet supported + - Training data must be uploaded to S3 + + ### Basic Command ```sh @@ -17,6 +25,8 @@ magemaker --train .magemaker_config/train-config.yaml ### Basic Training Configuration +Example for AWS SageMaker JumpStart model: + ```yaml training: !Training destination: aws @@ -26,11 +36,12 @@ training: !Training models: - !Model - id: your-model-id - source: huggingface + id: tensorflow-tc-bert-en-uncased-L-12-H-768-A-12-2 + version: 1.0.0 + source: sagemaker ``` -### Advanced Configuration +### Advanced Configuration with Hyperparameters ```yaml training: !Training @@ -38,15 +49,17 @@ training: !Training instance_type: ml.p3.2xlarge instance_count: 1 training_input_path: s3://your-bucket/data.csv + output_path: s3://your-bucket/model-output/ hyperparameters: !Hyperparameters epochs: 3 per_device_train_batch_size: 32 learning_rate: 2e-5 - weight_decay: 0.01 - warmup_steps: 500 - evaluation_strategy: "steps" - eval_steps: 500 - save_steps: 1000 + +models: +- !Model + id: tensorflow-tc-bert-en-uncased-L-12-H-768-A-12-2 + version: 1.0.0 + source: sagemaker ``` ## Data Preparation @@ -107,19 +120,21 @@ hyperparameters: !Hyperparameters batch_size: 32 ``` -### Advanced Tuning +### Supported Hyperparameters + +The following hyperparameters are supported and will be merged with the model's default hyperparameters: ```yaml hyperparameters: !Hyperparameters epochs: 3 - learning_rate: - min: 1e-5 - max: 1e-4 - scaling: log - batch_size: - values: [16, 32, 64] + per_device_train_batch_size: 32 + learning_rate: 2e-5 ``` + + Any hyperparameters not specified will use the SageMaker JumpStart model's default values. Additional hyperparameters beyond those defined in the schema may not be supported. + + ## Monitoring Training ### CloudWatch Metrics From 4832e33dfb9ffa43b9c3783cac4a43073f5c3f46 Mon Sep 17 00:00:00 2001 From: "pr-test1[bot]" <226697212+pr-test1[bot]@users.noreply.github.com> Date: Sat, 18 Oct 2025 22:36:23 +0000 Subject: [PATCH 9/9] docs: update concepts/models.mdx for changes #1760826977284 --- concepts/models.mdx | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/concepts/models.mdx b/concepts/models.mdx index 0161380..5f6303e 100644 --- a/concepts/models.mdx +++ b/concepts/models.mdx @@ -5,8 +5,13 @@ description: Guide to supported models and their requirements ## Supported Models +Magemaker currently supports two model sources for deployment: + +1. **Hugging Face Models** - Deploy any public or gated model from Hugging Face Hub +2. **Custom Models** - Deploy your own fine-tuned models from S3 or local paths + -Currently, Magemaker supports deployment of Hugging Face models only. Support for cloud provider marketplace models is coming soon! +AWS SageMaker JumpStart models are supported for **fine-tuning only**, not for direct deployment via Magemaker. Cloud provider marketplace models (GCP Vertex AI Model Garden, Azure ML Model Catalog) are not yet supported for deployment. ### Hugging Face Models @@ -65,17 +70,19 @@ We plan to add support for the following model sources: #### GCP Vertex AI 1. **Small Models** (n1-standard-4) ```yaml - machine_type: n1-standard-4 + instance_type: n1-standard-4 ``` 2. **Medium Models** (n1-standard-8 + GPU) ```yaml - machine_type: n1-standard-8 + instance_type: n1-standard-8 accelerator_type: NVIDIA_TESLA_T4 accelerator_count: 1 ``` -3. **Large Models** (a2-highgpu-1g) +3. **Large Models** (g2-standard-12 + GPU) ```yaml - machine_type: a2-highgpu-1g + instance_type: g2-standard-12 + accelerator_type: NVIDIA_L4 + accelerator_count: 1 ``` #### Azure ML @@ -155,8 +162,19 @@ deployment: !Deployment models: - !Model id: your-model-id - source: huggingface|sagemaker # we don't support vertex and azure specific models yet - revision: latest # Optional: specify model version + source: huggingface # or 'custom' for fine-tuned models +``` + +### Custom Model Parameters + +For deploying custom or fine-tuned models: + +```yaml +models: +- !Model + id: google-bert/bert-base-uncased # base model that was fine-tuned + source: custom + location: s3://my-bucket/my-model/ # S3 URI or local path ``` ### Advanced Parameters