Skip to content

Latest commit

 

History

History
283 lines (197 loc) · 15.6 KB

README.md

File metadata and controls

283 lines (197 loc) · 15.6 KB
Raw

English | 日本語 | 中文

ComfyUI on AWS

License: MIT

This sample repository provides a seamless and cost-effective solution to deploy ComfyUI, a powerful AI-driven image generation tool, on AWS. This repository provides a comprehensive infrastructure code and configuration setup, leveraging the power of ECS, EC2, and other AWS services. Experience a hassle-free deployment process while enjoying uncompromised security and scalability.

💡 Note: this solution will incur AWS costs. You can find more information about it in the costs section.

comfy comfy gallery

Solution Features

  1. Effortless Deployment 🚀: Harness the power of Cloud Development Kit (CDK) for a streamlined and automated deployment process.
  2. Cost Optimization 💰: Leverage cost-saving options like Spot Instances, Automatic Shutdown, and Scheduled Scaling to maximize your budget efficiency.
  3. Robust Security 🔒: Enjoy peace of mind with robust security measures, including Authentication (with SAML such as Microsoft Entra ID / Google Workspace), Email Domain Restriction, IP Restriction, Custom Domain SSL, Security Scans, etc.

Architecture Overview

AWS Architecture

Services

Getting Started

Prepare the AWS environment

For the sake of reproducability and consistency, we recommend using Amazon SageMaker Studio Code Editor for deploying and testing this solution.

ℹ️ You can use your local development environment, but you will need to make sure that you have AWS CLI, AWS CDK and Docker properly setup.

Click to see environment estup with Amazon SageMaker Studio Code Editor
  1. Launch Amazon SageMaker Studio Code Editor using CloudFormation template from link in sagemaker-studio-code-editor-template. (This template launches Code Editor with some necessary capabilities including Docker, auto termination)
  2. Open SageMaker Studio from url in CloudFormation Output.
  3. Navigate to Code Editor from Application section in top left.
Click to see environment setup with Local environment

If you do not have AWS CLI, follow AWS CLI Install Guide

If you do not have CDK, follow CDK Start Guide

If you do not have Docker follow Docker Install Guide

If you haven't setup AWS CLI after installation, execute the following commands on your local environment:

aws configure

When prompted, enter your AWS Access Key ID, Secret Access Key, and then the default region name (eg. us-east-1). You can leave the output format field as default or specify it as per your preference.

Note

Make sure your account has quota for GPU instance. Go to Service Quota and set All G and VT Spot Instance Requests to at least 4.

Deploying ComfyUI

  1. (First time only) Clone this repo (git clone https://github.com/aws-samples/cost-effective-aws-deployment-of-comfyui.git)
  2. (First time only) cd into repo directory (cd cost-effective-aws-deployment-of-comfyui)
  3. Run make

Depending on your custom_nodes and extenstions in the dockerfile, the deployment will take approx. 8-10 minutes to have ComfyUI ready

 ✅  ComfyUIStack

✨  Deployment time: 579.07s

Outputs:
ComfyUIStack.CognitoDomainName = comfyui-alb-auth-XXXXXXX
ComfyUIStack.Endpoint = ComfyUiALB-XXXXX.uw-west-2.elb.amazonaws.com
ComfyUIStack.UserPoolId = us-west-2_XXXXXXX
Stack ARN:
arn:aws:cloudformation:[us-east-1]:[your-account-id]:stack/ComfyUIStack/[uuid]

✨  Total time: 582.53s

You can access application from output value of ComfyUIStack.Endpoint.

Uploading models

  1. You can install models, loras, embedding, controlnets over ComfyUI-Manager or other extension (custom node). See User Guide for detail.
  2. You can extend (optional) and execute the upload script in this repo with a preselected list of models, controlnets etc. If the SSM command is not working, make sure that the role you are using is allowed to access the EC2. You'll find some additional examples in the /scripts/upload_models.sh file.
# 1. SSM into EC2
aws ssm start-session --target "$(aws ec2 describe-instances --filters "Name=tag:Name,Values=ComfyUIStack/Host" "Name=instance-state-name,Values=running" --query 'Reservations[].Instances[].[InstanceId]' --output text)" --region $AWS_DEFAULT_REGION

# 2. SSH into Container
container_id=$(sudo docker container ls --format '{{.ID}} {{.Image}}' | grep 'comfyui:latest$' | awk '{print $1}')
sudo docker exec -it $container_id /bin/bash

# 3. install models, loras, controlnets or whatever you need (you can also include all in a script and execute it to install)

# FACE SWAP EXAMPLE Upscaler - https://huggingface.co/ai-forever/Real-ESRGAN
wget -c https://huggingface.co/ai-forever/Real-ESRGAN/blob/main/RealESRGAN_x2.pth -P ./models/upscale_models/

Access ComfyUI

The deployed solution provides an EC2 accessible through an Application Load Balancer. The Load Balancer requires authentication through Amazon Cognito User Pool.

You may enable self-signup, enable SAML authentication, or manually create user in Cognito console.

User Guide

To unlock the full potential of ComfyUI and ensure a seamless experience, explore our detailed User Guide. This comprehensive resource will guide you through every step, from installation to advanced configurations, empowering you to harness the power of AI-driven image generation with ease.

Deploy Option

With our comprehensive Deploy Options, you have the power to craft a tailored solution that aligns perfectly with your security requirements, and budget constraints. Unlock the full potential of ComfyUI on AWS with unparalleled flexibility and control.You can enable following features with just few steps.

Delete deployments and cleanup resources

For the sake of preventing data loss from accidental deletions and keeping the example as straightforward as possible, the deletion of the complete deployment and resources is semi-automated. To cleanup and remove everything you've deployed you need to do following:

  1. Delete the Auto Scaling Group manually:
  • Login to your AWS console
  • Search for Auto Scaling Groups (EC2 featuer) in the search bar
  • Select ComfyASG
  • Press Actions and then delete
  • Confirm deletion
  1. After ASG deletion you just can run following command in your terminal. This command will delete all remaining resources, but EBS and the Cognito User pool.
npx cdk destroy
  1. Delete EBS Volume
  • Login to your AWS console
  • Search Volumes (EC2 featuer) in the search bar
  • Select ComfyUIVolume
  • Press Actions and then delete
  • Confirm deletion
  1. Delete Cognito User Pool
  • Login to your AWS console
  • Search for Cognito in the serach bar
  • Select ComfyUIuserPool..
  • Press delete
  • Confirm deletion
  1. Delete ECR Repository
  • Login to your AWS console
  • Search for ECR (Elastic Container Regsitry) in the search bar
  • Select comfyui
  • Press delete
  • Type delete to confirm deletion

Notes and Additional Information

Cost Estimation

This section provides cost estimations for running the application on AWS. Please note that these are rough estimations, and you should refine them based on your project's specific requirements and usage patterns.

Flexible Workload (Default)

For non-critical business workloads, which should apply to the majority of applications of this type, you can use Spot Instances to benefit from cost savings. Spot Instances offer an average historical discount of 71% (us-east-1, October 2024) for the g4dn.xlarge instance type. Additionally, you can replace the NAT Gateway with a NAT Instance to further reduce costs.

The following assumptions are made for the cost estimation:

  • No services from the AWS Free Tier are included.
  • Instance Type: g4dn.xlarge with 4 vCPU, 16 GiB memory, and 1 Nvidia T4 Tensor Core GPU (Spot Instance with 71% discount).
  • 250 GB SSD storage.
  • 1 Application Load Balancer.
  • VPC with NAT Instance.
  • Elastic Container Registry (ECR) with 10 GB of data stored per month.
  • 5 GB of logging data per month.
Service \ Runtime 2h/day Mon-Fri 8h/day Mon-Fri 12h/day Mon-Fri 24/7
Compute $7 $26 $40 $111
Storage - - - $20
ALB - - - $20
Networking - - - $6
Registry - - - $1
Logging - - - $3
Total Monthly Cost $60 $79 $93 $164

Business-Critical Workload

For business-critical workloads, you can use On-Demand instances and a NAT Gateway for high availability.

The following assumptions are made for the cost estimation:

  • Instance Type: g4dn.xlarge with 4 vCPU, 16 GiB memory, and 1 Nvidia T4 Tensor Core GPU (On-Demand pricing).
  • VPC with 50 GB of data processed per NAT Gateway per month.
  • Other assumptions are the same as the Flexible Workload scenario.
Service \ Runtime 2h/day Mon-Fri 8h/day Mon-Fri 12h/day Mon-Fri 24/7
Compute $23 $91 $137 $384
Storage - - - $20
ALB - - - $20
Networking - - - $70
Registry - - - $1
Logging - - - $3
Total Monthly Cost $137 $205 $251 $498

Useful Commands

  • npx cdk ls list all stacks in the app
  • npx cdk synth emits the synthesized CloudFormation template
  • npx cdk deploy deploy this stack to your default AWS account/region
  • npx cdk destroy destroy the deployed stack in your default AWS account/region
  • npx cdk diff compare deployed stack with current state
  • npx cdk docs open CDK documentation

Q&A

Does the Dockerfile already pre-install models?

Dockerfile includes only ComfyUI and ComfyUI-Manager. To install models either go over ComfyUI-Manager after deployment or over the section Upload Models.

Can I contribute to this project?

Yes, feel free to follow the contribution guide.

Can this be consiered for production deployments?

Consider this setup as an sample deployment for personal or non-production use.

Contributors

contributors

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.