Skip to content

Latest commit

 

History

History
138 lines (95 loc) · 4 KB

stable-diffusion-lambda-diffuser.mdx

File metadata and controls

138 lines (95 loc) · 4 KB
Raw
title description image authorUsername
Stable Diffusion tutorial: Stable Diffusion Image Variations using lambda diffusers
A step by step tutorial how to generate variations on an input image using a fine-tuned version of Stable Diffusion
omarsesa930

🧐 Short introduction to Stable Diffusion

Stable Diffusion is a text-to-image latent diffusion model created by the researchers and engineers from CompVis, Stability AI and LAION. It's trained on 512x512 images from a subset of the LAION-5B database.

About lambda diffusers

This version of Stable Diffusion has been fine tuned from CompVis/stable-diffusion-v1-3-original to accept CLIP image embedding rather than text embeddings. This allows the creation of "image variations" similar to DALLE-2 using Stable Diffusion. This version of the weights has been ported to huggingface Diffusers, to use this with the Diffusers library requires the Lambda Diffusers repo.

Let's get started and learn how to use Stable Diffusion Image Variations using lambda diffusers!

To run this tutorial we will use Google Colab and Google Drive

⚙️ Preparing dependencies

Download the necessary files:

%cd /content
!git clone https://github.com/LambdaLabsML/lambda-diffusers.git

%cd /content/lambda-diffusers
!git checkout 4d4386e5d2ae667d18987b1a505a47efc643e9c9

install the required libs:

%cd /content/lambda-diffusers


# for lambda-diffusers
!pip install -r requirements.txt

Import the required libs

%cd /content/lambda-diffusers

from pathlib import Path
from lambda_diffusers import StableDiffusionImageEmbedPipeline
from PIL import Image
import torch
from torch import autocast

from diffusers import StableDiffusionPipeline
from google.colab.patches import cv2_imshow
import cv2



device = "cuda" if torch.cuda.is_available() else "cpu"

🎥 Image to Image

Load the pipeline.

pipe = StableDiffusionImageEmbedPipeline.from_pretrained("lambdalabs/sd-image-variations-diffusers")
pipe = pipe.to(device)

download the intial image:

!rm -rf outputs
!rm -rf inputs
!mkdir outputs inputs

!wget -c https://raw.githubusercontent.com/Omarsesa/Lablab/main/R.jpg \
      -O inputs/test_01.jpg
cv2_imshow(cv2.imread('inputs/test_01.jpg'))

## Generate the images 
input_img_path = "inputs/test_01.jpg"

im = Image.open(input_img_path)
num_samples = 2
image = pipe(num_samples*[im], guidance_scale=3.0)
image = image["sample"]

base_path = Path("outputs")
base_path.mkdir(exist_ok=True, parents=True)
for idx, im in enumerate(image):
  im.save(base_path/f"{idx:06}.jpg")
  1. we load the image
  2. run the model
  3. save the output images

Show the image

Here we resize the images then concat them horizontally and show them

def hconcat_resize_min(im_list, interpolation=cv2.INTER_CUBIC):
  h_min = min(im.shape[0] for im in im_list)
  im_list_resize = [cv2.resize(im, (int(im.shape[1] * h_min / im.shape[0]), h_min), interpolation=interpolation) for im in im_list]
  
  return cv2.hconcat(im_list_resize)
kinput_cv_im = cv2.imread(input_img_path)

im_h = None
for i in range(num_samples):
  cv_im = cv2.imread(f"outputs/{i:06}.jpg")
  
  if i == 0:
    im_h = hconcat_resize_min([input_cv_im, cv_im])
  else:
    im_h = cv2.hconcat([im_h, cv_im])
    #pass

cv2_imshow(im_h)

As you can see here we have different variations of the intial image

Open in Colab

A big thank you to Hassen Shair for helping me write the tutorial!

Thanks for reading! Wait for next tutorials!