Stable Diffusion XL using DreamBooth and LoRA

This project is an implementation of fine-tuning a foundation SDXL model using DreamBooth and LoRA on custom data of interior room designs to generate:

• Text-to-image
• Image-to-image
• Image-to-video

Pipeline:

Text-to-image generation using fine-tuned LoRA

Prompt: A photo of TOK home, an Indian living room basking in Republic Day morning sun, adorned with saffron, white, and green decor, and vibrant festive accents. You can try out this model for inference here.

Image-to-image generation using fine-tuned LoRA

Input Image	Output Image

Stable Video Diffusion

Image to Video generation.

Resources:

1. Try it yourself on Hugging Face: https://huggingface.co/NSTiwari/SDXL_LoRA_model
2. Demo video: https://youtu.be/X_p0E2vBRjY
3. Blog: https://aashi-dutt3.medium.com/ai-architect-design-your-homes-with-stable-diffusion-b1c61644c61e