This repo compares images generated by awesome generative models on the DrawBench provided in Imagen. Currently, it includes images generated with some OpenSource image generative models (stable diffusion 1-4, 1-5, 2-1). One can browse the generated images and get a rough idea about the capability of these models.
- The repo includes results generated by 3 stable diffusion models (1-4, 1-5, 2-1) with 200 prompts from DrawBench, organized by prompt categories.
- The provided scripts can be used to generate images with additional prompts and private models and provide organized results (the README files in each sub-directory display the results).
- Currently, this repo only includes 3 OpenSource stable diffusion models.
- Currently, this repo only includes images generated with one random seed (
0), which makes the comparison not comprehensive.
- Create a conda environment:
conda env create -f bench.yaml. - Activate the conda environment:
conda activate bench. - Run the notebook
GenImages.ipynb. It takes ~30 minutes to generate a single image for 200 prompts with 3 different diffusion models on a single RTX3090.
Each sub-directory includes the generation results of a single prompt category. Some examples are provided below as a preview.
| Category: Color | ||
| 1:A red colored car. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |
| Category: Conflicting | ||
| 26:A horse riding an astronaut. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |
| Category: Counting | ||
| 36:One car on the street. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |
| Category: DALL-E | ||
| 55:A triangular purple flower pot. A purple flower pot in the shape of a triangle. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |
| Category: Descriptions | ||
| 75:A vehicle composed of two wheels held in a frame one behind the other, propelled by pedals and steered with handlebars attached to the front wheel. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |
| Category: Gary Marcus et al. | ||
| 95:Paying for a quarter-sized pizza with a pizza-sized quarter. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |
| Category: Misspellings | ||
| 105:Tcennis rpacket. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |
| Category: Positional | ||
| 115:A train on top of a surfboard. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |
| Category: Rare Words | ||
| 135:Acersecomicke. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |
| Category: Reddit | ||
| 142:A church with stained glass windows depicting a hamburger and french fries. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |
| Category: Text | ||
| 180:A storefront with 'Hello World' written on it. | ||
 |
 |
 |
| CompVis-stable-diffusion-v1-4 | runwayml-stable-diffusion-v1-5 | stabilityai-stable-diffusion-2-1 |