first commit

mapooon · Mar 11, 2024 · 310a25f · 310a25f
commit 310a25f
Show file tree

Hide file tree

Showing 17 changed files with 1,481 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -0,0 +1,60 @@
+# Face2Diffusion (CVPR2024)
+<a href='https://arxiv.org/abs/2403.05094'><img src='https://img.shields.io/badge/ArXiv-PDF-red'></a> &nbsp; 
+<a href='https://mapooon.github.io/Face2DiffusionPage'><img src='https://img.shields.io/badge/Project-Page-Green'></a> &nbsp; 
+![Overview](fig/teaser.png)
+The official PyTorch implementation for the following paper:
+> [**Face2Diffusion for Fast and Editable Face Personalization**](https://arxiv.org/abs/2403.05094),  
+> Kaede Shiohara, Toshihiko Yamasaki,   
+> *CVPR 2024*
+
+# Changelog
+2024/03/11: Released this repository and demo code.
+
+# Recomended Development Environment
+* GPU: NVIDIA A100
+* CUDA: 11.4
+
+
+# Setup
+## 1. Build Docker Image
+Build a docker image
+```bash
+docker build -t face2diffusion
+```
+Execute docker 
+```bash
+docker run -it --gpus all --shm-size 512G \
+-v /path/to/this/repository:/workspace \
+face2diffusion bash
+```
+Install some packages
+```bash
+bash install.sh
+```
+## 2. Download checkpoints
+We provide checkpoints for [mapping network](https://drive.google.com/file/d/1Lf_mwMgme_HVYJCkViGr4TfGOfKw9PhE/view?usp=sharing) and MSID encoder (Coming soon!) and place them to ```checkpoints/```.
+
+# Demo
+Currently, we provide pre-computed identity features from the MSID encoder. The extracted features and the original images can be seen in `./input/` folder. Here is an example to generate images:
+```bash
+python3 inference_f2d.py \
+-w checkpoints/mapping.pt \
+-i input/0.npy \ # input identity
+-p 'f l eating bread in front of the Eiffel Tower' \ # input prompt
+-o output.png \ # output file name
+-n 8 \ # num of images to generate
+```
+Note: The identifier S* should be represented as "f l".
+
+Full pipeline is coming soon!
+
+# Citation
+If you find our work useful for your research, please consider citing our paper:
+```bibtex
+@inproceedings{shiohara2024face2diffusion,
+  title={Face2Diffusion for Fast and Editable Face Personalization},
+  author={Shiohara, Kaede and Yamasaki, Toshihiko},
+  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
+  year={2024}
+}
+```
diff --git a/checkpoints/readme.md b/checkpoints/readme.md
@@ -0,0 +1 @@
+Please place downloaded checkpoints here.
diff --git a/dockerfiles/Dockerfile b/dockerfiles/Dockerfile
@@ -0,0 +1,21 @@
+FROM pytorch/pytorch:1.13.1-cuda11.6-cudnn8-runtime
+
+RUN conda install -c fvcore -c iopath -c conda-forge fvcore iopath
+RUN apt-get update
+RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tzdata
+RUN apt-get install ffmpeg libsm6 libxext6  git -y
+
+RUN pip install opencv-python-headless==4.5.5.64 matplotlib seaborn scikit-learn scikit-image kornia==0.6.8 chumpy
+
+RUN pip install omegaconf pytorch-lightning==1.7.0 einops av
+
+
+RUN pip install transformers==4.19.2
+RUN pip install diffusers invisible-watermark
+RUN pip install --upgrade transformers
+
+RUN python -m pip install --upgrade pip setuptools
+
+RUN pip install fire wandb safetensors
+RUN pip install accelerate==0.16.0 datasets==2.8.0 ftfy==6.1.1 tensorboard==2.11.2 Jinja2==3.1.2
+WORKDIR /workspace
diff --git a/fig/teaser.png b/fig/teaser.png
diff --git a/input/0.jpg b/input/0.jpg
diff --git a/input/0.npy b/input/0.npy
diff --git a/input/1.jpg b/input/1.jpg
diff --git a/input/1.npy b/input/1.npy
diff --git a/input/2.jpg b/input/2.jpg
diff --git a/input/2.npy b/input/2.npy
diff --git a/input/3.jpg b/input/3.jpg
diff --git a/input/3.npy b/input/3.npy
diff --git a/input/4.jpg b/input/4.jpg
diff --git a/input/4.npy b/input/4.npy
diff --git a/install.sh b/install.sh
@@ -0,0 +1,3 @@
+pip install transformers==4.30.2
+cp /workspace/src/modified_scripts/modeling_clip.py /opt/conda/lib/python3.10/site-packages/transformers/models/clip/
+pip install face_alignment timm