Skip to content

Avatar Generation For Characters and Game Assets Using Deep Fakes

License

Notifications You must be signed in to change notification settings

Yazdi9/Talking_Face_Avatar

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

30 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

"Buy Me A Coffee"

Open In Colab ย 

Talking Face Avatar:ย ย ย ย ย  single portrait image From Leonardo.ai API ๐Ÿ™Žโ€โ™‚๏ธ ย ย ย ย ย + ย ย ย ย ย  audio From ElevenLabs TTS API ๐ŸŽค ย ย ย ย ย  = ย ย ย ย ย  talking head video ๐ŸŽž.


Leonardo.ai

Go To Leonardo.Ai And Enter your Prompt And Negative Prompts To Generate Artistic Images

Here Some Recources :Leonardo.ai Youtube Video Leonardo.ai Youtube Video Toutorial

or you can use APIs Leonardo.Ai API Guide

Leonardo.ai Image Generation Leonardo.ai Image Generation Leonardo.ai Image Generation

ElevenLabs

Go To Eleven Labs And Enter your Text And Generate Beautiful Audios With Diffrent Pitchs and Speeckers. ElvenLabs also is Multilingual

Here Some Recources :ElevenLabs Youtube Video

or you can use APIs ElevenLabs API Guide

ElevenLabs Python Repo

Eleven Labs TTS Eleven Labs TTS Eleven Labs TTS
output-_6_.mp4
output.5.mp4
output.1.mp4

๐Ÿ”ฅ Highlight

-๐Ÿ”ฅ Scroll To left and Right To See All Videos

video 1 + enhancer(GFPGAN ) video 2 video 3
RPG_40_Female_Astronaut_model_soft_natural_lighting_forest_win_0.output5_enhanced.mp4
Deliberate_11_hyperrealistic_portrait_of_a_beautiful_white_wom_0.output-_6__enhanced.mp4
DreamShaper_32_Clara_Crawford_photorealistic_beautiful_woman_l_1.saba_enhanced.mp4
video 4 video 5 video 6
RPG_40_Portrait_of_beautiful_lady_little_pojatti_realistic_stu_0.output-_7__enhanced.mp4
Deliberate_11_a_hyper_realistic_ultra_detailed_photograph_of_a_1.output3_enhanced.mp4
RPG_40_hyperrealistic_photo_of_a_beautiful_white_woman_upper_b_0.output_enhanced.mp4
  • ๐Ÿ”ฅ Several new mode, eg, still mode, reference mode, resize mode are online for better and custom applications.

Our Diagram Approach

you_doodle_pro_2023-05-19t16_25_09z

Linux:

  1. Installing anaconda, python and git.

  2. Creating the env and install the requirements.

git clone https://github.com/saba99/Talking_Face_Avatar.git

cd SadTalker 

conda create -n sadtalker python=3.8

conda activate sadtalker

pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113

conda install ffmpeg

pip install -r requirements.txt

### tts is optional for gradio demo. 
### pip install TTS

UI + API:

look at index.html

Screenshot (4899)

๐Ÿ“ฅ 2. Download Trained Models.

You can run the following script to put all the models in the right place.

bash scripts/download_models.sh
Model Details

The final folder will be shown as:

image

Model explains:

Model Description
checkpoints/auido2exp_00300-model.pth Pre-trained ExpNet in Sadtalker.
checkpoints/auido2pose_00140-model.pth Pre-trained PoseVAE in Sadtalker.
checkpoints/mapping_00229-model.pth.tar Pre-trained MappingNet in Sadtalker.
checkpoints/mapping_00109-model.pth.tar Pre-trained MappingNet in Sadtalker.
checkpoints/facevid2vid_00189-model.pth.tar Pre-trained face-vid2vid model from the reappearance of face-vid2vid.
checkpoints/epoch_20.pth Pre-trained 3DMM extractor in Deep3DFaceReconstruction.
checkpoints/wav2lip.pth Highly accurate lip-sync model in Wav2lip.
checkpoints/shape_predictor_68_face_landmarks.dat Face landmark model used in dilb.
checkpoints/BFM 3DMM library file.
checkpoints/hub Face detection models used in face alignment.
gfpgan/weights Face detection and enhanced models used in facexlib and gfpgan.

๐Ÿ”ฎ 3. Quick Start (Best Practice).

WebUI Demos:

SDWebUI-Colab | Colab

## you need manually install TTS(https://github.com/coqui-ai/TTS) via `pip install tts` in advanced.
python app.py

Manually usages:

Animating a portrait image from default config:
python inference.py --driven_audio <audio.wav> \
                    --source_image <video.mp4 or picture.png> \
                    --enhancer gfpgan 

The results will be saved in results/$SOME_TIMESTAMP/*.mp4.

Full body/image Generation:

Using --still to generate a natural full body video. You can add enhancer to improve the quality of the generated video.

python inference.py --driven_audio <audio.wav> \
                    --source_image <video.mp4 or picture.png> \
                    --result_dir <a file to store results> \
                    --still \
                    --preprocess full \
                    --enhancer gfpgan 

Star History Chart