Retargeting Poses for Non-Standard Skeletons (e.g., Cats) Leading to Distorted Animations #5
Description
Issue Summary:
During my experimentation with DynamicPose, I encountered significant challenges when retargeting poses to non-standard skeletons, particularly with images of cats. The retargeting algorithm fails to accurately map the shorter leg bones of cats, resulting in disproportionately elongated legs in the generated animations. This leads to unrealistic and undesired video outputs.
Detailed Description:
When attempting to retarget a pose from a human to a cat, the algorithm does not account for the anatomical differences between humans and cats, especially the shorter and differently proportioned limbs of cats. As a result, the legs in the retargeted animation appear excessively long and unnatural, compromising the overall quality and realism of the animation.
In contrast, I tested MusePose with the same setup, and their retargeting algorithm handled the pose mapping for cats much more effectively. MusePose successfully preserved the correct limb proportions
Retarget in MusePose:
img_cat3_video_dance_1.mp4
Comparison dance_1 (First video is MusePose, Second video is DynamicPose)
cat3_img_cat3_video_Tik_0_old_3.5_20_1__.mp4
cat3_Tik_0_old_784x512_3_1030.mp4
Comparison dance_2 (First video is MusePose, Second video is DynamicPose)
cat3_img_cat3_video_dance_1_3.5_20_1__.mp4
cat3_dance_1_784x512_3_1030.mp4
Expected Behavior:
The retargeting algorithm should accurately map the human pose to the cat's skeleton, maintaining realistic limb proportions. Specifically, the shorter legs of the cat should be preserved in the animation, ensuring the resulting video appears natural and faithful to the original anatomical structure of the cat.
Actual Behavior:
The retargeted pose results in disproportionately long legs for the cat, making the animation appear unrealistic and unnatural. The anatomical discrepancies between humans and cats are not adequately addressed, leading to distorted animations.
I appreciate the hard work and dedication of the DynamicPose development team. Addressing this issue will significantly enhance the tool's flexibility and applicability across a broader range of projects. Thank you for considering this feedback, and I look forward to any updates or improvements in future releases.
Thank you!