ASAP: Aligning Simulation and Real-World Physics for

Learning Agile Humanoid Whole-Body Skills

[Website] [Arxiv] [Video]

                   

TODO

• Relaase code backbone
• Release phase-based motion tracking training pipeline
• Release ASAP motion datasets
• Release motion retargeting pipeline
• Release sim2sim in MuJoCo
• Release sim2real with UnitreeSDK
• Release ASAP delta action model training pipeline

Installation

ASAP codebase is built on top of HumanoidVerse (a multi-simulator framework for humanoid learning) and Human2Humanoid (our prior work on humanoid whole-body tracking).

HumanoidVerse allows you to train humanoid skills in multiple simulators including IsaacGym, IsaacSim and Genesis. We leverage this framework to develop ASAP and study how to best transfer policies across simulators, and real world.

IsaacGym Conda Env

Create mamba/conda environment, in the following we use conda for example, but you can use mamba as well.

conda create -n hvgym python=3.8
conda activate hvgym

Install IsaacGym

Download IsaacGym and extract:

wget https://developer.nvidia.com/isaac-gym-preview-4
tar -xvzf isaac-gym-preview-4

Install IsaacGym Python API:

pip install -e isaacgym/python

Test installation:

python 1080_balls_of_solitude.py  # or
python joint_monkey.py

For libpython error:

• Check conda path:

  conda info -e

• Set LD_LIBRARY_PATH:

  export LD_LIBRARY_PATH=</path/to/conda/envs/your_env/lib>:$LD_LIBRARY_PATH

Install HumanoidVerse

Install dependencies:

pip install -e .
pip install -e isaac_utils
pip install -r requirements.txt

Test with:

HYDRA_FULL_ERROR=1 python humanoidverse/train_agent.py \
+simulator=isaacgym \
+exp=locomotion \
+domain_rand=NO_domain_rand \
+rewards=loco/reward_g1_locomotion \
+robot=g1/g1_29dof_anneal_23dof \
+terrain=terrain_locomotion_plane \
+obs=loco/leggedloco_obs_singlestep_withlinvel \
num_envs=1 \
project_name=TestIsaacGymInstallation \
experiment_name=G123dof_loco \
headless=True

IsaacLab Environment

Install IsaacSim

1. Download Omniverse Launcher
2. Install Isaac Sim through launcher
3. Set environment variables:

export ISAACSIM_PATH="${HOME}/.local/share/ov/pkg/isaac-sim-4.2.0"
export ISAACSIM_PYTHON_EXE="${ISAACSIM_PATH}/python.sh"

Install IsaacLab

git clone https://github.com/isaac-sim/IsaacLab.git
cd IsaacLab && ./isaaclab.sh --conda hvlab
mamba activate hvlab
sudo apt install cmake build-essential
./isaaclab.sh --install

Setup HumanoidVerse

pip install -e .
pip install -e isaac_utils
conda install pinocchio -c conda-forge
pip install -r requirements.txt

Genesis Environment

mamba create -n hvgen python=3.10
mamba activate hvgen
pip install genesis-world torch

Install dependencies:

pip install -e .
pip install -e isaac_utils
pip install -r requirements.txt

Motion Tracking Training

Train a phase-based motion tracking policy to imitat

python humanoidverse/train_agent.py \
+simulator=isaacgym \
+exp=motion_tracking \
+domain_rand=NO_domain_rand \
+rewards=motion_tracking/reward_motion_tracking_dm_2real \
+robot=g1/g1_29dof_anneal_23dof \
+terrain=terrain_locomotion_plane \
+obs=motion_tracking/deepmimic_a2c_nolinvel_LARGEnoise_history \
num_envs=4096 \
project_name=MotionTracking \
experiment_name=MotionTracking_CR7 \
robot.motion.motion_file="humanoidverse/data/motions/g1_29dof_anneal_23dof/TairanTestbed/singles/0-TairanTestbed_TairanTestbed_CR7_video_CR7_level1_filter_amass.pkl" \
rewards.reward_penalty_curriculum=True \
env.config.resample_motion_when_training=False