For running the code, using Docker is recommended.
To start the container, run:
bash docker_run.sh
To start an experiment launch a command inside the docker like:
sh xvfb_run.sh python3 rl/train.py use_wandb=False project_name=er agent=sac configs=rlbench_both task=rlbench_stack_wine env.action_mode=erjoint seed=1,2,3 experiment=test --multirun &
The command above launches 3 seeds in parallel. You should set seed=SEED to a unique value and remove --multirun to run a single experiment.
Note that by default all log is redirected to the experiments folder in out.log and err.log, so no output will be logged in your console.
The possible action modes are: env.action_mode=joint,ee,erjoint,erangle
You can change the ERJ joint to the wrist one by setting: +env.erj_joint=6
- The implementation of SAC (with distributional critic) can be found at
rl/agent/sac.pyalong with the hyperparameters used in thesac.yamlfile. - The new RLBench tasks can be found at
rl/env/custom_rlbench_tasks.py - The new action modes can be found at
rl/env/custom_arm_action_modes.py
For the real-world experiments we used ACT: https://github.com/tonyzhaozh/act
Some additional notes about the setup are:
- Each action space model was trained for 3000 epochs using the hyper-parameters presented in the ACT paper (apart from using a batch size of 16, and chunk size of 20).
- Additionally, the EE pose was added to the state information.
- All quaternions in demonstrations and inference were forced to have a positive
w. Lastly, only a wrist camera (with resolution 224x224) was used.
For the IK in we used: pick_ik with default parameters for solving the IK for EE and ERJ.
Some additional notes:
- We followed the standard pick_ik Kinematics Solver" tutorial from the pick_ik documentation
- We only altered two parameters for the IK, from the standard parameters:
approximate_solution_position_threshold=0.01andapproximate_solution_orientation_threshold=0.01(both from the original value 0.05) to increase accuracy