Running simple_stacking.py with a new robot

[Follograph]

Hello,

I'm trying to run the simple_stacking.py with a new robot, the yaml file of which applies well to motion_gen_reacher.py. But it runs into different errors.

My robot's ee link is an artificial link in the center of two fingers, different from the franka's 'base_hand'. This leads to the CUDA error.

 
     Traceback (most recent call last):
  File "examples/isaac_sim/flexiv_stacking.py", line 602, in 
    art_action = my_controller.forward(sim_js, my_flexiv.dof_names)
  File "examples/isaac_sim/flexiv_stacking.py", line 334, in forward
    result = self.plan(ee_translation_goal, ee_orientation_goal, sim_js, js_names)
  File "examples/isaac_sim/flexiv_stacking.py", line 301, in plan
    result = self.motion_gen.plan_single(cu_js.unsqueeze(0), ik_goal, self.plan_config.clone())
  File "/home/uno/curobo/src/curobo/wrap/reacher/motion_gen.py", line 1230, in plan_single
    link_poses=link_poses,
  File "/home/uno/curobo/src/curobo/wrap/reacher/motion_gen.py", line 1049, in _plan_attempts
    link_poses,
  File "/home/uno/curobo/src/curobo/wrap/reacher/motion_gen.py", line 1530, in _plan_from_solve_state
    return_all_solutions=plan_config.parallel_finetune,
  File "/home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/python/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "/home/uno/curobo/src/curobo/wrap/reacher/motion_gen.py", line 921, in _solve_trajopt_from_solve_state
    newton_iters=newton_iters,
  File "/home/uno/curobo/src/curobo/wrap/reacher/trajopt.py", line 498, in solve_any
    newton_iters=newton_iters,
  File "/home/uno/curobo/src/curobo/wrap/reacher/trajopt.py", line 620, in solve_single
    newton_iters=newton_iters,
  File "/home/uno/curobo/src/curobo/wrap/reacher/trajopt.py", line 579, in solve_from_solve_state
    result = self.solver.solve(goal_buffer, seed_traj)
  File "/home/uno/curobo/src/curobo/wrap/wrap_base.py", line 141, in solve
    act_seq = self.optimize(seed, shift_steps=0)
  File "/home/uno/curobo/src/curobo/wrap/wrap_base.py", line 71, in optimize
    act_seq = opt.optimize(act_seq, shift_steps)
  File "/home/uno/curobo/src/curobo/opt/opt_base.py", line 98, in optimize
    torch.cuda.synchronize()
  File "/home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/cuda/__init__.py", line 566, in synchronize
    return torch._C._cuda_synchronize()
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Exception ignored in: ._Registry.__del__ at 0x7f413e6ca950>
Traceback (most recent call last):
  File "/home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/extscore/omni.kit.viewport.registry/omni/kit/viewport/registry/registry.py", line 103, in __del__
  File "/home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/extscore/omni.kit.viewport.registry/omni/kit/viewport/registry/registry.py", line 98, in destroy
TypeError: 'NoneType' object is not callable
Exception ignored in: ._Registry.__del__ at 0x7f413e6ca950>
Traceback (most recent call last):
  File "/home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/extscore/omni.kit.viewport.registry/omni/kit/viewport/registry/registry.py", line 103, in __del__
  File "/home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/extscore/omni.kit.viewport.registry/omni/kit/viewport/registry/registry.py", line 98, in destroy
TypeError: 'NoneType' object is not callable
Exception ignored in: 
Traceback (most recent call last):
  File "/home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/kernel/py/omni/kit/app/_impl/__init__.py", line 114, in __del__
AttributeError: 'NoneType' object has no attribute 'get_settings'
Exception ignored in: 
Traceback (most recent call last):
  File "/home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.kit.viewport.menubar.lighting-104.0.9/omni/kit/viewport/menubar/lighting/actions.py", line 347, in __del__
  File "/home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.kit.viewport.menubar.lighting-104.0.9/omni/kit/viewport/menubar/lighting/actions.py", line 352, in destroy
TypeError: 'NoneType' object is not callable
/home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/python.sh: line 41: 3379980 Segmentation fault      (core dumped) $python_exe "$@" $args
There was an error running python
  

pytorch 'c10' error

 
     terminate called after throwing an instance of 'c10::Error'
  what():  uc >= 0 INTERNAL ASSERT FAILED at "../c10/cuda/CUDACachingAllocator.cpp":1081, please report a bug to PyTorch.
Exception raised from notifyCaptureDestroy at ../c10/cuda/CUDACachingAllocator.cpp:1081 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7f58812a8457 in /home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, char const*) + 0x68 (0x7f58812724b5 in /home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/lib/libc10.so)
frame #2: c10::cuda::CUDACachingAllocator::notifyCaptureDestroy(int, std::pair) + 0x22f (0x7f5881320e1f in /home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/lib/libc10_cuda.so)
frame #3: at::cuda::CUDAGraph::reset() + 0x31 (0x7f584b946711 in /home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/lib/libtorch_cuda_cpp.so)
frame #4: at::cuda::CUDAGraph::~CUDAGraph() + 0xe (0x7f584b946ebe in /home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/lib/libtorch_cuda_cpp.so)
frame #5: std::_Sp_counted_ptr::_M_dispose() + 0x12 (0x7f585bbb2bd2 in /home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/lib/libtorch_python.so)
frame #6:  + 0xbb2c60 (0x7f585bbb2c60 in /home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/lib/libtorch_python.so)
frame #7:  + 0x3e4272 (0x7f585b3e4272 in /home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/lib/libtorch_python.so)
frame #8:  + 0x3e51af (0x7f585b3e51af in /home/uno/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/lib/libtorch_python.so)

  

When I try to take the base of the gripper as the ee link just like franka, it causes the warning "Plan did not converge to a solution".

 
     "Plan did not converge to a solution.  No action is being taken."
  

---

update:

Due to the problem with simple_stacking.py, I'm now trying to achieve the pick and place task by editing the motion_gen_reacher.py. The motion generation works but the cube cannot be picked up. I wonder whether the attach_objects_to_robots function in motion_gen is necessary for picking up objects. When I added the function, the error "MG: No spheres found" occurred.

Could you give me any suggestions on the possible problems above? Thanks a lot!

[Bailey-24]

hello, how do you solve the Plan did not converge to a solution problem.
i use my custom robot.

[Follograph]

In my case, the action command exceeds the robot's working space or fails to reach certain points. I haven't found out the problem with the certain position.