Hand-crafted reward and cost function & Sigwall lidar

[Mgineer117]

Hi developers!

I first appreciate all your efforts to construct this amazing environment for RL testing. I here brought two questions that is relevant to my current research: 1) hand-crafted reward and cost function and 2) Sigwall's lidar detection. The particular domain I am testing on is 'SafetyPointGoal1-v0" with sigwall(num=4) added.

1) hand-crafted reward and cost function
The reason I am trying to have such functions is because I want to see how different actions yield to different reward and cost. I already have referred to the environment duplication and rebuild the mujoco engine by overwriting current state, the first try of duplication takes significant computational cost during the duplication that makes it intractable and the second try just did not work. So, if any of y'all can provide those functions or insights for me to construct these functions, I would appreciate it.

2) Sigwall's lidar detection
To incorporate the cost calculation as the agent passes the sigwall into the cost function, I needed to make the sigwalls to be lidar-detectable. While I found that the environment successfully loaded its states into environment struct by calling env.obs_state_dict which shows "'sigwalls_lidar': Box(0.0, 1.0, (16,), float64)", I get the following error:

===========================================================================================
Exception has occurred: AssertionError (note: full exception trace is shown but execution is paused at: _run_module_as_main)
Bad pos [None]
File "/home/minjae/miniconda3/envs/sam/lib/python3.10/site-packages/safety_gymnasium/bases/base_task.py", line 542, in _ego_xy
assert pos.shape == (2,), f'Bad pos {pos}'
File "/home/minjae/miniconda3/envs/sam/lib/python3.10/site-packages/safety_gymnasium/bases/base_task.py", line 482, in _obs_lidar_pseudo
z = complex(*self._ego_xy(pos)) # X, Y as real, imaginary components
File "/home/minjae/miniconda3/envs/sam/lib/python3.10/site-packages/safety_gymnasium/bases/base_task.py", line 418, in _obs_lidar
return self._obs_lidar_pseudo(positions)
File "/home/minjae/miniconda3/envs/sam/lib/python3.10/site-packages/safety_gymnasium/bases/base_task.py", line 397, in obs
obs[obstacle.name + '_lidar'] = self._obs_lidar(obstacle.pos, obstacle.group)
File "/home/minjae/miniconda3/envs/sam/lib/python3.10/site-packages/safety_gymnasium/builder.py", line 212, in reset
return (self.task.obs(), info)
File "/home/minjae/.local/lib/python3.10/site-packages/gymnasium/utils/passive_env_checker.py", line 215, in env_reset_passive_checker
result = env.reset(**kwargs)
File "/home/minjae/.local/lib/python3.10/site-packages/gymnasium/wrappers/env_checker.py", line 55, in reset
return env_reset_passive_checker(self.env, **kwargs)
File "/home/minjae/.local/lib/python3.10/site-packages/gymnasium/wrappers/order_enforcing.py", line 61, in reset
return self.env.reset(**kwargs)
File "/home/minjae/.local/lib/python3.10/site-packages/gymnasium/wrappers/time_limit.py", line 75, in reset
return self.env.reset(**kwargs)
File "/home/minjae/Documents/Research/SAM4Meta/Adversary/simulator.py", line 21, in
state, info = env.reset(seed=i)
File "/home/minjae/miniconda3/envs/sam/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/minjae/miniconda3/envs/sam/lib/python3.10/runpy.py", line 196, in _run_module_as_main (Current frame)
return _run_code(code, main_globals, None,
AssertionError: Bad pos [None]
===========================================================================================

I would appreciate any response on this matter. Thanks again!

[muchvo]

I apologize for the delayed response. Here are the answers to your questions:

1. Both our reward function and cost function are customizable. You can find specific information in the code here and here. I am concerned that I might not have completely understood your needs. If that is indeed the case, please provide more background and details about your requirements. Due to the cost being related to the objects, the specific logic is encapsulated within the object classes. You can find the cost code for all objects at this path: safety_gymnasium\assets.
2. SigWalls does not support Pseudo Lidar because it is generated artificially based on the positions of objects, and walls cannot be abstracted into point masses. Therefore, our implementation does not include a function to determine the positions of walls. If you would like to use Pseudo Lidar to detect SigWalls, you will need to implement this function. For reference, please consult this file.