Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor #59

Closed
wants to merge 39 commits into from
Closed

Refactor #59

wants to merge 39 commits into from

Conversation

epignatelli
Copy link
Owner

No description provided.

epignatelli and others added 30 commits July 22, 2023 10:18
@epignatelli
refactor(envs): strip off all non essentials from env wrappers
cbc97d8
@epignatelli
refactor: remove random, stax and interop modules
8a25261
@epignatelli
refactor: everything
0abb43b
@epignatelli
ops(cd): cd now inputs manual version
8de5c00
@epignatelli
align install to new cd
a5a237b
@epignatelli
setup: add jax_enums to requirements
fcd4705
@epignatelli
remove logger
79e7afb
@epignatelli
sync
9a32114
@epignatelli
fix: importing procgen break OSX, import lazily
87028b5
@epignatelli
fix: pytype breaks with Shape=NoReturn from jax.core.ShapedArray
d3313d2
@epignatelli
refactor: ReplayBuffer now initialises using environment spec rather …
ced5570 
…than example item
@epignatelli
align dqn to the new changes
d7a9101
@alanlivio @epignatelli
add CITATION.cff
de812cb
@epignatelli
feat: add gymnax wrapper
ddda109
@epignatelli
refactor: log is not part of train state
e112af6
@epignatelli
add gymanx example
2e3dec5
@epignatelli
format: black
209c63e
@epignatelli
refactor: experiment.run now takes seed rather than key;
d2beaad
@epignatelli
align SAC
e2c824e
@epignatelli
align sacd
3888c4b
@epignatelli
brax and navix
2e980b2
@epignatelli
refactor: replace StepType with standard Array
1e72d67
@epignatelli
working dqn suite
6e1f0d7
@epignatelli
refactor: remove external downloads
8e4a9b6
@epignatelli
losses are not embedded into the agents
d30fedc
@epignatelli
add reporting;
83c0c0a
@epignatelli
fix copyright header
98dc23a
@epignatelli
add some missing copyright
0c38312
@epignatelli
fix merge module and add observations to navix example
1cec821
@epignatelli
feat: add navix wrapper
88857bc
@epignatelli epignatelli self-assigned this Dec 18, 2023
@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 18, 2023 •
edited
Loading 

************* Module ddqn_gymnax_catch
examples/ddqn_gymnax_catch.py:1:0: C0114: Missing module docstring (missing-module-docstring)
examples/ddqn_gymnax_catch.py:28:0: C0116: Missing function or method docstring (missing-function-docstring)
examples/ddqn_gymnax_catch.py:28:9: W0613: Unused argument 'argv' (unused-argument)
examples/ddqn_gymnax_catch.py:22:0: C0411: third party import "import wandb" should be placed before "import helx" (wrong-import-order)
************* Module dqn_bsuite_catch
examples/dqn_bsuite_catch.py:1:0: C0114: Missing module docstring (missing-module-docstring)
examples/dqn_bsuite_catch.py:28:0: C0116: Missing function or method docstring (missing-function-docstring)
examples/dqn_bsuite_catch.py:28:9: W0613: Unused argument 'argv' (unused-argument)
examples/dqn_bsuite_catch.py:22:0: C0411: third party import "import wandb" should be placed before "import helx" (wrong-import-order)
************* Module dueling_navix_empty
examples/dueling_navix_empty.py:1:0: C0114: Missing module docstring (missing-module-docstring)
examples/dueling_navix_empty.py:28:0: C0116: Missing function or method docstring (missing-function-docstring)
examples/dueling_navix_empty.py:28:9: W0613: Unused argument 'argv' (unused-argument)
examples/dueling_navix_empty.py:22:0: C0411: third party import "import wandb" should be placed before "import helx" (wrong-import-order)
************* Module sac_brax_halfcheetah
examples/sac_brax_halfcheetah.py:1:0: C0114: Missing module docstring (missing-module-docstring)
examples/sac_brax_halfcheetah.py:28:0: C0116: Missing function or method docstring (missing-function-docstring)
examples/sac_brax_halfcheetah.py:28:9: W0613: Unused argument 'argv' (unused-argument)
examples/sac_brax_halfcheetah.py:22:0: C0411: third party import "import wandb" should be placed before "import helx" (wrong-import-order)
************* Module sacd_navix_keydoor
examples/sacd_navix_keydoor.py:1:0: C0114: Missing module docstring (missing-module-docstring)
examples/sacd_navix_keydoor.py:26:0: C0116: Missing function or method docstring (missing-function-docstring)
examples/sacd_navix_keydoor.py:26:9: W0613: Unused argument 'argv' (unused-argument)
examples/sacd_navix_keydoor.py:20:0: C0411: third party import "import wandb" should be placed before "import helx" (wrong-import-order)
************* Module helx
helx/__init__.py:1:0: C0114: Missing module docstring (missing-module-docstring)
************* Module helx._version
helx/_version.py:1:0: C0114: Missing module docstring (missing-module-docstring)
************* Module helx.agents
helx/agents/__init__.py:1:0: C0114: Missing module docstring (missing-module-docstring)
************* Module helx.agents.agent
helx/agents/agent.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/agents/agent.py:27:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/agent.py:27:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/agent.py:35:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/agent.py:35:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/agent.py:41:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/agent.py:41:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/agent.py:47:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/agent.py:52:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/agents/agent.py:52:4: E0213: Method 'create' should have "self" as first argument (no-self-argument)
helx/agents/agent.py:56:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/agents/agent.py:60:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/agents/agent.py:61:69: W0622: Redefining built-in 'eval' (redefined-builtin)
helx/agents/agent.py:66:4: C0116: Missing function or method docstring (missing-function-docstring)
************* Module helx.agents.ddqn
helx/agents/ddqn.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/agents/ddqn.py:26:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/ddqn.py:26:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/ddqn.py:30:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/ddqn.py:30:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/ddqn.py:34:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/ddqn.py:34:0: R0903: Too few public methods (0/2) (too-few-public-methods)
************* Module helx.agents.dqn
helx/agents/dqn.py:133:0: C0301: Line too long (112/100) (line-too-long)
helx/agents/dqn.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/agents/dqn.py:36:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/dqn.py:36:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/dqn.py:57:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/dqn.py:57:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/dqn.py:62:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/dqn.py:62:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/dqn.py:80:4: W0221: Number of parameters was 1 in 'Agent.create' and is now 4 in overriding 'DQN.create' method (arguments-differ)
helx/agents/dqn.py:80:4: W0221: Variadics removed in overriding 'DQN.create' method (arguments-differ)
helx/agents/dqn.py:98:4: W0221: Number of parameters was 3 in 'Agent.init' and is now 3 in overriding 'DQN.init' method (arguments-differ)
helx/agents/dqn.py:121:4: W0237: Parameter 'agent_state' has been renamed to 'train_state' in overriding 'DQN.sample_action' method (arguments-renamed)
helx/agents/dqn.py:122:67: W0622: Redefining built-in 'eval' (redefined-builtin)
helx/agents/dqn.py:136:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/agents/dqn.py:158:4: W0237: Parameter 'agent_state' has been renamed to 'train_state' in overriding 'DQN.update' method (arguments-renamed)
helx/agents/dqn.py:158:4: W0237: Parameter 'timesteps' has been renamed to 'transition' in overriding 'DQN.update' method (arguments-renamed)
helx/agents/dqn.py:174:22: C3001: Lambda expression assigned to a variable. Define a function using the "def" keyword instead. (unnecessary-lambda-assignment)
helx/agents/dqn.py:188:12: W0108: Lambda may not be necessary (unnecessary-lambda)
helx/agents/dqn.py:33:0: W0611: Unused import losses (unused-import)
************* Module helx.agents.dueling_dqn
helx/agents/dueling_dqn.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/agents/dueling_dqn.py:27:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/dueling_dqn.py:27:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/dueling_dqn.py:31:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/dueling_dqn.py:31:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/dueling_dqn.py:35:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/dueling_dqn.py:35:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/dueling_dqn.py:39:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/dueling_dqn.py:21:0: C0412: Imports from package jax are not grouped (ungrouped-imports)
helx/agents/dueling_dqn.py:21:0: W0611: Unused jax.tree_util imported as jtu (unused-import)
************* Module helx.agents.random
helx/agents/random.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/agents/random.py:26:0: W0223: Method 'create' is abstract in class 'Agent' but is not overridden in child class 'Random' (abstract-method)
helx/agents/random.py:26:0: W0223: Method 'init' is abstract in class 'Agent' but is not overridden in child class 'Random' (abstract-method)
helx/agents/random.py:32:4: W0231: __init__ method from base class 'Agent' is not called (super-init-not-called)
helx/agents/random.py:41:4: W0237: Parameter 'agent_state' has been renamed to 'observation' in overriding 'Random.sample_action' method (arguments-renamed)
helx/agents/random.py:41:4: W0237: Parameter 'obs' has been renamed to 'eval' in overriding 'Random.sample_action' method (arguments-renamed)
helx/agents/random.py:41:48: W0622: Redefining built-in 'eval' (redefined-builtin)
helx/agents/random.py:41:28: W0613: Unused argument 'observation' (unused-argument)
helx/agents/random.py:41:0: W0613: Unused argument 'kwargs' (unused-argument)
helx/agents/random.py:44:4: W0221: Number of parameters was 4 in 'Agent.update' and is now 2 in overriding 'Random.update' method (arguments-differ)
helx/agents/random.py:44:21: W0613: Unused argument 'episode' (unused-argument)
helx/agents/random.py:16:0: W0611: Unused Any imported from typing (unused-import)
helx/agents/random.py:16:0: W0611: Unused Tuple imported from typing (unused-import)
************* Module helx.agents.sac
helx/agents/sac.py:202:0: C0301: Line too long (101/100) (line-too-long)
helx/agents/sac.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/agents/sac.py:42:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/sac.py:42:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/sac.py:59:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/sac.py:59:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/sac.py:69:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/sac.py:69:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/sac.py:108:4: W0221: Number of parameters was 1 in 'Agent.create' and is now 5 in overriding 'SAC.create' method (arguments-differ)
helx/agents/sac.py:108:4: W0221: Variadics removed in overriding 'SAC.create' method (arguments-differ)
helx/agents/sac.py:137:4: W0221: Number of parameters was 3 in 'Agent.init' and is now 3 in overriding 'SAC.init' method (arguments-differ)
helx/agents/sac.py:168:4: W0237: Parameter 'agent_state' has been renamed to 'train_state' in overriding 'SAC.sample_action' method (arguments-renamed)
helx/agents/sac.py:169:67: W0622: Redefining built-in 'eval' (redefined-builtin)
helx/agents/sac.py:180:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/agents/sac.py:180:4: R0914: Too many local variables (41/15) (too-many-locals)
helx/agents/sac.py:211:8: C0103: Variable name "qA_t" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sac.py:211:14: C0103: Variable name "qB_t" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sac.py:218:34: E0602: Undefined variable 'probs_a_tm1' (undefined-variable)
helx/agents/sac.py:221:8: C0103: Variable name "qA_tm1" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sac.py:221:16: C0103: Variable name "qB_tm1" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sac.py:222:8: C0103: Variable name "qA_t" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sac.py:222:14: C0103: Variable name "qB_t" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sac.py:225:50: E0602: Undefined variable 'discount_t' (undefined-variable)
helx/agents/sac.py:227:8: C0103: Variable name "qA_tm1" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sac.py:227:16: C0103: Variable name "qB_tm1" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sac.py:228:8: C0103: Variable name "qA_t" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sac.py:228:14: C0103: Variable name "qB_t" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sac.py:191:12: W0612: Unused variable 'k2' (unused-variable)
helx/agents/sac.py:207:8: W0612: Unused variable 'a_t' (unused-variable)
helx/agents/sac.py:256:4: W0237: Parameter 'agent_state' has been renamed to 'train_state' in overriding 'SAC.update' method (arguments-renamed)
helx/agents/sac.py:256:4: W0237: Parameter 'timesteps' has been renamed to 'transition' in overriding 'SAC.update' method (arguments-renamed)
helx/agents/sac.py:287:12: W0108: Lambda may not be necessary (unnecessary-lambda)
************* Module helx.agents.sacd
helx/agents/sacd.py:123:0: C0301: Line too long (101/100) (line-too-long)
helx/agents/sacd.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/agents/sacd.py:33:0: C0115: Missing class docstring (missing-class-docstring)
helx/agents/sacd.py:33:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/agents/sacd.py:77:67: W0622: Redefining built-in 'eval' (redefined-builtin)
helx/agents/sacd.py:84:4: W0237: Parameter 'timesteps' has been renamed to 'transition' in overriding 'SACD.loss' method (arguments-renamed)
helx/agents/sacd.py:84:4: W0221: Number of parameters was 4 in 'SAC.loss' and is now 3 in overriding 'SACD.loss' method (arguments-differ)
helx/agents/sacd.py:84:4: R0914: Too many local variables (34/15) (too-many-locals)
helx/agents/sacd.py:95:23: C3001: Lambda expression assigned to a variable. Define a function using the "def" keyword instead. (unnecessary-lambda-assignment)
helx/agents/sacd.py:114:8: C0103: Variable name "qA_tm1" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sacd.py:114:16: C0103: Variable name "qB_tm1" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sacd.py:115:8: C0103: Variable name "qA_t" doesn't conform to snake_case naming style (invalid-name)
helx/agents/sacd.py:115:14: C0103: Variable name "qB_t" doesn't conform to snake_case naming style (invalid-name)
************* Module helx.config
helx/config.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/config.py:27:0: C0115: Missing class docstring (missing-class-docstring)
helx/config.py:28:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/config.py:27:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/config.py:62:0: C0115: Missing class docstring (missing-class-docstring)
helx/config.py:63:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/config.py:62:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/config.py:102:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/config.py:103:4: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
helx/config.py:118:25: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/config.py:102:0: R0911: Too many return statements (7/6) (too-many-return-statements)
helx/config.py:121:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/config.py:130:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/config.py:130:0: R1711: Useless return at end of function or method (useless-return)
helx/config.py:160:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/config.py:161:48: W0621: Redefining name 'flags' from outer scope (line 21) (redefined-outer-name)
helx/config.py:171:29: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/config.py:178:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/config.py:180:4: W0101: Unreachable code (unreachable)
helx/config.py:180:4: C0415: Import outside toplevel (yaml) (import-outside-toplevel)
helx/config.py:182:9: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
************* Module helx.environment
helx/environment/__init__.py:1:0: C0114: Missing module docstring (missing-module-docstring)
************* Module helx.environment.brax
helx/environment/brax.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/environment/brax.py:35:4: W0221: Number of parameters was 2 in 'EnvironmentWrapper.wraps' and is now 2 in overriding 'BraxWrapper.wraps' method (arguments-differ)
************* Module helx.environment.bsuite
helx/environment/bsuite.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/environment/bsuite.py:42:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/bsuite.py:43:4: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
helx/environment/bsuite.py:53:12: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/environment/bsuite.py:62:4: W0221: Number of parameters was 2 in 'EnvironmentWrapper.wraps' and is now 2 in overriding 'BsuiteWrapper.wraps' method (arguments-differ)
************* Module helx.environment.dm_env
helx/environment/dm_env.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/environment/dm_env.py:45:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/dm_env.py:46:4: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
helx/environment/dm_env.py:58:12: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/environment/dm_env.py:62:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/dm_env.py:92:4: W0221: Number of parameters was 2 in 'EnvironmentWrapper.wraps' and is now 2 in overriding 'DmEnvWrapper.wraps' method (arguments-differ)
************* Module helx.environment.environment
helx/environment/environment.py:31:0: C0115: Missing class docstring (missing-class-docstring)
helx/environment/environment.py:37:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/environment.py:44:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/environment.py:57:0: C0115: Missing class docstring (missing-class-docstring)
helx/environment/environment.py:61:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/environment.py:64:4: C0116: Missing function or method docstring (missing-function-docstring)
************* Module helx.environment.gym
helx/environment/gym.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/environment/gym.py:45:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/gym.py:46:4: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
helx/environment/gym.py:56:12: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/environment/gym.py:60:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/gym.py:88:4: W0221: Number of parameters was 2 in 'EnvironmentWrapper.wraps' and is now 2 in overriding 'GymWrapper.wraps' method (arguments-differ)
helx/environment/gym.py:97:4: W0237: Parameter 'key' has been renamed to 'seed' in overriding 'GymWrapper.reset' method (arguments-renamed)
helx/environment/gym.py:18:0: W0611: Unused Tuple imported from typing (unused-import)
************* Module helx.environment.gymnasium
helx/environment/gymnasium.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/environment/gymnasium.py:43:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/gymnasium.py:44:4: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
helx/environment/gymnasium.py:54:12: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/environment/gymnasium.py:58:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/gymnasium.py:86:4: W0221: Number of parameters was 2 in 'EnvironmentWrapper.wraps' and is now 2 in overriding 'GymnasiumWrapper.wraps' method (arguments-differ)
helx/environment/gymnasium.py:97:4: W0237: Parameter 'key' has been renamed to 'seed' in overriding 'GymnasiumWrapper.reset' method (arguments-renamed)
helx/environment/gymnasium.py:17:0: W0611: Unused Tuple imported from typing (unused-import)
************* Module helx.environment.gymnax
helx/environment/gymnax.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/environment/gymnax.py:23:0: W0404: Reimport 'EnvParams' (imported line 22) (reimported)
helx/environment/gymnax.py:33:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/gymnax.py:38:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/gymnax.py:38:0: R0913: Too many arguments (7/5) (too-many-arguments)
helx/environment/gymnax.py:58:4: W0221: Number of parameters was 2 in 'EnvironmentWrapper.wraps' and is now 2 in overriding 'GymnaxWrapper.wraps' method (arguments-differ)
helx/environment/gymnax.py:80:34: W0612: Unused variable 'info' (unused-variable)
helx/environment/gymnax.py:25:0: C0412: Imports from package jax are not grouped (ungrouped-imports)
************* Module helx.environment.interop
helx/environment/interop.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/environment/interop.py:77:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/interop.py:85:4: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
helx/environment/interop.py:77:0: R0911: Too many return statements (8/6) (too-many-return-statements)
************* Module helx.environment.navix
helx/environment/navix.py:102:0: C0305: Trailing newlines (trailing-newlines)
helx/environment/navix.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/environment/navix.py:42:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/environment/navix.py:43:4: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
helx/environment/navix.py:53:12: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/environment/navix.py:62:4: W0221: Number of parameters was 2 in 'EnvironmentWrapper.wraps' and is now 2 in overriding 'NavixWrapper.wraps' method (arguments-differ)
************* Module helx.experiment
helx/experiment.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/experiment.py:27:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/experiment.py:31:4: W0622: Redefining built-in 'eval' (redefined-builtin)
helx/experiment.py:51:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/experiment.py:51:0: R0913: Too many arguments (7/5) (too-many-arguments)
helx/experiment.py:59:4: W0622: Redefining built-in 'eval' (redefined-builtin)
helx/experiment.py:76:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/experiment.py:104:0: C0116: Missing function or method docstring (missing-function-docstring)
************* Module helx.image
helx/image.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/image.py:24:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/image.py:29:7: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/image.py:64:12: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
helx/image.py:64:12: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
************* Module helx.logging
helx/logging.py:56:0: C0301: Line too long (126/100) (line-too-long)
helx/logging.py:62:0: C0301: Line too long (129/100) (line-too-long)
helx/logging.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/logging.py:4:0: W0404: Reimport 'logging' (imported line 1) (reimported)
helx/logging.py:18:0: C0115: Missing class docstring (missing-class-docstring)
helx/logging.py:18:0: R0903: Too few public methods (0/2) (too-few-public-methods)
helx/logging.py:32:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/logging.py:38:12: W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation)
helx/logging.py:50:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/logging.py:55:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/logging.py:56:4: W1201: Use lazy % formatting in logging functions (logging-not-lazy)
helx/logging.py:56:44: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/logging.py:61:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/logging.py:62:4: W1201: Use lazy % formatting in logging functions (logging-not-lazy)
helx/logging.py:62:44: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/logging.py:67:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/logging.py:4:0: C0412: Imports from package logging are not grouped (ungrouped-imports)
************* Module helx.losses
helx/losses.py:12:0: C0301: Line too long (117/100) (line-too-long)
helx/losses.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/losses.py:12:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/losses.py:21:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/losses.py:40:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/losses.py:40:0: R0914: Too many local variables (16/15) (too-many-locals)
helx/losses.py:58:0: C0116: Missing function or method docstring (missing-function-docstring)
helx/losses.py:58:0: R0913: Too many arguments (6/5) (too-many-arguments)
helx/losses.py:60:4: W0613: Unused argument 'critic' (unused-argument)
helx/losses.py:61:4: W0613: Unused argument 'params' (unused-argument)
helx/losses.py:62:4: W0613: Unused argument 'params_target' (unused-argument)
helx/losses.py:63:4: W0613: Unused argument 'entropy' (unused-argument)
helx/losses.py:66:4: W0612: Unused variable 's_tm1' (unused-variable)
helx/losses.py:66:11: W0612: Unused variable 's_t' (unused-variable)
helx/losses.py:66:16: W0612: Unused variable 'a_tm1' (unused-variable)
helx/losses.py:66:23: W0612: Unused variable 'r_t' (unused-variable)
helx/losses.py:66:28: W0612: Unused variable 'terminal_tm1' (unused-variable)
helx/losses.py:66:42: W0612: Unused variable 'discount_t' (unused-variable)
************* Module helx.mdp
helx/mdp.py:36:0: C0301: Line too long (105/100) (line-too-long)
helx/mdp.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/mdp.py:30:0: C0115: Missing class docstring (missing-class-docstring)
helx/mdp.py:30:0: R0903: Too few public methods (0/2) (too-few-public-methods)
************* Module helx.memory
helx/memory.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/memory.py:25:0: W0611: Unused Space imported from spaces (unused-import)
helx/memory.py:26:0: W0611: Unused TRANSITION imported from mdp (unused-import)
************* Module helx.modules
helx/modules.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/modules.py:25:0: C0115: Missing class docstring (missing-class-docstring)
helx/modules.py:25:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/modules.py:33:0: C0115: Missing class docstring (missing-class-docstring)
helx/modules.py:33:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/modules.py:43:0: C0115: Missing class docstring (missing-class-docstring)
helx/modules.py:43:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/modules.py:51:0: C0115: Missing class docstring (missing-class-docstring)
helx/modules.py:51:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/modules.py:59:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/modules.py:67:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/modules.py:75:0: C0115: Missing class docstring (missing-class-docstring)
helx/modules.py:75:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/modules.py:107:8: C0200: Consider using enumerate instead of iterating with range and len (consider-using-enumerate)
helx/modules.py:87:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/modules.py:148:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/modules.py:154:11: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/modules.py:160:8: W0201: Attribute 'modules' defined outside __init__ (attribute-defined-outside-init)
************* Module helx.preprocess
helx/preprocess.py:1:0: C0114: Missing module docstring (missing-module-docstring)
************* Module helx.spaces
helx/spaces.py:1:0: C0114: Missing module docstring (missing-module-docstring)
helx/spaces.py:25:0: W0404: Reimport 'Shape' (imported line 23) (reimported)
helx/spaces.py:34:0: C0115: Missing class docstring (missing-class-docstring)
helx/spaces.py:40:15: C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
helx/spaces.py:44:4: C0116: Missing function or method docstring (missing-function-docstring)
helx/spaces.py:48:0: C0115: Missing class docstring (missing-class-docstring)
helx/spaces.py:48:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/spaces.py:60:0: C0115: Missing class docstring (missing-class-docstring)
helx/spaces.py:60:0: R0903: Too few public methods (1/2) (too-few-public-methods)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==helx.environment.gym:[48:82]
==helx.environment.gymnasium:[46:80]
        return Continuous(
            shape=gym_space.shape,
            minimum=gym_space.low.min().item(),
            maximum=gym_space.high.max().item(),
        )
    else:
        raise NotImplementedError(
            "Cannot convert dm_env space of type {}".format(type(gym_space))
        )


def timestep_from_gym(gym_step: GymTimestep, action: Array, t: Array) -> Timestep:
    obs, reward, terminated, truncated, _ = gym_step

    if terminated:
        step_type = TERMINATION
    elif truncated:
        step_type = TRUNCATION
    else:
        step_type = TRANSITION

    obs = jnp.asarray(obs)
    reward = jnp.asarray(reward)
    action = jnp.asarray(action)
    return Timestep(
        observation=obs,
        reward=reward,
        step_type=step_type,
        action=action,
        t=t,
        state=None,
    )

 (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==ddqn_gymnax_catch:[32:63]
==dqn_bsuite_catch:[32:63]
    env = helx.environment.to_helx(env)

    # optimiser
    optimiser = optax.rmsprop(
        learning_rate=FLAGS.learning_rate,
        momentum=FLAGS.gradient_momentum,
        eps=FLAGS.min_squared_gradient,
        centered=True,
    )

    # agent
    hparams = helx.config.hparams_from_flags(
        helx.agents.DQNHParams,
        obs_space=env.observation_space,
        action_space=env.action_space,
        replay_start=10,
        batch_size=2,
    )

    backbone = nn.Sequential(
        [
            helx.modules.Flatten(),
            helx.modules.MLP(features=[32, 16]),
        ]
    )
    agent = helx.agents.DQN.create(
        hparams=hparams,
        optimiser=optimiser,
        backbone=backbone,
    )
 (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==sac_brax_halfcheetah:[37:59]
==sacd_navix_keydoor:[38:60]
    )

    # agent
    hparams = helx.config.hparams_from_flags(
        helx.agents.SACDHParams,
        obs_space=env.observation_space,
        action_space=env.action_space,
        replay_start=10,
        batch_size=2,
    )
    actor_backbone = nn.Sequential(
        [
            helx.modules.Flatten(),
            helx.modules.MLP(features=[32, 16]),
        ]
    )
    critic_backbone = nn.Sequential(
        [
            helx.modules.Flatten(),
            helx.modules.MLP(features=[32, 16]),
        ]
    ) (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==helx.environment.bsuite:[62:77]
==helx.environment.dm_env:[92:107]
        self = cls(
            env=env,
            observation_space=to_helx(env.observation_spec()),  # type: ignore
            action_space=to_helx(env.action_spec()),  # type: ignore
            reward_space=to_helx(env.reward_spec()),  # type: ignore
        )
        return self

    def reset(self, key: KeyArray | int) -> Timestep:
        next_step = self.env.reset()
        return timestep_to_helx(next_step, jnp.asarray(-1), jnp.asarray(0))

    def _step(self, key: KeyArray, timestep: Timestep, action: jax.Array) -> Timestep:
        next_step = self.env.step(action.item())
        return timestep_to_helx(next_step, action, timestep.t + 1) (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==helx.agents.dqn:[205:218]
==helx.agents.sac:[301:314]
            step_type=transition.step_type[-1],
            returns=train_state.log.returns
            + jnp.sum(
                self.hparams.discount ** transition.t[:-1] * transition.reward[:-1]
            ),
            buffer_size=buffer.size(),
        )

        # update train_state
        train_state = train_state.replace(
            iteration=iteration,
            opt_state=opt_state,
            params=params, (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==ddqn_gymnax_catch:[45:57]
==dueling_navix_empty:[45:57]
        obs_space=env.observation_space,
        action_space=env.action_space,
        replay_start=10,
        batch_size=2,
    )

    backbone = nn.Sequential(
        [
            helx.modules.Flatten(),
            helx.modules.MLP(features=[32, 16]),
        ]
    ) (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==helx.environment.dm_env:[76:86]
==helx.environment.gym:[72:82]
    return Timestep(
        observation=obs,
        reward=reward,
        step_type=step_type,
        action=action,
        t=t,
        state=None,
    )

 (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==sac_brax_halfcheetah:[60:71]
==sacd_navix_keydoor:[61:72]
        hparams=hparams,
        optimiser=optimiser,
        actor_backbone=actor_backbone,
        critic_backbone=critic_backbone,
    )

    helx.experiment.jrun(seed=FLAGS.seed, agent=agent, env=env, budget=20)


if __name__ == "__main__":
    app.run(main) (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==dqn_bsuite_catch:[32:44]
==sacd_navix_keydoor:[30:42]
    env = helx.environment.to_helx(env)

    # optimiser
    optimiser = optax.rmsprop(
        learning_rate=FLAGS.learning_rate,
        momentum=FLAGS.gradient_momentum,
        eps=FLAGS.min_squared_gradient,
        centered=True,
    )

    # agent
    hparams = helx.config.hparams_from_flags( (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==ddqn_gymnax_catch:[32:44]
==dueling_navix_empty:[32:44]
    env = helx.environment.to_helx(env)  # type: ignore

    # optimiser
    optimiser = optax.rmsprop(
        learning_rate=FLAGS.learning_rate,
        momentum=FLAGS.gradient_momentum,
        eps=FLAGS.min_squared_gradient,
        centered=True,
    )

    # agent
    hparams = helx.config.hparams_from_flags( (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==helx.environment.bsuite:[33:47]
==helx.environment.dm_env:[36:50]
    ...


@overload
def to_helx(dm_space: dm_env.specs.BoundedArray) -> Continuous:
    ...


def to_helx(dm_space: dm_env.specs.Array) -> Space:
    if isinstance(dm_space, dm_env.specs.DiscreteArray):
        return Discrete(dm_space.num_values)
    elif isinstance(dm_space, dm_env.specs.BoundedArray):
        return Continuous(
            shape=dm_space.shape, (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==helx.agents.ddqn:[49:57]
==helx.agents.dqn:[141:149]
        s_tm1 = timesteps.observation[:-1]
        s_t = timesteps.observation[1:]
        a_tm1 = timesteps.action[:-1][0]  # [0] because scalar
        r_t = timesteps.reward[:-1][0]  # [0] because scalar
        terminal_tm1 = timesteps.step_type[:-1] != TERMINATION
        discount_t = self.hparams.discount ** timesteps.t[:-1][0]  # [0] because scalar

        q_tm1 = jnp.asarray(self.critic.apply(params, s_tm1)) (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==ddqn_gymnax_catch:[58:68]
==dueling_navix_empty:[58:68]
        hparams=hparams,
        optimiser=optimiser,
        backbone=backbone,
    )

    helx.experiment.jrun(seed=FLAGS.seed, agent=agent, env=env, budget=20)


if __name__ == "__main__":
    app.run(main) (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==helx.agents.dqn:[186:194]
==helx.agents.sac:[285:293]
            cond,
            lambda p, o: _sgd_step(p, o),
            lambda p, o: (p, o, jnp.asarray(float("inf"))),
            train_state.params,
            train_state.opt_state,
        )

        # update log (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==helx.agents.dqn:[42:49]
==helx.agents.sac:[46:52]
    replay_start: int = 1000
    replay_memory_size: int = 1000
    update_frequency: int = 1
    target_network_update_frequency: int = 10000
    discount: float = 0.99
    n_steps: int = 1 (duplicate-code)
helx/spaces.py:1:0: R0801: Similar lines in 2 files
==helx.agents.ddqn:[49:54]
==helx.losses:[12:17]
    s_tm1 = timesteps.observation[:-1]
    s_t = timesteps.observation[1:]
    a_tm1 = timesteps.action[:-1][0]  # [0] because scalar
    r_t = timesteps.reward[:-1][0]  # [0] because scalar
    terminal_tm1 = timesteps.step_type[:-1] != TERMINATION (duplicate-code)


Report
======
1280 statements analysed.

Statistics by type
------------------

+---------+-------+-----------+-----------+------------+---------+
|type     |number |old number |difference |%documented |%badname |
+=========+=======+===========+===========+============+=========+
|module   |35     |NC         |NC         |2.86        |0.00     |
+---------+-------+-----------+-----------+------------+---------+
|class    |50     |NC         |NC         |36.00       |0.00     |
+---------+-------+-----------+-----------+------------+---------+
|method   |74     |NC         |NC         |81.08       |0.00     |
+---------+-------+-----------+-----------+------------+---------+
|function |66     |NC         |NC         |46.97       |3.03     |
+---------+-------+-----------+-----------+------------+---------+



External dependencies
---------------------
::

    helx (ddqn_gymnax_catch,dqn_bsuite_catch,dueling_navix_empty,sac_brax_halfcheetah,sacd_navix_keydoor)
      \-losses (helx.agents.dqn,helx.agents.sac)
      \-mdp (helx.agents.agent,helx.agents.ddqn,helx.agents.dqn,helx.agents.random,helx.agents.sac,helx.agents.sacd,helx.environment.brax,helx.environment.bsuite,helx.environment.dm_env,helx.environment.environment,helx.environment.gym,helx.environment.gymnasium,helx.environment.gymnax,helx.environment.navix)
      \-memory (helx.agents.dqn,helx.agents.sac)
      \-modules (helx.agents.dueling_dqn,helx.agents.sac,helx.agents.sacd)
      \-spaces (helx.agents.agent,helx.agents.dqn,helx.agents.sac,helx.agents.sacd,helx.environment.brax,helx.environment.bsuite,helx.environment.dm_env,helx.environment.environment,helx.environment.gym,helx.environment.gymnasium,helx.environment.gymnax,helx.environment.navix)
    yaml (helx.config)



3288 lines have been analyzed

Raw metrics
-----------

+----------+-------+------+---------+-----------+
|type      |number |%     |previous |difference |
+==========+=======+======+=========+===========+
|code      |2066   |62.83 |NC       |NC         |
+----------+-------+------+---------+-----------+
|docstring |204    |6.20  |NC       |NC         |
+----------+-------+------+---------+-----------+
|comment   |508    |15.45 |NC       |NC         |
+----------+-------+------+---------+-----------+
|empty     |510    |15.51 |NC       |NC         |
+----------+-------+------+---------+-----------+



Duplication
-----------

+-------------------------+------+---------+-----------+
|                         |now   |previous |difference |
+=========================+======+=========+===========+
|nb duplicated lines      |0     |NC       |NC         |
+-------------------------+------+---------+-----------+
|percent duplicated lines |0.000 |NC       |NC         |
+-------------------------+------+---------+-----------+



Messages by category
--------------------

+-----------+-------+---------+-----------+
|type       |number |previous |difference |
+===========+=======+=========+===========+
|convention |162    |NC       |NC         |
+-----------+-------+---------+-----------+
|refactor   |62     |NC       |NC         |
+-----------+-------+---------+-----------+
|warning    |78     |NC       |NC         |
+-----------+-------+---------+-----------+
|error      |3      |NC       |NC         |
+-----------+-------+---------+-----------+



Messages
--------

+-------------------------------+------------+
|message id                     |occurrences |
+===============================+============+
|missing-function-docstring     |49          |
+-------------------------------+------------+
|missing-module-docstring       |34          |
+-------------------------------+------------+
|missing-class-docstring        |32          |
+-------------------------------+------------+
|too-few-public-methods         |30          |
+-------------------------------+------------+
|duplicate-code                 |16          |
+-------------------------------+------------+
|arguments-differ               |15          |
+-------------------------------+------------+
|invalid-name                   |14          |
+-------------------------------+------------+
|consider-using-f-string        |13          |
+-------------------------------+------------+
|unused-argument                |12          |
+-------------------------------+------------+
|arguments-renamed              |11          |
+-------------------------------+------------+
|unused-variable                |9           |
+-------------------------------+------------+
|unused-import                  |8           |
+-------------------------------+------------+
|redefined-builtin              |7           |
+-------------------------------+------------+
|no-else-return                 |7           |
+-------------------------------+------------+
|line-too-long                  |7           |
+-------------------------------+------------+
|wrong-import-order             |5           |
+-------------------------------+------------+
|ungrouped-imports              |3           |
+-------------------------------+------------+
|too-many-locals                |3           |
+-------------------------------+------------+
|too-many-arguments             |3           |
+-------------------------------+------------+
|reimported                     |3           |
+-------------------------------+------------+
|unnecessary-lambda-assignment  |2           |
+-------------------------------+------------+
|unnecessary-lambda             |2           |
+-------------------------------+------------+
|undefined-variable             |2           |
+-------------------------------+------------+
|too-many-return-statements     |2           |
+-------------------------------+------------+
|logging-not-lazy               |2           |
+-------------------------------+------------+
|abstract-method                |2           |
+-------------------------------+------------+
|useless-return                 |1           |
+-------------------------------+------------+
|unspecified-encoding           |1           |
+-------------------------------+------------+
|unreachable                    |1           |
+-------------------------------+------------+
|trailing-newlines              |1           |
+-------------------------------+------------+
|super-init-not-called          |1           |
+-------------------------------+------------+
|redefined-outer-name           |1           |
+-------------------------------+------------+
|no-self-argument               |1           |
+-------------------------------+------------+
|logging-fstring-interpolation  |1           |
+-------------------------------+------------+
|logging-format-interpolation   |1           |
+-------------------------------+------------+
|import-outside-toplevel        |1           |
+-------------------------------+------------+
|consider-using-enumerate       |1           |
+-------------------------------+------------+
|attribute-defined-outside-init |1           |
+-------------------------------+------------+




-----------------------------------
Your code has been rated at 7.52/10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@epignatelli epignatelli

Labels
None yet
Projects
Helx roadmap
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@epignatelli @alanlivio