Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gymnasium v0.29.1 with additional fixes #32

Merged
merged 32 commits into from
Apr 2, 2024

Conversation

traversaro
Copy link
Contributor

@traversaro traversaro commented Feb 20, 2024

Based on #31, with an additional mujoco < 3.0.0 constraint due to Farama-Foundation/Gymnasium#749, that will be fixed in gymnasium 1.0.0 .

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe:

  • Recipes should usually depend on matplotlib-base as opposed to matplotlib so that runtime environments do not require large packages like qt.

@traversaro
Copy link
Contributor Author

@conda-forge-admin, please rerender

Copy link
Contributor

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/gymnasium-feedstock/actions/runs/7974980988.

@traversaro
Copy link
Contributor Author

The only remaining failures are in linux-aarch64 and linux-ppc64le:

=========================== short test summary info ============================
ERROR tests/envs/mujoco/test_mujoco_custom_env.py - AttributeError: 'NoneType' object has no attribute 'glGetError'
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
================== 6 skipped, 36 warnings, 1 error in 41.09s ===================

I wonder if there are any yum_requirements.txt or similar missing.

@traversaro
Copy link
Contributor Author

The only remaining failures are in linux-aarch64 and linux-ppc64le:

Obviously I was too optimistic, Windows tests are failing with error:

2024-02-20T15:04:48.5779497Z =========================== short test summary info ===========================
2024-02-20T15:04:48.5780939Z FAILED tests/spaces/test_spaces.py::test_roundtripping[MultiDiscrete([2 2])] - AssertionError: sample 1: [0 0], prime: [0 0]
2024-02-20T15:04:48.5781732Z assert False
2024-02-20T15:04:48.5782701Z  +  where False = data_equivalence(array([0, 0], dtype=int64), array([0, 0]))
2024-02-20T15:04:48.5783466Z FAILED tests/spaces/test_spaces.py::test_roundtripping[MultiDiscrete([[2 3]\n [3 2]])] - AssertionError: sample 1: [[0 2]
2024-02-20T15:04:48.5784068Z    [2 1]], prime: [[0 2]
2024-02-20T15:04:48.5784545Z    [2 1]]
2024-02-20T15:04:48.5784949Z assert False
2024-02-20T15:04:48.5785477Z  +  where False = data_equivalence(array([[0, 2],\n       [2, 1]], dtype=int64), array([[0, 2],\n       [2, 1]]))
2024-02-20T15:04:48.5786449Z FAILED tests/spaces/test_spaces.py::test_roundtripping[MultiDiscrete([2 2], start=[10 10])] - AssertionError: sample 1: [10 10], prime: [10 10]
2024-02-20T15:04:48.5787365Z assert False
2024-02-20T15:04:48.5787810Z  +  where False = data_equivalence(array([10, 10], dtype=int64), array([10, 10]))
2024-02-20T15:04:48.5788702Z FAILED tests/spaces/test_spaces.py::test_roundtripping[MultiDiscrete([[2 3]\n [3 2]], start=[[10 20]\n [30 40]])] - AssertionError: sample 1: [[10 22]
2024-02-20T15:04:48.5789318Z    [32 41]], prime: [[10 22]
2024-02-20T15:04:48.5789791Z    [32 41]]
2024-02-20T15:04:48.5790250Z assert False
2024-02-20T15:04:48.5790709Z  +  where False = data_equivalence(array([[10, 22],\n       [32, 41]], dtype=int64), array([[10, 22],\n       [32, 41]]))
2024-02-20T15:04:48.5791871Z FAILED tests/spaces/test_spaces.py::test_roundtripping[Sequence(Dict('a': Box(0.0, 1.0, (3,), float32), 'b': Discrete(5)), stack=True)] - AssertionError: sample 1: OrderedDict([('a', array([[0.23781213, 0.6477448 , 0.3832727 ],
2024-02-20T15:04:48.5792592Z          [0.15568377, 0.7579315 , 0.43943098],
2024-02-20T15:04:48.5793053Z          [0.1355174 , 0.60215604, 0.12298544],
2024-02-20T15:04:48.5793560Z          [0.18691972, 0.03824173, 0.9738922 ],
2024-02-20T15:04:48.5794006Z          [0.7367193 , 0.46722957, 0.70084214],
2024-02-20T15:04:48.5794511Z          [0.69305634, 0.9804614 , 0.15736328],
2024-02-20T15:04:48.5794955Z          [0.8837569 , 0.14653283, 0.07275112],
2024-02-20T15:04:48.5795535Z          [0.9505241 , 0.48309243, 0.47193125],
2024-02-20T15:04:48.5795983Z          [0.13466701, 0.5283474 , 0.504883  ],
2024-02-20T15:04:48.5797638Z          [0.17988856, 0.6406026 , 0.61961514],
2024-02-20T15:04:48.5798854Z          [0.8578526 , 0.6278885 , 0.51124865]], dtype=float32)), ('b', array([0, 2, 2, 3, 3, 3, 3, 0, 2, 1, 3], dtype=int64))]), prime: OrderedDict([('a', array([[0.23781213, 0.6477448 , 0.3832727 ],
2024-02-20T15:04:48.5799461Z          [0.15568377, 0.7579315 , 0.43943098],
2024-02-20T15:04:48.5799964Z          [0.1355174 , 0.60215604, 0.12298544],
2024-02-20T15:04:48.5800397Z          [0.18691972, 0.03824173, 0.9738922 ],
2024-02-20T15:04:48.5800945Z          [0.7367193 , 0.46722957, 0.70084214],
2024-02-20T15:04:48.5801397Z          [0.69305634, 0.9804614 , 0.15736328],
2024-02-20T15:04:48.5801912Z          [0.8837569 , 0.14653283, 0.07275112],
2024-02-20T15:04:48.5802411Z          [0.9505241 , 0.48309243, 0.47193125],
2024-02-20T15:04:48.5802978Z          [0.13466701, 0.5283474 , 0.504883  ],
2024-02-20T15:04:48.5803540Z          [0.17988856, 0.6406026 , 0.61961514],
2024-02-20T15:04:48.5804228Z          [0.8578526 , 0.6278885 , 0.51124865]], dtype=float32)), ('b', array([0, 2, 2, 3, 3, 3, 3, 0, 2, 1, 3]))])
2024-02-20T15:04:48.5804787Z assert False
2024-02-20T15:04:48.5807113Z  +  where False = data_equivalence(OrderedDict([('a', array([[0.23781213, 0.6477448 , 0.3832727 ],\n       [0.15568377, 0.7579315 , 0.43943098],\n       [0.1355174 , 0.60215604, 0.12298544],\n       [0.18691972, 0.03824173, 0.9738922 ],\n       [0.7367193 , 0.46722957, 0.70084214],\n       [0.69305634, 0.9804614 , 0.15736328],\n       [0.8837569 , 0.14653283, 0.07275112],\n       [0.9505241 , 0.48309243, 0.47193125],\n       [0.13466701, 0.5283474 , 0.504883  ],\n       [0.17988856, 0.6406026 , 0.61961514],\n       [0.8578526 , 0.6278885 , 0.51124865]], dtype=float32)), ('b', array([0, 2, 2, 3, 3, 3, 3, 0, 2, 1, 3], dtype=int64))]), OrderedDict([('a', array([[0.23781213, 0.6477448 , 0.3832727 ],\n       [0.15568377, 0.7579315 , 0.43943098],\n       [0.1355174 , 0.60215604, 0.12298544],\n       [0.18691972, 0.03824173, 0.9738922 ],\n       [0.7367193 , 0.46722957, 0.70084214],\n       [0.69305634, 0.9804614 , 0.15736328],\n       [0.8837569 , 0.14653283, 0.07275112],\n       [0.9505241 , 0.48309243, 0.47193125],\n       [0.13466701, 0.5283474 , 0.504883  ],\n       [0.17988856, 0.6406026 , 0.61961514],\n       [0.8578526 , 0.6278885 , 0.51124865]], dtype=float32)), ('b', array([0, 2, 2, 3, 3, 3, 3, 0, 2, 1, 3]))]))
2024-02-20T15:04:48.5809820Z ==== 5 failed, 2499 passed, 234 skipped, 391 warnings in 990.75s (0:16:30) ====

While macOS test (except on Python 3.8) are failing with:

2024-02-20T14:54:37.0727500Z FAILED tests/envs/test_action_dim_check.py::test_box_actions_out_of_bound[CarRacing-v2] - NotImplementedError: font module not available (ImportError: dlopen($PREFIX/lib/python3.11/site-packages/pygame/font.cpython-311-darwin.so, 2): Library not loaded: @rpath/libSDL2_ttf-2.0.15.0.0.dylib
2024-02-20T14:54:37.0728320Z   Referenced from: $PREFIX/lib/python3.11/site-packages/pygame/font.cpython-311-darwin.so
2024-02-20T14:54:37.0728800Z   Reason: image not found)
2024-02-20T14:54:37.0729550Z FAILED tests/envs/test_env_implementation.py::test_carracing_domain_randomize - NotImplementedError: font module not available (ImportError: dlopen($PREFIX/lib/python3.11/site-packages/pygame/font.cpython-311-darwin.so, 2): Library not loaded: @rpath/libSDL2_ttf-2.0.15.0.0.dylib
2024-02-20T14:54:37.0730490Z   Referenced from: $PREFIX/lib/python3.11/site-packages/pygame/font.cpython-311-darwin.so
2024-02-20T14:54:37.0731000Z   Reason: image not found)
2024-02-20T14:54:37.0731720Z FAILED tests/envs/test_envs.py::test_all_env_api[CarRacing-v2] - NotImplementedError: font module not available (ImportError: dlopen($PREFIX/lib/python3.11/site-packages/pygame/font.cpython-311-darwin.so, 2): Library not loaded: @rpath/libSDL2_ttf-2.0.15.0.0.dylib
2024-02-20T14:54:37.0732810Z   Referenced from: $PREFIX/lib/python3.11/site-packages/pygame/font.cpython-311-darwin.so
2024-02-20T14:54:37.0733140Z   Reason: image not found)
2024-02-20T14:54:37.0733830Z FAILED tests/envs/test_envs.py::test_all_env_passive_env_checker[CarRacing-v2] - NotImplementedError: font module not available (ImportError: dlopen($PREFIX/lib/python3.11/site-packages/pygame/font.cpython-311-darwin.so, 2): Library not loaded: @rpath/libSDL2_ttf-2.0.15.0.0.dylib
2024-02-20T14:54:37.0735930Z   Referenced from: $PREFIX/lib/python3.11/site-packages/pygame/font.cpython-311-darwin.so
2024-02-20T14:54:37.0736510Z   Reason: image not found)
2024-02-20T14:54:37.0737730Z FAILED tests/envs/test_envs.py::test_env_determinism_rollout[CarRacing-v2] - NotImplementedError: font module not available (ImportError: dlopen($PREFIX/lib/python3.11/site-packages/pygame/font.cpython-311-darwin.so, 2): Library not loaded: @rpath/libSDL2_ttf-2.0.15.0.0.dylib
2024-02-20T14:54:37.0738940Z   Referenced from: $PREFIX/lib/python3.11/site-packages/pygame/font.cpython-311-darwin.so
2024-02-20T14:54:37.0739270Z   Reason: image not found)

@traversaro
Copy link
Contributor Author

While macOS test (except on Python 3.8) are failing with:

This problem may be related to conda-forge/sdl2_ttf-feedstock#18 . If that is true, I guess the problem is also affecting older releases of gymnasium and the workaround 6cba969 should make the test pass.

@traversaro
Copy link
Contributor Author

While macOS test (except on Python 3.8) are failing with:

This problem may be related to conda-forge/sdl2_ttf-feedstock#18 . If that is true, I guess the problem is also affecting older releases of gymnasium and the workaround 6cba969 should make the test pass.

Unfortunately also that fails, with error:

traversaro@IITBMP014LW012:~$ CONDA_SUBDIR=osx-64 mamba create -n test636 pygame=2.1.3 sdl2_ttf==2.0.15

Looking for: ['pygame=2.1.3', 'sdl2_ttf==2.0.15']

warning  libmamba osx version not found (virtual package skipped)
conda-forge/noarch                                  13.6MB @   4.7MB/s  2.9s
conda-forge/osx-64                                  28.9MB @   6.3MB/s  4.6s
Could not solve for environment specs
The following packages are incompatible
├─ pygame 2.1.3**  is installable with the potential options
│  ├─ pygame 2.1.3.dev8 would require
│  │  └─ sdl2_ttf >=2.20.1,<3.0a0  with the potential options
│  │     ├─ sdl2_ttf 2.20.1, which can be installed;
│  │     ├─ sdl2_ttf 2.20.2, which can be installed;
│  │     └─ sdl2_ttf 2.22.0 would require
│  │        └─ harfbuzz >=8.3.0,<9.0a0 , which requires
│  │           └─ __osx >=10.9 , which is missing on the system;
│  └─ pygame 2.1.3 would require
│     └─ sdl2_ttf >=2.20.2,<3.0a0  with the potential options
│        ├─ sdl2_ttf 2.20.2, which can be installed;
│        └─ sdl2_ttf 2.22.0, which cannot be installed (as previously explained);
└─ sdl2_ttf 2.0.15  is not installable because it conflicts with any installable versions previously reported.

@stephane-caron stephane-caron mentioned this pull request Feb 21, 2024
3 tasks
@traversaro
Copy link
Contributor Author

The only remaining failures are in linux-aarch64 and linux-ppc64le:

=========================== short test summary info ============================
ERROR tests/envs/mujoco/test_mujoco_custom_env.py - AttributeError: 'NoneType' object has no attribute 'glGetError'
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
================== 6 skipped, 36 warnings, 1 error in 41.09s ===================

I wonder if there are any yum_requirements.txt or similar missing.

By looking at the discussion in #28 (comment) , this was supposed to fixed by Farama-Foundation/Gymnasium#480 that should be included in v0.29.1 . However, probably there was some regression between Farama-Foundation/Gymnasium#480 in May 2023 and v0.29.1 tag in August 2023.

@traversaro
Copy link
Contributor Author

However, probably there was some regression between Farama-Foundation/Gymnasium#480 in May 2023 and v0.29.1 tag in August 2023.

Indeed, the failing test was added ~10 days after Farama-Foundation/Gymnasium#480 : https://github.com/Farama-Foundation/Gymnasium/pull/525/files .

@pseudo-rnd-thoughts
Copy link
Contributor

@traversaro Farama-Foundation/Gymnasium#932 should fix the windows issues

@traversaro
Copy link
Contributor Author

@traversaro Farama-Foundation/Gymnasium#932 should fix the windows issues

Thanks!

@traversaro
Copy link
Contributor Author

After applying Farama-Foundation/Gymnasium#932 Windows is happy.

@pseudo-rnd-thoughts
Copy link
Contributor

@traversaro are the reminding issues related to Pygame?

@traversaro
Copy link
Contributor Author

@traversaro are the reminding issues related to Pygame?

The OpenGL failures are probably due to Farama-Foundation/Gymnasium#480 not being propagated to the test added in https://github.com/Farama-Foundation/Gymnasium/pull/525/files . macos failures are due to some strange combination of how sdl_* libraries export their CMake config files (just on macos) and the version of sdl_* libraries used to build pygame, see conda-forge/sdl2_ttf-feedstock#18 . I think I can tackle this, but it will probably take some time.

@pseudo-rnd-thoughts
Copy link
Contributor

@ChristofKaufmann Do you understand the issue with Farama-Foundation/Gymnasium#480 and Farama-Foundation/Gymnasium#525?

@traversaro
Copy link
Contributor Author

traversaro commented Mar 29, 2024

If you're starting to get the hang about these libraries, would you consider trying your hand at a PR to staged-recipes for that? 🙃

The amount of time I am hitting OpenGL-related problems (for example conda-forge/staged-recipes#25910 (comment)) may soon be pushing me toward that. However, I definitively need to understand more how a conda-forge libglvnd would find the system vendor-specific OpenGL libraries.

@traversaro
Copy link
Contributor Author

@conda-forge-admin, please rerender

Copy link
Contributor

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you but ran into some issues. Please check the output logs of the latest webservices GitHub actions workflow run for errors. You can also ping conda-forge/core for further assistance or you can try rerendeing locally.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/gymnasium-feedstock/actions/runs/8482911048.

@traversaro
Copy link
Contributor Author

Ok, network issues with anaconda.org website:

INFO:webservices_dispatch_action.rerendering:rerendering
Traceback (most recent call last):
  File "/opt/conda/bin/conda-smithy", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/conda_smithy/cli.py", line 737, in main
    args.subcommand_func(args)
  File "/opt/conda/lib/python3.11/site-packages/conda_smithy/cli.py", line 584, in __call__
    self._call(args, tmpdir)
  File "/opt/conda/lib/python3.11/site-packages/conda_smithy/cli.py", line 589, in _call
    configure_feedstock.main(
  File "/opt/conda/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 2615, in main
    exclusive_config_file, cf_pinning_ver = get_cached_cfp_file_path(
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 2436, in get_cached_cfp_file_path
    get_cfp_file_path(smithy_cache)
  File "/opt/conda/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 2360, in get_cfp_file_path
    pkg = get_most_recent_version("conda-forge-pinning")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 2299, in get_most_recent_version
    request.raise_for_status()
  File "/opt/conda/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 520 Server Error:  for url: https://api.anaconda.org/package/conda-forge/conda-forge-pinning
INFO:webservices_dispatch_action.utils:token can change workflows: False
INFO:webservices_dispatch_action.utils:pushing and commenting: branch|owner|repo = patch-1|traversaro|gymnasium-feedstock
Traceback (most recent call last):
  File "/opt/conda/bin/run-webservices-dispatch-action", line 33, in <module>
    sys.exit(load_entry_point('webservices-dispatch-action', 'console_scripts', 'run-webservices-dispatch-action')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/webservices_dispatch_action/webservices_dispatch_action/__main__.py", line 94, in main
    raise RuntimeError(
RuntimeError: Rerendering failed! error in push|rerender: False|True

@traversaro
Copy link
Contributor Author

@conda-forge-admin, please rerender

conda-forge-webservices[bot] and others added 2 commits March 29, 2024 16:14
@traversaro
Copy link
Contributor Author

Interestingly, re-adding the logic to ignore test_mujoco_custom_env does not fix the linux-aarch64 segfault, indicating that probably it is either related to mesalib being installed, or some other form of environment regression, as in #32 (comment) the tests were passing on linux-aarch64 .

@traversaro
Copy link
Contributor Author

@conda-forge-admin, please rerender

Copy link
Contributor

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/gymnasium-feedstock/actions/runs/8490282239.

@traversaro
Copy link
Contributor Author

If you're starting to get the hang about these libraries, would you consider trying your hand at a PR to staged-recipes for that? 🙃

The amount of time I am hitting OpenGL-related problems (for example conda-forge/staged-recipes#25910 (comment)) may soon be pushing me toward that. However, I definitively need to understand more how a conda-forge libglvnd would find the system vendor-specific OpenGL libraries.

OT, a first draft PR is available at conda-forge/staged-recipes#25919 .

@traversaro
Copy link
Contributor Author

Ok, a bit of recap. The aarch64 tests are crashing as it is trying to run the test_render_modes[tabular/Blackjack-v0] and test_render_modes[tabular/CliffWalking-v0], that require jax even if jax is not installed (instead, in linux-ppc64le these tests are correctly skipped). Note that I do not know why this result in a segfault, instead of a normal failure. Anyhow, by skipping test_render_modes (I was not able to just skip the specific tests) the compilation and the rest of the tests work fine, so just to draw a line, I think it is ok to merge this PR by just skipping the test_render_modes in linux-aarch64.

@traversaro
Copy link
Contributor Author

@conda-forge/gymnasium the PR is ready for review.

Copy link
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically LGTM aside from some nits.

Thanks so much for chewing through this! 🙏

@h-vetinari
Copy link
Member

I was not able to just skip the specific tests

If you use the serialisation of the parametrised test as you see it in the logs, you can skip specific combinations.

It would be great if we can robustify upstream to not segfault on missing jax, but in any case, skipping that test (even in its entirety) is a reasonable thing to do for now.

PS. Please consider adding yourself as a maintainer here! 🙃

@traversaro
Copy link
Contributor Author

Comments addressed in 1ad9513 .

@h-vetinari
Copy link
Member

@conda-forge-admin, please rerender

Copy link
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💪🙏

@h-vetinari h-vetinari added the automerge Merge the PR when CI passes label Apr 2, 2024
@github-actions github-actions bot merged commit b92012a into conda-forge:main Apr 2, 2024
26 checks passed
Copy link
Contributor

github-actions bot commented Apr 2, 2024

Hi! This is the friendly conda-forge automerge bot!

I considered the following status checks when analyzing this PR:

  • linter: passed
  • azure: passed

Thus the PR was passing and merged! Have a great day!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge Merge the PR when CI passes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants