BUG - Fails to install with Python 3.12

[Finwood]

Describe the bug

When inside a Python 3.12 environment, east fails to install.

/tmp/east > python -m pip install east-tool
Collecting east-tool
  Using cached east_tool-0.18.3-py3-none-any.whl.metadata (5.0 kB)
Collecting rich==12.5.1 (from east-tool)
  Using cached rich-12.5.1-py3-none-any.whl.metadata (18 kB)
Collecting click==8.1.3 (from east-tool)
  Using cached click-8.1.3-py3-none-any.whl.metadata (3.2 kB)
Collecting rich-click==1.5.2 (from east-tool)
  Using cached rich_click-1.5.2-py3-none-any.whl.metadata (18 kB)
Collecting requests==2.28.1 (from east-tool)
  Using cached requests-2.28.1-py3-none-any.whl.metadata (4.6 kB)
Collecting PyYAML==6.0 (from east-tool)
  Using cached PyYAML-6.0.tar.gz (124 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error


  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [54 lines of output]
      running egg_info
      writing lib/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/tmp/east/.venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>                                                            main()
        File "/tmp/east/.venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main                                                                json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/east/.venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel                                        return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel                                                  return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires                                                           self.run_setup()
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup                                                                     exec(code, locals())
        File "<string>", line 288, in <module>
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup                                                                           return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup                                                                    return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands                                                             dist.run_commands()
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 970, in run_commands                                                             self.run_command(cmd)
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 974, in run_command                                                                         super().run_command(command)
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 989, in run_command                                                              cmd_obj.run()
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run                                                                     self.find_sources()
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources                                                            mm.run()
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 550, in run                                                                     self.add_defaults()
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 588, in add_defaults                                                            sdist.add_defaults(self)
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/command/sdist.py", line 102, in add_defaults                                                               super().add_defaults()
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 250, in add_defaults                                                    self._add_defaults_ext()
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 335, in _add_defaults_ext                                               self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 204, in get_source_files
        File "/tmp/pip-build-env-tjclc1ny/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__                                                               raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Firmware/Software Version

east-tool 0.18.3

To Reproduce

1. Create a Python 3.12 environment, or start a python 3.12 container
2. Inside that environment, install pip
3. Try to install east with pip 3.12 (python -m pip install east-tool)

[MarkoSagadin]

@Finwood I have setup a Ubuntu docker image with the Python3.13 and have run into the same issue as you did above. The solution was just to bump up the PyYAML dependency. See #107.

Can you test building East on your machine just so that we are sure that this was also resolved on your side?

I expect that following commands will bring you to the point where you end up building the East:

git clone https://github.com/IRNAS/irnas-east-software
cd irnas-east-software
git checkout feature/python3.12+support
pip install virtualenv
virtualenv venv
source venv/bin/activate
make install

No error should be emitted by the last command.

[Finwood]

East builds successfully, all good here.

Running it results in a syntax warning, though:

irnas-east-software ❯ east --version
/tmp/irnas-east-software/src/east/__main__.py:93: SyntaxWarning: invalid escape sequence '\['
  """[bold]East[/] is a command line meta-tool, useful for creating, managing and
/tmp/irnas-east-software/src/east/helper_functions.py:269: SyntaxWarning: invalid escape sequence '\.'
  pattern = f"^{west_board}_[0-9]?[0-9]?_[0-9]?[0-9]?_[0-9]?[0-9]?\.conf"
/tmp/irnas-east-software/src/east/workspace_commands/codechecker_helpers.py:157: SyntaxWarning: invalid escape sequence '\('
  f".*__ASSERT.*\(.*{var}.*,.*",
/tmp/irnas-east-software/src/east/workspace_commands/codechecker_helpers.py:158: SyntaxWarning: invalid escape sequence '\('
  f".*LOG_.*\(.*,.*{var}.*",
0.18.3.post1+g3dadcb1

Fedora 39, Python 3.12.3

[MarkoSagadin]

Can you open the first offending file, irnas-east-software/src/east/__main__.py and add extra backslash before the original one, on the line 99?

It should look like this:

    Run [bold]east \\[command] --help[/] to show documentation for that command.

I don't get the error that you get, but I get a linter warning on this line and adding the backslash fixes this specific error.

If that is ok, I will fix the other ones.

[Finwood]

I can confirm, the extra backslash solves the issue.

The SyntaxWarning gets thrown upon compilation, that's why you may not see it. This should help:

> find src -type f -name '*.pyc' -delete  # remove all python byte code
> PYTHONWARNINGS=always::SyntaxWarning east --version
/tmp/irnas-east-software/src/east/helper_functions.py:269: SyntaxWarning: invalid escape sequence '\.'
  pattern = f"^{west_board}_[0-9]?[0-9]?_[0-9]?[0-9]?_[0-9]?[0-9]?\.conf"
/tmp/irnas-east-software/src/east/workspace_commands/codechecker_helpers.py:157: SyntaxWarning: invalid escape sequence '\('
  f".*__ASSERT.*\(.*{var}.*,.*",
/tmp/irnas-east-software/src/east/workspace_commands/codechecker_helpers.py:158: SyntaxWarning: invalid escape sequence '\('
  f".*LOG_.*\(.*,.*{var}.*",
0.18.3.post1+g3dadcb1

[MarkoSagadin]

@Finwood I pushed the other fixes to the feature/python3.12+support, please pull and let's see if this is now completely resolved.

[Finwood]

Everything resolved 👍

[MarkoSagadin]

Great, I will create now new release of east-tool.