Unable to install on airgapped machine with new python and pip installation (ESPTOOL-884)

[bryghtlabs-richard]

Operating System

Windows 11

Esptool Version

esptool.py v4.8.dev4

Python Version

Python 3.11.9

Full Esptool Command Line that Was Run

pip install --no-index --find-links="." esptool==4.8.dev4

Esptool Output

### On Source Machine ###

C:\Users\RichardAllen\esptoolUpdater>pip download esptool==4.8.dev4
Collecting esptool==4.8.dev4
  Using cached esptool-4.8.dev4.tar.gz (332 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting bitstring>=3.1.6 (from esptool==4.8.dev4)
  Using cached bitstring-4.2.3-py3-none-any.whl.metadata (5.0 kB)
Collecting cryptography>=2.1.4 (from esptool==4.8.dev4)
  Using cached cryptography-42.0.8-cp39-abi3-win_amd64.whl.metadata (5.4 kB)
Collecting ecdsa>=0.16.0 (from esptool==4.8.dev4)
  Using cached ecdsa-0.19.0-py2.py3-none-any.whl.metadata (29 kB)
Collecting pyserial>=3.3 (from esptool==4.8.dev4)
  Using cached pyserial-3.5-py2.py3-none-any.whl.metadata (1.6 kB)
Collecting reedsolo<1.8,>=1.5.3 (from esptool==4.8.dev4)
  Using cached reedsolo-1.7.0-py3-none-any.whl.metadata (23 kB)
Collecting PyYAML>=5.1 (from esptool==4.8.dev4)
  Using cached PyYAML-6.0.1-cp311-cp311-win_amd64.whl.metadata (2.1 kB)
Collecting intelhex (from esptool==4.8.dev4)
  Using cached intelhex-2.3.0-py2.py3-none-any.whl.metadata (2.7 kB)
Collecting bitarray<3.0.0,>=2.9.0 (from bitstring>=3.1.6->esptool==4.8.dev4)
  Using cached bitarray-2.9.2-cp311-cp311-win_amd64.whl.metadata (35 kB)
Collecting cffi>=1.12 (from cryptography>=2.1.4->esptool==4.8.dev4)
  Using cached cffi-1.16.0-cp311-cp311-win_amd64.whl.metadata (1.5 kB)
Collecting six>=1.9.0 (from ecdsa>=0.16.0->esptool==4.8.dev4)
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=2.1.4->esptool==4.8.dev4)
  Using cached pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
Using cached bitstring-4.2.3-py3-none-any.whl (71 kB)
Using cached cryptography-42.0.8-cp39-abi3-win_amd64.whl (2.9 MB)
Using cached ecdsa-0.19.0-py2.py3-none-any.whl (149 kB)
Using cached pyserial-3.5-py2.py3-none-any.whl (90 kB)
Using cached PyYAML-6.0.1-cp311-cp311-win_amd64.whl (144 kB)
Using cached reedsolo-1.7.0-py3-none-any.whl (32 kB)
Using cached intelhex-2.3.0-py2.py3-none-any.whl (50 kB)
Using cached bitarray-2.9.2-cp311-cp311-win_amd64.whl (126 kB)
Using cached cffi-1.16.0-cp311-cp311-win_amd64.whl (181 kB)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Saved c:\users\richardallen\esptoolupdater\esptool-4.8.dev4.tar.gz
Saved c:\users\richardallen\esptoolupdater\bitstring-4.2.3-py3-none-any.whl
Saved c:\users\richardallen\esptoolupdater\cryptography-42.0.8-cp39-abi3-win_amd64.whl
Saved c:\users\richardallen\esptoolupdater\ecdsa-0.19.0-py2.py3-none-any.whl
Saved c:\users\richardallen\esptoolupdater\pyserial-3.5-py2.py3-none-any.whl
Saved c:\users\richardallen\esptoolupdater\pyyaml-6.0.1-cp311-cp311-win_amd64.whl
Saved c:\users\richardallen\esptoolupdater\reedsolo-1.7.0-py3-none-any.whl
Saved c:\users\richardallen\esptoolupdater\intelhex-2.3.0-py2.py3-none-any.whl
Saved c:\users\richardallen\esptoolupdater\bitarray-2.9.2-cp311-cp311-win_amd64.whl
Saved c:\users\richardallen\esptoolupdater\cffi-1.16.0-cp311-cp311-win_amd64.whl
Saved c:\users\richardallen\esptoolupdater\six-1.16.0-py2.py3-none-any.whl
Saved c:\users\richardallen\esptoolupdater\pycparser-2.22-py3-none-any.whl
Successfully downloaded esptool bitstring cryptography ecdsa pyserial PyYAML reedsolo intelhex bitarray cffi six pycparser

C:\Users\RichardAllen\esptoolUpdater>dir
 Volume in drive C is OS
 Volume Serial Number is 561D-A61E

 Directory of C:\Users\RichardAllen\esptoolUpdater

07/08/2024  11:46 AM    <DIR>          .
07/08/2024  11:43 AM    <DIR>          ..
07/08/2024  11:46 AM           126,013 bitarray-2.9.2-cp311-cp311-win_amd64.whl
07/08/2024  11:46 AM            71,671 bitstring-4.2.3-py3-none-any.whl
07/08/2024  11:46 AM           181,495 cffi-1.16.0-cp311-cp311-win_amd64.whl
07/08/2024  11:46 AM         2,886,038 cryptography-42.0.8-cp39-abi3-win_amd64.whl
07/08/2024  11:46 AM           149,266 ecdsa-0.19.0-py2.py3-none-any.whl
07/08/2024  11:46 AM           332,041 esptool-4.8.dev4.tar.gz
07/08/2024  11:46 AM            50,914 intelhex-2.3.0-py2.py3-none-any.whl
07/08/2024  11:46 AM           117,552 pycparser-2.22-py3-none-any.whl
07/08/2024  11:46 AM            90,585 pyserial-3.5-py2.py3-none-any.whl
07/08/2024  11:46 AM           144,699 PyYAML-6.0.1-cp311-cp311-win_amd64.whl
07/08/2024  11:46 AM            32,360 reedsolo-1.7.0-py3-none-any.whl
07/08/2024  11:46 AM            11,053 six-1.16.0-py2.py3-none-any.whl
              12 File(s)      4,193,687 bytes
               2 Dir(s)  1,428,129,996,800 bytes free


### On Destination Machine ###

C:\Users\Tester7\Desktop\esptoolUpdater>pip install --no-index --find-links="." --no-build-isolation esptool==4.8.dev4
Looking in links: .
Processing c:\users\tester7\desktop\esptoolupdater\esptool-4.8.dev4.tar.gz
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      running dist_info
      creating C:\Users\Tester7\AppData\Local\Temp\pip-modern-metadata-f15w90hq\esptool.egg-info
      writing C:\Users\Tester7\AppData\Local\Temp\pip-modern-metadata-f15w90hq\esptool.egg-info\PKG-INFO
      writing dependency_links to C:\Users\Tester7\AppData\Local\Temp\pip-modern-metadata-f15w90hq\esptool.egg-info\dependency_links.txt
      writing entry points to C:\Users\Tester7\AppData\Local\Temp\pip-modern-metadata-f15w90hq\esptool.egg-info\entry_points.txt
      writing requirements to C:\Users\Tester7\AppData\Local\Temp\pip-modern-metadata-f15w90hq\esptool.egg-info\requires.txt
      writing top-level names to C:\Users\Tester7\AppData\Local\Temp\pip-modern-metadata-f15w90hq\esptool.egg-info\top_level.txt
      writing manifest file 'C:\Users\Tester7\AppData\Local\Temp\pip-modern-metadata-f15w90hq\esptool.egg-info\SOURCES.txt'
      reading manifest file 'C:\Users\Tester7\AppData\Local\Temp\pip-modern-metadata-f15w90hq\esptool.egg-info\SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      no previously-included directories found matching 'test'
      no previously-included directories found matching 'flasher_stub'
      no previously-included directories found matching '.github'
      no previously-included directories found matching 'docs'
      warning: no previously-included files found matching '.git*'
      adding license file 'LICENSE'
      writing manifest file 'C:\Users\Tester7\AppData\Local\Temp\pip-modern-metadata-f15w90hq\esptool.egg-info\SOURCES.txt'
      creating 'C:\Users\Tester7\AppData\Local\Temp\pip-modern-metadata-f15w90hq\esptool-4.8.dev4.dist-info'
      error: invalid command 'bdist_wheel'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

C:\Users\Tester7\Desktop\esptoolUpdater>pause
Press any key to continue . . .

What is the Expected Behaviour?

Running pip download esptool==4.8.dev4 should pull down all dependencies, including the wheel package, so that pip install --no-index --find-links="." --no-build-isolation can install the wheel dependency.

More Information

This is a dependency packaging issue, not an esptool issue directly.

Other Steps to Reproduce

The wheel package is quite common, I think we only found this because we were installing esptool on a new installation.

[bryghtlabs-richard]

I'm not very familiar with python packaging, but perhaps it is as simple as adding the wheel package to pyproject.toml? I'm not sure how to test this though, as the esptool-4.8.dev4.tar.gz created by running pip download esptool==4.8.dev4 does not contain a pyproject.toml.

[peterdragun]

Hi @bryghtlabs-richard

... pip download esptool==4.8.dev4 does not contain a pyproject.toml.

Yes, that is correct. 4.8.dev4 didn't use pyproject.toml yet (see https://github.com/espressif/esptool/tree/v4.8.dev4), we have switched to pyproject.toml after this release. I am not quite sure why in your case pip is trying to use pyproject.toml if it wasn't there back then. But that is an issue with either pip or environment.

Regarding your request, I am sorry but I don't think we should include the wheel package as a dependency for esptool, as it is not a dependency for the package itself but rather a dependency for the tools used for packaging. It is not common practice to include such tools as dependencies AFAIK.

We provide built binaries for people who don't want to use pip(or wheel) for installation, so I would recommend using those. You can find those in our releases (bottom part of the page after the changelog):
4.7.0: https://github.com/espressif/esptool/releases/tag/v4.7.0
for the latest dev release you can find them in the artifacts section of the build action:
4.8.dev4: https://github.com/espressif/esptool/actions/runs/9301686672

[bryghtlabs-richard]

Regarding your request, I am sorry but I don't think we should include the wheel package as a dependency for esptool, as it is not a dependency for the package itself but rather a dependency for the tools used for packaging.

Thanks @peterdragun , this makes sense to me. I'll look into trying the binaries.