Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VIO-3108 Upgrade breaking gRPC version #73

Merged
merged 1 commit into from
Oct 3, 2023
Merged

Conversation

DillonEnge
Copy link
Contributor

No description provided.

python/setup.py Outdated
@@ -34,7 +34,7 @@
zip_safe=False,
install_requires=[
'grpcio>=1.8.6',
'protobuf>=3.20.3, <4',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kamilovio gave a great overview of the problem space in slack.

The issue can be surfaced by:

$ cd python
$ make test

and seeing this error:

E   TypeError: Descriptors cannot not be created directly.
E   If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E   If you cannot immediately regenerate your protos, some other possible workarounds are:
E    1. Downgrade the protobuf package to 3.20.x or lower.
E    2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E   
E   More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Inspecting the toolchain to generate pb files via docker run --rm -it --entrypoint bash grpc/python:1.4, the general output is something like:

root@73ba0a48909b:/# pip show grpcio
Name: grpcio
Version: 1.4.0
Summary: HTTP/2-based RPC framework
Home-page: http://www.grpc.io
Author: The gRPC Authors
Author-email: grpc-io@googlegroups.com
License: 3-clause BSD
Location: /usr/local/lib/python3.6/site-packages
Requires: protobuf, six
Required-by: grpcio-tools

The Makefile also does some text replacement for the python imports.

Checking against the new image: docker run --rm -it --entrypoint bash namely/protoc-all:latest - we get protoc 3+ as recommended by the docs at the time of writing.

root@d9e18be909a3:/defs# protoc --version
libprotoc 3.21.6

docker run -v $(pwd):/defs namely/protoc-all -f synse.proto -l python will regenerate the pb files under /gen.

Remaining pieces here are:

  1. Moving the autogenerated code to overwrite the old ones.
  2. Update import path or address this in the tests
  3. Fix file permissions

If we wanted to continue using python to manage the toolchain, I am okay with that but would like to see a corresponding Dockerfile with this PR.

@DillonEnge DillonEnge changed the title VIO-3108 Upgrade protobuf past 3.20 VIO-3108 Upgrade breaking gRPC version Sep 27, 2023
Copy link
Contributor

@GuessWhoSamFoo GuessWhoSamFoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good. Next steps would be bumping to 3.2 and incrementing https://github.com/vapor-ware/synse-server/blob/develop/pyproject.toml#L23

@DillonEnge DillonEnge marked this pull request as ready for review October 3, 2023 13:42
@DillonEnge DillonEnge merged commit 88e5649 into master Oct 3, 2023
@DillonEnge DillonEnge deleted the de/protobuf-upgrade branch October 3, 2023 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants