Implement wire protocol version compatibility semantics#113
Draft
conradbzura wants to merge 6 commits intomainfrom
Draft
Implement wire protocol version compatibility semantics#113conradbzura wants to merge 6 commits intomainfrom
conradbzura wants to merge 6 commits intomainfrom
Conversation
Insert version as field 1 in Task (renumber existing fields) and Ack. Add TaskVersionEnvelope message for pre-deserialization version extraction by the server interceptor. Export TaskVersionEnvelope from the protobuf wrapper module. References #102
Workers whose major version differs from the local proxy are now excluded during discovery. Applies to both pool_uri and static workers paths, alongside the existing security filter. Workers with unparseable version strings are also rejected. References #102
VersionInterceptor extracts the version from raw request bytes via TaskVersionEnvelope before full deserialization. Incompatible major versions receive a Nack without attempting to deserialize the rest of the payload. Empty version fields pass through for backwards compatibility. Ack now carries wool.__version__ for observability. References #102
Task.to_protobuf() now stamps wool.__version__ into field 1 so the server interceptor can perform pre-deserialization version checking. WorkerConnection._dispatch() handles Nack responses by raising RpcError with the rejection reason. References #102
Cover discovery-time version filtering (property-based tests for compatible, incompatible, and unparseable versions plus combined filter composition), dispatch-time interceptor behavior (Ack version, empty version passthrough, incompatible version Nack), client-side Nack handling, and task serialization version round-trip. Fix two existing proxy tests that now need wool.__version__ patched to match test worker versions. References #102
Document the dispatch wire protocol sequence, hybrid serialization strategy, Python binding layout and regeneration, schema evolution rules, and version compatibility semantics including the TaskVersionEnvelope-based pre-deserialization interceptor. References #102
12815ea to
c4b4b25
Compare
conradbzura
commented
Feb 21, 2026
| @@ -0,0 +1,113 @@ | |||
| # Protobuf Subpackage | |||
|
|
|||
| Wire protocol definitions and generated Python bindings for wool's | |||
Contributor
Author
There was a problem hiding this comment.
When referring to "wool" in plain text, capitalize it.
conradbzura
commented
Feb 21, 2026
| @@ -0,0 +1,113 @@ | |||
| # Protobuf Subpackage | |||
Contributor
Author
There was a problem hiding this comment.
Let's go with "Wire protocol" for the heading. Also, a styling nit: section headings should only have the first word capitalized — see the other subpackage READMEs for reference.
conradbzura
commented
Feb 21, 2026
| Regenerate with: | ||
|
|
||
| ```bash | ||
| uv pip install -e './wool[dev]' |
Contributor
Author
There was a problem hiding this comment.
uv isn't necessary here. I happen to use it for development of the project, but users don't have to use it to install Wool themselves.
conradbzura
commented
Feb 21, 2026
Contributor
Author
There was a problem hiding this comment.
This document reads well for Wool contributors that will need to understand the wire protocol internals, but I'd prefer this document to be written for the user persona.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add protocol version awareness to Wool's worker communication layer. Incompatible workers (different major version) are filtered out during discovery before they ever reach the load balancer. As a secondary defense, the dispatch handshake carries version information in the
Ackresponse, with best-effortNackfor major-version mismatches when the wire format is still parseable.A new README for the protobuf subpackage documents the dispatch wire protocol, serialization strategy, Python binding layout, schema evolution rules, and the version compatibility semantics introduced here.
Closes #102
Proposed changes
Discovery-time version filtering
Add a major-version predicate to
WorkerProxy._create_security_filter()(or alongside it) inwool/src/wool/runtime/worker/proxy.py. The filter extracts the major version fromWorkerMetadata.versionand compares it againstwool.__version__. Workers with a different major version are excluded from the load balancer context, matching the existing security filter pattern.Proto schema changes
Add a
versionstring field toAckinwool/protobuf/worker.protoand aversionstring field toTaskinwool/protobuf/task.proto. Regenerate Python bindings. These are additive-only changes — older clients/workers that don't populate the new fields will see empty strings (proto3 defaults), which is safe.Worker service version population
In
wool/src/wool/runtime/worker/service.py, populateAck(version=wool.__version__)when acknowledging a task. Before executing, parse the client's version fromTask.version. If the major version doesn't match, yield aNackinstead. This is best-effort — truly incompatible wire formats will fail before reaching this path.Client-side ack version check
In
wool/src/wool/runtime/worker/connection.py, after receiving theAck, store or log the worker's version fromack.versionfor observability.Task version population
In
wool/src/wool/runtime/routine/task.py, populateTask.versionwithwool.__version__into_protobuf().Protobuf README
New file
wool/src/wool/runtime/protobuf/README.mddocumenting: dispatch wire protocol, serialization strategy (protobuf envelope + cloudpickle payloads), Python binding layout, schema evolution rules, and version compatibility semantics.Test cases
TestWorkerProxyTestWorkerProxyTestWorkerProxyTestWorkerProxyTestWorkerServicewool.__version__TestWorkerServiceTestWorkerServiceTestWorkerConnectiondispatch()is calledTestTaskto_protobuf()is calledwool.__version__in version fieldTestTaskfrom_protobuf()is calledImplementation plan
versionfield toAckinworker.protoandTaskintask.proto; regenerate bindingsproxy.pyAck.versionand implement best-effortNackinservice.pyTask.versionintask.py; handleack.versioninconnection.py