Skip to content

Comments

Implement wire protocol version compatibility semantics#113

Draft
conradbzura wants to merge 6 commits intomainfrom
protobuf-version-handshake
Draft

Implement wire protocol version compatibility semantics#113
conradbzura wants to merge 6 commits intomainfrom
protobuf-version-handshake

Conversation

@conradbzura
Copy link
Contributor

Summary

Add protocol version awareness to Wool's worker communication layer. Incompatible workers (different major version) are filtered out during discovery before they ever reach the load balancer. As a secondary defense, the dispatch handshake carries version information in the Ack response, with best-effort Nack for major-version mismatches when the wire format is still parseable.

A new README for the protobuf subpackage documents the dispatch wire protocol, serialization strategy, Python binding layout, schema evolution rules, and the version compatibility semantics introduced here.

Closes #102

Proposed changes

Discovery-time version filtering

Add a major-version predicate to WorkerProxy._create_security_filter() (or alongside it) in wool/src/wool/runtime/worker/proxy.py. The filter extracts the major version from WorkerMetadata.version and compares it against wool.__version__. Workers with a different major version are excluded from the load balancer context, matching the existing security filter pattern.

Proto schema changes

Add a version string field to Ack in wool/protobuf/worker.proto and a version string field to Task in wool/protobuf/task.proto. Regenerate Python bindings. These are additive-only changes — older clients/workers that don't populate the new fields will see empty strings (proto3 defaults), which is safe.

Worker service version population

In wool/src/wool/runtime/worker/service.py, populate Ack(version=wool.__version__) when acknowledging a task. Before executing, parse the client's version from Task.version. If the major version doesn't match, yield a Nack instead. This is best-effort — truly incompatible wire formats will fail before reaching this path.

Client-side ack version check

In wool/src/wool/runtime/worker/connection.py, after receiving the Ack, store or log the worker's version from ack.version for observability.

Task version population

In wool/src/wool/runtime/routine/task.py, populate Task.version with wool.__version__ in to_protobuf().

Protobuf README

New file wool/src/wool/runtime/protobuf/README.md documenting: dispatch wire protocol, serialization strategy (protobuf envelope + cloudpickle payloads), Python binding layout, schema evolution rules, and version compatibility semantics.

Test cases

Test Suite Test ID Given When Then Coverage Target
TestWorkerProxy VP-001 A proxy with wool version 1.x A worker with version 1.y is discovered Worker is accepted into the load balancer context Same major version passes filter
TestWorkerProxy VP-002 A proxy with wool version 1.x A worker with version 2.y is discovered Worker is rejected by the discovery filter Different major version filtered
TestWorkerProxy VP-003 A proxy with wool version 1.x A worker with version "unknown" is discovered Worker is rejected by the discovery filter Unparseable version filtered
TestWorkerProxy VP-004 A proxy with credentials and version 1.x A secure worker with version 2.y is discovered Worker is rejected (version filter takes precedence over security match) Combined filter interaction
TestWorkerService VP-005 A running worker service A task is dispatched Ack response contains wool.__version__ Ack version population
TestWorkerService VP-006 A running worker service with version 1.x A task with version 2.y is dispatched Worker responds with Nack citing version mismatch Best-effort Nack on major mismatch
TestWorkerService VP-007 A running worker service with version 1.x A task with empty version is dispatched Worker accepts and processes normally Backwards compat with old clients
TestWorkerConnection VP-008 A mock worker returning Ack with version dispatch() is called Ack is accepted and stream is returned Ack version field passthrough
TestTask VP-009 A Task instance to_protobuf() is called Protobuf Task contains wool.__version__ in version field Task version serialization
TestTask VP-010 A protobuf Task with version field set from_protobuf() is called Version field is preserved in roundtrip Task version deserialization

Implementation plan

    • Add version field to Ack in worker.proto and Task in task.proto; regenerate bindings
    • Write discovery-time version filter tests (VP-001 through VP-004)
    • Implement major-version filter in proxy.py
    • Write service-side version handling tests (VP-005 through VP-007)
    • Populate Ack.version and implement best-effort Nack in service.py
    • Write client-side and task serialization tests (VP-008 through VP-010)
    • Populate Task.version in task.py; handle ack.version in connection.py
    • Write protobuf subpackage README

@conradbzura conradbzura added documentation Indicates that a PR includes changes to documentation feature New feature or capability labels Feb 21, 2026
Insert version as field 1 in Task (renumber existing fields) and Ack.
Add TaskVersionEnvelope message for pre-deserialization version
extraction by the server interceptor. Export TaskVersionEnvelope from
the protobuf wrapper module.

References #102
Workers whose major version differs from the local proxy are now
excluded during discovery. Applies to both pool_uri and static
workers paths, alongside the existing security filter. Workers with
unparseable version strings are also rejected.

References #102
VersionInterceptor extracts the version from raw request bytes via
TaskVersionEnvelope before full deserialization. Incompatible major
versions receive a Nack without attempting to deserialize the rest of
the payload. Empty version fields pass through for backwards
compatibility. Ack now carries wool.__version__ for observability.

References #102
Task.to_protobuf() now stamps wool.__version__ into field 1 so the
server interceptor can perform pre-deserialization version checking.
WorkerConnection._dispatch() handles Nack responses by raising
RpcError with the rejection reason.

References #102
Cover discovery-time version filtering (property-based tests for
compatible, incompatible, and unparseable versions plus combined
filter composition), dispatch-time interceptor behavior (Ack version,
empty version passthrough, incompatible version Nack), client-side
Nack handling, and task serialization version round-trip. Fix two
existing proxy tests that now need wool.__version__ patched to match
test worker versions.

References #102
Document the dispatch wire protocol sequence, hybrid serialization
strategy, Python binding layout and regeneration, schema evolution
rules, and version compatibility semantics including the
TaskVersionEnvelope-based pre-deserialization interceptor.

References #102
@conradbzura conradbzura force-pushed the protobuf-version-handshake branch from 12815ea to c4b4b25 Compare February 21, 2026 18:43
@wool-labs wool-labs bot added the code-change Indicates that a PR should trigger a release label Feb 21, 2026
@@ -0,0 +1,113 @@
# Protobuf Subpackage

Wire protocol definitions and generated Python bindings for wool's
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When referring to "wool" in plain text, capitalize it.

@@ -0,0 +1,113 @@
# Protobuf Subpackage
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's go with "Wire protocol" for the heading. Also, a styling nit: section headings should only have the first word capitalized — see the other subpackage READMEs for reference.

Regenerate with:

```bash
uv pip install -e './wool[dev]'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uv isn't necessary here. I happen to use it for development of the project, but users don't have to use it to install Wool themselves.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This document reads well for Wool contributors that will need to understand the wire protocol internals, but I'd prefer this document to be written for the user persona.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code-change Indicates that a PR should trigger a release documentation Indicates that a PR includes changes to documentation feature New feature or capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement wire protocol version compatibility semantics

1 participant