Open
Conversation
33f014f to
955eddc
Compare
Introduce a namespace parameter to Task that controls global variable isolation during execution: - None (default): ephemeral isolated globals per invocation - "name": shared globals across tasks using the same namespace, enabling patterns like @lru_cache between related tasks - wool.WORKER: no isolation, runs in worker-level globals _IsolatedGlobals provides dict-subclass overlay semantics required by CPython's STORE_GLOBAL bytecode. Shared namespaces are managed via a ResourcePool-backed registry with TTL-based cleanup.
Cover _IsolatedGlobals overlay semantics, _namespace_registry lifecycle, Task.namespace serialization round-trips, and the three routine() call forms (bare, nullary, parameterized). Add WORKER to the expected public API surface.
955eddc to
eb4156c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a
namespaceparameter to the@wool.routinedecorator that controls a task's access to global variables on a given worker. Default to a unique, isolated namespace per task invocation to prevent global state leakage between unrelated tasks. Provide three modes: ephemeral (default), named shared, and worker-level. Also bump grpcio dependency to 1.78.0.Closes #104
Proposed changes
namespaceparameter on@wool.routineIntroduce an optional
namespaceparameter to the@wool.routinedecorator with three modes:Refactor the decorator to support bare
@routine,@routine(), and@routine(namespace=...)forms via@overloadsignatures._IsolatedGlobalsoverlay dictAdd an
_IsolatedGlobalsdict subclass intask.pythat provides read-through semantics to the original function's__globals__. Writes go to the overlay; reads fall through to the original globals when the key is not found locally. This must be adictsubclass (notMutableMapping) because CPython'sSTORE_GLOBALbytecode usesPyDict_SetItemat the C level.Namespace lifecycle via
ResourcePoolUse the existing
ResourcePoolfor named namespace lifecycle management with TTL-based cleanup (default 5 minutes). The_prepare_callableasync context manager onTaskhandles all three modes — ephemeral, named, and worker-level — constructing atypes.FunctionTypewith the appropriate globals dict.Namespace inheritance for shared namespaces
When a task enters a named namespace, merge the callable's module globals into the shared
_IsolatedGlobalsinstance without overwriting existing keys. This allows tasks from different modules to share a namespace while each bringing its required imports, and prevents later tasks from overwriting state (like caches) established by earlier tasks.Protobuf schema update
Add
optional string namespace = 14to theTaskmessage intask.proto. SerializeNoneas empty string; deserialize empty string back toNone.Test cases
TestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationfrom_protobufis calledTestNamespaceIsolationfrom_protobufis calledTestNamespaceIsolationfrom_protobufis calledTestNamespaceIsolationto_protobufis calledTestNamespaceIsolationto_protobufis calledTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestNamespaceIsolationTestRoutineDecorator@wool.routinebare decoratorTestRoutineDecorator@wool.routine()empty callTestRoutineDecorator@wool.routine(namespace="cache")TestRoutineDecorator@wool.routine(namespace=wool.WORKER)Implementation plan
optional string namespacefield toTaskmessage intask.proto; regenerate bindings_IsolatedGlobalsdict subclass and_namespace_registryResourcePooltoroutine/task.pynamespacefield toTaskdataclass and implement_prepare_callablecontext managerTask.from_protobufandto_protobuffor namespace serialization_prepare_callableinto_runand_streamexecution pathsroutinedecorator inroutine/wrapper.pywith@overloadsignatures andnamespaceparameter passthroughWORKERsentinel fromwool.__init__pyproject.toml