Skip to content

Comments

Add namespace isolation for routine tasks#111

Open
conradbzura wants to merge 4 commits intomainfrom
namespaced-tasks
Open

Add namespace isolation for routine tasks#111
conradbzura wants to merge 4 commits intomainfrom
namespaced-tasks

Conversation

@conradbzura
Copy link
Contributor

Summary

Add a namespace parameter to the @wool.routine decorator that controls a task's access to global variables on a given worker. Default to a unique, isolated namespace per task invocation to prevent global state leakage between unrelated tasks. Provide three modes: ephemeral (default), named shared, and worker-level. Also bump grpcio dependency to 1.78.0.

Closes #104

Proposed changes

namespace parameter on @wool.routine

Introduce an optional namespace parameter to the @wool.routine decorator with three modes:

@wool.routine  # namespace=None (default) — ephemeral isolated globals per invocation
async def isolated_task(): ...

@wool.routine(namespace="cached")  # shared globals with other "cached" tasks
@lru_cache
async def cached_computation(n): ...

@wool.routine(namespace=wool.WORKER)  # previous behavior — shared worker globals
async def worker_level_task(): ...

Refactor the decorator to support bare @routine, @routine(), and @routine(namespace=...) forms via @overload signatures.

_IsolatedGlobals overlay dict

Add an _IsolatedGlobals dict subclass in task.py that provides read-through semantics to the original function's __globals__. Writes go to the overlay; reads fall through to the original globals when the key is not found locally. This must be a dict subclass (not MutableMapping) because CPython's STORE_GLOBAL bytecode uses PyDict_SetItem at the C level.

Namespace lifecycle via ResourcePool

Use the existing ResourcePool for named namespace lifecycle management with TTL-based cleanup (default 5 minutes). The _prepare_callable async context manager on Task handles all three modes — ephemeral, named, and worker-level — constructing a types.FunctionType with the appropriate globals dict.

Namespace inheritance for shared namespaces

When a task enters a named namespace, merge the callable's module globals into the shared _IsolatedGlobals instance without overwriting existing keys. This allows tasks from different modules to share a namespace while each bringing its required imports, and prevents later tasks from overwriting state (like caches) established by earlier tasks.

Protobuf schema update

Add optional string namespace = 14 to the Task message in task.proto. Serialize None as empty string; deserialize empty string back to None.

Test cases

Test Suite Test ID Given When Then Coverage Target
TestNamespaceIsolation NS-001 A Task created without specifying namespace Task is instantiated The namespace field defaults to None Default field value
TestNamespaceIsolation NS-002 A Task created with namespace=None Task is instantiated The namespace field is None Explicit None
TestNamespaceIsolation NS-003 A Task created with namespace=WORKER Task is instantiated The namespace field is the WORKER sentinel WORKER sentinel
TestNamespaceIsolation NS-004 A Task created with namespace="cache" Task is instantiated The namespace field is "cache" Named namespace
TestNamespaceIsolation NS-005 A Task with namespace=None (default) The callable sets a global variable The global is not visible in the original namespace Coroutine isolation
TestNamespaceIsolation NS-006 A Task with namespace=WORKER The callable sets a global variable The global is visible in the original namespace Worker globals persist
TestNamespaceIsolation NS-007 A Task with async generator callable and namespace=None The callable sets a global variable The global is not visible in the original namespace Async generator isolation
TestNamespaceIsolation NS-008 A Task with async generator callable and namespace=WORKER The callable sets a global variable The global is visible in the original namespace Async generator worker globals
TestNamespaceIsolation NS-009 A Task with namespace=None The callable accesses a module-level import The import is accessible through the overlay Read-through semantics
TestNamespaceIsolation NS-010 A Task with namespace=None The callable shadows a module-level name The original module-level name is unchanged Write isolation
TestNamespaceIsolation NS-011 A Task with namespace=None and a closure The callable accesses closure variables The closure variables are accessible Closure preservation
TestNamespaceIsolation NS-012 A protobuf Task with namespace="cache" from_protobuf is called The deserialized Task has namespace="cache" Protobuf deserialization
TestNamespaceIsolation NS-013 A protobuf Task with namespace=WORKER from_protobuf is called The deserialized Task has namespace=WORKER Protobuf WORKER deserialization
TestNamespaceIsolation NS-014 A protobuf Task without namespace field set from_protobuf is called The deserialized Task has namespace=None Protobuf default
TestNamespaceIsolation NS-015 A Task with namespace="cache" to_protobuf is called The protobuf Task has namespace="cache" Protobuf serialization
TestNamespaceIsolation NS-016 A Task with namespace=None to_protobuf is called The protobuf Task has namespace="" Protobuf None serialization
TestNamespaceIsolation NS-EC-001 A Task with namespace=None accessing a mutable global The callable mutates the mutable object The mutation is visible (reference semantics) Mutable object edge case
TestNamespaceIsolation NS-EC-002 A Task with namespace=None The callable shadows then deletes a global name The original namespace is unaffected Deletion edge case
TestNamespaceIsolation NS-EC-003 A Task with namespace=None calling a nested function The nested function sets a global The global is isolated to the overlay Nested call isolation
TestNamespaceIsolation NS-EC-004 A Task with namespace=None that raises The callable sets a global then raises The global is still isolated Exception isolation
TestNamespaceIsolation NS-EC-005 A Task with namespace=None that gets cancelled The callable sets a global then is cancelled The global is still isolated Cancellation isolation
TestNamespaceIsolation NS-SHARE-001 Two Tasks with namespace="shared" First task sets a global, second task reads it The second task sees the global set by the first Named namespace sharing
TestNamespaceIsolation NS-SHARE-002 Two Tasks with different namespaces First task sets a global, second task reads it The second task does not see the global Cross-namespace isolation
TestNamespaceIsolation PBT-NS-001 Random namespace strings Serialization round-trip via protobuf Namespace value is preserved Property-based serialization
TestRoutineDecorator RD-NS-001 @wool.routine bare decorator Decorator is applied Function is wrapped with namespace=None Bare decorator form
TestRoutineDecorator RD-NS-002 @wool.routine() empty call Decorator is applied Function is wrapped with namespace=None Empty call form
TestRoutineDecorator RD-NS-003 @wool.routine(namespace="cache") Decorator is applied Function is wrapped with namespace="cache" Named namespace form
TestRoutineDecorator RD-NS-004 @wool.routine(namespace=wool.WORKER) Decorator is applied Function is wrapped with namespace=WORKER WORKER namespace form

Implementation plan

    • Add optional string namespace field to Task message in task.proto; regenerate bindings
    • Add _IsolatedGlobals dict subclass and _namespace_registry ResourcePool to routine/task.py
    • Add namespace field to Task dataclass and implement _prepare_callable context manager
    • Update Task.from_protobuf and to_protobuf for namespace serialization
    • Integrate _prepare_callable into _run and _stream execution paths
    • Refactor routine decorator in routine/wrapper.py with @overload signatures and namespace parameter passthrough
    • Export WORKER sentinel from wool.__init__
    • Write unit tests for all namespace modes, isolation, and serialization (NS-001 through PBT-NS-001)
    • Write decorator tests for all usage forms (RD-NS-001 through RD-NS-004)
    • Bump grpcio dependency to 1.78.0 in pyproject.toml

@wool-labs wool-labs bot added the code-change Indicates that a PR should trigger a release label Feb 21, 2026
Introduce a namespace parameter to Task that controls global variable
isolation during execution:

- None (default): ephemeral isolated globals per invocation
- "name": shared globals across tasks using the same namespace,
  enabling patterns like @lru_cache between related tasks
- wool.WORKER: no isolation, runs in worker-level globals

_IsolatedGlobals provides dict-subclass overlay semantics required by
CPython's STORE_GLOBAL bytecode. Shared namespaces are managed via a
ResourcePool-backed registry with TTL-based cleanup.
Extend routine() to accept an optional namespace keyword argument,
forwarded to Task during dispatch. Support bare @routine, nullary
@routine(), and parameterized @routine(namespace="name") call forms.
Cover _IsolatedGlobals overlay semantics, _namespace_registry lifecycle,
Task.namespace serialization round-trips, and the three routine()
call forms (bare, nullary, parameterized). Add WORKER to the expected
public API surface.
@conradbzura conradbzura added the feature New feature or capability label Feb 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code-change Indicates that a PR should trigger a release feature New feature or capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add namespace isolation for routine tasks

1 participant