What to do if task IDs are different based on paths and __code__ hashing?

[fhilde13]

What to do if task ID are different based on the __code__ object of a task function?

prefect/src/prefect/tasks.py

Line 268 in c0d49d3

def _generate_task_key(fn: Callable[..., Any]) -> str:

The Problem:

My flow and task workers are deployed differently using different images (flow uses git clone step, task workers uses a dedicated image). Because the task ID is derived from the file path in the __code__ object, even minor path differences break communication between the flow and task workers.
To me this limits the amount of the ways that Flows and Task-worker can be separated as well as making task workers very hard to communicate to by other services. Which is essential since I wanna scale the task worker horizontally and want multiple flows and services to interact with it.

Questions:

1. Is there an alternative way to make task workers callable from different applications with different deployment paths?
2. What's the reason for using the full file path in the hash instead of something more portable? Is it even the design as intended?

Thanks in advance for the support!

[bdalpe]

@fhilde13 see #12340 regarding this behavior

[zzstoatzz]

added some color there: #12340 (comment)

[fhilde13]

aweseome thanks for the good work!