feat: Add get_progress and set_progress to redis result backend

[sminnee]

Uses as standard suffix on the redis key (hardcoded as "__progress") to store progress results.

[sminnee]

A few questions about this:

• Is a key with suffix __progress a good default? Should the suffix be configurable?
• When should the data expire? Using redis.getdel() as with get_results is probably not appropriate as it's much more likely to be fetched repeatedly, but maybe it should have a shorter default expiry time? (2h?)
• Should it be possible to configure progress expiry separately from results expiry?

I also note that there's a fair bit of duplication in RedisAsyncClusterResultBackend, RedisAsyncResultBackend, and RedisAsyncSentinelResultBackend. They could possibly be refactored to have a common base class with most of the implementation, and the subclass just defining a method that provides the redis connection object as a resource (i.e. with enter() and exit() even if it's not strictly necessary). That refactor is beyond the scope of this PR, but I'm happy to do it.

CC @Sobes76rus as you did the original set_/get_progress implementation

[Sobes76rus]

Hello, I think the __progress suffix is good enough and doesn't need configuration
Expiration should be customisable, and it would be even better if it is customisable for each task for results and for progress based on labels

[sminnee]

OK so should I add 2 more arguments to the result backend(s) constructor?

• progress_ex_time: Optional[int] = 600
• progress_px_time: Optional[int] = None

I'm happy to add that to this PR, if @s3rius gives a 👍.

The bit I'm least confident about is the 600 default - it's a little arbitrary, but I think some default is better than no expiry. My rationale is that if there have been no progress updates in the last 10 minutes, something has probably gone wrong.

[sminnee]

FYI I believe I've fixed the build now, sorry about missing that in the first PR!

[sminnee]

@s3rius it would be great to get your input on this, if you have any? :)

[sminnee]

Hey, just giving this a nudge. I'm running my project on a fork so no great hurry, although it would be good to know if you're on board with the direction I took?

[s3rius]

It looks good to me. Let's check tests and release it if it's fine.

[s3rius]

Can you please rebase onto the main branch? I've fixed the workflows.

[sminnee]

done

[pdjohntony]

Any ETA on this?