Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make retries first class citizens in ControlFlow #393

Open
discdiver opened this issue Dec 5, 2024 · 0 comments
Open

Make retries first class citizens in ControlFlow #393

discdiver opened this issue Dec 5, 2024 · 0 comments
Labels
enhancement Enhance an existing feature feature New feature that doesn't exist today

Comments

@discdiver
Copy link
Collaborator

discdiver commented Dec 5, 2024

Enhancement Description

The ability to retry on task failure-with options for retry delays including exponential backoff-would be useful functionality to interface from ControlFlow directly. It's possible with ControlFlow and Prefect, but requires knowledge of Prefect.

Use Case

Retrying API calls is a common need with LLMs.

Proposed Implementation

Adding directly to ControlFlow task keyword arguments might be nice.

```python
task = cf.Task(
    objective="Write a poem about the provided topic",
    instructions="Write four lines that rhyme",
    context={"topic": "AI"},
    retries=4
)

Context from Prefect Community Slack:

quick question: in ControlFlow is there a way to apply exponential backoff to avoid rate limiting? I got a few error messages after the agents running for a few minutes:
...

Solution from @zzstoatzz:

i would just wrap the thing you want to retry in a prefect task (since prefect is already installed when you use cf)

from prefect.tasks import exponential_backoff
from prefect import task

t = task(lambda: 1/0)

with_retries = t.with_options(retries=10, retry_delay_seconds=exponential_backoff(3))

you can do this inlined task(**options)(some_callable) syntax or the more traditional decorator syntax:

python
@task(retries=..., ...)
def some_callable(): ...

@discdiver discdiver added the enhancement Enhance an existing feature label Dec 5, 2024
@github-actions github-actions bot added the feature New feature that doesn't exist today label Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhance an existing feature feature New feature that doesn't exist today
Projects
None yet
Development

No branches or pull requests

1 participant