You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
If an API call fails with a temporary error while running an extractor, we do not retry it. It could be improved.
Describe the solution you'd like
We should provide optional retries for extractors similar to sinks where the extractor call will be retried if the error returned is an instance of plugins.RetryError.
Describe alternatives you've considered
Each extractor could manage its own retries. But this does not seem to be a clean approach and is also likely to get missed when a new extractor is being added.
Additional context
We could make the retrier implementation available to extractors so that they can retry a specific intermediate step with limited retries without restarting the entire extract process.
The text was updated successfully, but these errors were encountered:
sudo-suhas
changed the title
Improve extractor retries
feat: improve extractor retries
Dec 1, 2022
sudo-suhas
changed the title
feat: improve extractor retries
Improve extractor retries
Dec 1, 2022
Is your feature request related to a problem? Please describe.
If an API call fails with a temporary error while running an extractor, we do not retry it. It could be improved.
Describe the solution you'd like
We should provide optional retries for extractors similar to sinks where the extractor call will be retried if the error returned is an instance of
plugins.RetryError
.Describe alternatives you've considered
Each extractor could manage its own retries. But this does not seem to be a clean approach and is also likely to get missed when a new extractor is being added.
Additional context
We could make the
retrier
implementation available to extractors so that they can retry a specific intermediate step with limited retries without restarting the entire extract process.The text was updated successfully, but these errors were encountered: