Science Museum requests can be flaky, add backoff
to all requests
#4710
Labels
💻 aspect: code
Concerns the software code in the repository
🛠 goal: fix
Bug fix
🟧 priority: high
Stalls work on the project or its dependents
🧱 stack: catalog
Related to the catalog and Airflow DAGs
🔧 tech: airflow
Involves Apache Airflow
Airflow log link
https://airflow.openverse.org/dags/science_museum_workflow/grid?dag_run_id=manual__2024-08-01T21%3A25%3A59%2B00%3A00&task_id=ingest_data.pull_image_data&map_index=-1&tab=logs
Description
We're seeing 3 kinds of failures, all occurring intermittently and interchangeably, with Science Museum DAG requests:
504 Server Error: Gateway Time-out for url
502 Server Error: Bad Gateway for url
503 Server Error: Service Unavailable for url
We should add
backoff
on all requests for these HTTP error types, almost exactly mirroring what we're doing in #4663 with Freesound.Reproduction
Since this is an upstream issue, it's very hard to reproduce consistently.
DAG status
We're currently skipping on ingestion errors with an attachment to #4013, so we'll leave this open.
The text was updated successfully, but these errors were encountered: