Migration 0108 runs slow with large data in TI and Dag_run table #63866
Description
Apache Airflow version
main (development)
If "Other Airflow 3 version" selected, which one?
No response
What happened?
Migration 0108_3_2_0_fix_migration_file_ORM_inconsistencies taking a lot of time to complete when we have alot data in TI and dag_run table.
I was testing this with 10M data in TI and dag_run table and migration taking more than an hour
What you think should happen instead?
No response
How to reproduce
- Start Airflow on 3.1.8 with PostgreSQL
- Have 1M+ task instances and dag runs
- Upgrade to 3.2.0 (airflow db migrate)
- Observe migration 0108 running for hour
Operating System
Mac
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct