backend: queue v2 migration steps

[uael]

No description provided.

[cloudflare-workers-and-pages]

Deploying windmill with    Cloudflare Pages

Latest commit:	8b1c81c
Status:	✅  Deploy successful!
Preview URL:	https://b2803a51.windmill.pages.dev
Branch Preview URL:	https://uael-queue-v2.windmill.pages.dev

View logs

[uael]

Because of how views works (not null constraints not reflected), this migrations make sqlx queries impossible to compile (e.g. Uuid/Option), but all the already running code remain 100% compatible, while using the new schema. The caveats are pretty small and are discarded when all running server(s)/workers(s) are up to date:

• Some columns and their data are duplicated (e.g. __flow_status in v2_job_queue and flow_status in v2_job_flow_runtime)
• Each insert/update on queue are reflected on v2_* tables using triggers -> overhead.
• Each insert on completed_job are reflected on v2_* tables using triggers -> overhead.
  I will add a custom migration step which once everything is up to date (i.e. all processes run under the new version), will drop the triggers, the compatibility views, and the old columns.
  After this migration step, no more overhead or duplicated and the migration is complete.

To resume:

• Transparant migration v1 -> v2
• Missing triggers for v2 -> v1 compatibility
• Missing indexes, policies and triggers
• Update code to use updated tables and not the compatibility views
• Add custom migration step to finish the migration when all workers are up to date

[uael]

Another option is to temporarily change all checked queries that does not compile because of views to unchecked query in order to merge this, which allow to discover if the transparent migration works correctly, and then proceed with next steps:

This PR:

• Transparant migration v1 -> v2
• Missing triggers for v2 -> v1 compatibility
• Missing indexes, policies and triggers
• Tmp checked queries to unchecked

Next PR(s):

• Update code to use updated tables and not the compatibility views
• Re-check queries
• Add custom migration step to finish the migration when all workers are up to date

[uael]

Text cast