Support for dry-running migrations

[marceliwac]

Problem statement

Running database migrations can frequently pose a problem in the form of breaking changes that happen when the state of the database does not match the state of the application code. This could happen when the migration code itself throws an error half-way through its execution and could result in the unknown state of the migration (i.e. partially migrated schema / partially seeded database).

Motivation

Running migrations as part of one's CI/CD pipeline usually means that the job has to be executed on par with the deployment stage (i.e. all-or-none). In this case, running a failing migration after the application is deployed can pose severe risk of bringing down the entire environment due to inconsistencies between the code and the state of the database. On the other hand, running a migration before the deployment can lead to the opposite problem - deployment script failure can break an existing environment, but the matching application code that has failed to deploy will also leave the environment in an out-of-sync state.

Solution

To decrease the likelihood of the above situations, I propose introducing a code that 'dry-runs' the migrations, giving the user ability to ensure that at the very least - the migration code will execute without syntax errors, even if external calls fail. This could be done in several ways.

Firstly the flag --dry, or similar one would have to be added to trigger the dry-run upon migration call. Secondly, the migration code itself should be checked for possible syntax errors etc. that would occur during runtime.

Alternatively, the migration code could explicitly specify the case of jobs that should run in case of dry-runs (as an example, external API calls that introduce the changes beyond the database itself). It would also be nice to perform full dry-runs in the way that does not alter the original database, but rather creates an exact copy of it and performs the migrations there.