Flask not creating migrations in Heroku app #468

acrainier1 · 2022-05-19T00:21:05Z

acrainier1
May 19, 2022

I made some new migrations in my local database. Then I pushed to Github. Then fired up my Heroku PostgreSQL terminal but I can't create a migration on it.

Before I pushe to Github, I did delete some old migrations and fiddled around in my local branch.

(venv) user1@DESKTOP-6GK323S:~/my_project/flask_app$ heroku run bash
Running bash on ⬢ my-project... up, run.9427 (Free)
~ $ flask db migrate -m "creating update_logs table";
[2022-05-19 00:16:14,685] INFO in __init__: Flask app startup
Flask app startup
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
/app/.heroku/python/lib/python3.7/site-packages/alembic/script/revision.py:154: UserWarning: Revision 6871d053d5c4 referenced from 6871d053d5c4 -> f90467badb79, update logs table is not present
  % (downrev, rev)
Traceback (most recent call last):
  File "/app/.heroku/python/bin/flask", line 8, in <module>
    sys.exit(main())
  File "/app/.heroku/python/lib/python3.7/site-packages/flask/cli.py", line 967, in main
    cli.main(args=sys.argv[1:], prog_name="python -m flask" if as_module else None)
  File "/app/.heroku/python/lib/python3.7/site-packages/flask/cli.py", line 586, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/app/.heroku/python/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/app/.heroku/python/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/app/.heroku/python/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/app/.heroku/python/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/app/.heroku/python/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/app/.heroku/python/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/app/.heroku/python/lib/python3.7/site-packages/flask/cli.py", line 426, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/app/.heroku/python/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/app/.heroku/python/lib/python3.7/site-packages/flask_migrate/cli.py", line 90, in migrate
    rev_id, x_arg)
  File "/app/.heroku/python/lib/python3.7/site-packages/flask_migrate/__init__.py", line 96, in wrapped
    f(*args, **kwargs)
  File "/app/.heroku/python/lib/python3.7/site-packages/flask_migrate/__init__.py", line 212, in migrate
    version_path=version_path, rev_id=rev_id)
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/command.py", line 214, in revision
    script_directory.run_env()
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
    module = load_module_py(module_id, path)
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/util/compat.py", line 184, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "migrations/env.py", line 96, in <module>
    run_migrations_online()
  File "migrations/env.py", line 90, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/runtime/migration.py", line 511, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/command.py", line 190, in retrieve_migrations
    revision_context.run_autogenerate(rev, context)
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/autogenerate/api.py", line 442, in run_autogenerate
    self._run_environment(rev, migration_context, True)
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/autogenerate/api.py", line 453, in _run_environment
    if set(self.script_directory.get_revisions(rev)) != set(
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/script/base.py", line 227, in get_revisions
    return self.revision_map.get_revisions(id_)
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/script/revision.py", line 319, in get_revisions
    return sum([self.get_revisions(id_elem) for id_elem in id_], ())
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/script/revision.py", line 319, in <listcomp>
    return sum([self.get_revisions(id_elem) for id_elem in id_], ())
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/script/revision.py", line 321, in get_revisions
    resolved_id, branch_label = self._resolve_revision_number(id_)
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/script/revision.py", line 501, in _resolve_revision_number
    self._revision_map
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/util/langhelpers.py", line 230, in __get__
    obj.__dict__[self.__name__] = result = self.fget(obj)
  File "/app/.heroku/python/lib/python3.7/site-packages/alembic/script/revision.py", line 156, in _revision_map
    down_revision = map_[downrev]
KeyError: '6871d053d5c4'

miguelgrinberg · 2022-05-19T09:08:48Z

miguelgrinberg
May 19, 2022
Maintainer

Did you notice this message?

UserWarning: Revision 6871d053d5c4 referenced from 6871d053d5c4 -> f90467badb79, update logs table is not present

This is saying that one of your migration scripts (the one with id f90467badb79) reports that its parent is 6871d053d5c4, but this migration does not exist in your repository.

0 replies

acrainier1 · 2022-05-20T13:23:47Z

acrainier1
May 20, 2022
Author

Thank you for the reply.

I did notice that. But I don't understand why that is happening. In Heroku PostgreSQL I manually updated my alembic_version table.

SELECT * FROM alembic_version; outputs -->

version_num  
--------------
 484a60f779cd
(1 row)

Currently the existing tables in the database are:

                 List of relations
 Schema |       Name       | Type  |     Owner      
--------+------------------+-------+----------------
 public | alembic_version  | table | *edited for privacy*
 public | user_messages    | table | *edited for privacy*
 public | user_notes       | table | *edited for privacy*
 public | content_data     | table | *edited for privacy*
 public | saved_info       | table | *edited for privacy*
(5 rows)

So of course I get the warning that update_logs is not present. I'm trying to create that table with my migration script.

UserWarning: Revision 6871d053d5c4 referenced from 6871d053d5c4 -> f90467badb79, update logs table is not present

A bit more background on the issue. On my local Sqlite I made some mistakes when creating the update_logs table. So then I manually dropped it with a DROP TABLE update_logs; command in bash. Then I generated a new migration script to recreate it.

Previous migration script revision = 'f90467badb79', down_revision = '6871d053d5c4'
Latest migration script revision = '484a60f779cd', down_revision = 'f90467badb79'

I deleted all the old migrations scripts prior to the previous one so there might be a disconnect between the last successfully applied migration script and the previous migration script.

Previous Migration Script

"""update logs table

Revision ID: f90467badb79
Revises: 6871d053d5c4
Create Date: 2022-05-17 10:04:34.389524

"""
from alembic import op
import sqlalchemy as sa


# revision identifiers, used by Alembic.
revision = 'f90467badb79'
down_revision = '6871d053d5c4'
branch_labels = None
depends_on = None


def upgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.create_table('update_logs',
    sa.Column('Id', sa.Integer(), nullable=False),
    ***...other columns...***
    sa.PrimaryKeyConstraint('Id')
    )
    # ### end Alembic commands ###


def downgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.drop_table('update_logs')
    # ### end Alembic commands ###

Latest Migration Script

"""create logs table

Revision ID: 484a60f779cd
Revises: f90467badb79
Create Date: 2022-05-17 10:33:10.003443

"""
from alembic import op
import sqlalchemy as sa


# revision identifiers, used by Alembic.
revision = '484a60f779cd'
down_revision = 'f90467badb79'
branch_labels = None
depends_on = None


def upgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.create_table('update_logs',
    sa.Column('Id', sa.Integer(), nullable=False),
    ***...other columns...***
    sa.PrimaryKeyConstraint('Id')
    )
    # ### end Alembic commands ###


def downgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.drop_table('update_logs')
    # ### end Alembic commands ###

0 replies

miguelgrinberg · 2022-05-20T13:38:53Z

miguelgrinberg
May 20, 2022
Maintainer

Let me remove the parts of the error message that aren't interesting, so that you see more clearly what's happenning:

UserWarning: Revision 6871d053d5c4 ... is not present

Alembic is saying you don't have this migration script, even though it is referenced in the other migration.

0 replies

acrainier1 · 2022-05-20T13:42:09Z

acrainier1
May 20, 2022
Author

I deleted all the old migrations scripts prior to the previous one so there might be a disconnect between the last successfully applied migration script and the previous migration script.

""Alembic is saying you don't have this migration script, even though it is referenced in the other migration.""

Okay thanks. So how do I fix this? I don't have the scripts anymore.

Also, I'm telling alembic to focus on '484a60f779cd' which points to down revision 'f90467badb79' which does exist.

0 replies

miguelgrinberg · 2022-05-20T14:07:30Z

miguelgrinberg
May 20, 2022
Maintainer

I can't give you the specific instructions because I don't know what you have and what you deleted, but what I can tell you is that the migration history needs to be a continuous chain, where each migration links to the previous one. You may need to rebuild the chain with what you have left, or start over from scratch.

0 replies

acrainier1 · 2022-05-20T14:24:11Z

acrainier1
May 20, 2022
Author

By start over from scratch you mean delete my /migrations folder and rerun flask db init?

0 replies

miguelgrinberg · 2022-05-20T14:42:20Z

miguelgrinberg
May 20, 2022
Maintainer

I mean delete your migrations, delete all your tables in your database (including alembic_version), and then start over by running the init, migrate and upgrade commands in that order.

0 replies

acrainier1 · 2022-05-20T15:20:19Z

acrainier1
May 20, 2022
Author

I fixed the issue. Just for future reference, you don't need to delete everything and drop all tables in the database. But you do need to start over as you said with a migration script whose down revision points to None. But be careful to edit the migration script so it adds only the desired tables and doesn't drop existing ones.

I did the following locally:

(1) Deleted migrations folder.
(2) Dropped alembic_version table manually in local database.
(3) flask db init
(4) flask db migrate "comment" so that downdown_revision = None
(5a) Manually edited new migration script so it would NOT drop any existing tables.
(5b) Manually added code in migration script to create the update_logs table I wanted.
(6) flask db upgrade

"""database update

Revision ID: f7bd52198464
Revises: 
Create Date: 2022-05-20 10:53:06.267641

"""
from alembic import op
import sqlalchemy as sa


# revision identifiers, used by Alembic.
revision = 'f7bd52198464' 
down_revision = None
branch_labels = None
depends_on = None


def upgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.create_table('update_logs',
    sa.Column('Id', sa.Integer(), nullable=False),
    ***...other columns...***
    sa.PrimaryKeyConstraint('Id')
    )
    # ### end Alembic commands ###


def downgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.drop_table('update_logs')
    # ### end Alembic commands ###

Finally I pushed to my Github repo:
(1) Push to github
(2) Dropped alembic_version table manually in Heroku database
(3) In Heroku bash ran flask db upgrade

0 replies

miguelgrinberg · 2022-05-20T17:05:32Z

miguelgrinberg
May 20, 2022
Maintainer

This is not a great fix, because your migration repository only has this last migration that you needed. Now you have no record of all the previous tables that you have in your database. The solution I proposed would have included all the other tables in your migration as well. If you ever need to generate a new database, such as when you have a new teammate, or if you decide to migrate your deployment to another platform, then you will have a hard time recreating your database from the migration repository.

0 replies

acrainier1 · 2022-05-20T18:30:40Z

acrainier1
May 20, 2022
Author

This is still in development so it doesn't matter. When I go live I will be starting from the beginning anyway. But you are right generally speaking.

0 replies

miguelgrinberg · 2022-05-20T18:36:14Z

miguelgrinberg
May 20, 2022
Maintainer

You could have backed up your data, then restore it once you have a complete migration repository. Or switch to a dummy empty database to create the migration repository, then go back to your real database once the migration script was generated. These are all better options that will allow you to regenerate your database if you need to in the future.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flask not creating migrations in Heroku app #468

{{title}}

Replies: 11 comments

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Flask not creating migrations in Heroku app #468

acrainier1 May 19, 2022

Replies: 11 comments

miguelgrinberg May 19, 2022 Maintainer

acrainier1 May 20, 2022 Author

miguelgrinberg May 20, 2022 Maintainer

acrainier1 May 20, 2022 Author

miguelgrinberg May 20, 2022 Maintainer

acrainier1 May 20, 2022 Author

miguelgrinberg May 20, 2022 Maintainer

acrainier1 May 20, 2022 Author

miguelgrinberg May 20, 2022 Maintainer

acrainier1 May 20, 2022 Author

miguelgrinberg May 20, 2022 Maintainer

acrainier1
May 19, 2022

miguelgrinberg
May 19, 2022
Maintainer

acrainier1
May 20, 2022
Author

miguelgrinberg
May 20, 2022
Maintainer

acrainier1
May 20, 2022
Author

miguelgrinberg
May 20, 2022
Maintainer

acrainier1
May 20, 2022
Author

miguelgrinberg
May 20, 2022
Maintainer

acrainier1
May 20, 2022
Author

miguelgrinberg
May 20, 2022
Maintainer

acrainier1
May 20, 2022
Author

miguelgrinberg
May 20, 2022
Maintainer