Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support the SQLAlchemy 2.0 DeclarativeBase models #1215

Merged
merged 27 commits into from
Aug 31, 2023
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
0150152
Possible way to mixin with DeclarativeBase
pamelafox May 24, 2023
7d0f5e5
Remove pdm-python file
pamelafox May 24, 2023
d073fa6
Try different approach to mixins for 2.0
pamelafox May 25, 2023
5ff15bc
Support MappedAsDataClass
pamelafox May 25, 2023
8d78e37
Update should check
pamelafox May 25, 2023
41fdd5d
Parameterizing all the tests
pamelafox May 31, 2023
31e8e2c
Remove redundant test
pamelafox May 31, 2023
b536d28
Updates to docs
pamelafox May 31, 2023
af5096d
Getting types to work (mostly)
pamelafox Jun 1, 2023
a8da619
Update sqlalchemy version
pamelafox Jun 13, 2023
d3a30c5
Mark the type classes as internal with an underscore in front
pamelafox Jun 13, 2023
73e0206
Fix tests
pamelafox Jul 4, 2023
2c3650a
Addressing docs note
pamelafox Jul 5, 2023
7610ea7
Add constraint of les than 2.1
pamelafox Jul 5, 2023
8fdc47a
Remove version cap per Davids comment
pamelafox Jul 6, 2023
1bbd722
Add disable_autonaming
pamelafox Jul 11, 2023
b052dd6
Update versionchanged for new mixins
pamelafox Aug 20, 2023
d8c31ca
Document refactor to emphasize 2.x, fix for bind key and metadata for…
pamelafox Aug 22, 2023
a89485b
Parameterize more tests for commit
pamelafox Aug 22, 2023
6f4c180
Parameterize more tests to use 2.x
pamelafox Aug 22, 2023
caee46a
Update README
pamelafox Aug 22, 2023
6d7a7e3
Style and sphinx checks
pamelafox Aug 22, 2023
1d877e8
Documentation updates- remove new mixins from ref, add some comments
pamelafox Aug 22, 2023
a2db216
Update abstract models and mixins
pamelafox Aug 22, 2023
cf4170a
Add test for declared_attr
pamelafox Aug 25, 2023
fe9e4de
Make suggested typing fix
pamelafox Aug 26, 2023
fdeec1d
Adjusted to work in 3.8
pamelafox Aug 26, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ Version 3.1.0

Unreleased

- Add support for the SQLAlchemy 2.x API via ``model_class`` parameter. :issue:`1140`
- Bump minimum version of SQLAlchemy to 2.0.16.
- Remove previously deprecated code.
- Pass extra keyword arguments from ``get_or_404`` to ``session.get``. :issue:`1149`

Expand Down
19 changes: 19 additions & 0 deletions docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,32 @@ Model
If the ``__table__`` or ``__tablename__`` is set explicitly, that will be used
instead.

Metaclass mixins (SQLAlchemy 1.x)
---------------------------------

If your code uses the SQLAlchemy 1.x API (the default for code that doesn't specify a ``model_class``),
then these mixins are automatically applied to the ``Model`` class. They can also be used
directly to create custom metaclasses. See :doc:`customizing` for more information.

.. autoclass:: DefaultMeta

.. autoclass:: BindMetaMixin

.. autoclass:: NameMetaMixin


Base class mixins (SQLAlchemy 2.x)
----------------------------------

If your code uses the SQLAlchemy 2.x API by passing a subclass of ``DeclarativeBase``
or ``DeclarativeBaseNoMeta`` as the ``model_class``, then the following classes
are automatically added as additional base classes.

.. autoclass:: BindMixin

.. autoclass:: NameMixin
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we dont put this in the reference?



Session
-------

Expand Down
68 changes: 11 additions & 57 deletions docs/customizing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -160,71 +160,25 @@ To customize only ``session.query``, pass the ``query_cls`` key to the
db = SQLAlchemy(session_options={"query_cls": GetOrQuery})


Model Metaclass
---------------
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a radical approach, but I've deleted the Model Metaclass section of the customizing doc, since:

  1. its main purpose was to explain table name generation, and I've now made that possible via a constructor parameter for both old and new sqlalchemy models
  2. going forward, new users will hopefully be using sqlalchemy 2.x, so less need for something as esoteric as metaclass customization


.. warning::
Metaclasses are an advanced topic, and you probably don't need to customize them to
achieve what you want. It is mainly documented here to show how to disable table
name generation.

The model metaclass is responsible for setting up the SQLAlchemy internals when defining
model subclasses. Flask-SQLAlchemy adds some extra behaviors through mixins; its default
metaclass, :class:`~.DefaultMeta`, inherits them all.

- :class:`.BindMetaMixin`: ``__bind_key__`` sets the bind to use for the model.
- :class:`.NameMetaMixin`: If the model does not specify a ``__tablename__`` but does
specify a primary key, a name is automatically generated.

You can add your own behaviors by defining your own metaclass and creating the
declarative base yourself. Be sure to still inherit from the mixins you want (or just
inherit from the default metaclass).

Passing a declarative base class instead of a simple model base class to ``model_class``
will cause Flask-SQLAlchemy to use this base instead of constructing one with the
default metaclass.

.. code-block:: python

from sqlalchemy.orm import declarative_base
from flask_sqlalchemy import SQLAlchemy
from flask_sqlalchemy.model import DefaultMeta, Model

class CustomMeta(DefaultMeta):
def __init__(cls, name, bases, d):
# custom class setup could go here

# be sure to call super
super(CustomMeta, cls).__init__(name, bases, d)

# custom class-only methods could go here

CustomModel = declarative_base(cls=Model, metaclass=CustomMeta, name="Model")
db = SQLAlchemy(model_class=CustomModel)

You can also pass whatever other arguments you want to
:func:`~sqlalchemy.orm.declarative_base` to customize the base class.


Disabling Table Name Generation
```````````````````````````````
-------------------------------

Some projects prefer to set each model's ``__tablename__`` manually rather than relying
on Flask-SQLAlchemy's detection and generation. The simple way to achieve that is to
set each ``__tablename__`` and not modify the base class. However, the table name
generation can be disabled by defining a custom metaclass with only the
``BindMetaMixin`` and not the ``NameMetaMixin``.
generation can be disabled by setting `disable_autonaming=True` in the `SQLAlchemy` constructor.

Example code using the SQLAlchemy 1.x (legacy) API:

.. code-block:: python

from sqlalchemy.orm import DeclarativeMeta, declarative_base
from flask_sqlalchemy.model import BindMetaMixin, Model
db = SQLAlchemy(app, disable_autonaming=True)

class NoNameMeta(BindMetaMixin, DeclarativeMeta):
pass
Example code using the SQLAlchemy 2.x declarative base:

CustomModel = declarative_base(cls=Model, metaclass=NoNameMeta, name="Model")
db = SQLAlchemy(model_class=CustomModel)
.. code-block:: python

class Base(sa_orm.DeclarativeBase):
pass

This creates a base that still supports the ``__bind_key__`` feature but does not
generate table names.
db = SQLAlchemy(app, model_class=Base, disable_autonaming=True)
13 changes: 13 additions & 0 deletions docs/models.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,19 @@ For convenience, the extension object provides access to names in the ``sqlalche
``sqlalchemy.orm`` modules. So you can use ``db.Column`` instead of importing and using
``sqlalchemy.Column``, although the two are equivalent.

It's also possible to use the SQLAlchemy 2.x style of defining models,
as long as you initialized the extension with an appropriate 2.x model base class
(as described in the quickstart).

.. code-block:: python

from sqlalchemy.orm import Mapped, mapped_column

class User(db.Model):
id: Mapped[int] = mapped_column(db.Integer, primary_key=True)
username: Mapped[str] = mapped_column(db.String, unique=True, nullable=False)
email: Mapped[str] = mapped_column(db.String)

Defining a model does not create it in the database. Use :meth:`~.SQLAlchemy.create_all`
to create the models and tables after defining them. If you define models in submodules,
you must import them so that SQLAlchemy knows about them before calling ``create_all``.
Expand Down
94 changes: 84 additions & 10 deletions docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,76 @@ For example, to install or update the latest version using pip:
.. _PyPI: https://pypi.org/project/Flask-SQLAlchemy/


Initialize the Extension
------------------------

First create the ``db`` object using the ``SQLAlchemy`` constructor.
The initialization step depends on which version of ``SQLAlchemy`` you're using.
This extension supports both SQLAlchemy 1 and 2, but defaults to SQLAlchemy 1.

.. _sqlalchemy1-initialization:

Using the SQLAlchemy 1 API
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defer all the SQLAlchemy 1 stuff into its own page, with a note at the top.

^^^^^^^^^^^^^^^^^^^^^^^^^^

To use the SQLAlchemy 1.x API, you do not need to pass any arguments to the ``SQLAlchemy`` constructor.

.. code-block:: python

from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy.orm import DeclarativeBase

db = SQLAlchemy()

.. _sqlalchemy2-initialization:

Using the SQLAlchemy 2 API
^^^^^^^^^^^^^^^^^^^^^^^^^^

To use the new SQLAlchemy 2.x API, pass a subclass of either `DeclarativeBase`_ or `DeclarativeBaseNoMeta`_
to the constructor.

.. code-block:: python

from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy.orm import DeclarativeBase

class Base(DeclarativeBase):
pass

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it needs to be clearer what adding a Base class does here instead of just doing SQLAlchemy(model_class=DeclarativeBase). Maybe put an example attribute in or a comment that your user defined attributes should go there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, my earlier version of this feature had you call SQLAlchemy(model_class=DeclarativeBase) but I wanted to support someone also subclassing MappedAsDataclass, so that's why I now ask developers to create the Base class. They can then choose whether to subclass DeclarativeBase, DeclarativeBaseNoMeta, MappedAsDataclass, etc.

However, I wouldn't typically expect them to put user-defined attributes in Base - those go in the table-specific models.

I'm debating what to write here that doesn't become an essay / repeat SQLAlchemy docs too much.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added more detail and an additional example, let me know what you think of the change.


db = SQLAlchemy(model_class=Base)

If desired, you can enable `SQLAlchemy's native support for data classes`_
by adding `MappedAsDataclass` as an additional parent class.

.. code-block:: python

from sqlalchemy.orm import DeclarativeBase, MappedAsDataclass

class Base(DeclarativeBase, MappedAsDataclass):
pass

db = SQLAlchemy(model_class=Base)

.. _DeclarativeBase: https://docs.sqlalchemy.org/en/20/orm/mapping_api.html#sqlalchemy.orm.DeclarativeBase
.. _DeclarativeBaseNoMeta: https://docs.sqlalchemy.org/en/20/orm/mapping_api.html#sqlalchemy.orm.DeclarativeBaseNoMeta
.. _SQLAlchemy's native support for data classes: https://docs.sqlalchemy.org/en/20/changelog/whatsnew_20.html#native-support-for-dataclasses-mapped-as-orm-models

About the ``SQLAlchemy`` object
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Once constructed, the ``db`` object gives you access to the :attr:`db.Model <.SQLAlchemy.Model>` class to
define models, and the :attr:`db.session <.SQLAlchemy.session>` to execute queries.

The :class:`SQLAlchemy` object also takes additional arguments to customize the
objects it manages.

Configure the Extension
-----------------------

The next step is to connect the extension to your Flask app.
The only required Flask app config is the :data:`.SQLALCHEMY_DATABASE_URI` key. That
is a connection string that tells SQLAlchemy what database to connect to.

Expand All @@ -53,24 +120,15 @@ which is stored in the app's instance folder.

.. code-block:: python

from flask import Flask
from flask_sqlalchemy import SQLAlchemy

# create the extension
db = SQLAlchemy()
# create the app
app = Flask(__name__)
# configure the SQLite database, relative to the app instance folder
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///project.db"
# initialize the app with the extension
db.init_app(app)

The ``db`` object gives you access to the :attr:`db.Model <.SQLAlchemy.Model>` class to
define models, and the :attr:`db.session <.SQLAlchemy.session>` to execute queries.

See :doc:`config` for an explanation of connections strings and what other configuration
keys are used. The :class:`SQLAlchemy` object also takes some arguments to customize the
objects it manages.
keys are used.


Define Models
Expand All @@ -81,6 +139,8 @@ Subclass ``db.Model`` to define a model class. The ``db`` object makes the names
The model will generate a table name by converting the ``CamelCase`` class name to
``snake_case``.

This example uses the SQLAlchemy 1.x style of defining models:

.. code-block:: python

class User(db.Model):
Expand All @@ -90,6 +150,20 @@ The model will generate a table name by converting the ``CamelCase`` class name

The table name ``"user"`` will automatically be assigned to the model's table.

It's also possible to use the SQLAlchemy 2.x style of defining models,
as long as you initialized the extension with an appropriate 2.x model base class
as described in :ref:`sqlalchemy2-initialization`.

.. code-block:: python

from sqlalchemy.orm import Mapped, mapped_column

class User(db.Model):
id: Mapped[int] = mapped_column(db.Integer, primary_key=True)
username: Mapped[str] = mapped_column(db.String, unique=True, nullable=False)
email: Mapped[str] = mapped_column(db.String)


See :doc:`models` for more information about defining and creating models and tables.


Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ classifiers = [
requires-python = ">=3.7"
dependencies = [
"flask>=2.2.5",
"sqlalchemy>=1.4.18",
"sqlalchemy>=2.0.16",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to set a max-version as well? Just incase SQL Alchemy 2.1+ introduces breaking changes.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this would mean that this extension is no longer compatible with SQL Alchemy 1. Users will have to install an older version (which is fine, but worth noting in the release notes)

Copy link
Contributor Author

@pamelafox pamelafox Jul 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's reasonable, I just got bit indirectly by Pillow 10.0 change in another repo, so it seems like the responsible thing for a package is to explicitly check compatibility with new major versions.

I've just changed it to:
"sqlalchemy>=2.0.16,<2.1",

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added note to the changelog

Copy link
Member

@davidism davidism Jul 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Libraries must never set upper bounds on dependencies, it makes it very difficult for installers to resolve dependency trees. We don't have any reason to expect SQLAlchemy 2.1 to break what we do, or to do so in a way that we won't address, so the upper bound would not be correct. See https://iscinumpy.dev/post/bound-version-constraints/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay, I've removed the version cap, still in process of reading the post, thanks for letting me know.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: Going to revert this, and make the tests run in both SQLAlchemy 1.4 and SQLAlchemy 2 (see my note in conftest.py)

Copy link
Contributor

@ThiefMaster ThiefMaster Jul 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"this" includes the >=2 requirement I assume? (i think it'll be a LONG time until everyone upgraded to SA2, for small apps it might be easy but for larger one, in particular if there's no perfect test coverage, it's not likely to happen quickly, so keeping support for SA 1.4 until upstream discontinues it is much better IMHO)

edit: ah i see you already removed the upper bound, so i guess this is indeed about the >=2

Copy link
Contributor Author

@pamelafox pamelafox Aug 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: I started along the path of removing the upper bound of 2, but realized that would require a lot more checks inside our codebase, since we wouldn't even be able to import the new classes without a version check.

After conferring with @davidism, we decided to stick with the upper bound of >=2.

There seems to be good adoption of SA2, and this version still supports all the 1.4 syntax, so it shouldn't break 1.4 code (if all goes as planned).

We wouldn't remove 1.4 support entirely until a major bump of this package, and we'd put deprecation warnings in before that happens.

SA version download charts: https://www.pepy.tech/projects/sqlalchemy?versions=2.*&versions=1.*

]
dynamic = ["version"]

Expand Down
2 changes: 0 additions & 2 deletions requirements/mypy.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@
#
# pip-compile-multi
#
greenlet==2.0.2
# via sqlalchemy
iniconfig==2.0.0
# via pytest
mypy==1.4.1
Expand Down
2 changes: 1 addition & 1 deletion requirements/tests-min.in
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
flask==2.2.5
werkzeug<2.3
sqlalchemy==1.4.18
sqlalchemy==2.0.16
8 changes: 4 additions & 4 deletions requirements/tests-min.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SHA1:46c532e5c6f44988d3c02d047503f87ea25a2f72
# SHA1:8cae0f9c9bfdb0c4ddf46f9fb41a746cc14300a7
#
# This file is autogenerated by pip-compile-multi
# To update, run:
Expand All @@ -9,8 +9,6 @@ click==8.1.3
# via flask
flask==2.2.5
# via -r requirements/tests-min.in
greenlet==2.0.2
# via sqlalchemy
itsdangerous==2.1.2
# via flask
jinja2==3.1.2
Expand All @@ -19,8 +17,10 @@ markupsafe==2.1.3
# via
# jinja2
# werkzeug
sqlalchemy==1.4.18
sqlalchemy==2.0.16
# via -r requirements/tests-min.in
typing-extensions==4.6.3
# via sqlalchemy
werkzeug==2.2.3
# via
# -r requirements/tests-min.in
Expand Down
Loading