how to type a function that returns new classes without writing a mypy plugin?

[zzzeek]

The goal is, a Python function returns a new class, that is a subclass of some arbitrary base. The returned object should be typed as such. pylance can handle this case, however in mypy I historically have to use the get_dynamic_class_hook, which actually refers to the code in question I'm trying to write without a plugin.

Example:

import typing
from typing import Any
from typing import Type


class MyBaseClass:
    some_attr = 'attribute'

    if typing.TYPE_CHECKING:
        def __init__(self, **kw: Any):
            ...

def make_a_base_class() -> Type[MyBaseClass]:
    return type("MyClass", (MyBaseClass,), {})


# this works:
# MyBase = MyBaseClass

# this doesn't
MyBase = make_a_base_class()


class MyConcreteClass(MyBase):
    pass

mc = MyConcreteClass()


if typing.TYPE_CHECKING:
    reveal_type(MyBase)

Output in pyright is correct:

/home/classic/dev/sqlalchemy/test6.py
  /home/classic/dev/sqlalchemy/test6.py:31:17 - info: Type of "MyBase" is "Type[MyBaseClass]"
0 errors, 0 warnings, 1 info 

mypy won't accept the base (but does know what it is!):

test6.py:24: error: Variable "test6.MyBase" is not valid as a type
test6.py:24: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
test6.py:24: error: Invalid base class "MyBase"
test6.py:31: note: Revealed type is "Type[test6.MyBaseClass]"
Found 2 errors in 1 file (checked 1 source file)

is this a bug in mypy?

[erictraut]

This should arguably generate an error in pyright/pylance as well. The statement MyBase = make_a_base_class() looks similar to a type alias declaration, but it's not really a legal type alias because it uses a call expression. Therefore it should be treated as a normal variable assignment. A variable shouldn't be allowed as a base class in a class declaration.

[sobolevn]

(a bit offtopic) it was proposed to use Literal[int] for literal ast.Int constants. Like def x(arg: Literal[int]): ..., where: x(1) is fine and i: int; x(i) is not

[erictraut]

Even if there was a legal way to express dynamic class hierarchies, they would still be problematic for static type checkers because they need to understand the hierarchy in detail. What you're trying to do here is "lie" to the type checker to get it to kind of do the right thing. I don't know if it's worth extending the type system to support this "lie". If you want your code to work with static type checking, I recommend that you avoid dynamic class hierarchies altogether.

[erictraut]

I added this check to pyright 1.1.207, but I had to revert it because there was too much internal Microsoft code relying on the current behavior. I'll need to think about whether there's a relatively safe way to introduce it back. If not, I'll probably just leave the behavior as is in pyright.

[zzzeek]

just so everyone doesnt need to worry about this, the discussion has only made it clear to me that this old "Base = declarative_base()" pattern we have (in SQLAlchemy) is pretty old and crusty in any case, I came up with a new version of it now that we are Py3 only that I think the users will like a lot better in any case, the essential idea looks like this

class DeclarativeBase:
     # declarations for ".metadata" etc. go here

    def __init_subclass__(cls) -> None:
        if DeclarativeBase in cls.__bases__:
            cls.registry = registry()
            cls.metadata = cls.registry.metadata
        else:
            cls.registry.mapped(cls)

so you make your Base by just subclassing that class, nothing with metaclasses etc.

from sqlalchemy.orm import DeclarativeBase

class Base(DeclarativeBase):
    pass

[erictraut]

Nice! That looks much cleaner, and it will make the static type checkers happy too.