Several related dataclass (mostly-)typing issues

[DougLeonard]

Hello, hope I found the right place...

I raised some of this on python.org in the context of a run-time dataclasses wrapper:
https://gitlab.com/douglas.s.leonard/strictdataclasses/

But beyond the idea of the wrapper it became clear that some of this is more about mypy than python. Here is a sample code:

#!/bin/python3
import subprocess
import sys

def install(package,source=None):
    if source is None:
        source=package
    try:
        __import__(package)
    except:
        subprocess.check_call([sys.executable, "-m", "pip", "install", source])

if __name__ == "__main__":
    install("mypy")
    from mypy import api  # type: ignore
    result=api.run([__file__, "--check-untyped-defs"])
    if not result[0].startswith("Success"):
        print(result[0])
#        exit()               #comment out for testing asserts...

from dataclasses import dataclass
from typing import Final

@dataclass
class MyClass:
    CONST : Final[int] = 5
    
    def SetConst(self,val):
        self.CONST=val     #  This gets caught now by mypy

    def InitNewvar(self):
        self.newvar = 5    #mypy allows this!!  

x=MyClass(CONST = 20)
assert(x.__class__.CONST == 5)    # these pass
assert(x.CONST == 20)                   # these pass

x.InitNewvar()
x.anothervar = 10              #mypy flags this.
assert(x.newvar == 5)        # works.

x.__dict__["CONST"] = 30     # This is also allowed, here or in the setter, but presently also in my runtime checker
assert(x.CONST == 30 )     # but that seems to fall under pythonic self-responsibility. 

output

selfcheck.py:30: error: Cannot assign to final attribute "CONST"
selfcheck.py:40: error: "MyClass" has no attribute "anothervar"
Found 2 errors in 1 file (checked 1 source file)

I'm using python 3.8.10 and mypy 0.761

So this just installs and runs mypy with --check-untyped-defs on the script itself, and runs some tests with dataclasses.

1. Mypy catches assignment to a non existent attribute (anothervar) from outside the class, but it doesn't stop it from methods of the class (newvar). That's not surprising since that's how instance variables must be declared for undecorated classes. This was actually a main motivation for the wrapper that started me on this. I wanted to have enforcement that object attributes should be declared in the class "header" and self-documenting, and it seems in spirit with dataclasses, but it's not any kind of requirement on them. So this is just my thought on a desirable feature or maybe potential typing spec.

2. PEP591 explicitly calls the x situation a typing error, because the class variable and instance variable shouldn't exist at the same time with Final. That's a clear stipulation in the PEP, but is also pretty legalistic here because the class attribute is shadowed from the start. There's no shift from seeing it to then seeing the instance attribute. Still, the legality is there.

3. The idea that CONST: Final[int] = 5 actually defines a constructor allowing initialization to 20 feels awkward and not so "final". It's less awkward when the CONST: Final[int] = field(default=5) syntax is used. PEP591 seems to have intended to prevent this sort of thing in normal classes (can't use a Final class variable as a default fall-back for the Final instance variable, as per point 1). And yet with dataclasses there's really no other way to provide a default value for the constructor parameter. I think this basically has to stand as legitimate, but... It can be explicitly disabled with init=False, so maybe it's a point for dataclasses that init=False should be the default for Final variables, so that, like in C++11, allowing the constructor to override the const default can be setup explicitly, but is not enabled by default. Then CONST: Final[int] = 5 would not actually define a contradicting constructor. CONST: Final[int] = field(default=5,init=True) would, which is more intuitive. That's not a typing solution though, and maybe it shouldn't be.

4. Very related, another thing not in the spec is if assignment to Final attributes should be allowed in __post_init__. Well... the spec by default says it shouldn't be allowed, and mypy doesn't allow it, but maybe the spec should be clarified? PEP591 clearly does allow assignment in __init__, and __post_init__ is the dataclass user-replacement for custom calculations in init.

Finally, my $0.02 is it would be great if mypy shipped by default and there was a shorthand to invoke it directly from a script.

[DougLeonard]

Of course the init=False default for Final would be a breaking change :( and would require python to interpret Final, so I don't expect that to happen.