Concerns about undefined behavior

[gvanrossum]

The PEP contains some pretty strong language (added fairly late in the process) that allows the compiler to use various optimizations (see "Performance considerations" and "Exceptions and side effects").

Some SC members don't seem happy with this. They are concerned about things like attributes implemented as properties with side effects, or isinstance operations with side effects. The PEP makes it undefined how many times such side effects occur, for example:

match a:
    case MyClass(x=1, y=2): ...
    case MyClass(x=1, y=3): ...
    case MyClass(x=2, y=y) if ...: ...

Here we don't know how many times MyClass is looked up (some crazy side effect could switch the class object), and more importantly we allow but don't require the interpreter to cache the outcome of isinstance(a, MyClass) as well as the result of a.x and a.y.

I think the SC's concern is that once CPython has settled on a specific strategy here, whatever it does will become part of the de-facto standard that other implementations (e.g. PyPy) will have to implement, regardless of whether the PEP allows them more freedom.

I personally have a performance concern about a match statement with many case SomeClass(...): cases (each with a different class name) -- I wish there was some kind of pre-computation that could be done to turn this into a single (weak) dict lookup (possibly with a PyPy-style guard, and dynamically updating the dict as subclasses are encountered. But that would definitely be optional behavior.

[Tobias-Kohn]

I fully understand the reservations and think we have to tread lightly here: we certainly need to leave the door open for optimisations, but having undefined behaviour has a history of quickly turning sour and backfiring.

In order to make pattern matching scale, we certainly want to keep the option to create a decision tree and memoize as much as possible, rather than go through it linearly case by case each time. But stipulating that it has to be so means a huge burden on the first implementation (and subsequently also for other Python implementations), hindering adoption in general. Finally, even with the idea of some optimised tree, we might still end up having an "unpredictable" number of isinstance-checks.

Here is a wild and vague idea (and probably completely unrealistic): if we look at how function and class definitions work, we find that they ingeniously combine static with dynamic elements. A def statement, for instance, creates a new object that uses the already existing and static code object. For classes, we find a similar combination of static and dynamic elements. Could we perhaps adopt a similar scheme here?

What I am proposing is that the match-block could be something like a lazily compiled/evaluated thing that gets compiled the first time the interpreter hits on it. But once it is in place, it will not bother looking up the class names etc. again (this is in itself a fairly commonly used and well understood technique for lazy evaluation). Hence, it would work a bit like a closure.

As I said, I am not sure how realistic it is, but I thought I throw it out there, notwithstanding.

[gvanrossum]

In the SC-VC we ended up agreeing that such optimized behaviors are desirable but not mandatory and we decided to keep this language.

[gvanrossum]

(Needs more PEP because I'm not yet happy with how this is spread across different sections of PEP 634.)