Skip to content

Conversation

@provinzkraut
Copy link

When using msgspec.Meta(pattern="something"), msgspec creates a leaky reference to an re.Pattern, by increasing its reference count too many times.

This is caused by the fact that it creates the Pattern object with a call to PyObject_CallOneArg, which returns a new strong reference. Msgspec then add this to the Meta object, during which the Pattern's refcount is increased again. Due to this, its refcount can never fall to 0, keeping it alive indefinitely.

We can show this with a simple script:

import gc
import re
import secrets
import msgspec

string_pattern = secrets.token_hex()
msgspec.Meta(pattern=string_pattern)
re.purge()

print(
    any(o for o in gc.get_objects() if isinstance(o, re.Pattern) and o.pattern == string_pattern)
)

this should print False, since there should be no such re.Pattern objects arounds anymore, but this is not the case.


I have fixed this by foregoing the SET_FIELD macro, and instead simply assigning the field directly, without calling Py_XINCREF on its value.

@provinzkraut provinzkraut force-pushed the fix-meta-pattern-refleak branch from f29a4d8 to cc60a08 Compare October 26, 2025 15:07
@provinzkraut provinzkraut force-pushed the fix-meta-pattern-refleak branch from cc60a08 to 2f8c030 Compare October 26, 2025 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant