From f2591d5d77b82a6f76ecdf55c8a1350852ab001b Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 27 Nov 2025 16:35:06 +0100 Subject: [PATCH] PEP 803: Updates based on packaging discussion --- peps/pep-0803.rst | 154 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 133 insertions(+), 21 deletions(-) diff --git a/peps/pep-0803.rst b/peps/pep-0803.rst index edece9a2c31..9b3e80c1dcb 100644 --- a/peps/pep-0803.rst +++ b/peps/pep-0803.rst @@ -63,12 +63,13 @@ to those builds. To build against the Stable ABI, the extension must use a *Limited API*, that is, only a subset of the functions, structures, etc. that CPython exposes. -The Limited API is versioned, and building against Limited API 3.X -yields an extension that is ABI-compatible with CPython 3.X and *any* later -version (though bugs in CPython sometimes cause incompatibilities in practice). -Also, the Limited API is not “stable”: newer versions may remove API that -were a part of older versions. +Both the Limited API and the Stable ABI are versioned, and building against +Stable ABI 3.X requires using only Limited API 3.X, and yields an extension +that is ABI-compatible with CPython 3.X and *any* later version +(though bugs in CPython sometimes cause incompatibilities in practice). +The Limited API is not “stable”: newer versions may remove API that +were a part of older versions. This PEP proposes the most significant such removal to date. @@ -88,6 +89,7 @@ No backwards compatibility now However, we won't block the possibility of extending compatibility to CPython 3.14 and below. + See a :ref:`rejected idea ` for how this could work. API changes are OK The new Limited API may require extension authors to make significant @@ -172,9 +174,12 @@ New Export Hook (PEP 793) ------------------------- Implementation of this PEP requires :pep:`793` (``PyModExport``): -A new entry point for C extension modules) to be -accepted, providing a new “export hook” for defining extension modules. -Using the new hook will become mandatory in Limited API 3.15. +A new entry point for C extension modules), which was accepted for Python +3.15. + +Since existing ways of defining modules use API that this PEP removes +(namely, :c:type:`PyModuleDef`), extensions will need to migrate to PEP 793's +new “export hook” when switching to Limited API 3.15. Runtime ABI checks @@ -185,6 +190,12 @@ will continue to be responsible for not putting incompatible extensions on Python's import paths. This decision makes sense since tools typically have much richer metadata than what CPython can check. +Typically, build tools and installers use `PyPA packaging metadata`_ and +`platform compatibility tags`_ to communicate compatibility details, but other +models are possible. + +.. _PyPA packaging metadata: https://packaging.python.org/en/latest/specifications/core-metadata/ +.. _platform compatibility tags: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/ However, CPython will add a line of defense against outdated or misconfigured tools, or human mistakes, in the form of a new *module slot* containing @@ -218,15 +229,45 @@ The ``abi3t`` wheel tag ----------------------- Wheels that use a stable ABI compatible with free-threading CPython builds -should use a new ABI tag: ``abi3t``. +should use a new `ABI tag`_: ``abi3t``. The name is chosen to reflect the fact that this ABI is similar to ``abi3``, -except changes necessary to support free-threading (which uses the letter ``t`` -in existing, version-specific ABI tags like ``cp314t``). +with limitations necessary to support free-threading (which uses the letter +``t`` in existing, version-specific ABI tags like ``cp314t``). + +.. _ABI tag: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#abi-tag + +Package installers should treat this tag as completely separate from ``abi3``. +They should allow ``abi3t``-tagged wheels for free-threaded builds wherever +they currently allow ``abi3``-tagged ones for (orherwise equal) non-free-threaded +builds. + +Build tools should generate ``abi3.abi3t`` instead of ``abi3`` when the Python +tag is ``cp315`` and above (or equivalently: when setting ``Py_LIMITED_API`` +to ``3.15`` (``0x030f0000``) or above). +``abi3.abi3t`` is a `compressed tag set`_ that signals compatibility with both +``abi3`` and ``abi3t``. + +.. _compressed tag set: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets + +.. note:: + + The version of the Stable ABI is indicated by the `Python wheel tag`_; this + PEP does not change that. + For example, a wheel tagged ``cp315-abi3.abi3t`` will be compatible with + 3.15, 3.16, and later versions; + ``cp317-abi3.abi3t`` will be compatible with 3.17+. -Since wheels built using Limited API 3.15 will be compatible with both -GIL-enabled builds and free-threaded ones, they should use the -`compressed ABI tag set `__ -``abi3.abi3t``. +.. _Python wheel tag: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#python-tag + +The ``abi3t`` tag can be used in extensions compatible with earlier versions of +free-threaded Python. +For example, an extension compatible with GIL-enabled *and* free-threaded +builds of *3.14*, 3.15, and higher versions would be tagged +``cpy314-abi3.abi3t``. +This PEP does not propose an official way to build such extensions, but since +a mechanism for that can be added later (see +:ref:`a Rejected idea `), installers should be ready to accept +the tag. New API @@ -297,6 +338,20 @@ free-threaded one. * ❌ * ✅ * ❌ + * * ``cp314-abi3t`` (*) + * ❌ + * ✅ + * ❌ + * ✅ + * ❌ + * ✅ + * * ``cp314-abi3.abi3t`` (*) + * ✅ + * ✅ + * ✅ + * ✅ + * ✅ + * ✅ * * ``cp315-cp315`` * ❌ * ❌ @@ -318,6 +373,13 @@ free-threaded one. * ❌ * ✅ * ❌ + * * ``cp315-abi3t`` (*) + * ❌ + * ❌ + * ❌ + * ✅ + * ❌ + * ✅ * * ``cp315-abi3.abi3t`` * ❌ * ❌ @@ -326,6 +388,8 @@ free-threaded one. * ✅ * ✅ +(*): Wheels with these tags cannot be built; see table below + The following table summarizes which wheel tag should be used for an extension built with a given interpreter and ``Py_LIMITED_API`` macro: @@ -349,6 +413,14 @@ built with a given interpreter and ``Py_LIMITED_API`` macro: * 3.14+ (GIL) * ``PY_PACK_VERSION(3, 14)`` * existing + * * ``cp314-abi3t`` + * N/A + * N/A + * out of spec + * * ``cp314-abi3.abi3t`` + * N/A + * N/A + * reserved * * ``cp315-cp315`` * 3.15 (GIL) * (unset) @@ -361,6 +433,10 @@ built with a given interpreter and ``Py_LIMITED_API`` macro: * 3.15+ (GIL) * ``PY_PACK_VERSION(3, 15)`` * discontinued + * * ``cp315-abi3t`` + * N/A + * N/A + * out of spec * * ``cp315-abi3.abi3t`` * 3.15+ (any) * ``PY_PACK_VERSION(3, 15)`` @@ -371,8 +447,16 @@ Values in the *Note* column: * *existing*: The wheel tag is currently in use * *continued*: The wheel tag continues the existing scheme * *discontinued*: The wheel tag continues the existing scheme, but it will - be discouraged. Older tools may still generate it. + be discouraged. Older tools may still generate it. Installers will + continue to accept it, but only for GIL-enabled builds, even though the wheel + would be compatible with free-threaded ones. * *new*: Proposed in this PEP. +* *reserved*: A mechanism to build a matching extension is not proposed in + this PEP, but may be added in the future. + Installers should be brepared to handle the tag. +* *out of spec*: Should not be used as-is: extensions should be tagged + ``abi3.abi3t`` rather than only ``abi3``. + The entry is included for installers that decompose compressed tag sets. Security Implications @@ -398,7 +482,8 @@ This PEP combines several pieces, implemented individually: ``_Py_OPAQUE_PYOBJECT`` macro. Implemented in GitHub pull request `python/cpython#136505 `__. -- For ``PyModExport``, see :pep:`793`. +- For ``PyModExport``, see :pep:`793` and + `GitHub issue #140550 `_. - A version-checking slot was implemented in GitHub pull request `python/cpython#137212 `__. - A check for older ``abi3`` was implemented in GitHub pull request @@ -430,9 +515,27 @@ It would also make the free-threading memory layout of ``PyObject`` part of the stable ABI, preventing future adjustments. +.. _pep803-no-shim: + Shims for compatibility with CPython 3.14 ----------------------------------------- +It’s possible to build a ``cp314-abi3.abi3t`` extenstion – one compatible +with 3.14 (both free-threaded build and default). +There are several challenges around this: + +* making it convenient and safe for general extensions +* testing it (as CPython’s test suite doesn’t involve other CPython versions + than the one being tested) + +So, providing a mechanism to build such extensions is best suited to an +external project (for example, one like `pythoncapi-compat`_). +It's out of scope for CPython’s C API, and this PEP. + +.. _pythoncapi-compat: https://github.com/python/pythoncapi-compat + +To sketch how such a mechanism could work: + The main issue that prevents compatibility with Python 3.14 is that with opaque ``PyObject`` and ``PyModuleDef``, it is not feasible to initialize an extension module. @@ -443,12 +546,11 @@ free-threading and GIL-enabled) are “frozen”, so it is possible for an extension to query the running interpreter, and for 3.14, use a ``struct`` definition corresponding to the detected build's ``PyModuleDef``. -This is too onerous to support and test in CPython's Limited API at this point, -but it may be allowed in the future. +.. _pep803-no-avoid-abi3t: -Using the Python wheel tag to determine compatibility ------------------------------------------------------ +Using the Python version wheel tag to determine compatibility +------------------------------------------------------------- A previous version of this PEP avoided adding a new wheel tag (``abi3t``), and specified that wheels tagged ``abi3`` would be compatible with @@ -477,6 +579,16 @@ an unnecessary technical change. Using ``abi3.abi4`` in wheel tags but only ``.abi3`` in filenames would look more inconsistent than ``abi3.abi3t`` and ``.abi3``. +If we add ``abi4`` tag, the ``Py_LIMITED_API`` value would either need to: + +* change to start with ``4`` to match ``abi4``, but no longer correspond + to ``PY_VERSION_HEX`` (making it harder to generate and check), or +* not change, making it inconsistent with ``abi4``. + +Adding ``abi3t`` is a smaller change than adding ``abi4``, making it work +better as a transitional state before larger changes like :pep:`809`'s +``abi2026``. + Copyright =========