Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add content from typeshed/CONTRIBUTING.md #1882

Merged
merged 15 commits into from
Dec 17, 2024
215 changes: 188 additions & 27 deletions docs/guides/writing_stubs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,9 @@ Liskov substitutability or detecting problematic overloads.
It may be instructive to examine `typeshed <https://github.com/python/typeshed/>`__'s
`setup for testing stubs <https://github.com/python/typeshed/blob/main/tests/README.md>`__.

To suppress type errors in stubs, use ``# type: ignore`` comments. Refer to the :ref:`type-checker-error-suppression` section of the style guide for
error suppression formats specific to individual typecheckers.

..
TODO: consider adding examples and configurations for specific type checkers

Expand All @@ -113,18 +116,6 @@ Stub Content
This section documents best practices on what elements to include or
leave out of stub files.

Modules excluded from stubs
---------------------------

Not all modules should be included in stubs.

It is recommended to exclude:

1. Implementation details, with `multiprocessing/popen_spawn_win32.py <https://github.com/python/cpython/blob/main/Lib/multiprocessing/popen_spawn_win32.py>`_ as a notable example
2. Modules that are not supposed to be imported, such as ``__main__.py``
3. Protected modules that start with a single ``_`` char. However, when needed protected modules can still be added (see :ref:`undocumented-objects` section below)
4. Tests

Public Interface
----------------

Expand All @@ -138,7 +129,17 @@ The following should always be included:
* All objects included in ``__all__`` (if present).

Other objects may be included if they are not prefixed with an underscore
or if they are being used in practice. (See the next section.)
or if they are being used in practice.

Modules excluded from stubs
---------------------------

The following should not be included in stubs:

1. Implementation details, with `multiprocessing/popen_spawn_win32.py <https://github.com/python/cpython/blob/main/Lib/multiprocessing/popen_spawn_win32.py>`_ as a notable example
2. Modules that are not supposed to be imported, such as ``__main__.py``
3. Protected modules that start with a single ``_`` char. However, when needed protected modules can still be added (see :ref:`undocumented-objects` section below)
4. Tests

.. _undocumented-objects:

Expand Down Expand Up @@ -212,23 +213,28 @@ to use them freely to describe simple structural types.
Incomplete Stubs
----------------

When writing new stubs, it is not necessary to fully annotate all arguments,
return types, and fields. Some items may be left unannotated or
annotated with ``_typeshed.Incomplete`` (`documentation <https://github.com/python/typeshed/blob/main/stdlib/_typeshed/README.md>`_)::

from _typeshed import Incomplete

field: Incomplete # unannotated

def foo(x): ... # unannotated argument and return type

``_typeshed.Incomplete`` can also be used for partially known types::

def foo(x: Incomplete | None = None) -> list[Incomplete]: ...

Partial stubs can be useful, especially for larger packages, but they should
follow the following guidelines:

* Included functions and methods should list all arguments, but the arguments
can be left unannotated.
* Do not use ``Any`` to mark unannotated or partially annotated values. Leave
function parameters and return values unannotated. In all other cases, use
``_typeshed.Incomplete``
(`documentation <https://github.com/python/typeshed/blob/main/stdlib/_typeshed/README.md>`_)::

from _typeshed import Incomplete

field1: Incomplete
field2: dict[str, Incomplete]

def foo(x): ...

``_typeshed.Incomplete``.
* Partial classes should include a ``__getattr__()`` method marked with
``_typeshed.Incomplete`` (see example below).
* Partial modules (i.e. modules that are missing some or all classes,
Expand All @@ -253,6 +259,17 @@ annotated function ``bar()``::

def bar(x: str, y, *, z=...): ...

``Any`` vs. ``Incomplete``
--------------------------

While ``Incomplete`` is a type alias of ``Any``, they serve different purposes:
``Incomplete`` is a placeholder where a proper type might be substituted.
It's a "to do" item and should be replaced if possible.

``Any`` is used when it's not possible to accurately type an item using the current
type system. It should be used sparingly, as described in the :ref:`using-any`
section of the style guide.

Attribute Access
----------------

Expand Down Expand Up @@ -475,6 +492,28 @@ and the :ref:`best-practices`. There are a few exceptions, outlined below, that
different structure of stub files into account and aim to create
more concise files.

Syntax Example
--------------

The below is an excerpt from the types for the ``datetime`` module.

MAXYEAR: int
MINYEAR: int

class date:
def __new__(cls, year: SupportsIndex, month: SupportsIndex, day: SupportsIndex) -> Self: ...
@classmethod
def fromtimestamp(cls, timestamp: float, /) -> Self: ...
@classmethod
def today(cls) -> Self: ...
@classmethod
def fromordinal(cls, n: int, /) -> Self: ...
@property
def year(self) -> int: ...
def replace(self, year: SupportsIndex = ..., month: SupportsIndex = ..., day: SupportsIndex = ...) -> Self: ...
def ctime(self) -> str: ...
def weekday(self) -> int: ...

Maximum Line Length
-------------------

Expand Down Expand Up @@ -506,7 +545,7 @@ No::

def time_func() -> None: ...

def date_func() -> None: ... # do no leave unnecessary empty lines
def date_func() -> None: ... # do not leave unnecessary empty lines

def ip_func() -> None: ...

Expand Down Expand Up @@ -563,7 +602,7 @@ Yes::

class Color(Enum):
# An assignment with no type annotation is a convention used to indicate
# an enum member.
# an enum member.
RED = 1

No::
Expand Down Expand Up @@ -633,6 +672,13 @@ No::
...
def to_int3(x: str) -> int: pass

Avoid invariant collection types (``list``, ``dict``) for function parameters,
in favor of covariant types like ``Mapping`` or ``Sequence``.

Avoid union return types. See https://github.com/python/mypy/issues/1693
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably for a future PR, but it would be nice if the content of python/mypy#1693 and python/mypy#7214 below could be copied into the guides. Having pieces of general typing advice hidden away in mypy issues feels sort of odd.


Use ``float`` instead of ``int | float`` for parameter annotations. See :pep:`484` for more details.

Language Features
-----------------

Expand Down Expand Up @@ -662,6 +708,14 @@ No::

class OtherClass: ...

Use variable annotations instead of type comments, even for stubs that target
older versions of Python.

Platform-dependent APIs
-----------------------

Use :ref:`platform checks<version-and-platform-checks>` like ``if sys.platform == 'win32'`` to denote platform-dependent APIs.

NamedTuple and TypedDict
------------------------

Expand Down Expand Up @@ -689,7 +743,7 @@ No::
Built-in Generics
-----------------

:pep:`585` built-in generics are supported and should be used instead
:pep:`585` built-in generics (such as ``list``, ``dict``, ``tuple``, ``set``) are supported and should be used instead
of the corresponding types from ``typing``::

from collections import defaultdict
Expand All @@ -707,8 +761,115 @@ generally possible and recommended::
Unions
------

Declaring unions with the shorthand `|` syntax is recommended and supported by
Declaring unions with the shorthand ``|`` syntax is recommended and supported by
all type checkers::

def foo(x: int | str) -> int | None: ... # recommended
def foo(x: Union[int, str]) -> Optional[int]: ... # ok

.. _using-any:

Using ``Any`` and ``object``
----------------------------

When adding type hints, avoid using the ``Any`` type when possible. Reserve
the use of ``Any`` for when:

* the correct type cannot be expressed in the current type system; and
* to avoid union returns (see above).

Note that ``Any`` is not the correct type to use if you want to indicate
that some function can accept literally anything: in those cases use
``object`` instead.

When using ``Any``, document the reason for using it in a comment. Ideally,
document what types could be used.

The ``Any`` Trick
-----------------

In cases where a function or method can return ``None``, but where forcing the
user to explicitly check for ``None`` can be detrimental, use
``_typeshed.MaybeNone`` (an alias to ``Any``), instead of ``None``.

Consider the following (simplified) signature of ``re.Match[str].group``::

class Match:
def group(self, group: str | int, /) -> str | MaybeNone: ...

This avoid forcing the user to check for ``None``::

match = re.fullmatch(r"\d+_(.*)", some_string)
assert match is not None
name_group = match.group(1) # The user knows that this will never be None
return name_group.uper() # This typo will be flagged by the type checker

In this case, the user of ``match.group()`` must be prepared to handle a ``str``,
but type checkers are happy with ``if name_group is None`` checks, because we're
saying it can also be something else than an ``str``.

This is sometimes called "the Any trick".

Context Managers
----------------

When adding type annotations for context manager classes, annotate
the return type of ``__exit__`` as bool only if the context manager
sometimes suppresses exceptions -- if it sometimes returns ``True``
at runtime. If the context manager never suppresses exceptions,
have the return type be either ``None`` or ``bool | None``. If you
are not sure whether exceptions are suppressed or not or if the
context manager is meant to be subclassed, pick ``bool | None``.
See https://github.com/python/mypy/issues/7214 for more details.

``__enter__`` methods and other methods that return ``self`` or ``cls(...)``
should be annotated with ``typing.Self``
(`example <https://github.com/python/typeshed/blob/3581846/stdlib/contextlib.pyi#L151>`_).

Naming
------

Type variables and aliases you introduce purely for legibility reasons
should be prefixed with an underscore to make it obvious to the reader
they are not part of the stubbed API.

A few guidelines for protocol names below. In cases that don't fall
into any of those categories, use your best judgement.

* Use plain names for protocols that represent a clear concept
(e.g. ``Iterator``, ``Container``).
* Use ``SupportsX`` for protocols that provide callable methods (e.g.
``SupportsInt``, ``SupportsRead``, ``SupportsReadSeek``).
* Use ``HasX`` for protocols that have readable and/or writable attributes
or getter/setter methods (e.g. ``HasItems``, ``HasFileno``).

.. _type-checker-error-suppression:

Type Checker Error Suppression Formats
--------------------------------------

* Use mypy error codes for mypy-specific ``# type: ignore`` annotations, e.g. ``# type: ignore[override]`` for Liskov Substitution Principle violations.
* Use pyright error codes for pyright-specific suppressions, e.g. ``# pyright: ignore[reportGeneralTypeIssues]``.
* If you need both on the same line, mypy's annotation needs to go first, e.g. ``# type: ignore[override] # pyright: ignore[reportGeneralTypeIssues]``.


``@deprecated``
---------------

The ``@typing_extensions.deprecated`` decorator (``@warnings.deprecated``
since Python 3.13) can be used to mark deprecated functionality; see
:pep:`702`.

Keep the deprecation message concise, but try to mention the projected
version when the functionality is to be removed, and a suggested
replacement.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we say that if a deprecated feature is still very widely used and was deprecated only recently, it may be best to wait before adding @deprecated?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. If the feature is still in wide use, that's an argument against deprecating the feature, but if the runtime code decides to deprecate something, the stubs should communicate that fact.


Docstrings
----------

There are several tradeoffs around including docstrings in type stubs. Consider the intended purpose
of your stubs when deciding whether to include docstrings in your project's stubs.

* They do not affect type checking results and will be ignored by type checkers.
* Docstrings can improve certain IDE functionality, such as hover info.
* Duplicating docstrings between source code and stubs requires extra work to keep them in sync.
Loading