Options for typing buffers

[hlovatt]

I’ve written typesheds for MicroPython (GitHub - hlovatt/PyBoardTypeshed: Typesheds (a.k.a.: interface stubs, pyi files, and type hints) for MicroPython.) to type buffers I currently use:

AnyReadableBuf: Final = TypeVar("AnyReadableBuf", bytearray, array, memoryview, bytes)

AnyWritableBuf: Final = TypeVar("AnyWritableBuf", bytearray, array, memoryview)

I like some aspects of this solution e.g.:

1. The names: AnyReadableBuf and AnyWritableBuf are consistent with AnyStr.
2. It support both read-write and read-only buffers.
3. It allows functions that accept a writable buffer and return that buffer to type correctly as the same type as the given buffer e.g.:

def mem_read(
        self,
        data: AnyWritableBuf,
        addr: int,
        memaddr: int,
        /,
        *,
        timeout: int = 5000,
        addr_size: int = 8,
    ) -> AnyWritableBuf

But I don’t like that someone can’t write their own buffer type.

1. Is there a better way without PEP 688?
2. Will PEP 688 support the use cases of read-only and read-write (point 2 above)?
3. Will PEP 688 support the use case of correctly returning the writable buffer type (point 3 above)?

[Gobot1234]

1. This is what typeshed does so I'd hazard a guess and say that this is the best thing you can do for now.
2. Currently no as there's no way to distinguish between read-only and writable buffers in the C API (https://peps.python.org/pep-0688/#read-only-and-writable-buffers)

3. BufferT = TypeVar("BufferT", bound=types.Buffer)
   
   def mem_read(
        self,
        data: BufferT,
        addr: int,
        memaddr: int,
        /,
        *,
        timeout: int = 5000,
        addr_size: int = 8,
   ) -> BufferT

   This should work, however, this doesn't currently make sure the buffer is writeable (see 2)

[hlovatt]

Thanks for your comments, I was suspecting as much.

Storchaka noted on https://discuss.python.org/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265 that "It is important that bf_releasebuffer is set to NULL in bytes and to non-NULL in mutable types. Some C code only accept types with this slot set to NULL." So re. Point 2, distinguishing between read-only and read-write, there is a sort of way.