astype with 'datetime64[us, UTC]'

[davetapley]

I'm trying to use astype with datetime64[us, UTC].

If I pass a string (as the docs):

astype('datetime64[us, UTC]')

I get:

error: No overloads for "astype" match the provided arguments (reportCallIssue)

I also tried:

from datetime import UTC
from numpy import datetime64

astype(datetime64['us', UTC])

 error: Expected no type arguments for class "datetime64" (reportInvalidTypeArguments)

numpy/numpy#16545 seems to imply that should work, but I thought I'd open a discussion here before an issue there.

[Dr-Irv]

With respect to doing pd.Series.astype("datetime64[us, UTC]"), that is something that we could handle in the stubs, but it would be a challenge to handle all combinations of precision (the 'us' part) and timezone (the 'UTC' part). We do allow a string such as "datetime64[us]", but don't have all the time zone combinations. We'd need to add a LOT of strings to the definition of TimestampDtypeArg in _typing.pyi in order to handle all possible combinations.

With respect to the second example, I don't think the expression datetime64['us', UTC] is valid. I tried it with python 3.11 and numpy 2.0, and got a runtime error.

[herebebeasties]

Given UTC is massively more common than any other timezone, and that the only alternative here for the user is to add # type: ignore annotations, perhaps we could be pragmatic and add just the UTC types, which would hopefully fix well over half the instances of this particular issue?

[Dr-Irv]

Given UTC is massively more common than any other timezone, and that the only alternative here for the user is to add # type: ignore annotations, perhaps we could be pragmatic and add just the UTC types, which would hopefully fix well over half the instances of this particular issue?

Sure, I'm open to a PR that does that.

[ashiq-km]

Hi all! 👋

Thanks for the detailed explanation. I totally understand the challenges with supporting all combinations of precision and timezone.

I agree that UTC is by far the most common, so adding just datetime64[us, UTC] to the stubs sounds very pragmatic. It would definitely reduce the number of # type: ignore annotations users have to sprinkle in their code.

From my side, I’d be happy to experiment with a small PR to include UTC in _typing.pyi. Even if it’s just a partial solution, it seems like it would fix a lot of real-world cases.

Curious to hear if anyone has tried this already or has suggestions for how to structure the PR so it doesn’t bloat the stub definitions too much.

[Dr-Irv]

So it turns out that we do support the following as strings:

        "datetime64[s, UTC]",
        "datetime64[ms, UTC]",
        "datetime64[us, UTC]",
        "datetime64[ns, UTC]",

What's missing are the tests. They should be added to test_astype_timestamp() in test_series.py .

That won't handle the original request, which was to handle it via numpy, and the UTC constant, but I think this is a reasonable workaround.