dataframe with cftime issues

[marikoll]

here is an example to reproduce the issue
https://abisko.uiogeo-apps.sigma2.no/user/a23bf869-f10d-4e06-b63a-3b201c67dda6/lab/tree/Arctic_haze.ipynb

[daliagachc]

It has to do with the index being a cftime object rather than pd.Timestamp.
we have similar issues with xarrays too. (see 102.
Ive created a function to solve this inside
negi-stuff (also instrucction to install update are here)

from negi_stuff.modules import koalas as kl
kl.check_transform_cftime_dim_2_timestamp(df_with_cftime)

[daliagachc]

@marikoll

[marikoll]

@daliagachc now it works with cftime.DatetimeNoLeap, but not cftime.Datetime360Day..

[daliagachc]

hi! i went into the file but im not sure where the error occurs. could you create a small notebook where you open a dataframe with cftime.Datetime360Day so that i can debug?
the send me the shareable file again
thanks!

[marikoll]

Thank you!

https://abisko.uiogeo-apps.sigma2.no/user/a23bf869-f10d-4e06-b63a-3b201c67dda6/lab/tree/Datetime360Day.ipynb

[daliagachc]

ok. now its fixed. the problem was that if cftime.Datetime360Day, then values like (2019,2,30) are allowed but that makes no sense in pd.Timestamp so we set it to 15 days to be in the middle and avoid the error.

new_df = kl.check_transform_cftime_dim_2_timestamp(_dsUKESM, middle_of_month=True)

make sure to update negi_stuff.
also you may want to resample to month after using the above line of code to make sure
that all monthly data is synced e.g.

new_df = new_df.resample('M').mean()

[marikoll]

Great, now it works! Thank you so much!