Collection of best einops-based repositories

[arogozhnikov]

There are already > 300 repositories using einops, some of them are of very high quality. Worth collecting a bunch to show off various use-cases

[arogozhnikov]

Currently done in https://github.com/arogozhnikov/einops/blob/master/docs/pages/projects.md

[Rishit-dagli]

@arogozhnikov I wouldn't say mine are super high quality but I do use einops in my implementations (definitely super useful) and would love to add them to the collection if these look good:

• https://github.com/Rishit-dagli/Fast-Transformer
• https://github.com/Rishit-dagli/Transformer-in-Transformer

[OriolAbril]

First of all, thanks for the amazing library. I started a library to use einops with xarray objects (among other things). The source is at https://github.com/arviz-devs/xarray-einstats but it's probably more interesting to look at the port of the einops basics tutorial, doing the same operations but on an xarray DataArray where dimensions are labeled and we can often skip the left side of the expression. It is still very recent, and I haven't yet added repeat for example but it would be great to get some feedback already.

[arogozhnikov]

Hi @OriolAbril,
Thanks for dropping a link.

I've briefly looked at the tutorial, I think it's very comprehensible and idea is quite clear, but I'm not convinced that tool will solve practical issues.

In particular, if result has axes names like 'b1,height', probably you just don't names at all? When decomposing such axes, you'll see conflicts with einsum patterns because of commas.

Also, how xarray indices are supported? What if one of combined axes has indices, second does not? How one can decomposed indexed axis? (sounds like that's impossible).

A tool that accepts xarray tensor, but returns just numpy, sounds more reasonable to me - basically you just define conversion from labelled tensor to numpy and identify layout of result, but still operating with names.

I'm also a bit concerned with interaction between repeat and (mostly missing) left hand side of pattern, because new axes now require some explicit mark.

I should probably warn I'm not an xarray user, and it's more important to start design from user cases that you want to address (which I know very little about!)

[OriolAbril]

Thanks so much for the feedback!

In particular, if result has axes names like 'b1,height', probably you just don't names at all? When decomposing such axes, you'll see conflicts with einsum patterns because of commas.

This feature is mostly useless and only somewhat convenient if those autonamed dimensions are only to be used once and reduced right after (with a function that is not compatible with reduce or as a facetting dimension when plotting). xarray requires dimensions to be named, because dimension order plays no role in any operation other than .values, so not naming is not an option, which is why this feature is mostly useless, you need dimension names to operate on xarray objects (generally use xarray because you want dimension names) and therefore it should and most probably will be named explicitly in the vast majority of cases.

It is true we do use commas for raw_einsum patterns and that could complicate things, thanks for poiting this out. I'll definitely change it to a dash instead.

This is only an issue for raw_ syntax though. xarray supports dimension names having spaces, commas, parenthesis... which is why the rearrange/reduce wrappers don't use strings and use lists and dictionaries to define the patterns. Even if more convenient, the raw_ ones will keep the prefix to signal they cover only a subset of the actual wrapper features.

Also, how xarray indices are supported? What if one of combined axes has indices, second does not? How one can decomposed indexed axis? (sounds like that's impossible).

reshaping is quite hard with xarray, so rearrange mostly solves problems I have faced, where I mostly do not care about coordinate values (if I have them). I know some ArviZ users with similar problems and I assume many of them will have similar issues too, but for now I am only addressing problems I have had multiple times and expect to have again in the future.

For now coordinate values are indeed ignored, but it would be possible to keep them by playing with pandas Multiindex and the different ways to initialize them. Currently xarray uses multiindex for combined/stacked dimensions, so decomposing is undoing the multiindex. There is an issue with an xarray version of numpy.reshape for example.

If I ever need to keep and combine coordinate values I'll look into it. I agree with you that this will be extremely challenging though, harder than the current "dimension guessing".

A tool that accepts xarray tensor, but returns just numpy, sounds more reasonable to me - basically you just define conversion from labelled tensor to numpy and identify layout of result, but still operating with names.

This is adding .values after the operation like I do in the notebook to take advantage of the image view for numpy arrays. Given how easy it is to do already and that I generally do want an xarray output this feels like shooting myself in the foot 😅

I'm also a bit concerned with interaction between repeat and (mostly missing) left hand side of pattern, because new axes now require some explicit mark.

Haven't yet had use for repeat and therefore have not looked into wrapping it yet, but it was the one that worried me the least. All the examples I have seen of repeat only use the left side to name the dimensions which is the main thing xarray does not need. Do you have a link you can share to cases where the left side of repeat is more complicated?