Conversion of pytorch-image-models to image-models with IVY

[Geeks-Sid]

Is your feature request related to a problem? Please describe.
Hi, As we know when latest papers come from different frameworks, we are bound to certain frameworks such a Torch, Tensofrlow, MXNet, JAX, Numpy etc and although the papers are great, A pytorch user would have to go through multiple loops to reproduce the paper and make the computational graph identical.

Describe the solution you'd like
I believe if we use IVY, a Unifying framework to convert pytorch-image-models, we could have this library create a broader impact. IVY supports PyTorch, TensorFlow, MXNet, JAX, and NumPy. IVY uses automatic code conversions between frameworks. This framework is being created by Danile Lenton and Ronnie Clark and many others and has now become a community effort.

I believe with community effort with everybody writing code for a set model family, pytorch-image-models can reach out to more researches and bolster research even more.

Additional context
Add any other context or screenshots about the feature request here.

[rwightman]

@Geeks-Sid moving to discussion as I try to keep the issues focused on bugs and concrete feature req

[rwightman]

@Geeks-Sid it looks like an interesting project, I'm sure some in the community would be interested to see it working with timm models.

A lot of people choose their frameworks for different reasons, some because they like the 'frontent' (modelling interface / API, often a driver for PyTorch or JAX), some because they need specific backend functionality (TF), or specific parallism / hardware support (TF or JAX + TPU @ scale).

However, for many of these needs there a desire to use canonical modelling code or framework specific functionality. When people want a PyTorch model, they want to work with the PyTorch code in canonical form, be able to modify, enhance, combine models/modules. If I'm working in JAX on a TPU POD I'd likely be using xmap/pjit for tranformers at scale... these sorts of things would be very challenging to capture with an intermediate interface that looks like it's capturing graphs via frontents and translating to backends for runtime?

If you could convert canonical PyTorch model code into canonical JAX or TF code. Or say, convert a canonical model in any framework without specific model parallelism support, and enhance it for any given framework to partion vertically or horizontally (ie Megatron for PyTorch + GPU, xmap/pjit for JAX + TPU) that'd be pretty amazing. But moving models back and forth at a lower level or requiring a different modelling lib would be less interesting to me.