Comparison with torch.compile instead of Eager

[bonlime]

📚 Documentation

Hey! I saw your tool and plots with "acceleration", but you compare to un-optimised eager torch, which is obviously slower. Could you provide a graph with comparison against basic native pytorch loop, where you torch.compile the model? It would be useful for people who already have some optimisations in their pipelines, but would like to try yours framework instead

Thanks

[github-actions]

Hi! thanks for your contribution!, great first issue!

[t-vi]

We love torch.compile and use it a lot, too, and when enabling it in thunder (and perhaps disabling other backends) we expect numbers to be in the same ballpark. Do try it out!
That said, PyTorch eager is a more stable anchor point for us (maybe like the Prototype Metre in olden times), so we think it is most useful to our measurement.

[lantiga]

Thank you @bonlime, as @t-vi wrote our baseline is eager.
We are all about compounding the effect of multiple optimizations and have torch.compile in the mix.

In the case of the plot it was roughly equivalent to the first bar in that particular experiment, but again, we are not interested in a head to head comparison as it would be misleading with respect to the goals of the project.