Thunder is 10**6 times slower than Lightning

[kurmukovai]

🐛 Bug

I feel like marketing is misleading, as thunder is 1_000_000 times slower than lightning^[1,2]

1. https://en.wikipedia.org/wiki/Speed_of_light
2. https://en.wikipedia.org/wiki/Speed_of_sound

[lantiga]

😁 Right but AI helps us defy the laws of physics (Karpathy) 😅

or more simply, the first run is slower due to compilation times (@nikitaved)

[nikitaved]

But on a serious note :) @kurmukovai , as @lantiga pointed out, the initial run could be slow because of the compilation/program analysis step. Note, however, that changes in inputs' meta-data (like device, shapes, but not the data itself) may trigger re-compilation.

Once meta-data is fixed and is not expected to change, the goal is to suffer a single initial warm-up perf hit for faster subsequent runs (for example, during inference).

If meta-data is expected to vary (say, like in the streaming setting), but this set is finite, one could cache compiled models to avoid non-stop re-compilations.

[Andrei-Aksionov]

Thunder, feel the thunder
Lightning, then the thunder

(Imaging Dragons - Thunder)