Improve Prefetching Primitives

[wagjamin]

At the moment our ROF backend doesn't perform better than the regular JIT backend. We need to get to a point where the vectorized hash table lookups actually help our overall performance. This PR makes steps in this direction:

Improve hash/prefetch primitives:

Until now we had a single primitive that would hash, and then another
primitive that would prefetch based on the hash. Looking at the ROF
paper their prefetching strategy is actually different: they fuse
hashing and prefetching into a single primitive. This allows
overlapping memory loads with computation, leading to better CPU
utilization. We now do the same and have a fused hash/prefetch
primitive.

Allow Inlining C++ Runtime Code in Vectorized Primitives

We use the link-time optimization of clang for this: we compile a new
inkfuse_runtime_static library with -flto.

When generating our C code for the interpreter and compiling it to a
shared library, we now link the objects from the
inkfuse_runtime_static library into the generated fragments.

This effectively removes the C/C++ compile-time/runtime optimization
boundary that existed so far. Up until now we would always generate call
statements into the runtime system.
We can now effectively inline functions from our runtime system into the
generated primitives.

Allow Inlining XXHASH calls in Vecotrized Prefetch Primitives

We add specialized hash primitives for fixed-width keys.
This is one of the biggest performance improvements we've shipped so far
for the vectorized interpreter.

We now for the first time manage to have vectorization outperform
compilation for some queries. This is possible because due to our new
link time optimizations, the vectorized primitives completely inline the
xxhash function calls. This leads to much more efficient hashing and
prefetching.

Improve the Hybrid Backend

We refine our Hybrid backend. So far, we always were in a
position where the JIT compiled code performs best. As a result, the
hybrid backend wasn't very dynamic.

As we now have a much faster interpreter, dynamic switching becomes much
more important.

We now collect exponentially decaying averages of the tuple throughput
of the different backend to use as a decision on which backend to prefer.

By default, we try 10% of morsels in the Interpreter, 10% of morsels in
the JIT compiled code (once it's ready), and the remaining 80% are run
in the backend with higher tuple throughput.
If we notice that one backend perform much better however we dynamically
increase the fraction of morsels we are running in the faster backend.
In extreme cases, we only run 2.5% of morsels in the slower backend.