Tutorial ~ Introduction

The `ildl-plugin` is a meta-programming technique aimed at allowing safe, custom data representation transformations across library boundaries. In practice, it allows programmers to improve the performance of their code by writing transformations which store data more efficiently.

Let's take tuples of two integers as an example:

scala> val x1 = (3, 5)
x1: (Int, Int) = (3,5)

scala> val x2 = (2, 8)
x2: (Int, Int) = (2,8)

In Scala, it is fairly easy to extend their functionality:

scala> type Complex = (Int, Int)
defined type alias Complex

scala> implicit class IntPairComplex(val p1: Complex) extends AnyVal {
     |   def *(p2: Complex): Complex = (p1._1 * p2._1 - p1._2 * p2._2,
     |                                  p1._1 * p2._2 + p1._2 * p2._1)
     | }
defined class IntPairComplex

scala> val x3 = x1 * x2
x3: Complex = (-34,34)

In two lines of code we defined the Complex type and added a multiplication method. It is great to be able to add semantics on top of existing objects and to easily compose a complex data types from small, generic and reusable bits, even across different libraries.

However, this opens up an invisible trap: we no longer have a way to control how our data is represented. And this can have surprisingly negative results in terms of performance. In our example, storing complex numbers as heap-based Scala tuples is very inefficient, and adds an almost 10x overhead to each operation compared to a compact stack-based encoding.

So what can we do about it? The most common solution is to manually transform the code to a C-like low-level equivalent. However, this loses the high-level nature of the code and makes it hard to maintain. It is also error-prone, as the low-level code avoids the use of type-safe data structures due to their overhead, preferring low-level, unchecked memory accesses and integer indices.

A better alternative would be if compilers analyzed the data in use and found the best representation automatically. This is commonly done for small DSLs, but the approach does not scale to complex structures built from components coming from different libraries, each with its own semantics.

Ultimately, it is the developer who knows exactly how would the data be represented and how it should be operated on. This is where the ildl-plugin project comes in: instead of going in and rewriting the code to a lower level, ildl allows programmers to define safe and composible transformations that are automatically applied to the code.

This is where the "data-centric metaprogramming" names comes from: in a typical meta-programming approach, the developer adds rewriting rules that operate directly on the program. But the fact that rewriting rules only touch the data makes them very specific and allows the compiler to offer correctness guarantees on the result.

The next section will show the example that motivated the creation of the ildl-plugin.

From Here:

• [[see an example of an ildl transformation|Tutorial-~-Example-(Part-1)]]
• return to the home page