Given a description of input/output types, ldmc generates code for transforming between different data representations of the same abstract type.
Based on ladder types, which enable the disambiguation of lower-level representations, while keeping the higher-level, abstract type structure intact, ldmc takes a library of primitive data transformation routines (morphisms) and tries to compose them into a path of morphisms such that the requested data transformation problem can be solved. This works by inducing a graph of morphisms (see picture below), inside which ldmc searches for the shortest path between the given input and output type.
Example: To solve the data transformation problem given by the type description
morph_angle1: Angle ~ Degrees ~ ℝ ~ native.Float64
--> Angle ~ Radians ~ ℝ ~ native.Float64
ldmc creates the following graph:
The solution found by ldmc contains two steps: first apply the morphism angle_as_degrees_to_turns_double
to convert the degrees into turns, and from there convert into radians by another morphism angle_as_turns_to_radians_dobule.
Try out the Live-Demo!
Inter-process communication (IPC) in distributed software systems requires marshaling, i.e. methods to encode/decode data for transmission. Traditional approaches rely on fixed wire formats, which may lead to unnecessary transformations and suboptimal balancing of transformation work, particularly under varying network setups and hardware constraints.
With ldmc, we propose a system that is able to optimize marshaling on a per-connection basis, such that firstly overall transformation overhead is minimized and secondly the required transformation work is balanced among the nodes to improve overall system performance. This will be achieved by choosing a wire-format adapted to a specific connection and the context of the application. In general, by selection of a wire-format, there is a trade-off between message size and encoding overhead, allowing to optimize different objectives such as energy efficiency, latency and throughput based on the interplay of CPU and network speeds. For example in the case that the sender and receiver run on compatible CPU-architectures, the encoding and decoding of primitive values could be eliminated, provided the network is fast enough (preserving endianness, padding, etc). Further, resource-constrained microcontrollers (MCUs) may be unable to support frameworks like Protobuf due to their memory requirements. Our system allows to move all required transformations away from the MCU to a server with more capacity by setting the wire format such that the MCU can send directly in its native format without additional encoding.
The system starts with a description of the desired internal data formats on both ends of the communication channel, given by ladder-types. Using a platform-wide library of morphisms (functions that transform between representations), a directed graph is induced whose vertices are ladder-types and its edges morphisms. Given this so called morphism-graph, the "ladder-typed data morphing compiler" (ldmc) is able to generate code for complex morphisms by compiling together multiple morphism definitions from the library: Finding a function to perform a data transformation between two ladder-types amounts to finding a path in the morphism-graph, whose edges can be annotated by (multi-dimensional) weights to optimize the path for an application specific goal.
Example: Morphism graph while searching for the conversion of an analog temperature reading (given as quantized voltage measurement) into a human readable representation of the temperature, in
particualar in degrees Celsius, expressed as decimal integer, encoded by a nullterminated ASCII-string.
To generate the marshaling & demarshaling stubs for the type A,
i.e. functions to convert (morph) between the Internal and Wire representations of the type, call ldmc with -m <REQUEST> for each requested morphism, which is given by the syntax of
<NAME> : SRC-TYPE --> DST-TYPE.
With -o, the output file is specified, where the generated C code is written.
In general, calling ldmc will look like the following:
ldmc -m "marshal_A: A ~ Internal --> A ~ Wire" \
-m "demarshal_A: A ~ Wire --> A ~ Internal" \
-o ldmc_morphisms.cIf no -o option is provided, the generated C source will be written to stdout.
To tell ldmc about where the morphism base is located, set the environment variable MORPHISM_BASE:
MORPHISM_BASE=<my/path/to/morphism-base> ldmc -m ...Alternatively, use the -b option:
ldmc -b <my/path/to/morphism-base> -m ...For debugging purposes it can be useful to inspect the current morphism-graph which ldmc is currently working on.
Launch with the option --serve <PORT> to start a webserver for viewing the state of the current graph search.
Example:
ldmc -m "morph_myfloat: ℝ ~ native.Float32 --> ℝ ~ native.Float64" \
--serve 8000