Spikes

Spikes

A Spike is simply a time-boxed experiment which aims to address specific issues or thorny design problems in a hands-on way, focusing experiment on the problem at hand and ignoring other concerns.

BDD Property Tests

Introduction

• Behaviour Driven Development is a variant of Test-Driven Development whereby tests are written in order to reflect high-level usage of the software, usually tied to a specific user story. BDD is highly effective both as a communication medium shared between business and tech in order to align precisely, unambiguously, and automatically on what the software should do and how users interact with it. In essence it's a way to incrementally build an executable specification of the software that can speak to non-technical and technical people alike.
• As the Hydra project is founded on research and some form of specification as pseudo-code or mathematical property, and there exists a Simulator that can play the role of a model against which to check correctness of the actual Node's code, it seemed interesting to try to combine a BDD approach with Property-Based Testing
• Thus the goal of this spike is to explore this approach and validate it can help us building a better system faster: Define high-level properties that make sense business-wise, cover a wider range of behaviour than traditional example-based BDD tests, can be checked against some "oracle", and can be executed with minimal operational overhead ; then run those properties against existing Node code and make the needed changes to pass the tests ; and iterate refining and extending the properties.
• Code is in the abailly-iohk/hydra-test-model branch in the repository, this is a summary of what's been done:
  • Defined a hydra-model whose main components are :
    • a Model module which provides types and functions to define expected state of the system and machinery to drive and observe behaviour of the System-under-Test (the Hydra nodes). The runModel function is the main entry point which runs a sequence of action against a cluster of Nodes and returns the ModelState
    • a ModelSpec which defines properties and stateful generators of actions
    • Execution is based on the machinery provided by Ouroboros io-sim-classes and io-sim in order to not depend on system-level behavior and be able to check tracesand concurrent behavior
    • Generators use the Shelley Properties infrastructure to provide arbitrary but valid UTXO and transactions to input to the nodes

Outcome & Comments

• Generating Mary transactions and UTXO was not mandatory for the spike as the Hydra nodes are supposed to be mostly agnostic about the actual ledger's implementation, but it was nevertheless interesting to provide more realistic data and understand how this works for future work integrating with PAB and mainchain. Generating an initial UTXO set is straightforward but I still have troubles with the transaction generation, probably because of issues with slot interval validity, but should be easy to sort out
  • It would be interesting to control the kind of transactions we generate and tie that to the actual ledger used by the nodes, which might be easier to do with an mock leder than with an actual Mary ledger
• Writing those tests exposed the need for a high-level Node structure that maintains the complete state of an instance and which is the main handle through which clients would interact, whether using a web-based API or a CLI
  • this interface is defined in the Run module, along with the code to actual start a node
• Changes to "production" code has been as minimal as possible but the need to observe the nodes' state and to have actions on one Node have impact on other nodes lead to a few interesting changes to the internal logic:
  • The state update loop now takes care of some OnChainEvent that change the head's state, namely the InitTx which marks the actual "opening" of the head and the CollectComTx which defines the initial set of UTXOs in the Head
  • I needed to observe the confirmed ledger's state in order to check the nodes were consistently updated when the CollectComTx was handled: It makes sense to provide some sort of query interface to clients
  • The init command was simplified and does not modify the state anymore: The Head state should only be updated in a single place, in this case the handleNextEvent, as this greatly simplifies the reasoning about each command's effects. I did not remove the putState in close but this should be the logical next step.
  • In the particular case of the init command, this was made necessary because we want to make sure initialisation is propagated to all nodes in the cluster which is done through the mainchain by posting relevant transactions, on whcih nodes can react.
• This approach makes it possible to completely switch to a different internal implementation without much hassle. For example, instead of having an Event loop based system we could have multiple threads each one handling the lifecycle of single "object" (transaction or snapshot) and interacting with a shared state. This would simplify implementation of waits and put the burden of queue management on the RTS, with queues being needed only at the borders of the system
• We can couple generator-based and a scenario-based (traditional BDD) tests, detailing a specific use case with scenarios and actors to help us have more concrete cases to discuss
• I did not yet use hydra-sim as an oracle to check our system is valid
• I did not implement the nodes-to-nodes networking. The idea is to reuse the Channels to interconnect nodes without having to setup a real network.
• Note that while the current Model's implementation works "in-process", the exact same principles could be used to run the exact same tests against an actual cluster of Hydra nodes connected through actual sockets, and even potentially connected PABs and a cardano-node. As our understanding evolves, we can remove some fake layers or provide the ability to swap them with more realisting components. This is a benefit of BDD: Writing tests "in the shoes" of a client makes it clearer what are the interfaces and interactions with the outside world, leading to Hexagonal architecture or functional core, imperative shell.