Core and Rule Syntax Harmonisation

Introduction

The Core syntax contains functionality for specifying models (e.g., types, attributes etc), expressions, and functions. The Core syntax is used in both the CDM and DRR models.

The Rule syntax contains functionality for specifying reports, and reporting rules. The Rule syntax is used only in the DRR model.

Although each syntax has a separate purpose, there is an amount of overlap where the same functionality has been implemented in both syntaxes with different keywords and different underlying code. The differing syntax can be confusing for users, and different underlying code is inefficient to support and maintain.

The functionality that is duplicated in the Rule syntax is mostly used in expressions for extracting and filtering data required for creating reports. This functionality will be moved to the Core syntax and removed from the Rule syntax so there is only one syntax for writing expressions.

The Rule syntax has some useful features that not currently supported in the Core. This proposal also recommends changes to migrate the any useful Rule syntax when it is migrated to the Core syntax, while removing any limitations. The benefits and limitations of the Rule syntax are summarised in the next section.

Rule Syntax

Limitations

Single Input Parameter

The Rule syntax has a limitation that a rule can only have one input parameter.

Single Item Processing

The Rule syntax has a limitation that a rule input parameter must have single cardinality, therefore no aggregate functions can be written.

Inefficient Type and Cardinality Validation

The Rule syntax does not allow the rule input or output parameters to have their type or cardinality specified. Although this makes for concise syntax, means that the logic cannot be validated without also analysing adjacent rules. Also, if syntax errors are present in any of the rules, then it is much harder to report accurate validation messages.

Benefits

Concise Syntax

The Rule syntax has concise syntax for calling rules where the parameter does not need to be specified.

If TradeForEvent was implemented as a rule:

extract TradeForEvent

If TradeForEvent was implemented as a func:

extract TradeForEvent( ReportableEvent )

List Keywords – `extract` and `filter when`

The Rule keywords extract and filter when are equivalent to the Rosetta keywords map and filter. Users have generally expressed a preference for extract over map, and somewhat mixed / neutral on filter when over filter.

Concatenate operations with `then` keyword

The Rule syntax supports the ability to chain together operations using the then keyword. This allows users to break larger operations into a set of smaller steps which can help with readability.

extract TradeForEvent( ReportableEvent )
    then extract Trade -> executionDetails -> executionVenue -> entityId

Syntax Proposal

This section details the proposed changes to the Core and Rule syntax.

Adopt List Keyword – `extract`

The Core syntax will adopt the preferred keyword for processing list items, e.g. extract replacing map.

The following syntax and examples show expressions that contain a default list parameter, item. There are further examples in the Appendix that use a named list parameter.

Syntax

Before

set outputList:
    inputList
        map [<Expression to modify item>]
        filter [<Boolean expression to filter item>]

After

set outputList:
    inputList
        extract [<Expression to modify item>]
        filter [<Boolean expression to filter item>]

Example

Before

func ExtractPriceType: 
    inputs:
        prices Price (0..*)
    output:
        priceTypeEnums PriceTypeEnum (0..*)

    set priceTypeEnums:
        prices 
            map [ item -> priceType ]
            filter [ item = PriceTypeEnum->AssetPrice ]

After

func ExtractPriceType: 
    inputs:
        prices Price (0..*)
    output:
        priceTypeEnums PriceTypeEnum (0..*)

    set priceTypeEnums:
        prices 
            extract [ item -> priceType ]
            filter [ item = PriceTypeEnum->AssetPrice ]

Implement CodeLens/Inlay feature to overlay type and cardinality in Rosetta UI

The DSL underlying technology and Rosetta application infrastructure both support CodeLens, which allows for information to be displayed over the code to assist the user.

In the example below, the type and cardinality information has been inferred and overlayed in grey text. Note that is a UI feature, and the appearance may not exactly match the example below.

Written code:

prices
    extract [ item -> priceType ]
    filter [ item = PriceTypeEnum->AssetPrice ]

CodeLens information overlayed in grey text over the written code:

This feature would require further analysis to determine feasibility and effort to implement.

Adopt feature to concatenate operations

The Core syntax will adopt the feature to concatenate operations.

Example

Before

Rule syntax:

extract TradeForEvent( ReportableEvent ) 
then extract Trade -> tradableProduct -> product
then extract InterestRateLeg1( Product )
then extract InterestRatePayout -> rateSpecification -> floatingRate -> initialRate
then filter when Price -> priceExpression -> spreadType = SpreadTypeEnum -> Spread
then extract Price -> amount

Core syntax (as an inline expression)

InterestRateLeg1( TradeForEvent( reportableEvent ) -> tradeableProduct -> product ) 
    -> rateSpecification -> floatingRate -> initialRate
        filter [ item -> priceExpression -> spreadType = SpreadTypeEnum -> Spread ]
        map [ item -> amount ]

Core syntax (as a function)

alias trade: TradeForEvent( reportableEvent )
alias product: trade -> tradeableProduct -> product
alias interestRateLeg1: InterestRateLeg1( product )
set result: 
    interestRateLeg1 -> rateSpecification -> floatingRate -> initialRate
        filter [ item -> priceExpression -> spreadType = SpreadTypeEnum -> Spread ]
        map [ item -> amount ]

After

Note that:

Single cardinality expressions can be concatenated with the extract keyword, no need for then
Multiple cardinality expression concatenation is already supported in Core syntax (no changes required). The extract step will process the list items one-by-one. See next section for processing as a list.

Written code:

extract [ TradeForEvent( item ) ]
extract [ item -> tradableProduct -> product ]
extract [ InterestRateLeg1( item) ]
extract [ item -> rateSpecification -> floatingRate -> initialRate ]
filter  [ item -> priceExpression -> spreadType = SpreadTypeEnum -> Spread ]
extract [ item -> amount ]

With CodeLens:

Adopt feature to concatenate operations (part 2)

The Core syntax will adopt the feature to concatenate operations where the previous step returns a list that must be processed as a list.

Example

Before

Rule syntax - the current rule expression concatenation syntax does not support processing as a list, therefore the extract of the initialRate attribute (e.g. a list of Price) cannot be separated from the function call.

extract FloatingRateSpread( InterestRatePayout -> rateSpecification -> floatingRate -> initialRate )

func FloatingRateSpread:
   inputs: prices Price (0..*)
   output: spread number (0..1)

After

The proposed concatenation syntax supports processing as a list using the extract all syntax.

Written code:

extract     [ item -> rateSpecification -> floatingRate -> initialRate ]
extract all [ FloatingRateSpread( item ) ]

With CodeLens:

When the CodeLens displays extra information when the cardinality is multi.

Adopt concise syntax to call a reporting rule

For a Reporting Rule that has one input parameter, the call to invoke the rule can omit the parameter.

Example

Before

Rule syntax - if the calls to TradeForEvent, ProductForTrade and InterestRateLeg1 were implemented as a reporting rule (rather than a func) then the parameter can be omitted.

extract TradeForEvent
then extract ProductForTrade
then extract InterestRateLeg1
then extract InterestRatePayout -> rateSpecification -> floatingRate -> initialRate

Core syntax - if the calls to TradeForEvent, ProductForTrade and InterestRateLeg1 were implemented as a func (rather than a reporting rule) then the parameter must be specified.

extract TradeForEvent( ReportableEvent )
then extract ProductForTrade( Trade )
then extract InterestRateLeg1( Product )
then extract InterestRatePayout -> rateSpecification -> floatingRate -> initialRate

After

For both func and reporting rule syntax, if there is a single input parameter then it can be omitted (because it can be inferred by the DSL).

Written code:

extract [ TradeForEvent ]
extract [ ProductForTrade ]
extract [ InterestRateLeg1 ]
extract [ item -> rateSpecification -> floatingRate -> initialRate ]

With CodeLens:

Reporting rules allow multiple inputs parameters / multiple cardinality

Add optional inputs and output declaration (same format as func) in the Reporting Rule syntax.

This has a few advantages:

Multiple inputs can be specified (currently only one allowed)
Input parameters can have multiple cardinality (currently only single cardinality allowed)
The rule expression can be validated against the declared input and output type/cardinality rather than analysing the surrounding rules.

Example

Before

reporting rule FloatingRateSpread
    [regulatoryReference ESMA EMIR Refit table "2h" dataElement "87"]
    extract ReportableEvent -> reportableTrade -> trade
        then extract Trade -> tradableProduct -> product
        then extract InterestRateLeg1( Product )
        then extract InterestRatePayout -> rateSpecification -> floatingRate -> initialRate
            then filter when Price -> priceExpression -> spreadType = Spread
            then extract Price -> amount
    as "9.999 Floating rate spread of leg 1"

After

With single input parameter - inputs/output syntax can be omitted.

Written code:

reporting rule FloatingRateSpread:
    [regulatoryReference ESMA EMIR Refit table "2h" dataElement "87"]
    // for single input, specifying inputs is not required
    extract [ TradeForEvent ]
    extract [ ProductForTrade ]
    extract [ InterestRateLeg1 ]
    extract [ item -> rateSpecification -> floatingRate -> initialRate ]
    filter  [ item -> priceExpression -> spreadType = SpreadTypeEnum->Spread ]
    extract [ item -> amount ]
    only-element
    as "9.999 Floating rate spread of leg 1"

With multiple input parameters - inputs/output syntax must be specified.

Written code:

reporting rule FloatingRateForSpreadType:
    [regulatoryReference ESMA EMIR Refit table "2h" dataElement "87"]
    inputs: 
        reportableEvent ReportableEvent (1..1)
        spreadType SpreadTypeEnum (1..1)
    output: 
        spread number (0..1)
    set spread:
      reportableEvent
          extract [ TradeForEvent ]
          extract [ ProductForTrade ]
          extract [ InterestRateLeg1 ]
          extract [ item -> rateSpecification -> floatingRate -> initialRate ]
          filter  [ item -> priceExpression -> spreadType = spreadType ]
          extract [ item -> amount ]
          only-element
    as "9.999 Floating rate spread of leg 1"

With CodeLens:

Auto-formatting

Implement auto-formatting so all code is uniform.

Implementation (Draft)

Support rules as either Rule or Core syntax
Top level report code generator
Release
Add CodeLen simple use case in DSL (CodeLens support in Rosetta can be investigated in parallel)
Reporting rule syntax optionally match function syntax with matching args
Add rule/function chaining functionality (optional use of extract)
Add extract all keyword
Release?
Add deprecation warnings on all blueprints syntax
Release
Update all DRR rules (using extract)
Delete Blueprints syntax
Update CDM and all other models (to use extract)
Replace top-level report code generator with func generator

Appendix

Adopt List Keywords – `extract` and `filter when`

The Core syntax will adopt the preferred keywords for processing lists, e.g. extract replacing map.

Syntax

Before

Using expression with named list parameter.

set outputList:
    inputList
        map <Named item> [<Expression to modify named item>]
        filter <Named item> [<Boolean expression to filter named item>]

After

Using expression with named list parameter.

set outputList:
    inputList
        extract <Named item> [<Expression to modify named item>]
        filter <Named item> [<Boolean expression to filter named item>]

Example

Before

Using expression with named list parameter.

func ExtractPriceType: 
    inputs:
        prices Price (0..*)
    output:
        priceTypeEnums PriceTypeEnum (0..*)

    set priceTypeEnums:
        prices 
            map price [ price -> priceType ]
            filter priceType [ priceType = PriceTypeEnum->AssetPrice ]

After

Using expression with named list parameter.

func ExtractPriceType: 
    inputs:
        prices Price (0..*)
    output:
        priceTypeEnums PriceTypeEnum (0..*)

    set priceTypeEnums:
        prices 
            extract price [ price -> priceType ]
            filter priceType [ priceType = PriceTypeEnum->AssetPrice ]

Core and Rule Syntax Harmonisation

Introduction

Rule Syntax

Limitations

Single Input Parameter

Single Item Processing

Inefficient Type and Cardinality Validation

Benefits

Concise Syntax

List Keywords – extract and filter when

Concatenate operations with then keyword

Syntax Proposal

Adopt List Keyword – extract

Syntax

Before

After

Example

Before

After

Implement CodeLens/Inlay feature to overlay type and cardinality in Rosetta UI

Adopt feature to concatenate operations

Example

Before

After

Adopt feature to concatenate operations (part 2)

Example

Before

After

Adopt concise syntax to call a reporting rule

Example

Before

After

Reporting rules allow multiple inputs parameters / multiple cardinality

Example

Before

After

Auto-formatting

Implementation (Draft)

Appendix

Adopt List Keywords – extract and filter when

Syntax

Before

After

Example

Before

After

Clone this wiki locally

List Keywords – `extract` and `filter when`

Concatenate operations with `then` keyword

Adopt List Keyword – `extract`

Adopt List Keywords – `extract` and `filter when`