Additional Semantic Information for Objects

[sAlexander]

(Feature request for consideration. This was based upon my reading of the previous spec, and may be out of sync with the concepts in this spec (such as the object tree))

I really like how the ThingSet protocol supports object names. But it would be nice to have other semantic information about fields, especially for input fields.

Examples of semantic information

To give some concrete examples of semantic information that could be very nice for inputs:

• name, binary id 0x00 (required): The human readable name of the device.
• value, binary id 0x01 (required): The current value of this object.
• minval, binary id 0x02 (optional): The minimum allowable value for the field. Uses the same datatype as the field itself.
• maxval, binary id 0x03 (optional): The maximum allowable value for the field. Uses the same datatype as the field itself.
• maxlen, binary id 0x04 (optional): The maximum allowable length for the field. Only applicable to string/byte fields.

An an example of semantic information that could be nice for outputs:

• readfreq_Hz, binary id 0x05 (optional): The internal read rate for this output. Sampling any more frequent than this value is not recommended.

Advantages of more semantic information

This additional semantic information could be very helpful for clients. For example, if clients know the minval and maxval for an input field, they can correctly send information within that bound. And if they know that output values will be clamped above some level, they can make more intelligent decisions based upon that.

This semantic information could also be very useful for user interface (UI) generation. For example, when displaying a UI to set the maximum allowable current for an MPPT solar converter, it would be nice to know that the minval for that field is 0A and the maxval for that field is 10A. Same with a thermostat or a lightbulb: that additional semantic information can make the UI much richer and more accurate. The UI may be able to rely entirely on the semantic information for generating a beautiful interface.

A possible implementation

With these semantic types, they could be queried as part of the "list" command for each type. For example, you could modify the list results as follows (this is Text Example 2 and Binary Example 3 on https://thingset.github.io/spec/functions):

Text Request:

!output {}

Text Response:

:0 Success. {
  3: {
    "name": "Bat_V", 
    "value": 12.2, 
    "readfreq_Hz": 1000}, 
  4: {
    "name": "Ambient_degC", 
    "value": 22,
    "minval": 0, 
    "maxval": 100},
}

Binary Request:

0x04        Function ID (output)
    0xA0    CBOR empty map

Binary Response:

0x80                # Status code (Success)
0xA2                  # A map with two data items              
   0x03 0xA3            # A map of properties for data item 0x3
      0x00                # Property: name
        0x65 0x4261745F56   # String "Bat_V"
      0x01                # Property: value
        0xFA 0x41633333     # Float 14.2
      0x05                # Property: readfreq_Hz
        0x19 0x03E8         # Unsigned Int: 1000
   0x04 0xA4            # A map of properties for data item 0x4
      0x00                # Property: name
        0x6C 0x416D6269656E745F64656743 # "Ambient_degC"
      0x01                # Property: value
        0x16                # Unsigned Int
      0x02                # Property: minval
        0x00                # Unsigned Int: 0
      0x03                # Property: maxval
        0x18 0x64           # Unsigned Int: 100

There is certainly more overhead associated with the proposal above. For Example 3 in the spec, the simple map{name: value} took up 26 bytes, while this updated map{id: map{properties}} would take 34 bytes to transmit the same information (name + value). With the additional information shown above (minval/maxval/readfreq_Hz), the message would take 43 bytes.

[martinjaeger]

This is a very interesting question and I also considered it some time ago, as it can be very useful to have information about the valid range of a value etc. But I decided that I would like to keep the implementation as simple as possible and provide just enough information for understanding the available data nodes and types.

With the new revision of the spec that is more compatible with CoAP and HTTP, one simple way to provide such features without losing the simplicity of the protocol would be with a query string in the GET request. For example, ?output?type=min ["Bat_V"] would respond with the min value instead of the actual value. But a device can also only support the basic features without the query string.

I would probably not like to implement this in the quite lightweight library now, as it would require enlarging the DataNode struct and use more memory for every node. Maybe there is a clever way around it so that this information is only stored for nodes that need it.
It could definitely be included in the spec so that anyone interested could implement it by extending the library. The query string could be used for lots of other (user-defined) stuff and is fully compatible with HTTP and CoAP aswell.

What do you think?

[sAlexander]

keep the implementation as simple as possible and provide just enough information for understanding the available data nodes and types.
I would probably not like to implement this in the quite lightweight library now

That makes total sense -- I'm a big fan of simplicity.

One thing that I was thinking could work for my application while using ThingSet: have a different limb of the tree that describes specifications:

{
    "output": {
        "Bat_V": 14.1,
        "Bat_A": 5.13,
        "Ambient_degC": 22
    },
    "spec": {
        "Bat_V": {
            "minval": 0,
            "maxval": 16,
            "readfreq_hz": 10,
        },
        "Bat_A": {
            "minval": -10,
            "maxval": 10,
            "readfreq_hz": 10,
        },
        "Ambient_degC": {
            "minval": 0,
            "maxval": 100,
            "readfreq_hz": 10,
        },
    }
}

IIUC, maps are a supported value type (or at least it's supported by Tiny-TP :)), and names only need to be unique if they are meant for publication. So I could perform the following text query for the specs of Bat_V:

?spec ["Bat_V"]
:85 Content. {"Bat_V":{"minval": 0, "maxval": 16, "readfreq_hz": 10}}

That's a pretty nice solution that would allow for improved UI presentations and additional client side validation (for devices that support specs on their properties).

(Things would be a bit more complex in the binary query language, but I think it would still work.)

[martinjaeger]

You understood correctly that maps are supported value types in general, but they are currently not implemented in the library (yet). Arrays were recently added.

It's also correct that the names only need to be unique if they are used for publication. Otherwise we couldn't have a separate "Enable" node under each publication channel.

I like your approach a lot, as it doesn't require any changes. Quickly tried it out and it works almost as you suggested. As a map as the value is currently not supported, I defined "Bat_V" as an internal (path) node.

Here is an example implemented in the C++ library: ThingSet/thingset-device-library@3c09bd3

The result is:

?spec/Bat_V
:85 Content. {"minval":0.0,"maxval":16.0,"readfreq_Hz":10}

If you have a linux system with installed GCC and PlatformIO it's quite easy to play around with the protocol now. You clone that repository and run following commands in its root folder:

pio run
.pio/build/native-std/program

It gives you a shell which reads requests from the terminal and prints the response.