[telemetry] Add new Telemetry implementation

[PeterJohnson]

Significantly simplified version of #6453. The design approach here is to make Telemetry purely imperative/immediate and write only. Read capabilities would be added as a separate Tunable implementation.

Telemetry is a utility class with only static functions to allow simple use such as Telemetry.log("name", value); (ala System.out.println()), and is intended as the primary user-facing class. Nested (structured) telemetry is available via TelemetryTable, instances of which can be gotten by calling Telemetry.getTable() (or TelemetryTable.getTable() for further nesting).

The Sendable concept equivalent is TelemetryLoggable (not a great name, but naming is hard). Unlike the current Sendable / SmartDashboard.putData(), this is designed to be immediate, not callback-based. The updateTelemetry(TelemetryTable) function of a TelemetryLoggable should immediately log the desired data to the provided TelemetryTable and not store the table for later use.

While we aim to fast-path most types through specific overloads, we do have a generic Object-taking overload to avoid potential overload ambiguity (particularly between StructSerializable and ProtobufSerializable, where many types implement both) and provide a fallback toString path. There's a type registry mechanism to register specific type handlers; the intent here is to use this for things like Unit types, where you might want to both set a property (indicating the unit type) and provide the value as a double.

Backends can be configured at any level; the most specific backend is used based on longest prefix match of the telemetry path (this allows for e.g. setting up NT logging at the top level, but making some tables DataLog-only). Backends are provided for both NetworkTables and DataLog, and there's also a discard backend (that throws away any logged data) as well as a mock backend (for unit testing).

The initial backend implementation takes the type of the first log() call as the "forever" type for that particular name. Trying to later log to the same name with a different type is ignored and emits a warning. Changing types dynamically both significantly increases implementation complexity and will likely result in difficult-to-debug behavior in downstream tooling; it's hard to see the user benefits to supporting this.

Dependency wise, the telemetry library only depends on wpiutil.

TODO:

• DataLog backend
• NT backend
• Discard backend
• Mock backend
• Introspection for struct/protobuf (to get .struct / .proto objects)
• C++ implementation
• Port current use cases of SmartDashboard/Sendable (and remove the implementations)
• Implement int and short arrays for DataLog and NT
• Registry should cache Entry so they are safe to use with ConcurrentMap and can be reset on reset()
• Registry should potentially cache SendableTable objects too?
• RobotBase set up NT as the default global backend
• Unit tests and examples
• Add early discard checks (avoid work if going to discard backend)
• Make discard easier to use (table function instead of needing to use backend API?)
• When new backend registered, remove matching cached entries from tables (or just invalidate all caches)
• Implement type handler(s) for unit types
• Implement warnings on type mismatch
• Port current use of DataLog/NT APIs (in particular, DS joysticks) (TBR)
• Add tunables (maybe separate PR?)
• Design document
• Update type strings for consistency (some have spaces between words, others don't)

Fixes #5513
Closes #5912
Closes #5481
Closes #5413

[alan412]

I don't see any locking to make sure this works from multiple threads.

I also don't understand why there are a lot of individual functions with things like "log" naming.

[Gold856]

I also don't understand why there are a lot of individual functions with things like "log" naming.

What do you mean by this? Are you asking why everything is called log or why there's so many overloads? log was chosen because lots of other 3rd party telemetry libraries use that name, and people seem to prefer the shorter name. If you're asking about the overloads, we have them to prevent accidental casting from, say, a double to a float, or a long to an int.

[alan412]

I also don't understand why there are a lot of individual functions with things like "log" naming.

What do you mean by this? Are you asking why everything is called log or why there's so many overloads? log was chosen because lots of other 3rd party telemetry libraries use that name, and people seem to prefer the shorter name. If you're asking about the overloads, we have them to prevent accidental casting from, say, a double to a float, or a long to an int.

I wasn't clear here. This is in reference to TelemetryBackEnd.java

[PeterJohnson]

I also don't understand why there are a lot of individual functions with things like "log" naming.

I wasn't clear here. This is in reference to TelemetryBackEnd.java

I haven't finished implementing the backends, but basically all the backends have type safety and type-specific functionality. So writing a double to a datalog file or NetworkTables is different than writing a string (or an integer). If you're asking why they are uniquely named instead of overloaded, overloading is bad practice with virtual functions (particularly in C++). A bunch of overloaded "log" functions on the user-facing side makes sense for ease of use, but is a potential footgun on the backend.

[PeterJohnson]

I don't see any locking to make sure this works from multiple threads.

It should be thread safe as written already. The backends will have locking/atomics included as needed (there are some synchronized blocks there already in the start of the DataLogSendableBackend implementation). We use ConcurrentMap etc to avoid explicit locking on the frontend (e.g. TelemetryTable caching uses ConcurrentMap), at least in Java--in C++, we'll use explicit mutexes.

[KangarooKoala]

Shouldn't this go into a separate file to match the other struct definitions?

[KangarooKoala]

Suggested change

	template <typename T>
	void ExpectLastValue(std::string_view path, const T& expected) {
	auto value = mock->GetLastValue<T>(path);
	ASSERT_TRUE(value);
	EXPECT_EQ(expected, *value);
	}

This should work and would reduce boilerplate, though it doesn't work for the types that gtest won't nicely format if the assertion fails. (If it doesn't recognize the type it outputs the hexademical of the raw memory contents- I remember encountering that for failing quaternion equality checks)

[spacey-sooty]

A log(name, measure) overload which includes unit metadata also be nice

[PeterJohnson]

Unit support will be implemented in a different way--the registry supports registering global type-specific handlers so we don't have to have telemetry depend on every library we want to provide overloads for.

[PeterJohnson]

Units support has been added.

In C++, it's integrated into units/base.h:

inline void LogEntry(wpi::TelemetryTable& table, std::string_view name, const nameSingular ## _t& value)
{
  table.SetProperty(name, "unit", " "#nameSingular);
  table.Log(name, value.value());
}

This will make it automatically work for all use cases.

In Java, it needs to be registered as a type handler, which is done in wpilibj RobotBase:

TelemetryRegistry.registerTypeEntry(Measure.class, (v, entry) -> {
  entry.setProperty("unit", v.unit().name());
  entry.logDouble(v.magnitude());
});

For Java, this means that user unit tests which want to log unit values in this way will need to similarly register the type entry. Unit tests will still compile and run without doing so, they'll just log a string value (via toString()) instead of a numeric value.

Currently, the callback is also a little different between C++ and Java; C++ gets the table and name, while Java gets the Entry. The reason for this is that we want to avoid referencing the virtual functions in Entry in C++, but it also adds a bit more flexibility and will let us unify the entry and table handler callbacks, so I'm considering changing the Java callback to also take the Table and name instead of the entry.

[KangarooKoala]

In C++, it's integrated into units/base.h:

inline void LogEntry(wpi::TelemetryTable& table, std::string_view name, const nameSingular ## _t& value)
{
  table.SetProperty(name, "unit", " "#nameSingular);
  table.Log(name, value.value());
}

This will make it automatically work for all use cases.

Before I forget, I think for mp-units we can use mp_units::unit_symbol(decltype(quantity)::unit) to get the unit symbol. I think we're also already planning to add a mp_units::quantity.value() (which can already be nicely expressed through the user-facing API), so logging the value shouldn't need to change.

Hard to say whether this or mp-units will happen first, but I just wanted to note this down before I forgot.

[KangarooKoala]

This class doesn't seem to exist?

[KangarooKoala]

FYI that this is cut off.