Data Gathering

Data Gathering

ExPresS XR provides an easy system for collecting and exporting data.

The DataGatherer is a component to extract and save data in a Unity Scene. All values that are specified in the DataGatherer will be stored in a CSV-semicolon-formatted file, which will be saved at the path and/or be sent via HTTP POST to a server.

The DataGatherer is built to support different commas (,) and semicolons (;), but also custom column separators as well as escaping columns containing these characters.
This is to comply with local standards and prevent issues with commas because of the heavy use of commas in the ToString()-methods.

To print or export (only at runtime) the current values, the buttons at the bottom of the editor can be used.

You will find a YouTube video at regarding this topic here.

Export Options

The Data Gatherer supports exports to the local file system and HTTP Posts.
Depending on your selection, the input field for the local and/or HTTP path will be shown, allowing the input of the paths.

If the local file path does not end with one of the following file extensions .txt, .log or .csv, the extension .csv will be added as per default. When used in the editor, the file will be saved at the specified path relative to the project location.

Using the built version on PC, the values will usually be stored in a subfolder of the AppData-Folder to ensure writing privileges. To open it, press Windows-Key + R, then enter %APPDATA% and confirm. This will open the AppData/Roaming folder. Go back one directory to Appdata and select LocalLow. Find the Folder named after your Company you set during the build (Default: DefaultCompany) and locate the folder named after the application.
When exporting to android headsets, you'll find the data in /storage/emulated/<your_user>/Android/data/<your_package_name>/files of your headsets file browser. <your_user> should be the account name that you use with your headset. <your_package_name can be found in Unity's Player Settings under the Android tab in the section "Other Settings" and usually consists of your Company Name and App Name, separated by dots. They can be retrieved either by connecting them to your PC and accessing the headsets file system or transmitting them via like bluetooth directly via the headset.

For separating values from different sessions, the option newExportFilePerPlaythrough will create a new CSV file for each playthrough. For each file, a timestamp in the form of _yyyy-MM-dd_HH-mm-ss.csv is added to the name.

The default separatorType is Semicolon as commas are used in the string representation of Vectors and other classes.
It is advised to enable escapeColumn to surround any columns containing the separator character (but also the escape character) with double-quotes (").
Since using Semicolons is also not guaranteed to be safe, it is advised to always keep escapeColumn enabled, unless your table-editor does not support it.

Triggers

The export can either be performed periodically at specific intervals, during FixedUpdate() and/or when an InputAction is performed.

Periodic exports will be performed when enabled and the export time is greater than zero.

The InputActions can be added to the InputActionTrigger-Array similar to the InputActions to export, but one with ActionType.Button must be selected.

Exporting Values

The DataGatherer provides the possibility to export public variables (members) of any component as the return values of functions and the values of InputActions.

If includeHumanReadableTimestamp is checked, a human-readable Timestamp in the form of yyyy-MM-dd HH-mm-ss (like 2023-12-01 23-59-00) will be added as one of the first columns of the CSV.
If includeTimestamp is checked, an UNIX timestamp will be added as one of the first columns of the CSV.
If includeUnityTime is checked, the Unity time (milliseconds since the app was started) will be added as one of the first columns of the CSV.
If includeDeltaTime is checked, the delta time (milliseconds since the last update) will be added as one of the first columns of the CSV.

Export Variables and Function Return Values

Adding new columns to be exported is handled via Drag & Drop similar to Unity's Event System: When +-Button in the editor for the dataBindings-array is pressed, a new entry is generated where a column name and a GameObject can be specified.
Once a GameObject is set, the Dropdown-Menu Values To Save can be used to select a value from a Component of the object, similar to how UnityEvents work. The selected value will be evaluated every time a new CSV-line is exported.
Exportable values are either public properties, members or functions. The latter may not have any (required) arguments but can be used on functions with default values. One notable exception is, if the first parameter is of type char. In these cases, the DataGatherer will call the function passing the columnSeparator. This is especially useful when returning multiple columns, as described below.

Please note that only primitive, positional values (Position, Rotation) and strings can be exported here.
Should other values be exported, a function converting a value into a string can be created for that purpose.
This also applies to values from static fields that do not have a GameObject associated with them (e.g., Application.version).

Be careful when changing the components of your target objects, as this might mess up the bindings. If you export the wrong values, simply remove and add the binding again.

class ExportWrapperComponent
{
	class MyObject myObject;
	
	public string GetApplicationVersion() => Application.version;
	
	// Assuming the class "MyObject" exists and has a function "getExportValue()"
	public string GetMyObjectExportValue() => myObject.getExportValue();
}

Exporting Multiple Columns

The DataGatherer supports a system for exporting multiple-columns at once. It is based on Attributes, the same system that is used for defining SerializeProperty by adding [SerializeProperty] in front of a function.

Multi-Column support provides three Attributes:

• MultiColumnValueAttribute: Marks an entry as multi-column. As the value will not be escaped even though it does contain the columnSeparator. The proper escaping of the individual columns must be done by the developer.
• HeaderReplacementAttribute: Automatically replaces the exportColumnName of the binding with the list of passed strings (See example below). Each string will be separated using the correct columnSeparator.
• HeaderReplacementNoticeAttribute: Will be printed when selecting this function to be exported. Can be used to convey information about the provided values.

Typically, you want to use MultiColumnValueAttribute and HeaderReplacementAttribute together:

[MultiColumnValueAttribute]
public string GetMultiColumn(char sep = CsvUtility.DEFAULT_COLUMN_SEPARATOR) 
	=> CSVUtility.JoinAsCsv(new string[] {"v1", "v2", "v3"}, sep);
// Result (sep = ','): v1,v2,v3
// Notice: value is not escaped with "

[HeaderReplacementAttribute("h1", "h2", "h3")]
public string GetMultiColumnWithHeaderReplacement(char sep)
	=> CSVUtility.JoinAsCsv(new string[] {"v1", "v2", "v3"}, sep);
// Header (sep = ','): h1,h2,h3
// Result (sep = ','): "v1,v2,v3"
// Notice: value is escaped with "

[MultiColumnValueAttribute]
[HeaderReplacementAttribute("h1", "h2", "h3")]
[HeaderReplacementNoticeAttribute("Careful! This function exports multiple lines!")]
public string GetMultiColumnWithHeaderReplacement()  => "v1,v2,v3\nv4,v5,v6";
// Header (sep = ','): h1,h2,h3
// Result (sep = ','): v1,v2,v3
// Prints to Console: "Careful! This function exports multiple lines!"

Quick Tip: If you want to combine multiple values from multiple sources, create a function that returns the values as a list of objects. Merge them and convert them using CSVUtility.JoinAsCSV().

Export Input Action Values

To export the values of InputActions, the InputActionBindings can be used. Add your desired InputAction via the ObjectPicker (Circle with a dot) to the list.
Usually there two are separate values for an action (e.g. Select and Select Value). The non-value action will return a simple Boolean (true/false) if that action is performed, while the second one will return a value (e.g. the grip strength).
All InputActions can be inspected at Assets/Settings/Input Mappings/XRI Default Input Actions.inputactions.

CSV Utility

The functions from the static class CsvUtility can be used to convert Unity Objects to generate CSV-compatible strings. This includes converting arrays to strings, joining multiple values as CSV-string and ensuring the values are safe to use with CSV.

For more guidance, refer to the documentation of the class CsvUtility.

Adding DataBindings via code

If you want to add new Bindings via code, you can do that by creating a new Binding with the constructor DataGatheringBinding(Component targetComponent, string valueName). However, using the inspector instead of code is recommended whenever possible.
The constructor creates a new Binding that should be bound to the valueName-member of the provided Component and its associated GameObject. The valueName must be a pretty name (i.e. as displayed in the menu, including the annotation for methods).
You do not need to provide the full name, a unique suffix suffices: "ButtonQuiz/string GetFullQuizCsvExportValues(char? sep)" and "GetFullQuizCsvExportValues(char? sep)" both work.
If valueName is invalid, it will be bound only to the GameObject, but no value will be selected.

Here is an example code on how to add a new Binding.

using System.Linq;

DataGatheringBinding binding = new(_quizGo, "ButtonQuiz/string GetFullQuizCsvExportValues(char? sep)");

DataGatheringBinding[] oldBindings = dataGatherer.dataBindings;
dataGatherer.dataBindings = oldBindings.Concat(new[] { binding }).ToArray();
// Or set it to a new array, if the old one was empty:
// dataGatherer.dataBindings = new[] { binding };