AudYoFlo: Recipes: From Simple Component to Matlab Full Feature Development Implementation

In this chapter, the transformation of a simple algorithm into a Matlab full feature algorithm development implementation shall be demonstrated. The example is based on the project ayfctc which contains the audio node ayfAuNCTC.

We start this demonstration in a moment where the new algorithm has not functional specific implementation yet but involves only an in-place audio processing algorithm in class

In order to extend our algorithm to run in the Matlab host, we refer to the (template) project ayfAuNMatlab,

which is also the basis for the explanations in the Wiki.

Getting the Component Running in Matlab

In the first part, we explain what is required to compile the component and to provide additional files which are useful for the Matlab host intergation.

Step I: Move Component into a Namespace

Our new algorithm component has a rather simple CMakeLists.txt build definition:

There is no hint to add any kind of Matlab support.

At first, we need to define a project namespace for our new class to prevent any duplicate symbols when linking statically:

and in the CMakeLists.txt file

Step II: Create two Versions of the Algorithm

Principally, all AudYoFlo components can be operated in the Matlab host. However, some instrumentations help to provide simplifications and hooks to better support functionality to better interact C/C++ code with Matlab. Therefore, next, we create two version of our algorithm by adding some lines in the CMakeLists.txt file:

Note that we keep two entries unused at this point:

# set(LOCAL_START_SCRIPT_MATLAB ${CMAKE_CURRENT_SOURCE_DIR}/scripts/${JVX_OS}/start_mat)
# set(LOCAL_SOURCES ${LOCAL_SOURCES} ${CMAKE_CURRENT_SOURCE_DIR}/src/CayfAuNMatlab_matlab.cpp)

Also, it is useful to create a build specific component descriptor in file componentEntry.cpp to better distinguish between the instrumented and the non-instrumented version of the algorithm:

The pre-processor macro JVX_EXTERNAL_CALL_ENABLED is set in case the algorithm provides Matlab instrumentation. In order to work properly, a project configuration file is required. We can add this configuration file by adding the subfolder matlab-in folder to our project. It is the easies approach to copy the folder in our template project jvxAuNMatlab,

to the project folder,

We need to run a full INSTALL build to review the output from the latest modifications in the release folder <BINARY_RELEASE>/release/runtime/matlab/m-files/subprojects-audionode/+ayfAuNCtc,

The file jvx_init_callback.m is the output of a code generation process in which the location of the development project is given,

Step III: Provide Callback Implementations for the Main Processing Hooks

While processing in the Matlab host, the system callbacks are linked in via the other files in the folder <BINARY_RELEASE>/release/runtime/matlab/m-files/subprojects-audionode/+ayfAuNCtc. For example, in the main processing call, the local function implementation refers to a function jvxProcessOffline_local,

This function is expected to be located in our project folder of project ayfAuNCtc in subfolder matlab-local. We therefore copy this folder from the template project ayfAuNMatlab

to our new project folder

Step IV: Provide a Start Script

In the next step, we need to provide a start script for our new project. This start script is mainly used to define a pre-defined configuration file and to output some project specific information. In order to provide this start script, again, we copy the folder scripts from the folder of project ayfAuNMatlab to the project folder,

In that file, we make modifications of the startup-script as required, e.g., we provide a project specific configuration file ayfctc-mat.jvx:

We need to activate the option to provide the start script in the CMakeLists.txt file in order to be deployed on build,

set(LOCAL_START_SCRIPT_MATLAB ${CMAKE_CURRENT_SOURCE_DIR}/scripts/${JVX_OS}/start_mat)

and run the INSTALL target build to let the latest modifications take effect. Note that in the section of the CMakeLists.txt file as shown, always the option JVX_ACTIVATE_VERSION_MATLAB(${PROJECT_NAME} JVX_EXTERNAL_CALL_ENABLED) must be last.

If everything has been setup properly, the runtime folder now contains the start script

which we should from now on use to start Matlab and our new algorithm. So far, we have not realized the config file ayfctc-mat.jvx.

Step VI: Starting the Matlab Host and Creating a Config File

Now, we can start the Matlab host and setup processing. Before we do so, however, we need to provide the location of Matlab in an environment variable:

We start the host by double-clicking the generated start script. This starts Matlab and displays the following text lines:

The options include only the line to start the offline host with the config file ayfctct-mat.jvx. The offline host is then started by copying the text and pasting it to the console:

The host pops up but it shows that no valid connection is currently available:

This is due to the fact that the provided config file does not exist yet.

In order to configure the host for later runs, we need to select the instrumented version of our new algorithm in the host:

Then, we finally save the current configuration by pushing the Save button in the lower left corner and shutdown the host.

Step VII: Adding the Config File to the Project

Whenever we start the Matlab again, the new module will be automatically selected due to the config file. Thwerefore, it is very useful to store the config file to be part of the project so that the host will start properly on every build system. We do so by adding the new config file in the install/common folder of our project. We therefore copy the file from the relase/runtime folder of our project,

to this location:

The file will be installed after the next call to the cmake build command and running the INSTALL build target.

Step VIII: Setting up the Host for Processing

In the last step, the new component will be involved in offline signal processing. For this purpose, we select a wav file as a source and specify two output channels. The selection of the input file is done by pushing the file selection button (...) in the Input Specification section in the left upper quarter of the user control.

We select the file wavs/music_stereo_48000Hz.wav

and acknowledge. Next, the waveform is viewed. For processing, we add two channels (+) in the Output Specification section of the UI:

The host is now ready for processing:

At first, we push the Save button to store the configuration for next time. Then, we can push the start button in the lower area of the host.

We will see the progress bar indicate the progress of processing,

and finally can view the output signals in the wavform viewer under ``Data Management'':

Functions to Set/Get Properties

The new components properties can be accessed from Matlab using an AudYoFlo system call. However, function calls can be generated by the build system to have easier access. Our new module has a property bypass which is defined in the pcg file description,

By adding the line

jvx_genMatProperties(${JVX_TARGET_NAME} "JVX_COMPONENT_AUDIO_NODE" "node" "${LOCAL_PCG_FILES}")

to the local CMakeLists.txt file,

activates the code generator to output function implementations to address the property from within Matlab. The generated functions are copied to the runtime relase folder on running the INSTALL target:

Instrumenting the C/C++ Code to Interct with Matlab

So far, the new component was involved as a pure C/C++ library - any kind of Matlab specific instrumentation has not yet taken place. The involved module is the exact same module that can also be used in the QT host. The instrumentation, however, allows many other functional extensions of algorithms and is therefore a very important step in the algorithm development process.

Code Modifications in Algorithm Project

In order to add Matlab instrumentation, our new class must be drived from an intermediate template class CjvxMexCallsTpl<CjvxBareNode1io> which sits on top of the original base class CjvxBareNode1io. Also, a new file export_project.mcg is added to our project that will allow us to add additional

In our class implementation, we add this extension as follows:

#include "jvxNodes/CjvxBareNode1io.h"
#include "pcg_exports_node.h"

#ifdef JVX_EXTERNAL_CALL_ENABLED
#include "CjvxMexCallsTpl.h" 
#define JVX_MAIN_BASE_CLASS CjvxMexCallsTpl<CjvxBareNode1io>
#else
#define JVX_MAIN_BASE_CLASS CjvxBareNode1io
#endif

class CAyfAuNCTC: public JVX_MAIN_BASE_CLASS, public genAuNCTC_node
{
public:

	JVX_CALLINGCONVENTION CAyfAuNCTC(JVX_CONSTRUCTOR_ARGUMENTS_MACRO_DECLARE);
	~CAyfAuNCTC();

       // ...
#ifdef JVX_EXTERNAL_CALL_ENABLED
	void initExternalCall() override;
	void terminateExternalCall() override;
	void onPrepareConnectMexCalls() override;
#endif

}; 

The pre-processor define JVX_EXTERNAL_CALL_ENABLED distinguishes between the version with Matlab instrumentation and the non-instrumented version. Accordingly, two options for the involved base classes are possible.

Besides the derivation, three functions need to be provided in case the Matlab instrumentation is chosen:

1. Member function initExternalCall
2. Member function terminateExternalCall
3. Member function onPrepareConnectMexCalls

It is a good idea to store the implementations of these functions in a dedicated file, e.g., CayfAuNCTC_matlab.cpp,

Activation of the Instrumented Implementation

Once the code modifications have been finalized, we need to link our new component to the correct Matlab entry functions. This is done in the Matlab host:

When starting the host,

there is now a new UI section holding the names of the Matlab Callbacks,

The right most combobox is setup to adress subfolder jvxDefault. This option must be modified to adress the current component ayfAuNCtc,

In the final step, we need to save the configuration such that the setup will be available at future starts. Note also that the modified configuration file may have to be stored in the project to be part of releases at other installation locations.

Signal Processing Modes

With the instrumented version of our algorithm we may run the signal processing either in C/C++ (as usual) or we may engage the Matlab callbacks. In the UI section for the Matlab callbacks the user may activate Matlab processing by activating the checkbox to Engage Matlab,

If the option is active, processing is always file or data based, that is, processing starts, procedes and ends. The output data is stored in a variable in the workspace. The sequence for processing involves the callbacks as listed in the UI:

1. When processing is started, at first the callback jvxStartOffline is called once. The callback may allocate storage and initialize everything for processing.
2. After the processing has been started, the callback 'jvxBeforeProcessingOffline' is called. In contrast to the callback jvxStartOffline in this callback all information about the following processing such as the number of frames that will be processed are provided. We may hence allocate data structures to hold all samples in all frames about to follow. This callback is always called only once.
3. Then, we run the main processing loop in which the actual processing callback jvxProcessingOffline is called for every input frame. Typically, the algorithm produces an output buffer and returns it to the host.
4. Once all frames had been processed, the callback 'jvxStopOffline` is called. In this callback, processing elements may be deallocated or otherwise cleaned up.

Note that, typically, the Matlab processing is much slower than the C/C++ based processing.

Switching Context: Calling C/C++ from within Matlab

Once our component is active in the Matlab host we may call C/C++ functions from within Matlab. In order to do so, the component defines entry points that we may address from within the Matlab environment. The definition of the entries is done in file codeGen/export_project.mcg. From this file, the AudYoFlo code generation tools generate the header mcg_export_project.h which we add to our project in the file CayfAuNCTC_matlab.cpp:

All functions,

• requestExternalCallHandle
• initExternalCallhandler
• terminateExternalCallhandler
• returnExternalCallHandle

are indeed output from the code generation procedure.

Example Function Definition

An example for a function definition is the following:

SECTION MATLAB_EXPORTS
{

	//USE_MATLAB_CALL_AS_LIBRARY = "YES";
	REFERENCE_CLASS = "CayfAuNCTC";
	OUTPUTFILE_NAME = "CayfAuNCTC_external";	

	SECTION myFirstAlgorithm
	{
		DESCRIPTION = "Christians Algo";
		ACCEPT_INPUT_TYPES = { "JVX_DATAFORMAT_DATA"};
		DIMENSION_INPUT_FIELD = {"1D"};
		DESCRIPTION_INPUT_PARAMETERS = { "Input Signal"};
		ACCEPT_INPUT_NUMBER_MIN = 1;
		ACCEPT_INPUT_NUMBER_MAX = 1;
		PRODUCE_OUTPUT_TYPES = {"JVX_DATAFORMAT_DATA"};
		DIMENSION_OUTPUT_FIELD = {"1D"};
		DESCRIPTION_OUTPUT_PARAMETERS = { "Output Signal"};
		ACCEPT_OUTPUT_NUMBER_MIN = 1;
		ACCEPT_OUTPUT_NUMBER_MAX = 1;

		//INPUT_OUTPUT_CROSS_REFERENCE_X = {1};
		//INPUT_OUTPUT_CROSS_REFERENCE_Y = {1};
	};
};

This entry defines a function entry with name myFirstAlgorithm that expects one input and one output argument. In order to function properly, we need to adapt the code of our algorithm to include the callback function in file CayfAuNCTC.h as a declaration,

and in file CayfAuNCTC_matlab.cpp as a function definition,

The callback function is implemented such that a (1xM) vector input is simply returned.

Calling Functions in our Component from within Matlab

With the new callback, dedicated functions can be directly called from Matlab. The component must be loaded (by starting the Matlab host) and we must access the backend via backend handle by using

global jvx_host_call_global;

in Matlab. Once the access function is available, we can use the function call

[str, b] = jvx_host_call_global('external_call');
b

to list all function entry points. We can use the object filter to only view the function entry points for our new module:

[str, b] = jvx_host_call_global('external_call', 'CayfAuNCTC');
b

We can finally call the entry function by running the following command:

[str, b] = jvx_host_call_global('external_call', 'CayfAuNCTC', 'myFirstAlgorithm', [1 2 3 4])

The returned vector is available in variable b.