Skip to content

VisualizationTools

George Bosilca edited this page Feb 22, 2022 · 5 revisions

Table of Contents

Streaming data out of the runtime

Starting from commit https://bitbucket.org/icldistcomp/parsec/commits/e14cc8332d05|e14cc8332d05 PaRSEC gained the capability to automatically generate and export properties out of the Parameterized Task Graph algorithms. In addition, a python visualizer is available for online monitoring of PaRSEC activity.

 * Adds: property feature for PTG. Properties can be attached to a function in the PTG/JDF file
 * Adds: properties can return type: int32_t, int64_t, float, double. (int32_t is the default)
 * Adds: multiple properties can be attached to a function

First, let's see how to add properties to an already existing JDF file. The properties for tasks are specified using the `[]` notation after the name and parameters of the function declaration as indicated below.

Every time a taskpool is pushed to a context, the dictionary will explore it, extract all the properties available, register them and compare them to what was requested by the user. Every propety that matches a user requested one is then streamed outside the runtime from each node.

How to request properties?

Easy, user can request properties using basic wildcards as MCA parameters in ${HOME}/.parsec/mca-params.conf:

To request properties, provide a string in the form, namespace:function:property. The namespace is the PTG name (inherited from the source file name), the function is the function that need to expose its properties and the property is the name of the property attached to the function to be exposed. Despite the use of the regexp `*` we are not providing full regexp support, `*` can only be used on the property name. As an example writing something like , will not work, the `*` in the middle of a string will not be treated as a wildcard. Use semi-colon ';' to form a list of properties.

Make the properties available for online use

The simplest way to access these properties is via a shared memory area, updated by the runtime with the current values. To activate a shared memory area named "parsec_shmem", and export data through it, you will need to activate the shmem, and the PINS alperf module, in ${HOME}/.parsec/mca-params.conf:

parsec_shmem is the default name when no name is specified. The alperf PINS module is in charge of evaluating function and accumulating, or setting, the values in the shmem. This data can be either used locally, or saved for offline processing or be used by online tools to provide a view on the ongoing activities of the runtime.

Online visualization of PaRSEC activity/progress

In tools/aggregator_visu, there is a small tool, the reader. This tool can read a shared memory exported by PaRSEC, single node for now. Once compiled, step that should be done automatically during the installation, it needs the name of the shared memory as first parameter, and the property to retrieve.

As an example the `header` argument will display something like this `./reader parsec_shmem header`

Passing data will send the client in a while loop until the application leaves (not working yet). `./reader parsec_shmem data`

Visualizing data streamed from the PaRSEC runtime

To visualize performances and, optionally to control the application resources, one need to start the following applications in this order: * Aggregator * Demo server * Application * GUI

Build PaRSEC

Compile PaRSEC with `PARSEC_PROF_TRACE` option turned ON, in order to activate the PINS modules. alperf module will be listed during configure, and therefore will be compiled.

Starting the aggregator process

Set up these environment variables for the PaRSEC runtime to know where to find the aggregator process:

Call the aggregator process:

(Optional) Using a server to send commands

The application needs to know where this server is located:

To start the demo server:

To control the demo server, the format of commands is simplistic and ceertainly not user friendly, but this server exists as a toy:

where

  • i is an integer, the rank of the target MPI rank to control, -1 is for all the ranks
  • c is a character for the command:
  • n: this command doesn't take the core rank, and will return the statuses of all the cores
  • s: this command will stop the target core on the target process
  • w: this command will start the target core on the target process
  • j is an integer, the rank of the target core to control
Starting the application

Start your favorite application running on top of PaRSEC (+argobots).

After starting the application, the aggregator should list the available keys {K} to plot.

Visualizing

In order to plot, you need to run:

The GUI application is connected to the aggregator, set of plotable keys has been shared and acknowledged with the user. You can ask to plot a key: