Skip to content

Latest commit

 

History

History
360 lines (251 loc) · 14.1 KB

RASynEdgeImpulseDataIngestion.md

File metadata and controls

360 lines (251 loc) · 14.1 KB
Raw

Using Edge Impulse RASynBoard application

The Edge Impulse team has developed a custom RASynBoard application that can capture audio or IMU data from the RASynBoard's onboard sensors and transmit the captured data directly into your Edge Impulse project using the Edge Impulse Daemon utility running on your development PC.

The Avnet Out-of-Box application has the ability to send IMU data to your Edge Impulse project using the Edge Impulse Data Forwarder utility, but if you are capturing audio data with the Avnet Out-of-Box application you need to record the data to the microSD card and then upload the audio file to Edge Impulse.

If you use the Edge Impulse RASynBoard application, you can capture IMU or audio data directly from your RASynBoard's sensors directly into your Edge Impulse project. This workflow is optimized to help developers capture training or testing data directly into your Edge Impulse project.

Edge Impulse RASynBoard Documentation

Table of Contents

Pros to using the Edge Impulse RASynBoard application:

  • Capture audio or IMU data directly to your Edge Impulse project
  • Drive the data capture workflow directly from the Edge Impulse project
    • Review the data and segment data into multiple files as you capture the data
    • Label the data as it's captured
  • Test your ML models without having to load the Avnet Out-of-Box application

Cons to using the Edge Impulse RASynBoard application:

  • You must flash the Edge Impulse RASynBoard application to your RASynBoard
  • The Avnet Out-of-Box features like cloud connectivity are not implemented in the Edge Impulse RASynBoard application

Prerequisites

Install NodeJS with npm

In order to use the data forwarder, you must have NodeJS with npm installed. Please refer to the NodeJS website for installation: https://nodejs.org/en/download/package-manager

Install the Edge Impulse CLI tools

Once you have installed NodeJS and npm, install the Edge Impulse CLI tools with the following command:

MacOS/Linux:

sudo npm install -g edge-impulse-cli

Windows:

npm install -g edge-impulse-cli

Load the Edge Impulse RASynBoard Application onto your device

Where to find the application

Every time your create a RASynBoard deployment in Edge Impulse Studio the Edge Impulse RASynBoard application is built and delivered as part of the deployment.



Starting with the Avnet Out-of-Box v.1.5.0 release, we include this application in the Release Assets section.

Note the *.srec image file is contained inside the *.zip file and must be extracted before loading onto the RASynBoard.



Load the latest microSD card files to your RASynBoard's microSD card

You should have the latest RASynBoard Out-of-Box release microSD card files on your microSD card. Unzip the compressed file and copy all the files to the root directory of your microSD card.



Flash the Edge Impulse RASynBoard application to your RASynBoard

The UsingRASynBoardReleases.md documentation includes details on . . .

  • Installing the Renesas Flash Programmer (RFP)
  • Using the RFP to flash a *.srec file to your board
    • Be sure to flash the firmware-avnet-rasyn*.srec file

Prepare the RASynBoard to auto run

Install the J3 Jumper and connect your USB-C cable to the core board as shown



Configure the application

Similar to the Avnet RASynBoard Avnet Out-of-Box application you can configure the Edge Impulse RASynBoard application by editing the config.ini file in the root directory of the microSD card.

Define the data output port [Debug Print]-->Port=x

The edge-impulse-daemon will connect to your RASynBoard using a COM port on your development PC. Just like we can define where debug data will be directed in the Avnet Out-of-Box application, we can define where the Edge Impulse RASynBoard application will output the sensor data. This is done using the [Debug Print]-->Port=x configuration item.

Note: You'll see higher data transfer rates if you use the VCOM port Port=2

  • Port=1 Outputs sensor to the PMOD connector signals
  • Port=2 (Preferred) Outputs sensor to the UART associated with the core board USB-C connector



Define the ML model to correspond with the data type to capture Mode=x

We need to tell the Edge Impulse RASynBoard application which ML model to load. If we want to collect IMU data, we must use an IMU model, and likewise, if we want to capture audio data we must load an audio ML model.

Capture audio data



Capture IMU data



Checkpoint

At this point we have . . .

  • Loaded the Edge Impulse RASynBoard application onto our board
  • Copied the microSD card files from the most recent Avnet Out-of-Box release to our microSD card
  • Configured the output port
    • [Debug Print]-->Port=2
  • Configured the ML model to correspond to the type of data we want to capture
    • Mode=4 to capture Audio data
  • Configured our board to auto run the application (installed the J3 jumper across pins 1-2)
  • Connected our USB-C cable to the core board's USB-C connector

Now we can collect some data!

Log into your Edge Impulse account

Edge Impulse

Create a new project

  • Select the + Create new project link



The Create a new project pop-up opens

  • Give your project a name
  • Select the Personal project type
  • Select the Public project setting
  • Click the Create new project button



Connect your board to the project

This is one of the coolest features of Edge Impulse. They provide all the tools I need to connect my device directly to my project. Once the connection is established, I can define the data I want to collect including how I want to label the data. Let's give it a shot . . .

For this example lets collect some audio data

Start the edge-impulse-daemon

The edge-impulse-daemon utility was installed with the Edge Impulse CLI tools.

  1. Open a terminal on your development PC
    1. Windows PowerShell is supported however my work laptop restricts me from running the edge-impulse-daemon
    2. I use the Windows command shell cmd
  2. The edge-impulse-daemon runs and prompts for . . .
    • Your Edge Impulse login credentials
    • Which COM port is connected to your board
      • If you have multiple COM ports you'll be prompted to select the correct port
      • If you only have one COM port it will automatically use that one
    • Which Edge Impulse project to connect to
      • Select from the list
    • Name the device
      • Enter a name for your board, this will show up in Edge Impulse Studio
  3. A URL is output that will take you directly to your Edge Impulse project and the Data acquisition tab!





Capture Audio Data

At this point we can capture data using the Collect data controls. In the Collect data window . . .

  1. Define a label for your data
    • Use a name that describes your data
    • Labels are case sensitive
  2. Select Microphone for the Sensor
  3. Leave the Sample length (10 seconds), and Frequency at the defaults
  4. Click the Start sampling link

This is where the magic happens!

  • Edge Impulse will send a message down to the daemon telling it to collect 10 seconds of audio data
  • The daemon will send a message to the RASynBoard app, telling it to collect 10 seconds of audio data
  • The RASynBoard app starts to flash the Blue LED indicating that it's recording
  • Start to repeat the keyword until the Blue LED stops flashing "Avnet Avnet Avnet . . ."
  • The RASynBoard collects the data and transfers it to the daemon
  • The daemon sends the data to your Edge Impulse project where it's displayed!

Once the data is in my project I can . . .

  • Play back the audio using the audio control
  • View what my data looks like



I can also drill down into the data and split my sample into multiple files, each containing one instance of my keyword.

  • In the RAW DATA audio control, click on the vertical ellipsis (upper right hand corner)
  • Select the Split sample option



The interface opens a new window and tries to identify unique segments and allows you to . . .

  • Play each segment
    • Select a segment to play it
  • Drag the segment windows around
  • Remove segment windows
  • Add segment windows
  1. Click on the Split link



The Dataset list updates. Notice that the single 10 second sample is now replaced with multiple 1 second samples.



Capture 6-Axis IMU Data

To capture IMU data from the RASynBoard we need to change the ML model that gets loaded

  1. Open the config.ini file on the microSD card
  2. Change Mode=5 to load the IMU model



  1. Power cycle your RASynBoard to take the new configuration
  2. Start the edge-impulse-daemon
  3. If you want to connect the edge-impulse-daemon to a different Edge Impulse project use the --clean argument
    • edge-impulse-daemon --clean
  4. In Edge Impulse define a new label SideToSide
  5. Set the sensor to Inertial
  6. Click the Start sampling link



  1. The magic happens again!
  2. Move your device side to side while the blue LED flashes

The data is captured and sent up to your Edge Impulse Project!



Conclusion

I hope you were able to experience how the Edge Impulse RASynBoard application can optimize your data collection workflow. If you have any issues or questions, please open a new issue in the GitHub Repo.

Next Steps

Now that you have your training data in Edge Impulse Studio, you can follow the Edge Impulse Tutorials detailing how to create ML models targeting the RASynBoard's Syntiant NDP120 Neural Decision Processor.

Troubleshooting Tips

I've tried to anticipate issues that developers may encounter with this tutorial and captured some troubleshooting tips.

edge-impulse-daemon does not connect to the board

  • Verify that the microSD card is inserted into the I/O board
    • Since the config.ini file in on the microSD card, it must be present
  • Verify you've selected the correct COM port when starting the edge-impulse-daemon
    • Verify that the COM port selected is connected to the RASynBoard on the port indicated by the [Debug Print]-->Port=x setting
      • [Debug Print]-->Port=2 the data will be on the USB-C connector on the core board (the smaller board)
      • [Debug Print]-->Port=1 the data will be on the PMOD connector and you must use a USB to TTL serial cable, connect it to the PMOD connector as shown below



  • Try rebooting your PC to clean up any COM port errors

Sampling Failed Error

Trying to capture audio data with an IMU model loaded

If you try to capture audio data when an IMU model is loaded, you'll see this error



  • Edit the config.ini file and set Mode=4 to load the audio model

Trying to capture IMU data with an audio model loaded

If you try to capture IMU data when an audio model is loaded, you'll see this error



  • Edit the config.ini file and set Mode=5 to load the IMU model