This project provides a real-time embedded environment for STM32 microcontrollers to execute STEdgeAI generated models, specifically targeting the instance segmentation application. The code prioritizes clarity and understandability over performance, making it an ideal starting point for further development.
Detected classes are displayed on the bounding boxes. The instance segmentation is displayed in overlay.
This is a standalone project that can be deployed directly to hardware. It is also integrated into the ST ModelZoo repository, and is required to deploy the instance segmentation use case. The ModelZoo enables you to train, evaluate, and automatically deploy any supported model. If you wish to use this project as part of the ModelZoo, please refer to the Quickstart using stm32ai-modelzoo-services section for instructions.
This README provides an overview of the application. Additional documentation is available in the Doc folder.
Note: Please note that the NUCLEO-N657X0-Q board does not support any model for this use case at the moment.
- Features Demonstrated
- Hardware Support
- Tools Version
- Boot Modes
- Quickstart using stm32ai-modelzoo-services
- Quickstart using Prebuilt Binaries
- Quickstart using Source Code
- Known Issues and Limitations
Documentation Folder:
- Application Overview
- Boot Overview
- Camera Build Options
- Camera Orientation
- Aspect Ratio Mode
- Deploying tflite Model on STM32N6570-DK
- Deploying tflite Model on NUCLEO-N657X0-Q
- Programming Hex Files with STM32CubeProgrammer
- Sequential application flow
- NPU-accelerated quantized AI model inference
- Dual DCMIPP pipelines
- DCMIPP cropping, decimation, and downscaling
- DCMIPP ISP usage
- LTDC dual-layer implementation
- Development mode
- Boot from external flash
Supported development platforms:
- STM32N6570-DK Discovery Board
- Connect to the onboard ST-LINK debug adapter (CN6) using a USB-C to USB-C cable for sufficient power.
- OTP fuses are configured for xSPI IOs to achieve maximum speed (200MHz) on xSPI interfaces.
- NUCLEO-N657X0-Q Nucleo Board
- Connect to the onboard ST-LINK debug adapter (CN9) using a USB-C to USB-C cable for sufficient power.
- OTP fuses are configured for xSPI IOs to achieve maximum speed (200MHz) on xSPI interfaces.
STM32N6570-DK board with MB1854B IMX335.
Supported camera modules:
- Provided IMX335 camera module
- STEVAL-55G1MBI
- STEVAL-66GYMAI1
For the Nucleo board, one of the following displays is required:
- A USB host for data transmission via USB/UVC (using the USB OTG port CN8)
NUCLEO-N657X0-Q board with USB/UVC display.
- X-NUCLEO-GFX01M2 SPI display
NUCLEO-N657X0-Q board with SPI display.
- STM32CubeIDE (v1.17.0)
- STM32CubeProgrammer (v2.18.0)
- STEdgeAI (v2.2.0)
The STM32N6 series does not have internal flash memory. To retain firmware after a reboot, program it into the external flash. Alternatively, you can load firmware directly into SRAM (development mode), but note that the program will be lost if the board is powered off in this mode.
Development Mode: used for loading firmware into RAM during a debug session or for programming firmware into external flash.
Boot from Flash: used to boot firmware from external flash.
| STM32N6570-DK | NUCLEO-N657X0-Q | |
|---|---|---|
| Boot from flash |  |
 |
| Development mode |  |
 |
This application is a C-based project required by the deployment service in the ModelZoo. The ModelZoo enables you to train, evaluate, and automatically deploy any supported model.
To deploy your model using the ModelZoo, refer to the Deployment README for STM32N6 for detailed instructions on deploying to either the STM32N6570-DK or the NUCLEO-N657X0-Q.
Note: This C-based application is already included in the ModelZoo repository under STM32N6 instance segmentation application code folder.
To program the board's external flash, follow these steps:
- Set the board to development mode.
- Program
Binary/ai_fsbl.hex(First Stage Boot Loader). - Program
Binary/STM32N6570-DK_network_data.hex(network parameters; only required when the network changes). - Program
Binary/STM32N6570-DK_GettingStarted_InstanceSegmentation.hex(firmware application). - Set the board to boot from flash mode.
- Power cycle the board.
Note: The Binary/STM32N6570-DK_GettingStarted_InstanceSegmentation.hex firmware is built for MB1939 STM32N6570-DK REV C02 with any of the listed camera modules.
To program the board's external flash, follow these steps:
- Set the board to development mode.
- Program
Binary/ai_fsbl.hex(First Stage Boot Loader). - Program
Binary/NUCLEO-N657X0-Q_network_data.hex(network parameters; only required when the network changes). - Program
Binary/NUCLEO-N657X0-Q_GettingStarted_InstanceSegmentation.hex(firmware application). - Set the board to boot from flash mode.
- Connect a USB cable to the USB OTG port (CN8), next to the RJ45 port. Connect the other end to a USB host (PC, USB hub, etc.) for data transmission via USB/UVC.
- Power cycle the board.
- Start the camera application on the host. On Windows, search for "camera" in the Start menu.
Note: The Binary/NUCLEO-N657X0-Q_GettingStarted_InstanceSegmentation.hex firmware is built for MB1940 NUCLEO-N657X0-Q REV C01 with USB/UVC host display and any of the listed camera modules.
See How to program hex files STM32CubeProgrammer.
Ensure the STM32CubeProgrammer bin folder is in your PATH.
export DKEL="<STM32CubeProgrammer_N6 Install Folder>/bin/ExternalLoader/MX66UW1G45G_STM32N6570-DK.stldr"
# First Stage Boot Loader
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $DKEL -hardRst -w Binary/ai_fsbl.hex
# Network parameters and biases
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $DKEL -hardRst -w Binary/STM32N6570-DK_network_data.hex
# Application Firmware
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $DKEL -hardRst -w Binary/STM32N6570-DK_GettingStarted_InstanceSegmentation.hexEnsure the STM32CubeProgrammer bin folder is in your PATH.
export NUEL="<STM32CubeProgrammer_N6 Install Folder>/bin/ExternalLoader/MX25UM51245G_STM32N6570-NUCLEO.stldr"
# First Stage Boot Loader
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $NUEL -hardRst -w Binary/ai_fsbl.hex
# Network parameters and biases
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $NUEL -hardRst -w Binary/NUCLEO-N657X0-Q_network_data.hex
# Application Firmware
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $NUEL -hardRst -w Binary/NUCLEO-N657X0-Q_GettingStarted_InstanceSegmentation.hexBefore building and running the application, you must program <board_name>_network_data.hex (model weights and biases). This only needs to be done once unless you change the AI model. See Quickstart using prebuilt binaries for details.
For more information about boot modes, see Boot Overview.
Note: To select the NUCLEO-N657X0-Q display interface, use the appropriate build configuration in CubeIDE, or specify SCR_LIB_SCREEN_ITF=UVCL or SCR_LIB_SCREEN_ITF=SPI as a Makefile option (default is UVCL).
Go to the ST Ultralytics fork and download Yolo v8 seg 256x256, or click this link to download the model directly. Place the downloaded model in the Model/ directory, then run the following commands:
cd Model
./generate-n6-model_<board_name>.sh
cd ..This process will generate the Model/network.c and Model/network_data.hex files.
Set your board to development mode.
Double-click Application/<board_name>/STM32CubeIDE/.project to open the project in STM32CubeIDE. Build and run the project.
Navigate to Application/<board_name>/ and run the following commands (ensure required tools are in your PATH):
- Build the project:
make -j8
- Start a GDB server connected to the STM32 target:
ST-LINK_gdbserver -p 61234 -l 1 -d -s -cp <path-to-stm32cubeprogramer-bin-dir> -m 1 -g
- In a separate terminal, launch a GDB session to load the firmware:
$ arm-none-eabi-gdb build/Application/<board_name>/Project.elf (gdb) target remote :61234 (gdb) monitor reset (gdb) load (gdb) continue
Set your board to development mode.
Double-click Application/<board_name>/STM32CubeIDE/.project to open the project in STM32CubeIDE. Build and run the project.
Ensure all required tools are in your PATH, then build the project:
make -j8After building the application, you can add a signature to the binary file:
STM32_SigningTool_CLI -bin build/Application/<board_name>/Project.bin -nk -t ssbl -hv 2.3 -o build/Application/<board_name>/Project_sign.binProgram the signed binary at address 0x70100000.
On STM32N6570-DK:
export DKEL="<STM32CubeProgrammer_N6 Install Folder>/bin/ExternalLoader/MX66UW1G45G_STM32N6570-DK.stldr"
# Adjust build path as needed
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $DKEL -hardRst -w build/Application/<board_name>/Project_sign.bin 0x70100000On NUCLEO-N657X0-Q:
export NUEL="<STM32CubeProgrammer_N6 Install Folder>/bin/ExternalLoader/MX25UM51245G_STM32N6570-NUCLEO.stldr"
# Adjust build path as needed
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $NUEL -hardRst -w build/Application/<board_name>/Project_sign.bin 0x70100000Note: Only the application binary needs to be programmed if fsbl and network_data.hex have already been programmed.
Set your board to boot from flash mode and power cycle to boot from external flash.
- Only RGB888 format for neural network input has been tested.
- Only UINT8 format for neural network input is supported.