(Work in progress, migrating RGB565 to YUV4:2:2: code in development)
I am just interested in computer vision at lower resolutions (32x32 or 128x128) for edge AI.
- Since the RAM size of STM32F4 Discovery board is very limited (around 100KBytes), I mainly handle 32x32 resolution images in this project.
- Although deep learning is quite a heavy task, it is possible to run deep learing on 32x32 resolution images with a limited number of classes on my PC.
STM32L4 Discovery
...................
: :
[OV7670]----->[DCMI] [UASRT]------>[Python program on my PC]
: :
...................
I captured a hundred of images for each class (rock, paper, scissors) by using the viewer, then trained CNN with the data: Rock-paper-scissors.
I made X-CUBE-AI/CubeMX to generate image classification code based on the Keras model. The code could classify "rock", "paper" and "scissors".
--- Inference ---
PAPER 1%
ROCK 26%
SCISSORS 71%
--- Inference ---
PAPER 0%
ROCK 53%
SCISSORS 46%
--- Inference ---
PAPER 54%
ROCK 13%
SCISSORS 31%
:
I honestly admit that the result above was the best case, because the view angle was too small (25 degrees is not enough for hand gesture recognition).
I exchanged a camera with another one that supports 170 degrees view angle. Since the image sensor is not big enough, I could only get around 60 degrees of view. But the inference result was significantly improved.
=> Visualization of convolution layers
Sensors used for edge AI:
- 9-axis sensor (MEMS)
- Mic sensor (MEMS)
- Infrared array sensor
- Image sensor
- Camera (OV7670 CMOS image sensor)
- AI-enabled rock-scissor-papers (infrared array sensor)
- Acoustic features (Mic sensor)
- Wireless network of infrared array sensors (infrared array sensor)
- Motion logger with PIC16F18326, MPU9255, A1324LUA-T and EEPROM (9-axis sensor)
- My original DCT Type-II implementation on CMSIS-DSP