Delay in frame grabbed using blaze

[Shedino]

Describe what you want to implement and what the issue & the steps to reproduce it are:

Using the latest version of the Humble branch with ros2 humble, and also making sure that there is no more frame available to grab by applying some modifications in the blaze grab frame implementation, there is a delay of about 300msec between the real actual scene and the blaze depth and intensity images. Is this expected (as camera image processing time) or someone has any suggestions to improve this latency? Our exposure time is well below 1 msec (about 200microsec).

Using blaze viewer it is not clear by human eye the actual delay. Images react similarly to what is viewed using rviz2.

Hardware setup description

PC with Intel core i5, x86_64 architecture
Ubuntu 22.04 LTS
16GB RAM

Blaze-101 camera with gigabit ethernet connection

Runtime information

ROS2 Humble version

Is your camera operational with the Basler pylon Viewer on your platform?

Yes

[SMA2016a]

do you see this effect also in blazeviewer?

[Shedino]

I cannot really tell. These kind of delays are difficult to see by human eye...
I can say that the statistics does not show any packet errors or any retries in the communication.

[Shedino]

We've profiled the whole code, and we cannot figure out where this delay can occour. We're not triggering the camera by software, trying to grab frames in free-run.
We tried also to start the actual grabbing in a new thread, so to avoid any delay in the computation, trying to use software triggering at the appropriate frame rate, but detaching the spin function seems to not work because of some device usage. Probably only the grabbing itself (with the subsequent process) may be detached? Otherwise the computation required does not permit to achieve the full frame rate. This can be also related with another issue opened: #237

[Shedino]

Do you have any hints on how to make the framegrabbing multithread in a correct way? I tried to detach the spin function to a separate thread, but it seems that there is a conflict while trying to initialize the camera.