Question- How is near-real time streaming achieved?

[poornimajd]

Hello,
Thank you for the great work!
I wanted to clarify the following points-
1.In the paper it is mentioned that by using a data-compressed streaming pipeline for video,you are able to transmit the video at 20fps.Does this apply to single camera or is it that all 6 cameras are able to transmit simultaneously at 20 fps?

2.Since ROS cannot transmit data over different networks,is this also one of the reasons for choosing gstreamer so that you can connect the remote driver to a network A and vehicle to network B?

Let me know your thoughts on this.
Thanks in advance.

[hofbi]

Thanks for asking.

1. First of all the 20fps are defined by the carla ros-bridge which is so far its maximum frequency. The streaming pipeline itself is able to process even higher fps. For a single camera view the ros-bridge also achieves the 20fps. For 6 camera views reaching the 20fps requires a quite powerful PC as the carla simulator has to render 6 camera view. So here the bottleneck is the carla ros-bridge. If you are only interested in the video transmission you can overcome this by recording a 6 camera view rosbag and then replay this rosbag as input to the streaming pipeline. The streaming pipeline itself is able to transmit the 6 camera views in parallel with 20fps.
2. I guess is should be also possible to transmit ros messages between 2 networks. The main reason why choosing GStreamer as video streaming framework is to make use of the video compression features. The transmission in ros is based on ros messages. With the image transport node you are even able to compress image messages. However, this is only an image compression (JPEG). On the other side, video compression does not only make use of compressing in the spatial domain (within one image), but also on the temporal domain (redundancies between 2 frames). This compression on the temporal domain does not really match with the ros message concept and as GStreamer (or alternatively ffmpeg) are frameworks for exactly this case, we decided for them.

Hopefully, I could answer all your questions. Feel free to ask if there are more unclear points.

[poornimajd]

Thanks a ton for those answers and the quick reply.

1. For 6 camera views reaching the 20fps requires a quite powerful PC as the carla simulator has to render 6 camera view.

Oh okay,may be a powerful compute or a gpu may help to render 6 streams.
The usage of Gstreamer is well justified.We have tried sending messages using ROS when on different networks,but could not really succeed at it.

I just wanted to clarify one last point.
1.The streaming experiment performed in the paper,is done when the remote driver is on network A and the cameras are on network B right? or are both remote driver and the vehicle on the same network?

2.Have you tried experimenting the telecarla with ROS2,or do you have any idea if this framework will work with ROS2,if we use the telecarla in our case?

Thank you.

[hofbi]

1. It was the same network but 2 different machines within this network
2. So far I did not use ROS2 and as all nodes are written for ROS1 they won't work with ROS2 out of the box. The carla ros bridge recently added support for both ROS1 and ROS2. This is achieved among others with the ros_compatibility package. So if you are interested in using telecarla with ROS2 you would need to adapt the code by yourself, but use the carla ros bridge as template on how to support both version. If you plan to do this, I would be glad to answer further questions you might have.

[poornimajd]

Thanks for the prompt reply.Yes we might have to use ROS2 with telecarla in the near future.

2\. So if you are interested in using telecarla with ROS2 you would need to adapt the code by yourself, but use the carla ros bridge as template on how to support both version. If you plan to do this, I would be glad to answer further questions you might have.

Got it.
Thank you

[hofbi]

Just let me know when there are some unclear point. And feel free to provide your ROS2 changes as PR.