Cannot connect and get images from Blaze over a router

[Srijal97]

Describe what you want to implement and what the issue & the steps to reproduce it are:

I initially described this in Issue #191.

We are using the Blaze 101 camera on a Boston Dynamics Spot robot, and the camera has to be connected through the robot's internal network, and not directly to the computer. Basically, the robot acts as a router and the camera and our computer are clients on the network. They are all configured to be in the same IP and subnet, and I am able to ping the camera from the computer through the robot. However, we are not able to connect to the camera both through the SDK and also the Blaze Viewer App.

I found that there exists this function AnnounceRemoteDevice() as part of Pylon::IGigETransportLayer that can be be used in this situation. The documentation mentions that this should be applicable to the Blaze cameras as well:

This applies to all Basler GigE Vision devices, including the Basler blaze, which isn't usually considered by the GigE transport layer.

However, I am not sure how to use this correctly. Here's what I have tried:

I have made changes to function PylonROS2Camera::create on line 180 in file pylon_ros2_camera.cpp to call the AnnounceRemoteDevice() method:

PylonROS2Camera* PylonROS2Camera::create(const std::string& device_user_id_to_open)
{
    try
    {
        // Before using any pylon methods, the pylon runtime must be initialized.
        Pylon::PylonInitialize();
        Pylon::CTlFactory& tl_factory = Pylon::CTlFactory::GetInstance();

        Pylon::ITransportLayer* pTL =  tl_factory.CreateTl(Pylon::BaslerGigEDeviceClass);
        Pylon::ITransportLayer* pTL_blaze =  tl_factory.CreateTl(Pylon::BaslerGenTlBlazeDeviceClass);

        Pylon::IGigETransportLayer* gTL = dynamic_cast<Pylon::IGigETransportLayer*>(pTL);

        // make sure the GigE transport layer was created
        if (gTL == NULL) {
            RCLCPP_ERROR_ONCE(LOGGER, "No GigE transport layer available.");
            return nullptr;
        }

        // create a blank device info object 
        Pylon::CDeviceInfo device_info = pTL_blaze ->CreateDeviceInfo();

        // Announce remote device connected to this IP
        const Pylon::String_t CAMERA_IP = "192.168.50.7";
        gTL->AnnounceRemoteDevice("192.168.50.7", &device_info);

        // Make sure correct device is found 
        RCLCPP_INFO_STREAM(LOGGER, "Remote device found with IP: " << device_info.GetIpAddress());
        RCLCPP_INFO_STREAM(LOGGER, "Remote device has serial: " << device_info.GetSerialNumber());

       // Then try to enumerate devices as usual
        ...

Note that I could not cast the Pylon::BaslerGenTlBlazeDeviceClass pTL_blaze object to Pylon::IGigETransportLayer* gTL, and so I had to create a separate Pylon::BaslerGigEDeviceClass pTL.
When I run this I get the following output:

[INFO] Trying to connect the camera device with the following device user id: blaze-balto
[INFO] Remote device found with IP: 192.168.50.7
[INFO] Remote device has serial: 24486655
[ERROR] No available camera device

The AnnounceRemoteDevice() method works as it can now find the camera with the correct serial number on the network. But I just can't enumerate and connect to it.

Next, I tried to directly tl_factory.createDevice() with the found Pylon::CDeviceInfo device_info by doing:

PylonROS2Camera* PylonROS2Camera::create(const std::string& device_user_id_to_open)
{
    try
    {
        // Before using any pylon methods, the pylon runtime must be initialized.
        Pylon::PylonInitialize();
        Pylon::CTlFactory& tl_factory = Pylon::CTlFactory::GetInstance();

        Pylon::ITransportLayer* pTL =  tl_factory.CreateTl(Pylon::BaslerGigEDeviceClass);
        Pylon::ITransportLayer* pTL_blaze =  tl_factory.CreateTl(Pylon::BaslerGenTlBlazeDeviceClass);

        Pylon::IGigETransportLayer* gTL = dynamic_cast<Pylon::IGigETransportLayer*>(pTL);

        // make sure the GigE transport layer was created
        if (gTL == NULL) {
            RCLCPP_ERROR_ONCE(LOGGER, "No GigE transport layer available.");
            return nullptr;
        }

        // create a blank device info object 
        Pylon::CDeviceInfo device_info = pTL_blaze ->CreateDeviceInfo();

        // Announce remote device connected to this IP
        const Pylon::String_t CAMERA_IP = "192.168.50.7";
        gTL->AnnounceRemoteDevice("192.168.50.7", &device_info);

        // Make sure correct device is found 
        RCLCPP_INFO_STREAM(LOGGER, "Remote device found with IP: " << device_info.GetIpAddress());
        RCLCPP_INFO_STREAM(LOGGER, "Remote device has serial: " << device_info.GetSerialNumber());

        RCLCPP_INFO_STREAM(LOGGER, "Trying to create camera_device object...");

        Pylon::IPylonDevice* camera_device = tl_factory.CreateDevice(device_info);

        RCLCPP_INFO_STREAM(LOGGER, "camera_device object created!");

        PylonROS2Camera* new_cam_ptr = createFromDevice(BLAZE, camera_device);

        return new_cam_ptr;  

    }
     ...

Here, the output is:

[INFO] Trying to connect the camera device with the following device user id: blaze-balto
[INFO] Remote device found with IP: 192.168.50.7
[INFO] Remote device has serial: 24486655
[INFO] Trying to create camera_device object...
[INFO] camera_device object created!
[ERROR] An exception occurred while opening the specified camera device with Device User ID: blaze-balto: 
[pylon_ros2_camera_wrapper-2] The attached pylon device type cannot be used with this type of camera class.

The camera device does get successfully created, but we are unable to createFromDevice() as BLAZE.

Finally, I tried changing the line PylonROS2Camera* new_cam_ptr = createFromDevice(BLAZE, camera_device); to PylonROS2Camera* new_cam_ptr = createFromDevice(GIGE, camera_device);, and got the following output:

[INFO] Trying to connect the camera device with the following device user id: blaze-balto
[INFO] Remote device found with IP: 192.168.50.7
[INFO] Remote device has serial: 24486655
[INFO] Trying to create camera_device object...
[INFO] camera_device object created!
[DEBUG] Pylon camera instance created with the following user id: blaze-balto
[INFO] Startup user profile set to ""
[WARN] Unsupported startup user profile "", ignoring
[WARN] No image encoding provided -> Will use 'mono8' or 'rgb8' as fallback
[ERROR] Couldn't find a fallback solution!
[ERROR] Error when trying to start grabbing. Shutting down now.
[pylon_ros2_camera_wrapper-2] Impossible to spin
[pylon_ros2_camera_wrapper-2] failed to create guard condition: the given context is not valid, either rcl_init() was not called or rcl_shutdown() was called., at ./src/rcl/guard_condition.c:67
[ERROR] [pylon_ros2_camera_wrapper-2]: process has died [pid 73266, exit code 1, cmd '/home/cdcl/cdcl_ws/install/pylon_ros2_camera_wrapper/lib/pylon_ros2_camera_wrapper/pylon_ros2_camera_wrapper --ros-args -r __node:=pylon_ros2_camera_node -r __ns:=/my_blaze --params-file /home/cdcl/cdcl_ws/install/pylon_ros2_camera_wrapper/share/pylon_ros2_camera_wrapper/config/my_blaze.yaml --params-file /tmp/launch_params_01u83z6y'].

Which makes me think that using GIGE we are able to connect to the camera, but it is not the correct way to get images from a Blaze camera.

How would I be able to use the AnnounceRemoteDevice() method to successfully connect to a Blaze camera with this node? Any thoughts or guidance would be very helpful!

Thank you for your time!

Hardware setup description

Hardware Connection:
Blaze 101 (192.168.50.7/24) <--eth--> Boston Dynamics Spot robot (192.168.50.3/24) <--eth--> HP Z2 mini G9 computer (192.168.50.5/24)

HP Z2 can ping and see the camera through the robot.

HP Z2 mini G9 Specs:

• i9 13900K CPU (x86)
• RTX A2000 GPU
• 32GB DDR5 RAM
• Ubuntu 22.04 with ROS2 Humble

Runtime information

Running on Ubuntu 22.04 with ROS2 Humble

Using:
- pylon_7.4.0.14900-deb0_amd64
- codemeter_7.40.4997.501_amd64
- pylon-supplementary-package-for-blaze-1.5.0.def07388_amd64

Camera has firmware 5.0.0 installed from file: blaze-image-fw-5.0.0-20231009085059.swu

Is your camera operational with the Basler pylon Viewer on your platform?

Yes

[SMA2016a]

you are announcing on one transport layer and trying to find the camera object on another one. This will not work.

What might work is:

Pylon::CDeviceInfo device_info = pTL_blaze ->CreateDeviceInfo();
device_info .SetIPAddress("192.168.50.7");

pass the this info object to createdevice.

[Srijal97]

Thank you for your response @SMA2016a. Sorry I only just had the time to try out your suggestion.

Based on your comment, I tried the following:

PylonROS2Camera* PylonROS2Camera::create(const std::string& device_user_id_to_open)
{
    try
    {
        // Before using any pylon methods, the pylon runtime must be initialized.
        Pylon::PylonInitialize();
        Pylon::CTlFactory& tl_factory = Pylon::CTlFactory::GetInstance();

        Pylon::ITransportLayer* pTL =  tl_factory.CreateTl(Pylon::BaslerGigEDeviceClass);
        Pylon::ITransportLayer* pTL_blaze =  tl_factory.CreateTl(Pylon::BaslerGenTlBlazeDeviceClass);

        Pylon::IGigETransportLayer* gTL = dynamic_cast<Pylon::IGigETransportLayer*>(pTL);

        // make sure the GigE transport layer was created
        if (gTL == NULL) {
            RCLCPP_ERROR_ONCE(LOGGER, "No GigE transport layer available.");
            return nullptr;
        }

        // create a blank device info object 
        Pylon::CDeviceInfo device_info = pTL_blaze ->CreateDeviceInfo();

        // Announce remote device connected to this IP
        const Pylon::String_t CAMERA_IP = "192.168.50.7";
        const Pylon::String_t SERIAL = "24486655";
        gTL->AnnounceRemoteDevice(CAMERA_IP, &device_info);

        // Make sure correct device is found 
        RCLCPP_INFO_STREAM(LOGGER, "Remote device found with IP: " << device_info.GetIpAddress());
        RCLCPP_INFO_STREAM(LOGGER, "Remote device has serial: " << device_info.GetSerialNumber());

        RCLCPP_INFO_STREAM(LOGGER, "Trying to create camera_device object...");

        // Create a new device info object using the Blaze TL
        Pylon::CDeviceInfo device_info_blaze = pTL_blaze->CreateDeviceInfo();

        device_info_blaze.SetIpAddress(CAMERA_IP);
        // device_info_blaze.SetSerialNumber(SERIAL);  // also tried setting: Serial, IP and Serial, None

        Pylon::IPylonDevice* camera_device = tl_factory.CreateDevice(device_info_blaze);

        RCLCPP_INFO_STREAM(LOGGER, "camera_device object created!");

        PylonROS2Camera* new_cam_ptr = createFromDevice(BLAZE, camera_device);

        return new_cam_ptr;  

    }
     ...

But the createFromDevice() method is not able to find the camera even though AnnounceRemoteDevice() does:

[INFO] Trying to connect the camera device with the following device user id: blaze_balto
[INFO] Remote device found with IP: 192.168.50.7
[INFO] Remote device has serial: 24486655
[INFO] Trying to create camera_device object...
[ERROR] An exception occurred while opening the specified camera device with Device User ID: blaze_balto: 
[pylon_ros2_camera_wrapper-2] No device is available or no device contains the provided device info properties
[DEBUG] Pylon camera instance created with the following user id: blaze_balto
[WARN] Failed to connect camera device with device user id: blaze_balto. Wait and retry to connect until the specified camera is available...

Did I understand your suggestion correctly?

[Srijal97]

I added some additional print statements to see what each of the device info objects actually contains.

The variable device_info contains the following values after AnnounceRemoteDevice(CAMERA_IP, &device_info) is called and the camera connected through the robot/router:

[basler.pylon.ros2.pylon_ros2_camera] [INFO] Device info contains the following values: 
[pylon_ros2_camera_wrapper-2] SerialNumber: 24486655
[pylon_ros2_camera_wrapper-2] UserDefinedName: blaze_balto
[pylon_ros2_camera_wrapper-2] ModelName: blaze-101-GEV
[pylon_ros2_camera_wrapper-2] DeviceVersion: 107796-17
[pylon_ros2_camera_wrapper-2] DeviceFactory: GigE/BaslerGigE 7.4.0.38864
[pylon_ros2_camera_wrapper-2] XMLSource: N/A
[pylon_ros2_camera_wrapper-2] InterfaceID: DefaultInterface
[pylon_ros2_camera_wrapper-2] Address: 192.168.50.7:3956
[pylon_ros2_camera_wrapper-2] IpAddress: 192.168.50.7
[pylon_ros2_camera_wrapper-2] SubnetAddress: 192.168.50.255
[pylon_ros2_camera_wrapper-2] DefaultGateway: 0.0.0.0
[pylon_ros2_camera_wrapper-2] SubnetMask: 255.255.255.0
[pylon_ros2_camera_wrapper-2] PortNr: 3956
[pylon_ros2_camera_wrapper-2] MacAddress: 0030534475FF
[pylon_ros2_camera_wrapper-2] Interface: 0.0.0.0
[pylon_ros2_camera_wrapper-2] IpConfigOptions: 7
[pylon_ros2_camera_wrapper-2] IpConfigCurrent: 5

Meanwhile, device_info_blaze contains the following values when connected directly to the computer (as it does not work through the our robot/router). These were obtained by adding additional print statements in the applyCamSpecificStartupSettings() function in pylon_ros2_camera_blaze.hpp.

[DEBUG] Pylon camera instance created with the following user id: blaze_balto
[DEBUG] Connected to camera blaze_balto (24486655)
[DEBUG] Infos about connected camera:
[DEBUG] -> ID:                Basler blaze-101 (24486655)
[DEBUG] -> Model name:        blaze-101
[DEBUG] -> Display name:      blaze_balto (24486655)
[DEBUG] -> Serial number:     24486655
[DEBUG] -> User defined name: blaze_balto
[DEBUG] -> DeviceVersion:        N/A
[DEBUG] -> DeviceFactory:        GenTL/GenTL Producer for Basler blaze-101 cameras 4.5.0.7
[DEBUG] -> XMLSource:        Device
[DEBUG] -> InterfaceID:        Basler blaze GenTL Interface Module
[DEBUG] -> Address:        N/A
[DEBUG] -> IpAddress:        N/A
[DEBUG] -> SubnetAddress:        N/A
[DEBUG] -> DefaultGateway:        N/A
[DEBUG] -> SubnetMask:        N/A
[DEBUG] -> PortNr:        N/A
[DEBUG] -> MacAddress:        N/A
[DEBUG] -> Interface:        N/A
[DEBUG] -> IpConfigOptions:        N/A
[DEBUG] -> IpConfigCurrent:        N/A

I understand that the Blaze uses a specific/exclusive Device Factory value, but what is interesting and concerning to me is that most IP, Subnet and Port specific details are not populated for Blaze. How would I ever be able to connect to Blaze through a router without these values?