BUG: Image Rectify OpenCV Error and Crash

[alexwonseokcho]

Issue description

When running the driver using the command ros2 launch pylon_ros2_camera_wrapper pylon_ros2_camera.launch.py with no calibration file linked in the config file, the driver publishes the raw images properly under pylon_ros2_camera_node/image_rect which I have confirmed using RViz. When I provide the drive the calibration file (contents below), the driver does create a topic called image_rect. However, when I subscribe to this topic from RViz, the driver node immediately crashes. The error is attached below as well. The crash appears to come from this->pinhole_model_->rectifyImage(cv_img_raw->image, this->cv_bridge_img_rect_->image);.

Any help would be greatly appreciated. Thank you.

Code example about how to reproduce the issue

camera_calibration.yaml:

image_width: 1920
image_height: 1200
camera_name: my_camera
camera_matrix:
  rows: 3
  cols: 3
  data: [484.2153 ,   0.     , 950.62755,
           0.     , 484.17352, 587.00057,
           0.     ,   0.     ,   1.     ]
distortion_model: plumb_bob
distortion_coefficients:
  rows: 1
  cols: 5
  data: [-0.008885, -0.000410, -0.001960, 0.001645, 0.000000]
rectification_matrix:
  rows: 3
  cols: 3
  data: [1., 0., 0.,
         0., 1., 0.,
         0., 0., 1.]
projection_matrix:
  rows: 3
  cols: 4
  data: [461.9892 ,   0.     , 961.20259,   0.     ,
           0.     , 476.66439, 581.9262 ,   0.     ,
           0.     ,   0.     ,   1.     ,   0.     ]

Error message

[INFO] [launch]: All log files can be found below /home/utfr-dv/.ros/log/2025-02-28-22-05-46-075944-ubuntu-31360
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [pylon_ros2_camera_wrapper-1]: process started with pid [31361]
[pylon_ros2_camera_wrapper-1] 1740798346.388597873 [basler.pylon.ros2.pylon_ros2_camera_parameter] [INFO] Autoflash: 0, line2: 1, line3: 1
[pylon_ros2_camera_wrapper-1] 1740798346.389003895 [basler.pylon.ros2.pylon_ros2_camera_parameter] [INFO] No Device User ID set -> Will connect the first available camera device
[pylon_ros2_camera_wrapper-1] 1740798348.170012285 [basler.pylon.ros2.pylon_ros2_camera] [INFO] Found camera device! Device Model: a2A1920-160ucBAS with Device User Id: 
[pylon_ros2_camera_wrapper-1] 1740798348.554412199 [basler.pylon.ros2.pylon_ros2_camera_node] [INFO] No user id for the camera has been set
[pylon_ros2_camera_wrapper-1] 1740798348.647703052 [basler.pylon.ros2.pylon_ros2_usb_camera] [INFO] Startup user profile set to "CurrentSetting"
[pylon_ros2_camera_wrapper-1] 1740798348.649812104 [basler.pylon.ros2.pylon_ros2_usb_camera] [INFO] Cam has upper exposure value limit range: [19 - 1e+07 (max possible value from cam is 1e+07)].
[pylon_ros2_camera_wrapper-1] 1740798348.649907785 [basler.pylon.ros2.pylon_ros2_usb_camera] [INFO] CurrentSetting loaded
[pylon_ros2_camera_wrapper-1] 1740798348.742675976 [basler.pylon.ros2.pylon_ros2_camera_node] [WARN] [] name not valid for camera_info_manager
[pylon_ros2_camera_wrapper-1] 1740798348.743027469 [my_camera.pylon_ros2_camera_node] [INFO] camera calibration URL: package://perception/config/camera_calibration.yaml
[pylon_ros2_camera_wrapper-1] 1740798348.744552225 [my_camera.pylon_ros2_camera_node] [WARN] [camera] does not match my_camera in file /home/utfr-dv/utfr_dv/install/perception/share/perception/config/camera_calibration.yaml
[pylon_ros2_camera_wrapper-1] 1740798348.747952557 [basler.pylon.ros2.pylon_ros2_camera_base] [WARN] Camera does not support binning (X). Will keep the current settings.
[pylon_ros2_camera_wrapper-1] 1740798348.748104911 [basler.pylon.ros2.pylon_ros2_camera_node] [INFO] Setting horizontal binning_x to 1
[pylon_ros2_camera_wrapper-1] 1740798348.748128079 [basler.pylon.ros2.pylon_ros2_camera_base] [WARN] Camera does not support binning (Y). Will keep the current settings.
[pylon_ros2_camera_wrapper-1] 1740798348.748210864 [basler.pylon.ros2.pylon_ros2_camera_node] [INFO] Setting vertical binning_y to 1
[pylon_ros2_camera_wrapper-1] 1740798348.750188522 [basler.pylon.ros2.pylon_ros2_camera_node] [INFO] Attempted to set exposure to 10000, reached: 10000
[pylon_ros2_camera_wrapper-1] 1740798348.751587581 [basler.pylon.ros2.pylon_ros2_camera_node] [INFO] Startup settings: encoding = 'rgb8', binning = [1, 1], exposure = 10000, gain = 0, gamma = 1, shutter mode = global
[pylon_ros2_camera_wrapper-1] 1740798348.751908929 [basler.pylon.ros2.pylon_ros2_camera_node] [INFO] Start image grabbing if node connects to topic with a spinning rate of: 30 Hz
[pylon_ros2_camera_wrapper-1] 1740798348.785414648 [basler.pylon.ros2.pylon_ros2_camera_node] [INFO] Camera is calibrated
[pylon_ros2_camera_wrapper-1] 1740798348.836937690
[pylon_ros2_camera_wrapper-1] Impossible to spin
[pylon_ros2_camera_wrapper-1] Cannot call rectifyImage when distortion is unknown.
[ERROR] [pylon_ros2_camera_wrapper-1]: process has died [pid 31361, exit code 1, cmd '/home/utfr-dv/utfr_dv/install/pylon_ros2_camera_wrapper/lib/pylon_ros2_camera_wrapper/pylon_ros2_camera_wrapper --ros-args -r __node:=pylon_ros2_camera_node -r __ns:=/my_camera --params-file /home/utfr-dv/utfr_dv/install/pylon_ros2_camera_wrapper/share/pylon_ros2_camera_wrapper/config/basler_config.yaml --params-file /tmp/launch_params_4fdvbsgd'].

Hardware setup description

Running on a Jetson AGX Orin, which has an ARM64 CPU with 64GB of RAM. Running on Ubuntu 22.04 ROS2 Humble. The camera in use is the ace 2 R a2A1920-160ucBAS connected via a USB 3.0 port directly plugged into the Jetson.

Runtime information

Pylon release 8.0.1.16188
pylon c++ SDK 9.0.3.215
pylon Application 3.3.0.16267
pylon Viewer 9.0.2.16267
ROS2 Humble

Is your camera operational with the Basler pylon Viewer on your platform?

Yes