camera_calib_matlab

This is a camera calibration toolbox. It's partly based on Bouguet's toolbox and Zhang's camera calibration paper, but with more functionality:

• Setup is based on an input configuration file which allows for easy tweaking and modification of algorithms and allows for greater reproducibility. If you save the images, configuration file, and script, the calibration will be repeatable.
• Includes fiducial marker recognition which makes the calibration fully automatic.
• The distortion function is input as a symbolic function (via configuration file) and is therefore very easily modifiable. Two distortion functions are provided already ("heikkila97" and "wang08"); this toolbox uses symbolic differentiation to compute the updated jacobians/hessians/gradients automatically.
• Supports multi-camera calibration.
• Implements both "distortion refinement" and "frontal refinement" techniques.
• Supports multiple calibration board targets (checkers, circles, etc...) and correctly accounts for "center of ellipse" vs "projected center of circle" for circular targets depending on the type of calibration (i.e. "frontal refinement" or "distortion refinement").
• Supports custom calibration board geometries by overriding an abstract calibration board geometry class.
• Supports (optional) covariance optimization (i.e. generalized least squares) based on uncertainties computed during target localization.
• Supports calibration board going partially "out of frame" which improves robustness and allows for bigger calibration boards to be used.
• Code is organized, documented, and utilizes object oriented principles for code reuse.

Installation instructions:

1. Clone the repo:

git clone https://github.com/justinblaber/camera_calib_matlab.git

Example:

1. First, download images/config/script from here.
2. Unzip, navigate to the folder in matlab, and check the configuration script, dotvision_checker.conf:

% Checkerboard target
target              = checker
target_optimization = edges

% Checkerboard geometry
height_cb           = 50.8
width_cb            = 50.8
num_targets_height  = 16
num_targets_width   = 16
target_spacing      = 2.032
height_fp           = 42.672
width_fp            = 42.672
obj_cb_geom         = class.cb_geom.csgrid_cfp

% Optimization
calib_optimization  = distortion_refinement

% Plotting
camera_size         = 20

The targets are checkers and the target optimization is a cool "edges" refinement algorithm from Mallon07. The calibration board geometry class used is "csgrid_cfp" which is a centered square grid with centered "four point" fiducial markers.

3. Next, open the dotvision_checker.m script in matlab, which should include:

clear; clc;

% Read calibration config
calib_config = intf.load_calib_config('dotvision_checker.conf');

% Set images
path_cbs(:,1) = {
'SERIAL_19061245_DATETIME_2019-05-30-01:17:49-279539_CAM_1_FRAMEID_0_COUNTER_1.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:19:10-092852_CAM_1_FRAMEID_0_COUNTER_2.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:20:05-049316_CAM_1_FRAMEID_0_COUNTER_3.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:20:53-252540_CAM_1_FRAMEID_0_COUNTER_4.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:21:26-059003_CAM_1_FRAMEID_0_COUNTER_5.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:22:22-406834_CAM_1_FRAMEID_0_COUNTER_6.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:22:55-725990_CAM_1_FRAMEID_0_COUNTER_7.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:23:34-075146_CAM_1_FRAMEID_0_COUNTER_8.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:24:13-309275_CAM_1_FRAMEID_0_COUNTER_9.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:24:50-385473_CAM_1_FRAMEID_0_COUNTER_10.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:25:35-707981_CAM_1_FRAMEID_0_COUNTER_11.png', ...
'SERIAL_19061245_DATETIME_2019-05-30-01:26:07-053008_CAM_1_FRAMEID_0_COUNTER_12.png'
};
             
path_cbs(:,2) = {
'SERIAL_16276941_DATETIME_2019-05-30-01:17:49-279573_CAM_2_FRAMEID_0_COUNTER_1.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:19:10-092894_CAM_2_FRAMEID_0_COUNTER_2.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:20:05-049342_CAM_2_FRAMEID_0_COUNTER_3.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:20:53-252565_CAM_2_FRAMEID_0_COUNTER_4.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:21:26-059023_CAM_2_FRAMEID_0_COUNTER_5.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:22:22-406855_CAM_2_FRAMEID_0_COUNTER_6.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:22:55-726022_CAM_2_FRAMEID_0_COUNTER_7.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:23:34-075170_CAM_2_FRAMEID_0_COUNTER_8.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:24:13-309309_CAM_2_FRAMEID_0_COUNTER_9.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:24:50-385500_CAM_2_FRAMEID_0_COUNTER_10.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:25:35-708005_CAM_2_FRAMEID_0_COUNTER_11.png', ...
'SERIAL_16276941_DATETIME_2019-05-30-01:26:07-053021_CAM_2_FRAMEID_0_COUNTER_12.png', ...
};

path_cbs(:,3) = {
'SERIAL_16276942_DATETIME_2019-05-30-01:17:49-279580_CAM_3_FRAMEID_0_COUNTER_1.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:19:10-092897_CAM_3_FRAMEID_0_COUNTER_2.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:20:05-049351_CAM_3_FRAMEID_0_COUNTER_3.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:20:53-252567_CAM_3_FRAMEID_0_COUNTER_4.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:21:26-059040_CAM_3_FRAMEID_0_COUNTER_5.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:22:22-406871_CAM_3_FRAMEID_0_COUNTER_6.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:22:55-726019_CAM_3_FRAMEID_0_COUNTER_7.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:23:34-075180_CAM_3_FRAMEID_0_COUNTER_8.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:24:13-309307_CAM_3_FRAMEID_0_COUNTER_9.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:24:50-385526_CAM_3_FRAMEID_0_COUNTER_10.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:25:35-708001_CAM_3_FRAMEID_0_COUNTER_11.png', ...
'SERIAL_16276942_DATETIME_2019-05-30-01:26:07-053028_CAM_3_FRAMEID_0_COUNTER_12.png', ...
};

% Validate all calibration board images
img_cbs = intf.validate_multi_path_imgs(path_cbs);

% Display calib config
intf.plot_cb_geom(calib_config);

% Detect four points
[p_fpss, debug_fp] = intf.fp_detect(img_cbs, calib_config);

% four point gui
intf.gui_fp_detect(p_fpss, img_cbs, debug_fp, calib_config)

% Perform calibration
calib = intf.calib_fp(img_cbs, p_fpss, calib_config);
                           
% Calib gui
intf.gui_calib(calib);

Make sure to first add the library to path with something like: addpath('~/camera_calib')

After running this script, the first figure to appear should be:

This gui is useful for debugging the detection of the four fiducial markers. You can toggle through the images by pressing the left and right arrow keys.

The next figure should be:

This gui is useful for debugging the calibration. The first thing to do is to double check the calibration board geometry to make sure it's correct. Next, check the residuals to make sure they are reasonably small. I would also rotate the extrinsics to confirm the relative pose of the cameras makes sense. Lastly, you can toggle "w" and zoom into the calibration point with the largest residual.