From c6d8ecba6665c1b7fbc20c6475c206a056642e0d Mon Sep 17 00:00:00 2001 From: Oleg Alexandrov Date: Fri, 30 Aug 2024 20:55:37 -0700 Subject: [PATCH] Documentation for --reference-terrain --- NEWS.rst | 1 + docs/tools/bundle_adjust.rst | 31 ++++---- docs/tools/jitter_solve.rst | 135 ++++++++++++++++++++++++++++++++--- 3 files changed, 140 insertions(+), 27 deletions(-) diff --git a/NEWS.rst b/NEWS.rst index 56d96897a..52832adfc 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -15,6 +15,7 @@ jitter_solve (:numref:`jitter_solve`): * Can model rig constraints between sensors (:numref:`jitter_rig`). * Added an example for the Kaguya Terrain Camera (:numref:`jitter_kaguya`). * Added the option ``--camera-position-uncertainty`` (:numref:`jitter_camera`). + * Can constrain against a sparse point cloud (:numref:`jitter_ref_terrain`). * Can use GCP files. * Can read a control network from an nvm file. diff --git a/docs/tools/bundle_adjust.rst b/docs/tools/bundle_adjust.rst index 610b20ff0..490e3e976 100644 --- a/docs/tools/bundle_adjust.rst +++ b/docs/tools/bundle_adjust.rst @@ -935,10 +935,11 @@ Command-line options Choose a cost function from: Cauchy, PseudoHuber, Huber, L1, L2 --robust-threshold - Set the threshold for robust cost functions. Increasing this - makes the solver focus harder on the larger errors. - See the `Google Ceres `_ - documentation on robust cost functions. + Set the threshold for the robust reprojection error cost function. + Increasing this makes the solver focus harder on the larger errors while + becoming more sensitive to outliers. See the `Google Ceres + `_ documentation on robust cost + functions. --datum Set the datum. This will override the datum from the input images and also @@ -1229,10 +1230,6 @@ Command-line options cameras are optimized) are outside of this range. Specify as four values. ---reference-terrain-weight - How much weight to give to the cost function terms involving - the reference terrain. - --heights-from-dem Assuming the cameras have already been bundle-adjusted and aligned to a known DEM, constrain the triangulated points to be close to the DEM. See @@ -1270,8 +1267,8 @@ Command-line options again measured from planet center. --csv-srs - The PROJ or WKT string to use to interpret the entries in input CSV - files, if those files contain Easting and Northing fields. + The PROJ or WKT string for interpreting the entries in input CSV + files. --update-isis-cubes-with-csm-state Save the model state of optimized CSM cameras as part of the .cub @@ -1476,13 +1473,15 @@ Command-line options to ensure the parameters are big enough to be optimized. Can be negative. Applies to Pinhole cameras (all distortion models) and CSM (radial-tangential distortion only). Does not apply to optical bar models. - + --reference-terrain - An externally provided trustworthy 3D terrain, either as a DEM - or as a lidar file, very close (after alignment) to the stereo - result from the given images and cameras that can be used as a - reference, instead of GCP, to optimize the intrinsics of the - cameras. + An externally provided trustworthy reference terrain to use as a constraint. + It can be either a DEM or a point cloud in CSV format. It must be + well-aligned with the input cameras (:numref:`reference_terrain`). + +--reference-terrain-weight + How much weight to give to the cost function terms involving + the reference terrain. --max-num-reference-points Maximum number of (randomly picked) points from the reference diff --git a/docs/tools/jitter_solve.rst b/docs/tools/jitter_solve.rst index daceaae5b..405e21d6c 100644 --- a/docs/tools/jitter_solve.rst +++ b/docs/tools/jitter_solve.rst @@ -86,6 +86,9 @@ cameras is available. The implementation is the same as for bundle adjustment (:numref:`heights_from_dem`). +This solver can also use a sparse point cloud as a constraint. This is +an advanced option. See :numref:`jitter_ref_terrain`. + Ground control points ^^^^^^^^^^^^^^^^^^^^^ @@ -530,12 +533,12 @@ DEMs can later be hillshaded. .. figure:: ../images/jitter_dem_diff.png :name: ctx_jitter_dem_diff_error - From left to right are shown colorized absolute differences of - (a) jitter-unoptimized but aligned DEM and MOLA (b) - jitter-optimized DEM and MOLA - (c) unoptimized DEM and reference DEM (d) jitter-optimized - DEM and reference DEM. Then, (e) hillshaded optimized DEM (f) - hillshaded reference DEM . The max shade of red is 20 m difference. + From left to right are shown colorized absolute differences of (a) + jitter-unoptimized but aligned DEM and ungridded MOLA (b) jitter-optimized + DEM and ungridded MOLA (c) unoptimized DEM and gridded MOLA (d) + jitter-optimized DEM and gridded MOLA. Then, (e) hillshaded optimized DEM (f) + hillshaded gridded MOLA (which is the reference DEM). The max shade of red is + 20 m difference. It can be seen that the banded systematic error due to jitter is gone, both in the triangulation error maps and DEM differences. The produced @@ -1737,9 +1740,10 @@ images and cameras are created with ``sat_sim`` (:numref:`sat_sim_rig`). -o jitter/run The options ``--use-initial-rig-transforms``, ``--fix-rig-rotations``, -``--fix-rig-translations``, and ``--camera-position-uncertainty`` can constrain the -solution in various ways. A rig can be created by hand or generated by -``sat_sim`` to desired specifications (:numref:`sat_sim_rig`). +``--fix-rig-translations``, and ``--camera-position-uncertainty`` / +``--camera-position-weight`` can constrain the solution in various ways. A rig +can be created by hand or generated by ``sat_sim`` to desired specifications +(:numref:`sat_sim_rig`). The optimized rig will be saved in the output directory and can be inspected. @@ -1758,6 +1762,80 @@ and that the errors are small and uniform. The ``orbit_plot`` program (:numref:`orbit_plot`) can inspect the camera orientations before and after optimization. +.. _jitter_ref_terrain: + +Point cloud constraint +~~~~~~~~~~~~~~~~~~~~~~ + +In this scenario it is assumed that a point cloud is available that can be used +to constrain the jitter solution. The cloud can be in CSV format or be a DEM. +*The cloud must be well-aligned with the input cameras.* + +This workflow requires having filtered stereo disparity files (``F.tif``, +:numref:`outputfiles`) as made by ``parallel_stereo`` +(:numref:`parallel_stereo`). For the moment, *ony a single stereo run is +supported*. + +The algorithm projects a reference terrain point into one camera, propagates it +through the stereo disparity to the other camera, and takes the pixel difference +with the direct projection in the other camera (the *residual* further down). +The option ``--reference-terrain-uncertainty`` can set the uncertainty, with a +higher uncertainty resulting in less weight for this constraint. The weight is +also adjusted for the ground sample distance at each reference terrain point. + +The stereo runs should be produced either with the same images as invoked by the +jitter solver, or with mapprojected versions of those (the same order of images +need not be kept). The stereo run produces a file named ``{output prefix}-info.txt`` +that has some information which ``jitter_solve`` will use. + +It is not important that the cameras or bundle-adjust prefixes passed to the +jitter solving be the same as the ones for the stereo runs, but all the relevant +files must be available, and the inputs to stereo should not be changed after +the stereo runs are created. + +The list of stereo prefixes (:numref:`tutorial`), should be set via +``--stereo-prefix-list``. The jitter solver will peek in those runs and figure +out how they relate to the images passed in to the solver. It will undo any +alignment or mapprojection transforms as appropriate. + +This workflow does not preclude using the ``--heights-from-dem`` option or +anchor points. It assumes that no rig is present and that all cameras are linescan. + +If the images are mapprojected, *this option loads fully in memory the DEM that +was used for mapprojecting them*, for performance reasons, so it should not be +too large. Both mapprojected images must use the same DEM. The solver will not +fully load in memory the stereo disparity file (``F.tif``), but only portions as +needed. + +Example usage:: + + echo stereo_run/run > stereo_list.txt + + jitter_solve \ + --max-pairwise-matches 50000 \ + --match-files-prefix ba/run \ + --num-iterations 20 \ + --reference-terrain lidar.csv \ + --max-num-reference-points 50000 \ + --reference-terrain-uncertainty 10 \ + --csv-format 1:lon,2:lat,3:height_above_datum \ + --stereo-prefix-list stereo_list.txt \ + --num-anchor-points-per-tile 50 \ + --num-anchor-points-extra-lines 1000 \ + --anchor-weight 0.1 \ + --anchor-dem anchor_dem.tif \ + left.tif right.tif \ + left.json right.json \ + -o jitter/run + +If the reference terrain is a CSV file rather than a DEM, and it has a a custom +projection, rather than geographic coordinates, the option ``--csv-srs`` must be +set to specify the projection. Then adust ``--csv-format`` accordingly above. + +The initial and final residuals for the reference terrain points are saved to +disk in CSV format and should be examined +(:numref:`jitter_ref_terrain_residuals`). + .. _jitter_out_files: Output files @@ -1883,6 +1961,22 @@ confusing. Yet in the final (optimized) file these points have moved, so then the result makes more sense. When using the ``--tri-weight`` option the true initial triangulated points and errors are used. +.. _jitter_ref_terrain_residuals: + +Reference terrain residuals +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If ``jitter_solve`` is run with the option ``--reference-terrain`` +(:numref:`jitter_ref_terrain`), the pixel residuals for each reference terrain +point are saved to:: + + {output-prefix}-initial_residuals_ref_terrain.csv + {output-prefix}-final_residuals_ref_terrain.csv + +The format is the same as for pixel reprojection errors +(:numref:`ba_err_per_point`). These files can be inspected in ``stereo_gui`` as +well. + Image and camera lists ^^^^^^^^^^^^^^^^^^^^^^ @@ -1912,8 +2006,8 @@ Command-line options for jitter_solve options. --robust-threshold - Set the threshold for robust cost functions. Increasing this - makes the solver focus harder on the larger errors. + Set the threshold for the robust reprojection error cost function, in + pixels. Increasing this makes the solver focus harder on the larger errors. --min-matches Set the minimum number of matches between images that will be @@ -2163,6 +2257,25 @@ Command-line options for jitter_solve rig given by ``--rig-config``, instead of computing them from the poses of individual cameras. +--reference-terrain + An externally provided trustworthy reference terrain to use as a constraint. It can + be either a DEM or a point cloud in CSV format. It must be well-aligned with the + input cameras (:numref:`jitter_ref_terrain`). + +--reference-terrain-uncertainty + The uncertainty of the reference terrain, in meters. A smaller value will + result in a stronger constraint. It is suggested to not make this too small + as it may prevent convergence. + +--max-num-reference-points + Maximum number of (randomly picked) points from the reference + terrain to use. A large value will greatly slow down the solver. + +--reference-terrain-robust-threshold + The robust threshold, in pixels, for the option ``--reference-terrain``. It + is suggested to not modify this value, and adjust instead + ``--reference-terrain-uncertainty``. + --overlap-limit Limit the number of subsequent images to search for matches to the current image to this value. By default try to match all