IMP mode support.

* IMP mode support.
* Azimuth multilooking.
* Slant range to ground range conversion.
* Outputs real valued detected image.
* Fixed issues with IMP mode metadata.
* Added range spreading loss correction.
* Version 0.3.0 release notes.
* Documentation updated.

CMakeLists.txt

@@ -9,6 +9,8 @@ set(CMAKE_CXX_STANDARD_REQUIRED ON)
 set(CMAKE_POSITION_INDEPENDENT_CODE ON)
 
 set(SAR_SOURCES
+        sar/az_multilook.cuh
+        sar/az_multilook.cu
         sar/fractional_doppler_centroid.cu
         sar/fractional_doppler_centroid.cuh
         sar/iq_correction.cu
@@ -21,9 +23,13 @@ set(SAR_SOURCES
         sar/range_compression.cuh
         sar/range_doppler_algorithm.cu
         sar/range_doppler_algorithm.cuh
+        sar/range_spreading_loss.cuh
+        sar/range_spreading_loss.cu
         sar/sar_chirp.h
         sar/sar_chirp.cpp
         sar/sar_metadata.h
+        sar/srgr.cuh
+        sar/srgr.cu
 )
 
 set(SOURCES

RELEASE.md

@@ -1,3 +1,34 @@
+# Release 0.3.0
+
+## Breaking changes
+
+## Known Caveats
+* Only IMS and IMP product processing
+* Azimuth compression windowing is yet to be done - https://github.com/cgi-estonia-space/asar-focus/issues/2
+* Processing speed (Vr) and Doppler centroid changes in azimuth direction yet to be done - https://github.com/cgi-estonia-space/asar-focus/issues/2
+* Packets' ISP sensing time handling might not be 100% correct
+  * It is observed for the reference products that it calculates new ISP sensing times based on PRI
+  * Therefore products by this processor differ in sensing start/stop and first/last line times (always inside the specified sensing filter)
+  * Best knowledge/effort basis changes has been implemented - https://github.com/cgi-estonia-space/asar-focus/issues/17 and https://github.com/cgi-estonia-space/asar-focus/issues/16
+* Various metadata fields needs further work (Some were adressed during this release)
+* Final results' scaling is yet to be determined, currently it is not matching exactly the reference processor
+  * With the current experience/knowledge there is a "best guess" implemented
+
+## Major Features and Improvements
+* ERS time is corrected according to PATC files - https://github.com/cgi-estonia-space/asar-focus/issues/15
+* IMP mode support along with metadata enhancements - https://github.com/cgi-estonia-space/asar-focus/issues/4
+  * Ground detected metadata created
+  * azimuth spacing, chirp ADS corrected
+
+## Bug Fixes and Other Changes
+* `SOFTWARE_VER` in the dataset changed to `asar_gpu/M.m.p`
+
+## Thanks to our Contributors
+
+Kajal Haria, Fabiano Costantini from Telespazio UK\
+Sabrina Pinori, Marco Galli from SERCO\
+Andrea Recchia from aresys
+
 # Release 0.2.0
 
 ## Breaking changes

doc/add/add_esa.md

@@ -30,9 +30,10 @@
 
 <div align="center">
 
-| Issue/Revision | Date       | Pages | Description                        |
-|----------------|------------|-------|------------------------------------|
-| 1/A            | 08/01/2024 | All   | Initial document for version 0.2.0 |
+| Issue/Revision | Date       | Pages/Paragraphs | Description                         |
+|----------------|------------|------------------|-------------------------------------|
+| 1/A            | 08/01/2024 | All              | Initial document for version 0.2.0  |
+| 1/B            | 19/03/2024 | 1.2, 5.2         | Version 0.3.0 supports IMP and PATC |
 
 </div>
 
@@ -104,7 +105,6 @@ The project could be run on other Linux distributions as well, with possible twe
 Current state of the focusser will be able to focus ERS-1/2 and Envisat's synthetic aperture radar (SAR) instrument's 
 data for the imaging mode (IM) datasets only. It lacks some functionality in order to match the baseline PF-ASAR
 processor functionality:
-* PATC/PATN support for ERS time synchronization is missing
 * Sections of metadata is not appropriately formed (enough for products to be further processed in ESA SNAP)
 * Focussing quality is not matched (visually good, but histograms of focussed I/Q raster are not so well-formed)
 
@@ -217,7 +217,7 @@ It contains the following sub-functionality:
 * DORIS orbit file search and parsing
 * Auxiliary files' search and parsing
 * Level 0 imaging mode dataset parsing
-* Level 1 image mode single look complex dataset structure and writing
+* Level 1 image mode single look complex and ground range precision datasets structure and writing
 * GPU computation kernels to condition the measurements
 
 ## 5.3 Boost log
@@ -280,7 +280,7 @@ Example GPU profiling chart is presented below
 *Figure 6.1 - Processor profiling*
 
 Test setup:
-* RTX3060 Laptop GPU<
+* RTX3060 Laptop GPU
 * AMD Ryzen 7 5800H CPU
 * Fast NVMe SSD
 

doc/sum/sum.md

@@ -121,7 +121,7 @@ asar_focus/0.2.0
                         verbose|debug|info|warning|error
 ```
 
-As of release version 0.2 only single `type` is supported which is `IMS`. Any other value would result in termination
+As of release version 0.3 `type` supports `IMS` or `IMP`. Any other value would result in termination
 of processing with accompanying message on console.
 
 ### 3.1.1 Hidden CLI features
@@ -163,16 +163,17 @@ DOR_VOR_AXVF-P20120424_125300_20040228_215528_20040301_002328
 
 ## 3.2 Input datasets
 
-Single level 0 **envisat format** ERS-1/2 and Envisat mission dataset is required. As of version 0.2 only imaging mode
+Single level 0 **envisat format** ERS-1/2 and Envisat mission dataset is required. As of version 0.3 only imaging mode
 datasets are supported. The specification is given in document `PO-RS-MDA-GS-2009 4/C`.
 
 ## 3.3 Auxiliary files
 
-As of version 0.2 the following auxiliary files are used:
+As of version 0.3 the following auxiliary files are used:
 * DORIS orbit files in envisat format, see [DOR_VOR_AX](https://earth.esa.int/eogateway/catalog/envisat-doris-precise-orbit-state-vectors-dor-vor_ax-)
 * Processor configuration file (ASA_CON_AX/ER_CON_AX)
 * Instrument characterization file (AUX_INS_AX/ER_INS_AX)
 * External calibration data (ASA_XCA_AX/ER_XCA)
+* PATC for ERS onboard time corrections
 
 For auxiliary file access and documentation please visit [ERS aux](https://earth.esa.int/eogateway/instruments/sar-ers/auxiliary-data)
 and [ENVISAT aux](https://earth.esa.int/eogateway/instruments/asar/auxiliary-data) pages at ESA.

doc/sum/sum_esa.md

@@ -30,9 +30,10 @@
 
 <div align="center">
 
-| Issue/Revision | Date       | Pages | Description                        |
-|----------------|------------|-------|------------------------------------|
-| 1/A            | 08/01/2024 | All   | Initial document for version 0.2.0 |
+| Issue/Revision | Date       | Pages/Paragraphs | Description                         |
+|----------------|------------|------------------|-------------------------------------|
+| 1/A            | 08/01/2024 | All              | Initial document for version 0.2.0  |
+| 1/B            | 19/03/2024 | 3.1, 3.2, 3.3    | Version 0.3.0 supports IMP and PATC |
 
 </div>
 
@@ -74,6 +75,7 @@
 
 <div style="page-break-after: always;"></div>
 
+
 # 1 Introduction
 
 Following information describes the usage of the ERS and Envisat focusser utilizing GPU. It is implemented in C/C++/CUDA
@@ -110,8 +112,8 @@ Based on target platform one can conveniently choose intended scripts. Subchapte
 ### 2.1.1 Enabling containers
 
 There are many options to choose from which are all summarized on Nvidia's official CDI page at https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/index.html.
-The scenario could vary from the type of operating system to orchestration framework used. This means there could be 
-different engines used - docker, containerd, CRI-O and podman - which all need customized setup. 
+The scenario could vary from the type of operating system to orchestration framework used. This means there could be
+different engines used - docker, containerd, CRI-O and podman - which all need customized setup.
 
 Currently utilized and tested approaches within this project are represented below:
 * [podman on Ootpa](https://github.com/cgi-estonia-space/ALUs-platform/blob/main/rhel/8/ootpa/install_container_toolkit.sh)
@@ -162,7 +164,7 @@ The developed processor called `asar_focus` is a single executable GPU ERS and E
 user interface or any other hidden requirements are present. Below will be described the arguments, formats and input
 datasets/auxiliary files. All the produced files will be written to the location specified by the `output` argument.
 
-If the program has successfully processed the product the return code of the binary execution will be **0**. Any other 
+If the program has successfully processed the product the return code of the binary execution will be **0**. Any other
 exit code means error during processing which SHALL be accompanied by the message on the console. Unsuccessful
 processing could occur due to the invalid input(s), programming error or host machine's/OS errors/congestion.
 
@@ -197,14 +199,14 @@ asar_focus/0.2.0
                         verbose|debug|info|warning|error
 ```
 
-As of release version 0.2 only single `type` is supported which is `IMS`. Any other value would result in termination
+As of release version 0.3 `type` supports `IMS` or `IMP`. Any other value would result in termination
 of processing with accompanying message on console.
 
 ### 3.1.1 Hidden CLI features
 
 For debugging and research possibilities couple of 'unofficial' command line arguments are present which are given below.
 * `--plot` - create HTML files consisting graphs of the following processing properties - processing velocity,
-chirp signal, doppler centroid and scene geolocation properties on earth
+  chirp signal, doppler centroid and scene geolocation properties on earth
 * `--intensity` - create GeoTIFF files of the raw measurements, range compressed results and fully compressed intensity
 
 All the created files would be saved in the same directory as the final product given by the `output` argument. They are
@@ -239,23 +241,24 @@ DOR_VOR_AXVF-P20120424_125300_20040228_215528_20040301_002328
 
 ## 3.2 Input datasets
 
-Single level 0 **envisat format** ERS-1/2 and Envisat mission dataset is required. As of version 0.2 only imaging mode 
+Single level 0 **envisat format** ERS-1/2 and Envisat mission dataset is required. As of version 0.3 only imaging mode
 datasets are supported. The specification is given in document `PO-RS-MDA-GS-2009 4/C`.
 
 ## 3.3 Auxiliary files
 
-As of version 0.2 the following auxiliary files are used:
+As of version 0.3 the following auxiliary files are used:
 * DORIS orbit files in envisat format, see [DOR_VOR_AX](https://earth.esa.int/eogateway/catalog/envisat-doris-precise-orbit-state-vectors-dor-vor_ax-)
 * Processor configuration file (ASA_CON_AX/ER_CON_AX)
 * Instrument characterization file (AUX_INS_AX/ER_INS_AX)
 * External calibration data (ASA_XCA_AX/ER_XCA)
+* PATC for ERS onboard time corrections
 
 For auxiliary file access and documentation please visit [ERS aux](https://earth.esa.int/eogateway/instruments/sar-ers/auxiliary-data)
 and [ENVISAT aux](https://earth.esa.int/eogateway/instruments/asar/auxiliary-data) pages at ESA.
 
 # 4 Requirements
 
-Below are specified minimum requirements for hardware. It is based on the resources needed to focus 16 seconds of 
+Below are specified minimum requirements for hardware. It is based on the resources needed to focus 16 seconds of
 level 0 imaging mode dataset. Resource like CPU is out of scope, because all modern CPUs are suitable. Disk storage
 resource is dismal, since only level 1 dataset storage is required also no intermediate files are stored
 (except for debugging features discussed in [3.1.1 Hidden CLI features](#311-hidden-cli-features)).

envisat_format/include/asar_constants.h

@@ -27,6 +27,18 @@ enum ProductTypes { SAR_IM0, ASA_IM0, SAR_IMS, SAR_IMP, ASA_IMS, ASA_IMP, UNIDEN
 
 enum Instrument { SAR, ASAR };
 
+constexpr double REFERENCE_RANGE = 800000;
+
+inline bool IsSLCProduct(ProductTypes product)
+{
+    return product == SAR_IMS || product == ASA_IMS;
+}
+
+inline bool IsDetectedProduct(ProductTypes product)
+{
+    return product == SAR_IMP || product == ASA_IMP;
+}
+
 constexpr std::string_view PRODUCT_NAME_SAR_IM0{"SAR_IM__0"};
 constexpr std::string_view PRODUCT_NAME_ASA_IM0{"ASA_IM__0"};
 

envisat_format/include/envisat_format_kernels.h

@@ -15,7 +15,10 @@
 namespace alus::asar::envformat {
 
 // dest_buffer must be on device and have enough capacity to accommodate original image (complex float) without padding.
-void ConditionResults(DevicePaddedImage& img, char* dest_space, size_t record_header_size, float calibration_constant);
+void ConditionResultsSLC(DevicePaddedImage& img, char* dest_space, size_t record_header_size,
+                         float calibration_constant);
+void ConditionResultsDetected(const float* d_img, char* d_dest_space, int range_size, int azimuth_size,
+                              size_t record_header_size, float calibration_constant);
 
 void ConvertErsImSamplesToComplex(const uint8_t* samples, size_t sample_count_per_range, size_t packets,
                                   const uint16_t* swst_codes, uint16_t min_swst, cufftComplex* gpu_buffer,

envisat_format/include/envisat_lvl0_parser.h

@@ -31,6 +31,7 @@
 struct ASARMetadata {
     std::string lvl0_file_name;
     std::string product_name;
+    alus::asar::specification::ProductTypes target_product_type;
     std::string instrument_file;
     std::string configuration_file;
     std::string external_calibration_file;

envisat_format/include/envisat_lvl1_ads.h

@@ -667,4 +667,26 @@ struct __attribute__((packed)) GeoLocationADSR {
     }
 };
 
-static_assert(sizeof(GeoLocationADSR) == 521);
+static_assert(sizeof(GeoLocationADSR) == 521);
+
+
+struct __attribute__((packed)) SrGrADSR {
+    mjd zero_doppler_time;
+    uc attach_flag;
+    fl slant_range_time;
+    fl ground_range_origin;
+    fl srgr_coeffs[5]; //NB! weird naming, named SRGR... but polynomial is for converting GR to SR
+    uc spare_2[14];
+
+    void SetDefaults() { memset(this, 0, sizeof(*this)); }
+
+    void BSwap() {
+        zero_doppler_time = bswap(zero_doppler_time);
+
+        slant_range_time = bswap(slant_range_time);
+        ground_range_origin = bswap(ground_range_origin);
+        BSWAP_ARR(srgr_coeffs);
+    }
+};
+
+static_assert(sizeof(SrGrADSR) == 55);

envisat_format/include/envisat_lvl1_writer.h

@@ -18,14 +18,15 @@
 
 #include "envisat_lvl0_parser.h"
 
-struct __attribute__((packed)) EnvisatIMS {
+struct __attribute__((packed)) EnvisatSubFiles {
     Lvl1MPH mph;
     Lvl1SPH sph;
     SummaryQualityADS summary_quality;
     MainProcessingParametersADS main_processing_params;
     DopplerCentroidParameters dop_centroid_coeffs;
     ChirpParameters chirp_params;
     GeoLocationADSR geolocation_grid[12];
+    SrGrADSR srgr_params;
 };
 
 struct MDS {
@@ -39,5 +40,5 @@ struct MDS {
     MDS& operator=(const MDS&) = delete;
 };
 
-void ConstructIMS(EnvisatIMS& ims, const SARMetadata& sar_meta, const ASARMetadata& asar_meta, const MDS& mds,
+std::vector<uint8_t> ConstructEnvisatFileHeader(EnvisatSubFiles& ims, const SARMetadata& sar_meta, const ASARMetadata& asar_meta, const MDS& mds,
                   std::string_view software_ver);

envisat_format/src/asar_constants.cc

@@ -72,12 +72,18 @@ ProductTypes TryDetermineTargetProductFrom(ProductTypes in_product, std::string_
         if (boost::iequals(user_defined_target_type, "IMS")) {
             return ProductTypes::SAR_IMS;
         }
+        if (boost::iequals(user_defined_target_type, "IMP")) {
+            return ProductTypes::SAR_IMP;
+        }
     }
 
     if (in_product == ProductTypes::ASA_IM0) {
         if (boost::iequals(user_defined_target_type, "IMS")) {
             return ProductTypes::ASA_IMS;
         }
+        if (boost::iequals(user_defined_target_type, "IMP")) {
+            return ProductTypes::ASA_IMP;
+        }
     }
 
     throw std::runtime_error("Focussing '" + std::string(user_defined_target_type) +

envisat_format/src/envisat_format_kernels.cu

@@ -18,7 +18,7 @@
 
 namespace alus::asar::envformat {
 
-__global__ void CalibrateClampToBigEndian(cufftComplex* data, int x_size_stride, int x_size, int y_size,
+__global__ void CalibrateClampToBigEndian(const cufftComplex* data, int x_size_stride, int x_size, int y_size,
                                           char* dest_space, int record_header_size, float calibration_constant) {
     const int x = threadIdx.x + blockIdx.x * blockDim.x;
     const int y = threadIdx.y + blockIdx.y * blockDim.y;
@@ -54,7 +54,37 @@ __global__ void CalibrateClampToBigEndian(cufftComplex* data, int x_size_stride,
     }
 }
 
-void ConditionResults(DevicePaddedImage& img, char* dest_space, size_t record_header_size, float calibration_constant) {
+__global__ void CalibrateClampToBigEndianDetected(const float* data, int x_size, int y_size, char* dest_space,
+                                                  int record_header_size, float calibration_constant) {
+    const int x = threadIdx.x + blockIdx.x * blockDim.x;
+    const int y = threadIdx.y + blockIdx.y * blockDim.y;
+
+    const int src_idx = (y * x_size) + x;
+    const int mds_x_size = x_size * sizeof(uint16_t) + record_header_size;
+
+    if (x < x_size && y < y_size) {
+        auto pix = data[src_idx];
+        float val = pix * calibration_constant;
+        val = 10 * log10f(val);  // is IMP log scaled? S1 GRD is? TODO investigate
+        const uint16_t result = static_cast<uint16_t>(std::min<float>(val, UINT16_MAX));
+
+        // |HEADER...|packet data....................................|
+        // |HEADER...|packet data....................................|
+        // ...
+        const int dest_idx = (y * mds_x_size) + record_header_size + (x * sizeof(uint16_t));
+        if (x == 0) {
+            for (int i{record_header_size}; i >= 0; i--) {
+                dest_space[dest_idx - i] = 0x00;
+            }
+        }
+
+        dest_space[dest_idx] = static_cast<char>(result & 0x00FF);
+        dest_space[dest_idx + 1] = static_cast<char>((result >> 8) & 0x00FF);
+    }
+}
+
+void ConditionResultsSLC(DevicePaddedImage& img, char* dest_space, size_t record_header_size,
+                         float calibration_constant) {
     const auto x_size_stride = img.XStride();  // Need to count for FFT padding when rows are concerned
     const auto x_size = img.XSize();
     const auto y_size = img.YSize();  // No need to calculate on Y padded FFT data
@@ -66,6 +96,16 @@ void ConditionResults(DevicePaddedImage& img, char* dest_space, size_t record_he
     CHECK_CUDA_ERR(cudaGetLastError());
 }
 
+void ConditionResultsDetected(const float* d_img, char* d_dest_space, int range_size, int azimuth_size,
+                              size_t record_header_size, float calibration_constant) {
+    dim3 block_sz(16, 16);
+    dim3 grid_sz((range_size + 15) / 16, (azimuth_size + 15) / 16);
+    CalibrateClampToBigEndianDetected<<<grid_sz, block_sz>>>(d_img, range_size, azimuth_size, d_dest_space,
+                                                             record_header_size, calibration_constant);
+    CHECK_CUDA_ERR(cudaDeviceSynchronize());
+    CHECK_CUDA_ERR(cudaGetLastError());
+}
+
 __global__ void ConvertErsImSamplesToComplexKernel(const uint8_t* samples, int samples_range_size_bytes, int packets,
                                                    const uint16_t* swst_codes, uint16_t min_swst,
                                                    cufftComplex* target_buf, int target_range_width) {