Offset applying on Element84 products

[bcerripromethee]

Hey,

I looked at your code for the next release concerning the reading of Elements 84 items. I saw that you apply this formula for all the products : band_arr * band_mtd["scale"] + band_mtd["offset"].

Before, on Sentinel-2 Granule products, you were checking if the processing baseline was greater than 4.0. Only if True, you were applying the offset.

So I don't know if I misunderstood your code, but it seems that you always apply the offset now ? Even if the product was already BOA_OFFSET was applied ?

Let's take this product S2A_30TXQ_20220801_0_L2A, it was captured after the 25 Jan 2022 (date of processing baseline 4.0), and the BOA_OFFSET has been applied :

So you compute an offset on a product already offset corrected or am i wrong probably ?

An another product (S2A_30TXQ_20220712_0_L2A. This one was captured few days before, but still after the 25 Jan 2022, so it should be corrected. Apparently it wasn't but your code does it so perfect :

Thanks again for eoreader,
Bastien

[remi-braun]

Hey Bastien,

Thanks for this analysis, this support is still in an early stage and I'm a bit swamped by other things in my work right now.
You are probably right, I'll need to investiguate it further !

Don't hesitate to investiguate also on your side. If you come up with specification, it'll be easier for me 😉

Thanks !

[bcerripromethee]

Hey,

I dived into the problem. EarthSearch updated their S2 items products by applying the -1000 offset. So they created the tag boa_offset_applied set to True per default for all the products created after the 25th Jan 2022.
In reality, some items still appears with the tag setted as False even if the offset was applied to it. So, you should consider if the tag s2:processing_baseline is at least 4.00.

I've meditated a little about the issue, and i thought about three use cases and what it should be done for those ones. Imagine you have two images to compare :

• 10 jan 21 / 20 jan 21 : no offset to apply because the processing baseline is at least 3.07
• 20 jan 21 / 30 jan 22 : the offset should be applied to avoid a bias of 1000 between the images.
• 30 jan 22 / 10 fev 22 : no offset to apply because it was already done by earthsearch

What do you think about it ?

Bastien

One source : https://github.com/cirrus-geo/cirrus-earth-search/issues/21

[remi-braun]

So if I understand correctly:

• if PB >= 4.00 and boa_offset_applied==False: Apply offset and gain
• else: Only apply gain

[bcerripromethee]

No, i think you never have to apply the offset for a single image analysis.

You only need to do it if you have a stack of multiple image where they have different processing baseline (<4.00 and > 4.00)

[bcerripromethee]

Hey,

I dived into the problem. EarthSearch updated their S2 items products by applying the -1000 offset. So they created the tag boa_offset_applied set to True per default for all the products created after the 25th Jan 2022. In reality, some items still appears with the tag setted as False even if the offset was applied to it. So, you should consider if the tag s2:processing_baseline is at least 4.00.

I've meditated a little about the issue, and i thought about three use cases and what it should be done for those ones. Imagine you have two images to compare :

• 10 jan 21 / 20 jan 21 : no offset to apply because the processing baseline is at least 3.07
• 20 jan 21 / 30 jan 22 : the offset should be applied to avoid a bias of 1000 between the images.
• 30 jan 22 / 10 fev 22 : no offset to apply because it was already done by earthsearch

What do you think about it ?

Bastien

One source : cirrus-geo/cirrus-earth-search#21

Hey,

I dived into the problem. EarthSearch updated their S2 items products by applying the -1000 offset. So they created the tag boa_offset_applied set to True per default for all the products created after the 25th Jan 2022. In reality, some items still appears with the tag setted as False even if the offset was applied to it. So, you should consider if the tag s2:processing_baseline is at least 4.00.

I've meditated a little about the issue, and i thought about three use cases and what it should be done for those ones. Imagine you have two images to compare :

• 10 jan 21 / 20 jan 21 : no offset to apply because the processing baseline is at least 3.07
• 20 jan 21 / 30 jan 22 : the offset should be applied to avoid a bias of 1000 between the images.
• 30 jan 22 / 10 fev 22 : no offset to apply because it was already done by earthsearch

What do you think about it ?

Bastien

One source : cirrus-geo/cirrus-earth-search#21

What i wanted to say here is that the tag boa_offset_applied is not stable

[remi-braun]

Hmmm I think there is a gain, I have images scaled between 0 and 10000 (or a bit more)?
I would also divide in anycase by 10,000, isn't it ?

[bcerripromethee]

Yeah the process they do to obtain reflectance after 25th Jan 2022 is :
Reflectance = (DN - 1000) / 10000
On Element84, for post 25Jan2022 images, the offset is already applied even if the the tag boa_offset_applied is set to False. So the only operation you have to apply is the / 10000

[remi-braun]

Top be sure we are aligned: for E84 products, EOReader will do:

    def _to_reflectance(
        self,
        band_arr: xr.DataArray,
        band_path: AnyPathType,
        band: BandNames,
        **kwargs,
    ) -> xr.DataArray:
        """
        Converts band to reflectance

        Args:
            band_arr (xr.DataArray): Band array to convert
            band_path (AnyPathType): Band path
            band (BandNames): Band to read
            **kwargs: Other keywords

        Returns:
            xr.DataArray: Band in reflectance
        """
        # The offset is always applied for E84 products: https://github.com/sertit/eoreader/discussions/120#discussioncomment-7751885
        # offset = 0
        quantif_value = 10000.0

        # Compute the correct radiometry of the band
        band_arr = band_arr / quantif_value

        return band_arr.astype(np.float32)

[bcerripromethee]

Yep, i'm agree with this code!!! I hope i was right !
I checked 3 images, compared it with granules, i've ridden git issues, it must be right !

[remi-braun]

If not, you'll be blamed and sent straight to hell 😆

[bcerripromethee]

argggggggh,

i'll delete my git account asap 😆