Fmask4 runs indefinitely with some Sentinel2 scenes.

[sharkinsspatial]

Fmask4 appears to run indefinitely with some Sentinel 2 scenes. The initial processing start is logged

------------------------------------------
Setting up environment variables
---
LD_LIBRARY_PATH is .:/usr/local/MATLAB/v95/runtime/glnxa64:/usr/local/MATLAB/v95/bin/glnxa64:/usr/local/MATLAB/v95/sys/os/glnxa64:/usr/local/MATLAB/v95/sys/opengl/lib/glnxa64
Cloud/cloud shadow/snow dilated by 3/3/0 pixels.
Cloud probability threshold of 20.00%.
Load or calculate TOA reflectances, Detect potential clouds, cloud shadows, snow, and water.
Match cloud shadows with clouds.

But the process continues to run (we terminate the container process after 90 minutes). Here is a list of problem scenes we have encountered.

S2B_MSIL1C_20200224T214859_N0209_R043_T02KMG_20200224T230359
S2B_MSIL1C_20200224T214859_N0209_R043_T02LNL_20200224T230359
S2B_MSIL1C_20200224T214859_N0209_R043_T02LMJ_20200224T230359
S2B_MSIL1C_20200224T214859_N0209_R043_T02LMK_20200224T230359
S2B_MSIL1C_20200224T214859_N0209_R043_T02LMH_20200224T230359

I'll track these failures during testing and continue to add failing scenes here in the comments for tracking purposes.

[j08lue]

Thanks for raising this, we have had that issue a lot of times. Always in winter (low Sun angle) when potential shadows are long and on scenes with many cloud objects that need to be traced.

I can see the code has a bunch of checks in place to avoid too much matching, but those obviously need improvement (2 min runtime?!). Anyone into Matlab? 😄

Fmask/MatchCloudShadow.m

Line 68 in db9f01f

max_match_num =1000000; %more than 1 million pixels will result in ~2 mins runtime.

[qsly09]

This usually occurs over water, where the cloud shadow matching process will be very time-consuming. The new version 4.1 does not mask out cloud shadows over water at default. Thank you.

[j08lue]

The scenes you named, @sharkinsspatial, had loads of water, but certainly no low Sun angles... So perhaps your issue is solved.

[sharkinsspatial]

Thanks @qsly09. @j08lue We are in the process of doing large scale testing with version 4.1 right now so hopefully we'll have some better metrics on runtimes for different water content soon.