diff --git a/05_zarr_tools/57_rstac_gdalcubes.qmd b/05_zarr_tools/57_rstac_gdalcubes.qmd index 4bf068b..84eebe6 100644 --- a/05_zarr_tools/57_rstac_gdalcubes.qmd +++ b/05_zarr_tools/57_rstac_gdalcubes.qmd @@ -4,6 +4,13 @@ format: html self-contained: true --- + + + +
+ **By:** *[@DaChro](https://github.com/DaChro)* ### Introduction diff --git a/06_eopf_zarr_in_action/67_reservoir_surface_monitoring.ipynb b/06_eopf_zarr_in_action/67_reservoir_surface_monitoring.ipynb index 6e4c3ca..79f6ec5 100644 --- a/06_eopf_zarr_in_action/67_reservoir_surface_monitoring.ipynb +++ b/06_eopf_zarr_in_action/67_reservoir_surface_monitoring.ipynb @@ -26,7 +26,7 @@ } }, "source": [ - "\n", + "\n", " \n", diff --git a/06_eopf_zarr_in_action/68_vegetation_anomalies.ipynb b/06_eopf_zarr_in_action/68_vegetation_anomalies.ipynb index df24846..d245710 100644 --- a/06_eopf_zarr_in_action/68_vegetation_anomalies.ipynb +++ b/06_eopf_zarr_in_action/68_vegetation_anomalies.ipynb @@ -12,7 +12,7 @@ "---\n", "title: \"Analyzing Forest Vegetation Anomalies Using Sentinel-2 Zarr Data Cubes\" ### Add here the title of the notebook as displayed on the left-side menu\n", "execute:\n", - " enabled: false\n", + " enabled: true\n", "format: html\n", "---" ] @@ -26,7 +26,7 @@ } }, "source": [ - "\n", + "\n", " \n", diff --git a/06_eopf_zarr_in_action/69_coastal_water_dynamics_s1.ipynb b/06_eopf_zarr_in_action/69_coastal_water_dynamics_s1.ipynb index 2859941..a92fa1e 100644 --- a/06_eopf_zarr_in_action/69_coastal_water_dynamics_s1.ipynb +++ b/06_eopf_zarr_in_action/69_coastal_water_dynamics_s1.ipynb @@ -22,7 +22,11 @@ "id": "1", "metadata": {}, "source": [ - "**By:** *[@WalidGharianiEAGLE](https://github.com/WalidGharianiEAGLE)*" + "\n", + " \n", + "" ] }, { @@ -30,13 +34,21 @@ "id": "2", "metadata": {}, "source": [ - "### Introduction" + "**By:** *[@WalidGharianiEAGLE](https://github.com/WalidGharianiEAGLE)*" ] }, { "cell_type": "markdown", "id": "3", "metadata": {}, + "source": [ + "### Introduction" + ] + }, + { + "cell_type": "markdown", + "id": "4", + "metadata": {}, "source": [ "Water is a vital part of Earth ecosystems and life, supporting biodiversity, and sustaining human livelihoods. Monitoring surface water dynamics (occurence, frequency and change) is important for managing resources, and mitigating natural hazards such as floods and droughts. Wetlands in particular are unique ecosystems under the influence of precipitaion, hydrological processes and coastal dynamics, which contribute in shaping the habitats, and species diversity. Within this context, the [Keta](https://rsis.ramsar.org/ris/567) and [Songor](https://rsis.ramsar.org/ris/566) [Ramsar](https://www.ramsar.org/) Sites in southeastern Ghana present a dynamic coastal wetland system where water levels fluctuate due to rainfall, tidal influence, and lagoon-river interactions. \n", "\n", @@ -49,7 +61,7 @@ }, { "cell_type": "markdown", - "id": "4", + "id": "5", "metadata": {}, "source": [ "
" @@ -57,7 +69,7 @@ }, { "cell_type": "markdown", - "id": "5", + "id": "6", "metadata": {}, "source": [ "### What we will learn" @@ -65,7 +77,7 @@ }, { "cell_type": "markdown", - "id": "6", + "id": "7", "metadata": {}, "source": [ "- 🛰️ Accessing Sentinel-1 Ground Range Detected (GRD) `.zarr` data via the EODC STAC API using `pystac-client`.\n", @@ -76,7 +88,7 @@ }, { "cell_type": "markdown", - "id": "7", + "id": "8", "metadata": {}, "source": [ "#### Import libraries" @@ -85,7 +97,7 @@ { "cell_type": "code", "execution_count": null, - "id": "8", + "id": "9", "metadata": {}, "outputs": [], "source": [ @@ -108,7 +120,7 @@ }, { "cell_type": "markdown", - "id": "9", + "id": "10", "metadata": {}, "source": [ "#### Helper functions" @@ -116,7 +128,7 @@ }, { "cell_type": "markdown", - "id": "10", + "id": "11", "metadata": {}, "source": [ "Before starting the analysis workflow, we import several helper functions from **`zarr_s1_utils.py`** that implement key processing steps for Sentinel-1." @@ -125,7 +137,7 @@ { "cell_type": "code", "execution_count": null, - "id": "11", + "id": "12", "metadata": {}, "outputs": [], "source": [ @@ -141,7 +153,7 @@ { "cell_type": "code", "execution_count": null, - "id": "12", + "id": "13", "metadata": {}, "outputs": [], "source": [ @@ -150,7 +162,7 @@ }, { "cell_type": "markdown", - "id": "13", + "id": "14", "metadata": {}, "source": [ "
" @@ -158,7 +170,7 @@ }, { "cell_type": "markdown", - "id": "14", + "id": "15", "metadata": {}, "source": [ "## Data search" @@ -166,7 +178,7 @@ }, { "cell_type": "markdown", - "id": "15", + "id": "16", "metadata": {}, "source": [ "As we are interested into processing data that covers [Keta](https://rsis.ramsar.org/ris/567) and [Songor](https://rsis.ramsar.org/ris/566) [Ramsar](https://www.ramsar.org/) Sites in southeastern Ghana we define our interest parameters for filering the Sentinel-1 GRD data using `pystac-client` in the EOPF STAC Catalog." @@ -175,7 +187,7 @@ { "cell_type": "code", "execution_count": null, - "id": "16", + "id": "17", "metadata": {}, "outputs": [], "source": [ @@ -188,7 +200,7 @@ { "cell_type": "code", "execution_count": null, - "id": "17", + "id": "18", "metadata": {}, "outputs": [], "source": [ @@ -203,7 +215,7 @@ }, { "cell_type": "markdown", - "id": "18", + "id": "19", "metadata": {}, "source": [ "To have an overview of the retrieved scenes, we can inspect the first item" @@ -212,7 +224,7 @@ { "cell_type": "code", "execution_count": null, - "id": "19", + "id": "20", "metadata": {}, "outputs": [], "source": [ @@ -222,7 +234,7 @@ }, { "cell_type": "markdown", - "id": "20", + "id": "21", "metadata": {}, "source": [ "### Data exploration" @@ -230,7 +242,7 @@ }, { "cell_type": "markdown", - "id": "21", + "id": "22", "metadata": {}, "source": [ "To have an overview of Sentinel-1's `.zarr` product, we can navigate its hierarchical structure, and extract the data, geolocation conditions, and calibration metadata for the polarization we are interested in." @@ -239,7 +251,7 @@ { "cell_type": "code", "execution_count": null, - "id": "22", + "id": "23", "metadata": {}, "outputs": [], "source": [ @@ -255,7 +267,7 @@ { "cell_type": "code", "execution_count": null, - "id": "23", + "id": "24", "metadata": {}, "outputs": [], "source": [ @@ -267,7 +279,7 @@ { "cell_type": "code", "execution_count": null, - "id": "24", + "id": "25", "metadata": {}, "outputs": [], "source": [ @@ -277,7 +289,7 @@ { "cell_type": "code", "execution_count": null, - "id": "25", + "id": "26", "metadata": {}, "outputs": [], "source": [ @@ -287,7 +299,7 @@ { "cell_type": "code", "execution_count": null, - "id": "26", + "id": "27", "metadata": {}, "outputs": [], "source": [ @@ -296,7 +308,7 @@ }, { "cell_type": "markdown", - "id": "27", + "id": "28", "metadata": {}, "source": [ "## Preprocessing" @@ -304,7 +316,7 @@ }, { "cell_type": "markdown", - "id": "28", + "id": "29", "metadata": {}, "source": [ "### Spatial subset" @@ -312,7 +324,7 @@ }, { "cell_type": "markdown", - "id": "29", + "id": "30", "metadata": {}, "source": [ "To focus our analysis over the chosen AOI, we can efficiently crop our dataset using a spatial subset. The function **`subset()`** determine the slices in azimuth_time and ground_range that cover the AOI, and then extract and mask the corresponding portion of the GRD dataset.\n", @@ -328,7 +340,7 @@ { "cell_type": "code", "execution_count": null, - "id": "30", + "id": "31", "metadata": {}, "outputs": [], "source": [ @@ -338,7 +350,7 @@ { "cell_type": "code", "execution_count": null, - "id": "31", + "id": "32", "metadata": {}, "outputs": [], "source": [ @@ -348,7 +360,7 @@ { "cell_type": "code", "execution_count": null, - "id": "32", + "id": "33", "metadata": {}, "outputs": [], "source": [ @@ -358,7 +370,7 @@ }, { "cell_type": "markdown", - "id": "33", + "id": "34", "metadata": {}, "source": [ "### Radiometric calibration" @@ -366,7 +378,7 @@ }, { "cell_type": "markdown", - "id": "34", + "id": "35", "metadata": {}, "source": [ "In order to get a meanigful physical properties of features in the SAR scene that could be used for quantitative analysis, we need to apply a radiometric calibration on the backscatter values. This step converts the backscatter into a calibrated normalized radar cross section, correcting for incidence angle and sensor characteristics ensuring SAR images from different dates or viewing geometries are directly comparable. \n", @@ -382,7 +394,7 @@ }, { "cell_type": "markdown", - "id": "35", + "id": "36", "metadata": {}, "source": [ "To have a look into the available data vars within the calibration dataset that could be used for the radiometric calibration" @@ -391,7 +403,7 @@ { "cell_type": "code", "execution_count": null, - "id": "36", + "id": "37", "metadata": {}, "outputs": [], "source": [ @@ -400,7 +412,7 @@ }, { "cell_type": "markdown", - "id": "37", + "id": "38", "metadata": {}, "source": [ "We apply the calibration, obtaining:" @@ -409,7 +421,7 @@ { "cell_type": "code", "execution_count": null, - "id": "38", + "id": "39", "metadata": {}, "outputs": [], "source": [ @@ -421,7 +433,7 @@ { "cell_type": "code", "execution_count": null, - "id": "39", + "id": "40", "metadata": {}, "outputs": [], "source": [ @@ -431,7 +443,7 @@ { "cell_type": "code", "execution_count": null, - "id": "40", + "id": "41", "metadata": {}, "outputs": [], "source": [ @@ -440,7 +452,7 @@ }, { "cell_type": "markdown", - "id": "41", + "id": "42", "metadata": {}, "source": [ "### Speckel filtering" @@ -448,7 +460,7 @@ }, { "cell_type": "markdown", - "id": "42", + "id": "43", "metadata": {}, "source": [ "Raw SAR imagery is characterized by \"grainy\" or \"salt and pepper\" effect caused by random constructive and destructive interference, known as **speckle**. In order to reduce this effect and noise, we apply the spatial **Lee Filter** ([Lee et al., 2009](https://doi.org/10.1080/02757259409532206)) that averages the pixel values while preserving edges.\n", @@ -462,7 +474,7 @@ { "cell_type": "code", "execution_count": null, - "id": "43", + "id": "44", "metadata": {}, "outputs": [], "source": [ @@ -472,7 +484,7 @@ { "cell_type": "code", "execution_count": null, - "id": "44", + "id": "45", "metadata": {}, "outputs": [], "source": [ @@ -481,7 +493,7 @@ }, { "cell_type": "markdown", - "id": "45", + "id": "46", "metadata": {}, "source": [ "Obtaining a cleaner image" @@ -490,7 +502,7 @@ { "cell_type": "code", "execution_count": null, - "id": "46", + "id": "47", "metadata": {}, "outputs": [], "source": [ @@ -501,7 +513,7 @@ }, { "cell_type": "markdown", - "id": "47", + "id": "48", "metadata": {}, "source": [ "### Georefrecing and regredding" @@ -509,7 +521,7 @@ }, { "cell_type": "markdown", - "id": "48", + "id": "49", "metadata": {}, "source": [ "Sentinel-1 GRD Zarr comes in irregular image geometry that does not align with common geographic grids. In order to conduct spatial analysis, and scenes comparison, and mapping, we need to georefrence and resample the data onto a regular grid.
\n", @@ -525,7 +537,7 @@ { "cell_type": "code", "execution_count": null, - "id": "49", + "id": "50", "metadata": {}, "outputs": [], "source": [ @@ -544,7 +556,7 @@ { "cell_type": "code", "execution_count": null, - "id": "50", + "id": "51", "metadata": {}, "outputs": [], "source": [ @@ -554,7 +566,7 @@ { "cell_type": "code", "execution_count": null, - "id": "51", + "id": "52", "metadata": {}, "outputs": [], "source": [ @@ -564,7 +576,7 @@ { "cell_type": "code", "execution_count": null, - "id": "52", + "id": "53", "metadata": {}, "outputs": [], "source": [ @@ -575,7 +587,7 @@ }, { "cell_type": "markdown", - "id": "53", + "id": "54", "metadata": {}, "source": [ "### Convert backscatter to dB" @@ -583,7 +595,7 @@ }, { "cell_type": "markdown", - "id": "54", + "id": "55", "metadata": {}, "source": [ "We convert the regridded `sigma_0` backscatter intensity from Linear scale to decibels (dB) using a logarithmic transformation. This enhances contrast and simplifies statistical analysis and interpretation of the image. It is considered a standard approach for representing SAR intensity." @@ -592,7 +604,7 @@ { "cell_type": "code", "execution_count": null, - "id": "55", + "id": "56", "metadata": {}, "outputs": [], "source": [ @@ -602,7 +614,7 @@ { "cell_type": "code", "execution_count": null, - "id": "56", + "id": "57", "metadata": {}, "outputs": [], "source": [ @@ -612,7 +624,7 @@ { "cell_type": "code", "execution_count": null, - "id": "57", + "id": "58", "metadata": {}, "outputs": [], "source": [ @@ -626,7 +638,7 @@ { "cell_type": "code", "execution_count": null, - "id": "58", + "id": "59", "metadata": {}, "outputs": [], "source": [ @@ -641,7 +653,7 @@ }, { "cell_type": "markdown", - "id": "59", + "id": "60", "metadata": {}, "source": [ "## Water mask " @@ -649,7 +661,7 @@ }, { "cell_type": "markdown", - "id": "60", + "id": "61", "metadata": {}, "source": [ "To separate water from non-water surfaces, we first inspect the distribution of backscatter values using a histogram. In the following histogram we could choose -19 as therhold." @@ -658,7 +670,7 @@ { "cell_type": "code", "execution_count": null, - "id": "61", + "id": "62", "metadata": {}, "outputs": [], "source": [ @@ -669,7 +681,7 @@ }, { "cell_type": "markdown", - "id": "62", + "id": "63", "metadata": {}, "source": [ "Since SAR water thresholds vary across scenes and times, a fixed cutoff is unreliable. Threfore, we apply an adaptative thresholding method using **Otsu** algorithm provided by [skimage](https://scikit-image.org/docs/stable/api/skimage.filters.html#skimage.filters.threshold_otsu), which automatically determines an optimal threshold from the intensity distribution and create a water mask accordingly. This algorithm is available within **`xr_threshold_otsu`** function which takes the following keyword arguments:\n", @@ -683,7 +695,7 @@ { "cell_type": "code", "execution_count": null, - "id": "63", + "id": "64", "metadata": {}, "outputs": [], "source": [ @@ -696,7 +708,7 @@ { "cell_type": "code", "execution_count": null, - "id": "64", + "id": "65", "metadata": {}, "outputs": [], "source": [ @@ -706,7 +718,7 @@ { "cell_type": "code", "execution_count": null, - "id": "65", + "id": "66", "metadata": {}, "outputs": [], "source": [ @@ -718,7 +730,7 @@ { "cell_type": "code", "execution_count": null, - "id": "66", + "id": "67", "metadata": {}, "outputs": [], "source": [ @@ -733,7 +745,7 @@ }, { "cell_type": "markdown", - "id": "67", + "id": "68", "metadata": {}, "source": [ "## Time series analysis " @@ -741,7 +753,7 @@ }, { "cell_type": "markdown", - "id": "68", + "id": "69", "metadata": {}, "source": [ "So far we have walked through each processing step separately. To automate the workflow and apply it efficiently across Sentinel-1 acquisitions, we could now wrap all these operations into a single processing function **`process_item`**. This allows us to automatically generate water masks for every item in our STAC collection." @@ -750,7 +762,7 @@ { "cell_type": "code", "execution_count": null, - "id": "69", + "id": "70", "metadata": {}, "outputs": [], "source": [ @@ -793,7 +805,7 @@ }, { "cell_type": "markdown", - "id": "70", + "id": "71", "metadata": {}, "source": [ "Now we loop through all STAC items and use the `process_item` to build the full water-mask dataset." @@ -802,7 +814,7 @@ { "cell_type": "code", "execution_count": null, - "id": "71", + "id": "72", "metadata": {}, "outputs": [], "source": [ @@ -823,7 +835,7 @@ { "cell_type": "code", "execution_count": null, - "id": "72", + "id": "73", "metadata": {}, "outputs": [], "source": [ @@ -833,7 +845,7 @@ { "cell_type": "code", "execution_count": null, - "id": "73", + "id": "74", "metadata": {}, "outputs": [], "source": [ @@ -846,7 +858,7 @@ { "cell_type": "code", "execution_count": null, - "id": "74", + "id": "75", "metadata": {}, "outputs": [], "source": [ @@ -856,7 +868,7 @@ }, { "cell_type": "markdown", - "id": "75", + "id": "76", "metadata": {}, "source": [ "Lets inspect the water mask for the first 4 dates." @@ -865,7 +877,7 @@ { "cell_type": "code", "execution_count": null, - "id": "76", + "id": "77", "metadata": {}, "outputs": [], "source": [ @@ -874,7 +886,7 @@ }, { "cell_type": "markdown", - "id": "77", + "id": "78", "metadata": {}, "source": [ "We found out that the RAW Sentinel-1 GRD scene for 2024-06-07 has issues and artificat that will lead to incorrect water classification. Therefore, we will exclude it from our water frequency analysis." @@ -883,7 +895,7 @@ { "cell_type": "code", "execution_count": null, - "id": "78", + "id": "79", "metadata": {}, "outputs": [], "source": [ @@ -896,7 +908,7 @@ { "cell_type": "code", "execution_count": null, - "id": "79", + "id": "80", "metadata": {}, "outputs": [], "source": [ @@ -905,7 +917,7 @@ }, { "cell_type": "markdown", - "id": "80", + "id": "81", "metadata": {}, "source": [ "### Monthly surface water frequency" @@ -913,7 +925,7 @@ }, { "cell_type": "markdown", - "id": "81", + "id": "82", "metadata": {}, "source": [ "Now that we have the water masks, we can group them by month, compute the average water presence for each pixel, and convert it to a percentage (%) to represent monthly water frequency." @@ -922,7 +934,7 @@ { "cell_type": "code", "execution_count": null, - "id": "82", + "id": "83", "metadata": {}, "outputs": [], "source": [ @@ -941,7 +953,7 @@ { "cell_type": "code", "execution_count": null, - "id": "83", + "id": "84", "metadata": {}, "outputs": [], "source": [ @@ -950,7 +962,7 @@ }, { "cell_type": "markdown", - "id": "84", + "id": "85", "metadata": {}, "source": [ "### Annual surface water frequency" @@ -958,7 +970,7 @@ }, { "cell_type": "markdown", - "id": "85", + "id": "86", "metadata": {}, "source": [ "Lets calculates the average water occurence across the entire time series over the year." @@ -967,7 +979,7 @@ { "cell_type": "code", "execution_count": null, - "id": "86", + "id": "87", "metadata": {}, "outputs": [], "source": [ @@ -981,7 +993,7 @@ { "cell_type": "code", "execution_count": null, - "id": "87", + "id": "88", "metadata": {}, "outputs": [], "source": [ @@ -993,7 +1005,7 @@ { "cell_type": "code", "execution_count": null, - "id": "88", + "id": "89", "metadata": {}, "outputs": [], "source": [ @@ -1008,7 +1020,7 @@ }, { "cell_type": "markdown", - "id": "89", + "id": "90", "metadata": {}, "source": [ "### Annual water change" @@ -1016,7 +1028,7 @@ }, { "cell_type": "markdown", - "id": "90", + "id": "91", "metadata": {}, "source": [ "We could also computes the standard deviation of water presence over time. This highlights areas with high water variability (seasonal or dynamic water bodies)." @@ -1025,7 +1037,7 @@ { "cell_type": "code", "execution_count": null, - "id": "91", + "id": "92", "metadata": {}, "outputs": [], "source": [ @@ -1039,7 +1051,7 @@ { "cell_type": "code", "execution_count": null, - "id": "92", + "id": "93", "metadata": {}, "outputs": [], "source": [ @@ -1051,7 +1063,7 @@ { "cell_type": "code", "execution_count": null, - "id": "93", + "id": "94", "metadata": {}, "outputs": [], "source": [ @@ -1066,7 +1078,7 @@ }, { "cell_type": "markdown", - "id": "94", + "id": "95", "metadata": {}, "source": [ "
" @@ -1074,7 +1086,7 @@ }, { "cell_type": "markdown", - "id": "95", + "id": "96", "metadata": {}, "source": [ "## 💪 Now it is your turn \n", @@ -1110,7 +1122,7 @@ }, { "cell_type": "markdown", - "id": "96", + "id": "97", "metadata": {}, "source": [ "
" @@ -1118,7 +1130,7 @@ }, { "cell_type": "markdown", - "id": "97", + "id": "98", "metadata": {}, "source": [ "## Conclusion" @@ -1126,7 +1138,7 @@ }, { "cell_type": "markdown", - "id": "98", + "id": "99", "metadata": {}, "source": [ "In this notebook, we demonstrated how to use Sentinel-1 data in `.zarr` format for time-series analysis of surface water dynamics in a wetland coastal and cloudy-prone area. The zarr structure is especially useful for efficient extraction of data over a specific area of interest without loading the full dataset.\n", @@ -1138,7 +1150,7 @@ }, { "cell_type": "markdown", - "id": "99", + "id": "100", "metadata": {}, "source": [ "
" @@ -1146,7 +1158,7 @@ }, { "cell_type": "markdown", - "id": "100", + "id": "101", "metadata": {}, "source": [ "## What's next?\n",