diff --git a/05_zarr_tools/51_eopf_stac_r.qmd b/05_zarr_tools/51_eopf_stac_r.qmd index c7a7e31..d4b4877 100644 --- a/05_zarr_tools/51_eopf_stac_r.qmd +++ b/05_zarr_tools/51_eopf_stac_r.qmd @@ -19,6 +19,8 @@ jupyterlab_link <- paste0(link_start, folder, separator, file, link_end) +**By:** *[@sharlagelfand](https://github.com/sharlagelfand)* + ### Introduction In this section, we will explore programmatic access of the EOPF Zarr Collections available in the [EOPF Sentinel Zarr Sample Service STAC Catalog](https://stac.browser.user.eopf.eodc.eu/) using R. We will introduce R packages that enable us to effectively access and search through STAC catalogs. diff --git a/05_zarr_tools/52_eopf_stac_qgis.qmd b/05_zarr_tools/52_eopf_stac_qgis.qmd index aed4f4b..72f6c4b 100644 --- a/05_zarr_tools/52_eopf_stac_qgis.qmd +++ b/05_zarr_tools/52_eopf_stac_qgis.qmd @@ -2,6 +2,8 @@ title: "Access EOPF Zarr in QGIS Using STAC" --- +**By:** *[@captaincoordinates](https://github.com/captaincoordinates)* + ### Introduction The following page will demonstrate how to access the [EOPF STAC Catalog](https://stac.browser.user.eopf.eodc.eu/?.language=en) in QGIS. We will utilise QGIS's native **STAC** capabilities to add **Sentinel-2 L2A Zarr** items delivered from **EOPF STAC Catalog** to a **QGIS** project. diff --git a/05_zarr_tools/53_eopf_zarr_r.qmd b/05_zarr_tools/53_eopf_zarr_r.qmd index b6e7420..2ae539a 100644 --- a/05_zarr_tools/53_eopf_zarr_r.qmd +++ b/05_zarr_tools/53_eopf_zarr_r.qmd @@ -24,6 +24,8 @@ jupyterlab_link <- paste0(link_start, folder, separator, file, link_end) +**By:** *[@sharlagelfand](https://github.com/sharlagelfand)* + ### Introduction In this tutorial, we will demonstrate how to access EOPF Zarr products directly from the [EOPF Sentinel Zarr Sample Service STAC Catalog](https://stac.browser.user.eopf.eodc.eu/) using R. We will introduce R packages that enable us to effectively get an overview of and read Zarr arrays. diff --git a/05_zarr_tools/54_eopf_zarr_r_examples.qmd b/05_zarr_tools/54_eopf_zarr_r_examples.qmd index d46450b..fae4675 100644 --- a/05_zarr_tools/54_eopf_zarr_r_examples.qmd +++ b/05_zarr_tools/54_eopf_zarr_r_examples.qmd @@ -23,6 +23,8 @@ jupyterlab_link <- paste0(link_start, folder, separator, file, link_end) +**By:** *[@sharlagelfand](https://github.com/sharlagelfand)* + ## Introduction This tutorial expands on the previous tutorials ([Access the EOPF Zarr STAC API with R](https://eopf-toolkit.github.io/eopf-101/05_zarr_tools/51_eopf_stac_r.html) and [Access and analyse EOPF STAC Zarr data with R](https://eopf-toolkit.github.io/eopf-101/05_zarr_tools/53_eopf_zarr_r.html)), going into further details on analysing and visualising Zarr data from the [EOPF Sample Service STAC catalog](https://stac.browser.user.eopf.eodc.eu/) programmatically using R. We recommend reviewing the previous tutorials if you have not done so already. diff --git a/05_zarr_tools/57_rstac_gdalcubes.qmd b/05_zarr_tools/57_rstac_gdalcubes.qmd index 6d203a8..4bf068b 100644 --- a/05_zarr_tools/57_rstac_gdalcubes.qmd +++ b/05_zarr_tools/57_rstac_gdalcubes.qmd @@ -4,6 +4,7 @@ format: html self-contained: true --- +**By:** *[@DaChro](https://github.com/DaChro)* ### Introduction diff --git a/06_eopf_zarr_in_action/61_sardinia_s2_tfci.ipynb b/06_eopf_zarr_in_action/61_sardinia_s2_tfci.ipynb index f30c35a..6f37e78 100644 --- a/06_eopf_zarr_in_action/61_sardinia_s2_tfci.ipynb +++ b/06_eopf_zarr_in_action/61_sardinia_s2_tfci.ipynb @@ -34,13 +34,21 @@ "id": "2", "metadata": {}, "source": [ - "### Introduction" + "**By:** *[@gisromerocandanedo](https://github.com/gisromerocandanedo)*" ] }, { "cell_type": "markdown", "id": "3", "metadata": {}, + "source": [ + "### Introduction" + ] + }, + { + "cell_type": "markdown", + "id": "4", + "metadata": {}, "source": [ "Communities and ecosystems worldwide are under **increasing threat** from **wildfires**, a problem that is being made worse by **climate change**.
\n", "\n", @@ -50,7 +58,7 @@ }, { "cell_type": "markdown", - "id": "4", + "id": "5", "metadata": {}, "source": [ "### The Fire Event\n", @@ -63,7 +71,7 @@ }, { "cell_type": "markdown", - "id": "5", + "id": "6", "metadata": {}, "source": [ "### The case study\n", @@ -79,7 +87,7 @@ }, { "cell_type": "markdown", - "id": "6", + "id": "7", "metadata": { "vscode": { "languageId": "raw" @@ -95,7 +103,7 @@ }, { "cell_type": "markdown", - "id": "7", + "id": "8", "metadata": {}, "source": [ "### What we will learn" @@ -103,7 +111,7 @@ }, { "cell_type": "markdown", - "id": "8", + "id": "9", "metadata": {}, "source": [ "- 💨 Create cloud-free layers from Sentinel-2 L2A data for a clear view of the surface at a fire location.\n", @@ -113,7 +121,7 @@ }, { "cell_type": "markdown", - "id": "9", + "id": "10", "metadata": {}, "source": [ "
" @@ -121,7 +129,7 @@ }, { "cell_type": "markdown", - "id": "10", + "id": "11", "metadata": {}, "source": [ "#### Import libraries" @@ -130,7 +138,7 @@ { "cell_type": "code", "execution_count": null, - "id": "11", + "id": "12", "metadata": {}, "outputs": [], "source": [ @@ -150,7 +158,7 @@ }, { "cell_type": "markdown", - "id": "12", + "id": "13", "metadata": {}, "source": [ "#### Helper functions" @@ -158,7 +166,7 @@ }, { "cell_type": "markdown", - "id": "13", + "id": "14", "metadata": {}, "source": [ "This notebook makes use of a set of functions that are all listed inside the [zarr_wf_utils.py](./zarr_wf_utils.py) script. Inside the script, we will find costumised functions that allow us to mask, normalise and extract specific areas of our items of interest." @@ -167,7 +175,7 @@ { "cell_type": "code", "execution_count": null, - "id": "14", + "id": "15", "metadata": {}, "outputs": [], "source": [ @@ -184,7 +192,7 @@ }, { "cell_type": "markdown", - "id": "15", + "id": "16", "metadata": {}, "source": [ "
" @@ -192,7 +200,7 @@ }, { "cell_type": "markdown", - "id": "16", + "id": "17", "metadata": {}, "source": [ "## Setting up the environment" @@ -200,7 +208,7 @@ }, { "cell_type": "markdown", - "id": "17", + "id": "18", "metadata": {}, "source": [ "### Defining parameters for querying the EOPF STAC catalog" @@ -208,7 +216,7 @@ }, { "cell_type": "markdown", - "id": "18", + "id": "19", "metadata": {}, "source": [ "As a first step, we need to define specific parameters for our query, including:
\n", @@ -223,7 +231,7 @@ { "cell_type": "code", "execution_count": null, - "id": "19", + "id": "20", "metadata": {}, "outputs": [], "source": [ @@ -245,7 +253,7 @@ }, { "cell_type": "markdown", - "id": "20", + "id": "21", "metadata": {}, "source": [ "### Initiate a Dask cluster\n", @@ -258,7 +266,7 @@ { "cell_type": "code", "execution_count": null, - "id": "21", + "id": "22", "metadata": {}, "outputs": [], "source": [ @@ -272,7 +280,7 @@ }, { "cell_type": "markdown", - "id": "22", + "id": "23", "metadata": {}, "source": [ "### Establish a connection to the EOPF STAC Catalog\n", @@ -282,7 +290,7 @@ { "cell_type": "code", "execution_count": null, - "id": "23", + "id": "24", "metadata": {}, "outputs": [], "source": [ @@ -292,7 +300,7 @@ }, { "cell_type": "markdown", - "id": "24", + "id": "25", "metadata": {}, "source": [ "## Pre-Fire Visualisation\n", @@ -306,7 +314,7 @@ { "cell_type": "code", "execution_count": null, - "id": "25", + "id": "26", "metadata": {}, "outputs": [], "source": [ @@ -325,7 +333,7 @@ }, { "cell_type": "markdown", - "id": "26", + "id": "27", "metadata": {}, "source": [ "Now, we can retrieve the item and directly and open it as a `xarray.DataTree`. In addition, key information from the item's properties is also extracted to verify key properties of the item." @@ -334,7 +342,7 @@ { "cell_type": "code", "execution_count": null, - "id": "27", + "id": "28", "metadata": {}, "outputs": [], "source": [ @@ -355,7 +363,7 @@ }, { "cell_type": "markdown", - "id": "28", + "id": "29", "metadata": {}, "source": [ "\n", @@ -370,7 +378,7 @@ { "cell_type": "code", "execution_count": null, - "id": "29", + "id": "30", "metadata": {}, "outputs": [], "source": [ @@ -386,7 +394,7 @@ }, { "cell_type": "markdown", - "id": "30", + "id": "31", "metadata": {}, "source": [ "The visualisation we are intending to create covers a larger extent than the specific fire area. This helps us to better understand the event’s overall spatial extent. For this, we generate a bounding box to visually pinpoint the fire’s precise location within a wider composite image." @@ -395,7 +403,7 @@ { "cell_type": "code", "execution_count": null, - "id": "31", + "id": "32", "metadata": {}, "outputs": [], "source": [ @@ -408,7 +416,7 @@ }, { "cell_type": "markdown", - "id": "32", + "id": "33", "metadata": {}, "source": [ "In next step, we need to reproject the area of interest from EPSG: **4326** to UTM coordinates.\n", @@ -418,7 +426,7 @@ { "cell_type": "code", "execution_count": null, - "id": "33", + "id": "34", "metadata": {}, "outputs": [], "source": [ @@ -442,7 +450,7 @@ }, { "cell_type": "markdown", - "id": "34", + "id": "35", "metadata": {}, "source": [ "### Pre-Fire True Colour Image\n", @@ -455,7 +463,7 @@ { "cell_type": "code", "execution_count": null, - "id": "35", + "id": "36", "metadata": {}, "outputs": [], "source": [ @@ -476,7 +484,7 @@ }, { "cell_type": "markdown", - "id": "36", + "id": "37", "metadata": {}, "source": [ "The next step is to clip the retrieved asset to our area of interest which we defined earlier with specific bounding box information.\n", @@ -492,7 +500,7 @@ { "cell_type": "code", "execution_count": null, - "id": "37", + "id": "38", "metadata": {}, "outputs": [], "source": [ @@ -518,7 +526,7 @@ }, { "cell_type": "markdown", - "id": "38", + "id": "39", "metadata": {}, "source": [ "To create the composite image, we need to **normalise** each of the input assets. Normalisation ensures that the bands have a consistent and predictable range of values. This supports optimal data processing and removes the influence of external factors (like changing light conditions) allowing for a meaningful comparison among generated composites.\n", @@ -531,7 +539,7 @@ { "cell_type": "code", "execution_count": null, - "id": "39", + "id": "40", "metadata": {}, "outputs": [], "source": [ @@ -554,7 +562,7 @@ }, { "cell_type": "markdown", - "id": "40", + "id": "41", "metadata": {}, "source": [ "The image is currently displayed with a neutral colour ramp and with the non-valid masked pixels. Some of the details can be enhanced based on the information the overall composite contains.\n", @@ -566,7 +574,7 @@ { "cell_type": "code", "execution_count": null, - "id": "41", + "id": "42", "metadata": {}, "outputs": [], "source": [ @@ -579,7 +587,7 @@ }, { "cell_type": "markdown", - "id": "42", + "id": "43", "metadata": {}, "source": [ "### Pre-Fire False Colour Image" @@ -587,7 +595,7 @@ }, { "cell_type": "markdown", - "id": "43", + "id": "44", "metadata": {}, "source": [ "Next, a **False Colour Image** (FCI) is created to provide a clearer overview of vegetation health.
\n", @@ -600,7 +608,7 @@ { "cell_type": "code", "execution_count": null, - "id": "44", + "id": "45", "metadata": {}, "outputs": [], "source": [ @@ -612,7 +620,7 @@ }, { "cell_type": "markdown", - "id": "45", + "id": "46", "metadata": {}, "source": [ "Following the same principle as of the creation of the True Colour composite, we can choose the relevant bands and apply the masking and clipping steps." @@ -621,7 +629,7 @@ { "cell_type": "code", "execution_count": null, - "id": "46", + "id": "47", "metadata": {}, "outputs": [], "source": [ @@ -639,7 +647,7 @@ }, { "cell_type": "markdown", - "id": "47", + "id": "48", "metadata": {}, "source": [ "Then, we can apply the normalisation function, followed by the stacking of the three bands. The False Colour composite $$$ explain the colouring" @@ -648,7 +656,7 @@ { "cell_type": "code", "execution_count": null, - "id": "48", + "id": "49", "metadata": {}, "outputs": [], "source": [ @@ -667,7 +675,7 @@ }, { "cell_type": "markdown", - "id": "49", + "id": "50", "metadata": {}, "source": [ "We then continue and also apply the equalisation function." @@ -676,7 +684,7 @@ { "cell_type": "code", "execution_count": null, - "id": "50", + "id": "51", "metadata": {}, "outputs": [], "source": [ @@ -691,7 +699,7 @@ }, { "cell_type": "markdown", - "id": "51", + "id": "52", "metadata": {}, "source": [ "## Post-Fire Visualisation" @@ -699,7 +707,7 @@ }, { "cell_type": "markdown", - "id": "52", + "id": "53", "metadata": {}, "source": [ "Now, we will replicate the same visualisation for a specific time after the fire: 10 June 2025.\n", @@ -709,7 +717,7 @@ { "cell_type": "code", "execution_count": null, - "id": "53", + "id": "54", "metadata": {}, "outputs": [], "source": [ @@ -726,7 +734,7 @@ }, { "cell_type": "markdown", - "id": "54", + "id": "55", "metadata": {}, "source": [ "We open again the the retrieved item from our filtered results, followed by the masking and validation steps to ensure a clean, cloud-free composite." @@ -735,7 +743,7 @@ { "cell_type": "code", "execution_count": null, - "id": "55", + "id": "56", "metadata": {}, "outputs": [], "source": [ @@ -755,7 +763,7 @@ }, { "cell_type": "markdown", - "id": "56", + "id": "57", "metadata": {}, "source": [ "### Post-Fire True Colour Image" @@ -763,7 +771,7 @@ }, { "cell_type": "markdown", - "id": "57", + "id": "58", "metadata": {}, "source": [ "Once invalid pixels are filtered out, we can generate the corresponding True-Color composite to get a view of our area of interest after the fire event. We again clip the retrieved assets to our specific bounding box." @@ -772,7 +780,7 @@ { "cell_type": "code", "execution_count": null, - "id": "58", + "id": "59", "metadata": {}, "outputs": [], "source": [ @@ -801,7 +809,7 @@ }, { "cell_type": "markdown", - "id": "59", + "id": "60", "metadata": {}, "source": [ "And, once the new area is defined, we normalise, stack and equalize the composite." @@ -810,7 +818,7 @@ { "cell_type": "code", "execution_count": null, - "id": "60", + "id": "61", "metadata": {}, "outputs": [], "source": [ @@ -831,7 +839,7 @@ }, { "cell_type": "markdown", - "id": "61", + "id": "62", "metadata": {}, "source": [ "### Post-Fire False Colour Image" @@ -839,7 +847,7 @@ }, { "cell_type": "markdown", - "id": "62", + "id": "63", "metadata": {}, "source": [ "And as the last processing step, we create the False Colour composite for the same day, to clearly visualise the extent of the burn scars and vegetation recovery." @@ -848,7 +856,7 @@ { "cell_type": "code", "execution_count": null, - "id": "63", + "id": "64", "metadata": {}, "outputs": [], "source": [ @@ -883,7 +891,7 @@ }, { "cell_type": "markdown", - "id": "64", + "id": "65", "metadata": {}, "source": [ "## Compare pre- and post-fire composites" @@ -891,7 +899,7 @@ }, { "cell_type": "markdown", - "id": "65", + "id": "66", "metadata": {}, "source": [ "As a last step, we will **georeference** and **visualise** the created composites together, presenting it in a way that makes it easier to recognise and understand the monitored areas.\n", @@ -904,7 +912,7 @@ { "cell_type": "code", "execution_count": null, - "id": "66", + "id": "67", "metadata": {}, "outputs": [], "source": [ @@ -964,7 +972,7 @@ }, { "cell_type": "markdown", - "id": "67", + "id": "68", "metadata": {}, "source": [ "## Calculating processing time" @@ -973,7 +981,7 @@ { "cell_type": "code", "execution_count": null, - "id": "68", + "id": "69", "metadata": {}, "outputs": [], "source": [ @@ -986,7 +994,7 @@ }, { "cell_type": "markdown", - "id": "69", + "id": "70", "metadata": {}, "source": [ "As our plots show, the **True Colour Image** reveals a clear change in the state of the vegetation, with an evident burn scar visible on the ground. The **False Colour Image** also highlights a significant change in the spectral response, which precisely encloses the spot where the fire occurred.
\n", @@ -996,7 +1004,7 @@ }, { "cell_type": "markdown", - "id": "70", + "id": "71", "metadata": {}, "source": [ "
" @@ -1004,7 +1012,7 @@ }, { "cell_type": "markdown", - "id": "71", + "id": "72", "metadata": {}, "source": [ "## Conclusion" @@ -1012,7 +1020,7 @@ }, { "cell_type": "markdown", - "id": "72", + "id": "73", "metadata": {}, "source": [ "Having processed the key spectral bands, we have successfully established a visual baseline for **our monitoring workflow**.
\n", @@ -1024,7 +1032,7 @@ }, { "cell_type": "markdown", - "id": "73", + "id": "74", "metadata": {}, "source": [ "
" @@ -1032,7 +1040,7 @@ }, { "cell_type": "markdown", - "id": "74", + "id": "75", "metadata": {}, "source": [ "## What’s next?\n", diff --git a/06_eopf_zarr_in_action/62_sardinia_s3_lst.ipynb b/06_eopf_zarr_in_action/62_sardinia_s3_lst.ipynb index c220122..3ae6292 100644 --- a/06_eopf_zarr_in_action/62_sardinia_s3_lst.ipynb +++ b/06_eopf_zarr_in_action/62_sardinia_s3_lst.ipynb @@ -32,6 +32,14 @@ { "cell_type": "markdown", "id": "2", + "metadata": {}, + "source": [ + "**By:** *[@gisromerocandanedo](https://github.com/gisromerocandanedo)*" + ] + }, + { + "cell_type": "markdown", + "id": "3", "metadata": { "vscode": { "languageId": "raw" @@ -57,7 +65,7 @@ }, { "cell_type": "markdown", - "id": "3", + "id": "4", "metadata": {}, "source": [ "### What we will learn" @@ -65,7 +73,7 @@ }, { "cell_type": "markdown", - "id": "4", + "id": "5", "metadata": {}, "source": [ "- ✂️ Extract and clip data from Sentinel-3 SLSTR L2 \n", @@ -75,7 +83,7 @@ }, { "cell_type": "markdown", - "id": "5", + "id": "6", "metadata": {}, "source": [ "
" @@ -83,7 +91,7 @@ }, { "cell_type": "markdown", - "id": "6", + "id": "7", "metadata": {}, "source": [ "#### Import libraries" @@ -92,7 +100,7 @@ { "cell_type": "code", "execution_count": null, - "id": "7", + "id": "8", "metadata": {}, "outputs": [], "source": [ @@ -112,7 +120,7 @@ }, { "cell_type": "markdown", - "id": "8", + "id": "9", "metadata": {}, "source": [ "#### Helper functions" @@ -120,7 +128,7 @@ }, { "cell_type": "markdown", - "id": "9", + "id": "10", "metadata": {}, "source": [ "This notebook makes use of a set of functions that are all listed inside the [zarr_wf_utils.py](./zarr_wf_utils.py) script. Inside the script, we will find costumised functions that allow us to mask, normalise and extract specific areas of our items of interest." @@ -129,7 +137,7 @@ { "cell_type": "code", "execution_count": null, - "id": "10", + "id": "11", "metadata": {}, "outputs": [], "source": [ @@ -147,7 +155,7 @@ }, { "cell_type": "markdown", - "id": "11", + "id": "12", "metadata": {}, "source": [ "
" @@ -155,7 +163,7 @@ }, { "cell_type": "markdown", - "id": "12", + "id": "13", "metadata": {}, "source": [ "## Setting up the environment" @@ -163,7 +171,7 @@ }, { "cell_type": "markdown", - "id": "13", + "id": "14", "metadata": {}, "source": [ "### Initiate a Dask cluster" @@ -171,7 +179,7 @@ }, { "cell_type": "markdown", - "id": "14", + "id": "15", "metadata": {}, "source": [ "The first step is to initiate a **virtual Dask cluster**. This cluster consists of a scheduler (the \"brain\") and several workers (the \"hands\"), which enables faster processing of large datasets by breaking down tasks and running them in parallel.\n", @@ -182,7 +190,7 @@ { "cell_type": "code", "execution_count": null, - "id": "15", + "id": "16", "metadata": {}, "outputs": [], "source": [ @@ -196,7 +204,7 @@ }, { "cell_type": "markdown", - "id": "16", + "id": "17", "metadata": {}, "source": [ "### Establish a connection to the EOPF STAC Catalog" @@ -204,7 +212,7 @@ }, { "cell_type": "markdown", - "id": "17", + "id": "18", "metadata": {}, "source": [ "Data is retrieved from the [EOPF STAC Catalogue](https://stac.browser.user.eopf.eodc.eu/?.language=en) endpoint. Once the connection is established, we can query the catalog based on specific search criteria." @@ -213,7 +221,7 @@ { "cell_type": "code", "execution_count": null, - "id": "18", + "id": "19", "metadata": {}, "outputs": [], "source": [ @@ -224,7 +232,7 @@ }, { "cell_type": "markdown", - "id": "19", + "id": "20", "metadata": {}, "source": [ "### Define search paramters" @@ -233,7 +241,7 @@ { "cell_type": "code", "execution_count": null, - "id": "20", + "id": "21", "metadata": {}, "outputs": [], "source": [ @@ -263,7 +271,7 @@ }, { "cell_type": "markdown", - "id": "21", + "id": "22", "metadata": {}, "source": [ "## Overview of processing steps" @@ -271,7 +279,7 @@ }, { "cell_type": "markdown", - "id": "22", + "id": "23", "metadata": {}, "source": [ "In the following, we will go through three main processing steps:
\n", @@ -283,7 +291,7 @@ }, { "cell_type": "markdown", - "id": "23", + "id": "24", "metadata": {}, "source": [ "### Retrieve Land Surface Temperature (LST) from Sentinel-3 SLSTR L2" @@ -291,7 +299,7 @@ }, { "cell_type": "markdown", - "id": "24", + "id": "25", "metadata": {}, "source": [ "Land Surface Temperature (LST) data, can be retrieved from the Sentinel-3 SLSTR L2 collection. This data helps to identify temperature anomalies over the Earth's Surface, which can be a strong indicator of an active fire.\n", @@ -304,7 +312,7 @@ { "cell_type": "code", "execution_count": null, - "id": "25", + "id": "26", "metadata": {}, "outputs": [], "source": [ @@ -329,7 +337,7 @@ }, { "cell_type": "markdown", - "id": "26", + "id": "27", "metadata": {}, "source": [ "After filtering the catalog, we open the **first** available Sentinel-3 SLSTR item, which corresponds to our specific timeframe of the selected day.\n", @@ -340,7 +348,7 @@ { "cell_type": "code", "execution_count": null, - "id": "27", + "id": "28", "metadata": {}, "outputs": [], "source": [ @@ -362,7 +370,7 @@ }, { "cell_type": "markdown", - "id": "28", + "id": "29", "metadata": {}, "source": [ "To effectively overlay the data, we first need to process the Land Surface Temperature (LST) asset to cover the same area of interest.\n", @@ -372,7 +380,7 @@ { "cell_type": "code", "execution_count": null, - "id": "29", + "id": "30", "metadata": {}, "outputs": [], "source": [ @@ -396,7 +404,7 @@ }, { "cell_type": "markdown", - "id": "30", + "id": "31", "metadata": {}, "source": [ "After clipping the data to our defined area of interest, we apply the **temperature threshold** to the data, filtering for only those pixels with **temperatures above 312 Kelvin**. This temperature range is a strong indicator of heat anomalies, which are often associated with active or developing fires." @@ -405,7 +413,7 @@ { "cell_type": "code", "execution_count": null, - "id": "31", + "id": "32", "metadata": {}, "outputs": [], "source": [ @@ -419,7 +427,7 @@ }, { "cell_type": "markdown", - "id": "32", + "id": "33", "metadata": {}, "source": [ "\n", @@ -429,7 +437,7 @@ { "cell_type": "code", "execution_count": null, - "id": "33", + "id": "34", "metadata": {}, "outputs": [], "source": [ @@ -448,7 +456,7 @@ }, { "cell_type": "markdown", - "id": "34", + "id": "35", "metadata": {}, "source": [ "The final step of the Sentinel-3 SLSTR data processing is to **visualise** the temperature anomalies. We will prepare the filtered LST data to be overlaid over the True-Color composite of Sentinel-2 L2 data.\n" @@ -457,7 +465,7 @@ { "cell_type": "code", "execution_count": null, - "id": "35", + "id": "36", "metadata": {}, "outputs": [], "source": [ @@ -477,7 +485,7 @@ }, { "cell_type": "markdown", - "id": "36", + "id": "37", "metadata": {}, "source": [ "### Create Sentinel-2 L2A True-Color composite\n", @@ -486,7 +494,7 @@ }, { "cell_type": "markdown", - "id": "37", + "id": "38", "metadata": {}, "source": [ "Following the parameters we defined before and the workflow described in the [first part](./61_sardinia_s2_tfci.ipynb) of this notebook series, we will filter the **Sentinel-2 L2A** data collection to match our event and AOI. This ensures that the visualisations we create are directly relevant to the fire event and set the stage for comparing it with the Land Surface Temperature data from Sentinel-3.\n" @@ -495,7 +503,7 @@ { "cell_type": "code", "execution_count": null, - "id": "38", + "id": "39", "metadata": {}, "outputs": [], "source": [ @@ -516,7 +524,7 @@ }, { "cell_type": "markdown", - "id": "39", + "id": "40", "metadata": {}, "source": [ "As we can see, there is no capture available for the day on which the fire occurred. This is because **Sentinel-2 L2A** has a **revisit time** of **five** days at the equator, making it possible that, even when the constellation is synchronously retrieving data, the day in question may not be available.
\n", @@ -526,7 +534,7 @@ { "cell_type": "code", "execution_count": null, - "id": "40", + "id": "41", "metadata": {}, "outputs": [], "source": [ @@ -545,7 +553,7 @@ }, { "cell_type": "markdown", - "id": "41", + "id": "42", "metadata": {}, "source": [ "Once we have obtained the available items from the **Sentinel-2 L2A** collection, we can open the asset as a xarray.DataTree.\n", @@ -563,7 +571,7 @@ }, { "cell_type": "markdown", - "id": "42", + "id": "43", "metadata": {}, "source": [ "#### Masking out invalid pixels" @@ -572,7 +580,7 @@ { "cell_type": "code", "execution_count": null, - "id": "43", + "id": "44", "metadata": {}, "outputs": [], "source": [ @@ -597,7 +605,7 @@ }, { "cell_type": "markdown", - "id": "44", + "id": "45", "metadata": {}, "source": [ "#### Clipping to AOI" @@ -605,7 +613,7 @@ }, { "cell_type": "markdown", - "id": "45", + "id": "46", "metadata": {}, "source": [ "In a next step, we clip the retrieved item to our defined area of interest (AOI)." @@ -614,7 +622,7 @@ { "cell_type": "code", "execution_count": null, - "id": "46", + "id": "47", "metadata": {}, "outputs": [], "source": [ @@ -637,7 +645,7 @@ }, { "cell_type": "markdown", - "id": "47", + "id": "48", "metadata": {}, "source": [ "#### Band selection, normalisation composite creation and equalisation" @@ -645,7 +653,7 @@ }, { "cell_type": "markdown", - "id": "48", + "id": "49", "metadata": {}, "source": [ "In the next step, we proceed to select the relevant bands from the item, apply normalisation and equalisation, in order to visualise the True-Color composite of 11 June 2025 over the Nuoto region in Sardinia." @@ -654,7 +662,7 @@ { "cell_type": "code", "execution_count": null, - "id": "49", + "id": "50", "metadata": {}, "outputs": [], "source": [ @@ -700,7 +708,7 @@ }, { "cell_type": "markdown", - "id": "50", + "id": "51", "metadata": {}, "source": [ "### Overlay Sentinel-2 True-Color composite with Sentinel-3 LST data" @@ -708,7 +716,7 @@ }, { "cell_type": "markdown", - "id": "51", + "id": "52", "metadata": {}, "source": [ "And finally, we can **georeference** and overlay the two datasets, the True-Color composite from Sentinel-2 as well as the Land Surface Temperature information from Sentinel-3. \n" @@ -717,7 +725,7 @@ { "cell_type": "code", "execution_count": null, - "id": "52", + "id": "53", "metadata": {}, "outputs": [], "source": [ @@ -753,7 +761,7 @@ }, { "cell_type": "markdown", - "id": "53", + "id": "54", "metadata": {}, "source": [ "The **Sentinel-2 L2A** provides the geographical context of the landscape, while the **Sentinel-3 SLSTR** LST information provides an indication of the hottest areas on that day. This allows for a more accurate and immediate understanding of a fire's behaviour during the event. We can see that the hottest detected spot over the area of interest is indeed aligned with the fire event location.
\n", @@ -763,7 +771,7 @@ }, { "cell_type": "markdown", - "id": "54", + "id": "55", "metadata": {}, "source": [ "### Calculating processing time" @@ -772,7 +780,7 @@ { "cell_type": "code", "execution_count": null, - "id": "55", + "id": "56", "metadata": {}, "outputs": [], "source": [ @@ -785,7 +793,7 @@ }, { "cell_type": "markdown", - "id": "56", + "id": "57", "metadata": {}, "source": [ "A significant takeaway from this process is its remarkable speed. The entire workflow, from data access to visualisation, is completed in under two minutes. Here is the key evident advantage of using `.zarr` encoding for wildfire detection.
\n", @@ -794,7 +802,7 @@ }, { "cell_type": "markdown", - "id": "57", + "id": "58", "metadata": {}, "source": [ "
" @@ -802,7 +810,7 @@ }, { "cell_type": "markdown", - "id": "58", + "id": "59", "metadata": {}, "source": [ "## Conclusion" @@ -810,7 +818,7 @@ }, { "cell_type": "markdown", - "id": "59", + "id": "60", "metadata": {}, "source": [ "By integrating items from **Sentinel-2** with critical thermal data from **Sentinel-3** through the [EOPF STAC Catalog]() Collections, this notebook has demonstrated a complete and efficient workflow for analysing a real-world wildfire event.
\n", @@ -820,7 +828,7 @@ }, { "cell_type": "markdown", - "id": "60", + "id": "61", "metadata": {}, "source": [ "
" @@ -828,7 +836,7 @@ }, { "cell_type": "markdown", - "id": "61", + "id": "62", "metadata": {}, "source": [ "## What’s next?\n", diff --git a/06_eopf_zarr_in_action/63_sardinia_dNBR.ipynb b/06_eopf_zarr_in_action/63_sardinia_dNBR.ipynb index ac64210..6154e0e 100644 --- a/06_eopf_zarr_in_action/63_sardinia_dNBR.ipynb +++ b/06_eopf_zarr_in_action/63_sardinia_dNBR.ipynb @@ -33,6 +33,14 @@ "cell_type": "markdown", "id": "2", "metadata": {}, + "source": [ + "**By:** *[@gisromerocandanedo](https://github.com/gisromerocandanedo)*" + ] + }, + { + "cell_type": "markdown", + "id": "3", + "metadata": {}, "source": [ "### Introduction\n", "This notebook introduces the **Normalised Burn Ratio** (NBR), a key index used in satellite remote sensing to assess burn severity. The **NBR** quantifies the severity of damage from wildfires. The NBR is calculated using near-infrared and shortwave infrared bands from Sentinel-2 Level 2 data to highlight vegetation and burned areas.\n", @@ -55,7 +63,7 @@ }, { "cell_type": "markdown", - "id": "3", + "id": "4", "metadata": {}, "source": [ "### What we will learn" @@ -63,7 +71,7 @@ }, { "cell_type": "markdown", - "id": "4", + "id": "5", "metadata": {}, "source": [ "- 🛰️ Access, compute and relate the **SWIR** and **NIR** bands from Sentinel-2 mission\n", @@ -73,7 +81,7 @@ }, { "cell_type": "markdown", - "id": "5", + "id": "6", "metadata": {}, "source": [ "
" @@ -81,7 +89,7 @@ }, { "cell_type": "markdown", - "id": "6", + "id": "7", "metadata": {}, "source": [ "#### Import libraries" @@ -90,7 +98,7 @@ { "cell_type": "code", "execution_count": null, - "id": "7", + "id": "8", "metadata": {}, "outputs": [], "source": [ @@ -110,7 +118,7 @@ }, { "cell_type": "markdown", - "id": "8", + "id": "9", "metadata": {}, "source": [ "#### Helper functions" @@ -118,7 +126,7 @@ }, { "cell_type": "markdown", - "id": "9", + "id": "10", "metadata": {}, "source": [ "This notebook makes use of a set of functions that are all listed inside the [zarr_wf_utils.py](./zarr_wf_utils.py) script. Inside the script, we will find costumised functions that allow us to mask, normalise and extract specific areas of our items of interest." @@ -127,7 +135,7 @@ { "cell_type": "code", "execution_count": null, - "id": "10", + "id": "11", "metadata": {}, "outputs": [], "source": [ @@ -141,7 +149,7 @@ }, { "cell_type": "markdown", - "id": "11", + "id": "12", "metadata": {}, "source": [ "
" @@ -149,7 +157,7 @@ }, { "cell_type": "markdown", - "id": "12", + "id": "13", "metadata": {}, "source": [ "## Setting up the environment" @@ -157,7 +165,7 @@ }, { "cell_type": "markdown", - "id": "13", + "id": "14", "metadata": {}, "source": [ "### Initiate a Dask cluster" @@ -165,7 +173,7 @@ }, { "cell_type": "markdown", - "id": "14", + "id": "15", "metadata": {}, "source": [ "The first step is to initiate a virtual Dask cluster consisting of a scheduler that manages tasks and multiple workers that process them." @@ -174,7 +182,7 @@ { "cell_type": "code", "execution_count": null, - "id": "15", + "id": "16", "metadata": {}, "outputs": [], "source": [ @@ -185,7 +193,7 @@ }, { "cell_type": "markdown", - "id": "16", + "id": "17", "metadata": {}, "source": [ "### Define workflow variables" @@ -193,7 +201,7 @@ }, { "cell_type": "markdown", - "id": "17", + "id": "18", "metadata": {}, "source": [ "Following [Part 1](./61_sardinia_s2_tfci.ipynb) of, we need to define the specific **spatial** and **temporal** parameters for our analysis. We will set the same area of interest over the Province of Nuoro Sardinia, Italy. We define the dates for our pre-fire and post-fire period.\n", @@ -214,7 +222,7 @@ { "cell_type": "code", "execution_count": null, - "id": "18", + "id": "19", "metadata": {}, "outputs": [], "source": [ @@ -253,7 +261,7 @@ }, { "cell_type": "markdown", - "id": "19", + "id": "20", "metadata": {}, "source": [ "### Establish a connection to the EOPF STAC Catalog\n" @@ -261,7 +269,7 @@ }, { "cell_type": "markdown", - "id": "20", + "id": "21", "metadata": {}, "source": [ "Additionally, we also need to establish a connection to the EOPF STAC catalog." @@ -270,7 +278,7 @@ { "cell_type": "code", "execution_count": null, - "id": "21", + "id": "22", "metadata": {}, "outputs": [], "source": [ @@ -281,7 +289,7 @@ }, { "cell_type": "markdown", - "id": "22", + "id": "23", "metadata": {}, "source": [ "## Calculation of NBR - Pre-Fire" @@ -289,7 +297,7 @@ }, { "cell_type": "markdown", - "id": "23", + "id": "24", "metadata": {}, "source": [ "Now we can use our defined parameters to query the STAC catalog and to retrieve all available **Sentinel-2** imagery that correspond to our query arguments." @@ -298,7 +306,7 @@ { "cell_type": "code", "execution_count": null, - "id": "24", + "id": "25", "metadata": {}, "outputs": [], "source": [ @@ -316,7 +324,7 @@ }, { "cell_type": "markdown", - "id": "25", + "id": "26", "metadata": {}, "source": [ "Now we open the retrieved item as a xarray.DataTree. This image will serve as our baseline for the pre-fire conditions, providing a clean snapshot of the landscape before the event.\n", @@ -326,7 +334,7 @@ { "cell_type": "code", "execution_count": null, - "id": "26", + "id": "27", "metadata": {}, "outputs": [], "source": [ @@ -346,7 +354,7 @@ }, { "cell_type": "markdown", - "id": "27", + "id": "28", "metadata": {}, "source": [ "To calculate the **Normalised Burn Ratio** (NBR), we will access the specific spectral bands we need: the **Near-Infrared** (NIR) band (B8A) and the **Shortwave Infrared** (SWIR) band (B12).\n", @@ -357,7 +365,7 @@ { "cell_type": "code", "execution_count": null, - "id": "28", + "id": "29", "metadata": {}, "outputs": [], "source": [ @@ -380,7 +388,7 @@ }, { "cell_type": "markdown", - "id": "29", + "id": "30", "metadata": {}, "source": [ "Once we obtain the masked assets, we can visualise them before calculating the **NBR**." @@ -389,7 +397,7 @@ { "cell_type": "code", "execution_count": null, - "id": "30", + "id": "31", "metadata": {}, "outputs": [], "source": [ @@ -408,7 +416,7 @@ }, { "cell_type": "markdown", - "id": "31", + "id": "32", "metadata": {}, "source": [ "With our `swir_pre` and `nir_pre` bands ready, we can now calculate the pre-fire NBR image.\n", @@ -417,7 +425,7 @@ }, { "cell_type": "markdown", - "id": "32", + "id": "33", "metadata": {}, "source": [ "$$\\frac{(NIR - SWIR)} {(NIR + SWIR)}$$" @@ -425,7 +433,7 @@ }, { "cell_type": "markdown", - "id": "33", + "id": "34", "metadata": {}, "source": [ "An important consideration for this process is the possibility of division by zero when calculating the composite. To prevent this, we create a mask that assigns a value of `1` to any pixel that is `0`, while keeping the original values for all other pixels." @@ -434,7 +442,7 @@ { "cell_type": "code", "execution_count": null, - "id": "34", + "id": "35", "metadata": {}, "outputs": [], "source": [ @@ -459,7 +467,7 @@ }, { "cell_type": "markdown", - "id": "35", + "id": "36", "metadata": {}, "source": [ "Now, we can visualise the calculated pre-fire NBR image." @@ -468,7 +476,7 @@ { "cell_type": "code", "execution_count": null, - "id": "36", + "id": "37", "metadata": {}, "outputs": [], "source": [ @@ -479,7 +487,7 @@ }, { "cell_type": "markdown", - "id": "37", + "id": "38", "metadata": {}, "source": [ "## Calculation of NBR - Post-Fire" @@ -487,7 +495,7 @@ }, { "cell_type": "markdown", - "id": "38", + "id": "39", "metadata": {}, "source": [ "The next step is to repeat the same steps as above and to calculate a **post-fire** NBR image.\n", @@ -498,7 +506,7 @@ { "cell_type": "code", "execution_count": null, - "id": "39", + "id": "40", "metadata": {}, "outputs": [], "source": [ @@ -526,7 +534,7 @@ }, { "cell_type": "markdown", - "id": "40", + "id": "41", "metadata": {}, "source": [ "In a next step, we follow the same **masking** and **clipping** functions to this new dataset to ensure it is focused on our area of interest." @@ -535,7 +543,7 @@ { "cell_type": "code", "execution_count": null, - "id": "41", + "id": "42", "metadata": {}, "outputs": [], "source": [ @@ -562,7 +570,7 @@ }, { "cell_type": "markdown", - "id": "42", + "id": "43", "metadata": {}, "source": [ "After our masking, we obtain the following bands" @@ -571,7 +579,7 @@ { "cell_type": "code", "execution_count": null, - "id": "43", + "id": "44", "metadata": {}, "outputs": [], "source": [ @@ -590,7 +598,7 @@ }, { "cell_type": "markdown", - "id": "44", + "id": "45", "metadata": {}, "source": [ "With our post-fire bands, we can now calculate the Post-Fire NBR. We apply the **0** division consideration masking and the **NBR** formula again to these bands to get the post-fire **NBR** image." @@ -599,7 +607,7 @@ { "cell_type": "code", "execution_count": null, - "id": "45", + "id": "46", "metadata": {}, "outputs": [], "source": [ @@ -624,7 +632,7 @@ }, { "cell_type": "markdown", - "id": "46", + "id": "47", "metadata": {}, "source": [ "## Calculate Differenced NBR (dNBR)" @@ -632,7 +640,7 @@ }, { "cell_type": "markdown", - "id": "47", + "id": "48", "metadata": {}, "source": [ "The last calculation step for our analysis is the **delta NBR** (dNBR). This index is calculated by **subtracting** the **post-fire** NBR **from** the **pre-fire** NBR. Higher dNBR values indicate more severe damage, providing a quantitative measure of the fire's impact." @@ -640,7 +648,7 @@ }, { "cell_type": "markdown", - "id": "48", + "id": "49", "metadata": {}, "source": [ "$$dNBR = prefireNBR - postfireNBR$$" @@ -649,7 +657,7 @@ { "cell_type": "code", "execution_count": null, - "id": "49", + "id": "50", "metadata": {}, "outputs": [], "source": [ @@ -662,7 +670,7 @@ }, { "cell_type": "markdown", - "id": "50", + "id": "51", "metadata": {}, "source": [ "As our calculation was based on dask array for a faster computation, we bring it back to memory and multiply it by 1000 to be able to compare it to **Key & Benson (2006) severity thresholds**, proposed by [EFFIS](https://forest-fire.emergency.copernicus.eu/about-effis/technical-background/fire-severity#:~:text=The%20proposed%20methodology%20is%20recommended,and%20post%2Dfire%20NBR%20composites)." @@ -671,7 +679,7 @@ { "cell_type": "code", "execution_count": null, - "id": "51", + "id": "52", "metadata": {}, "outputs": [], "source": [ @@ -683,7 +691,7 @@ }, { "cell_type": "markdown", - "id": "52", + "id": "53", "metadata": {}, "source": [ "Finally, we will plot our results to visualise the **burn severity**. Using the `cartopy` library, we can create a georeferenced map that accurately displays our data based on its CRS and geospatial bounding box.\n", @@ -694,7 +702,7 @@ { "cell_type": "code", "execution_count": null, - "id": "53", + "id": "54", "metadata": {}, "outputs": [], "source": [ @@ -731,7 +739,7 @@ }, { "cell_type": "markdown", - "id": "54", + "id": "55", "metadata": {}, "source": [ "As the plot shows, the burn scar provides a spectral response through **dNBR**, even after the asset was clipped to remove clouds and water bodies.
\n", @@ -748,7 +756,7 @@ }, { "cell_type": "markdown", - "id": "55", + "id": "56", "metadata": {}, "source": [ "### Calculating processing time" @@ -756,7 +764,7 @@ }, { "cell_type": "markdown", - "id": "56", + "id": "57", "metadata": {}, "source": [ "Besides the asset's accessibility, it is important to note the time efficiency of this monitoring workflow. The entire process, from defining the area of interest to searching, accessing, processing, and visualising the data takes **less** than **30 seconds**.
" @@ -765,7 +773,7 @@ { "cell_type": "code", "execution_count": null, - "id": "57", + "id": "58", "metadata": {}, "outputs": [], "source": [ @@ -776,7 +784,7 @@ }, { "cell_type": "markdown", - "id": "58", + "id": "59", "metadata": {}, "source": [ "
" @@ -784,7 +792,7 @@ }, { "cell_type": "markdown", - "id": "59", + "id": "60", "metadata": {}, "source": [ "## Conclusion" @@ -792,7 +800,7 @@ }, { "cell_type": "markdown", - "id": "60", + "id": "61", "metadata": {}, "source": [ "This series of notebooks demonstrated a complete workflow for monitoring fire events using data from the Sentinel-2 and Sentinel-3 missions, providing a comprehensive view of the wildfire stages, **before**, **during** and **after**.\n", @@ -804,7 +812,7 @@ }, { "cell_type": "markdown", - "id": "61", + "id": "62", "metadata": {}, "source": [ "
" @@ -812,7 +820,7 @@ }, { "cell_type": "markdown", - "id": "62", + "id": "63", "metadata": {}, "source": [ "## What’s next?\n", diff --git a/06_eopf_zarr_in_action/64_flood_mapping_valencia.ipynb b/06_eopf_zarr_in_action/64_flood_mapping_valencia.ipynb index a595018..d8ec403 100644 --- a/06_eopf_zarr_in_action/64_flood_mapping_valencia.ipynb +++ b/06_eopf_zarr_in_action/64_flood_mapping_valencia.ipynb @@ -34,12 +34,20 @@ "id": "2", "metadata": {}, "source": [ - "### Introduction" + "**By:** *[@beatrizbsperes](https://github.com/beatrizbsperes)*" ] }, { "cell_type": "markdown", "id": "3", + "metadata": {}, + "source": [ + "### Introduction" + ] + }, + { + "cell_type": "markdown", + "id": "4", "metadata": { "vscode": { "languageId": "raw" @@ -53,7 +61,7 @@ }, { "cell_type": "markdown", - "id": "4", + "id": "5", "metadata": {}, "source": [ "#### The Flooding Event" @@ -61,7 +69,7 @@ }, { "cell_type": "markdown", - "id": "5", + "id": "6", "metadata": {}, "source": [ "On October 29, 2024, the city of Valencia (Spain) was hit by catastrophic flooding caused by intense storms, leaving over 230 deaths and billions in damages. This disaster was part of Europe’s worst flood year in over a decade, with hundreds of thousands affected continent-wide. Such events highlight the urgent need for reliable flood monitoring to support **emergency response**, damage assessment and long-term resilience planning.\n", @@ -85,7 +93,7 @@ }, { "cell_type": "markdown", - "id": "6", + "id": "7", "metadata": {}, "source": [ "#### What we will learn" @@ -93,7 +101,7 @@ }, { "cell_type": "markdown", - "id": "7", + "id": "8", "metadata": {}, "source": [ "- 🌊 How to create a workflow to map flood events.\n", @@ -103,7 +111,7 @@ }, { "cell_type": "markdown", - "id": "8", + "id": "9", "metadata": {}, "source": [ "
" @@ -111,7 +119,7 @@ }, { "cell_type": "markdown", - "id": "9", + "id": "10", "metadata": {}, "source": [ "#### Import libraries" @@ -120,7 +128,7 @@ { "cell_type": "code", "execution_count": null, - "id": "10", + "id": "11", "metadata": {}, "outputs": [], "source": [ @@ -138,7 +146,7 @@ }, { "cell_type": "markdown", - "id": "11", + "id": "12", "metadata": {}, "source": [ "
" @@ -146,7 +154,7 @@ }, { "cell_type": "markdown", - "id": "12", + "id": "13", "metadata": { "vscode": { "languageId": "raw" @@ -158,7 +166,7 @@ }, { "cell_type": "markdown", - "id": "13", + "id": "14", "metadata": {}, "source": [ "To search and load the data needed for the analysis, we will follow the processes we presented in [Sentinel-1 GRD structure tutorial](./../02_about_eopf_zarr/22_zarr_structure_S1GRD.ipynb) and [S1 basic operations tutorial](./../02_about_eopf_zarr/23_S1_basic_operations.ipynb).\n", @@ -169,7 +177,7 @@ }, { "cell_type": "markdown", - "id": "14", + "id": "15", "metadata": {}, "source": [ "### Loading the datatree" @@ -177,7 +185,7 @@ }, { "cell_type": "markdown", - "id": "15", + "id": "16", "metadata": {}, "source": [ "The list below shows the names of the products we will use for the flood mapping and time series analysis.
\n", @@ -187,7 +195,7 @@ { "cell_type": "code", "execution_count": null, - "id": "16", + "id": "17", "metadata": {}, "outputs": [], "source": [ @@ -214,7 +222,7 @@ }, { "cell_type": "markdown", - "id": "17", + "id": "18", "metadata": {}, "source": [ "Next, we will load all `zarr` datasets as xarray.Datatrees. Here **we are not reading** the entire dataset from the store; but, creating a set of references to the data, which enables us to access it efficiently later in the analysis." @@ -223,7 +231,7 @@ { "cell_type": "code", "execution_count": null, - "id": "18", + "id": "19", "metadata": {}, "outputs": [], "source": [ @@ -242,7 +250,7 @@ }, { "cell_type": "markdown", - "id": "19", + "id": "20", "metadata": {}, "source": [ "Each element inside the `datatree` list is a datatree and corresponds to a Sentinel-1 GRD scene datatree present on the list above." @@ -251,7 +259,7 @@ { "cell_type": "code", "execution_count": null, - "id": "20", + "id": "21", "metadata": {}, "outputs": [], "source": [ @@ -261,7 +269,7 @@ }, { "cell_type": "markdown", - "id": "21", + "id": "22", "metadata": {}, "source": [ "### Defining variables" @@ -270,7 +278,7 @@ { "cell_type": "code", "execution_count": null, - "id": "22", + "id": "23", "metadata": {}, "outputs": [], "source": [ @@ -280,7 +288,7 @@ }, { "cell_type": "markdown", - "id": "23", + "id": "24", "metadata": {}, "source": [ "If we run the following commented out code line we will be able to see how each datatree is organized within its groups and subgroups (as explained in this [section](./../02_about_eopf_zarr/22_zarr_structure_S1GRD.ipynb)). From this datatree, we took the groups and subgroups constant `ID` numbers used to open specific groups and variables such as:\n", @@ -294,7 +302,7 @@ { "cell_type": "code", "execution_count": null, - "id": "24", + "id": "25", "metadata": {}, "outputs": [], "source": [ @@ -304,7 +312,7 @@ { "cell_type": "code", "execution_count": null, - "id": "25", + "id": "26", "metadata": {}, "outputs": [], "source": [ @@ -315,7 +323,7 @@ { "cell_type": "code", "execution_count": null, - "id": "26", + "id": "27", "metadata": {}, "outputs": [], "source": [ @@ -327,7 +335,7 @@ }, { "cell_type": "markdown", - "id": "27", + "id": "28", "metadata": {}, "source": [ "We now define the thresholds that will be used for the flood mapping analysis. These values are not fixed and they can be calibrated and adjusted to achieve a better fit for different regions or flood events.
\n", @@ -338,7 +346,7 @@ { "cell_type": "code", "execution_count": null, - "id": "28", + "id": "29", "metadata": {}, "outputs": [], "source": [ @@ -347,7 +355,7 @@ }, { "cell_type": "markdown", - "id": "29", + "id": "30", "metadata": {}, "source": [ "It is interesting to study the flood event over a specific point within the area of interest.
\n", @@ -357,7 +365,7 @@ { "cell_type": "code", "execution_count": null, - "id": "30", + "id": "31", "metadata": {}, "outputs": [], "source": [ @@ -370,7 +378,7 @@ }, { "cell_type": "markdown", - "id": "31", + "id": "32", "metadata": {}, "source": [ "## Extracting information from the `.zarr`" @@ -378,7 +386,7 @@ }, { "cell_type": "markdown", - "id": "32", + "id": "33", "metadata": {}, "source": [ "As explained in the [S1 basic operations tutorial](23_S1_basic_operations.ipynb), we will perform over all the selected data the following operations:\n", @@ -391,7 +399,7 @@ }, { "cell_type": "markdown", - "id": "33", + "id": "34", "metadata": {}, "source": [ "### Slicing and decimating GRD variable" @@ -399,7 +407,7 @@ }, { "cell_type": "markdown", - "id": "34", + "id": "35", "metadata": {}, "source": [ "To begin with, we access all our `.zarr` items `measurements` groups by creating a list storing all of them." @@ -408,7 +416,7 @@ { "cell_type": "code", "execution_count": null, - "id": "35", + "id": "36", "metadata": {}, "outputs": [], "source": [ @@ -421,7 +429,7 @@ }, { "cell_type": "markdown", - "id": "36", + "id": "37", "metadata": {}, "source": [ "We continue by slicing `grd`'s data to focus on a specific area (Valencia). \n", @@ -432,7 +440,7 @@ { "cell_type": "code", "execution_count": null, - "id": "37", + "id": "38", "metadata": {}, "outputs": [], "source": [ @@ -446,7 +454,7 @@ { "cell_type": "code", "execution_count": null, - "id": "38", + "id": "39", "metadata": {}, "outputs": [], "source": [ @@ -456,7 +464,7 @@ }, { "cell_type": "markdown", - "id": "39", + "id": "40", "metadata": {}, "source": [ "We use GCPs to create a spatial mask based on our bounding box, then use that mask to find the corresponding slice in SAR geometry (azimuth_time and ground_range). This approach ensures that each pixel represents the same ground location across different products, which is crucial for time series analysis." @@ -465,7 +473,7 @@ { "cell_type": "code", "execution_count": null, - "id": "40", + "id": "41", "metadata": {}, "outputs": [], "source": [ @@ -543,7 +551,7 @@ { "cell_type": "code", "execution_count": null, - "id": "41", + "id": "42", "metadata": {}, "outputs": [], "source": [ @@ -555,7 +563,7 @@ }, { "cell_type": "markdown", - "id": "42", + "id": "43", "metadata": {}, "source": [ "### Assigning latitude and longitude coordinates" @@ -563,7 +571,7 @@ }, { "cell_type": "markdown", - "id": "43", + "id": "44", "metadata": {}, "source": [ "We will execute the following step to assign latitude and longitude coordinates to our datasets:\n", @@ -576,7 +584,7 @@ { "cell_type": "code", "execution_count": null, - "id": "44", + "id": "45", "metadata": {}, "outputs": [], "source": [ @@ -590,7 +598,7 @@ { "cell_type": "code", "execution_count": null, - "id": "45", + "id": "46", "metadata": {}, "outputs": [], "source": [ @@ -602,7 +610,7 @@ }, { "cell_type": "markdown", - "id": "46", + "id": "47", "metadata": {}, "source": [ "### Computing backscatter" @@ -610,7 +618,7 @@ }, { "cell_type": "markdown", - "id": "47", + "id": "48", "metadata": {}, "source": [ "Again, the following steps are just recreating what was done before, but this time over more datasets. For further detailed information, take a look at this [chapter](./../02_about_eopf_zarr/23_S1_basic_operations.ipynb).\n", @@ -623,7 +631,7 @@ { "cell_type": "code", "execution_count": null, - "id": "48", + "id": "49", "metadata": {}, "outputs": [], "source": [ @@ -636,7 +644,7 @@ { "cell_type": "code", "execution_count": null, - "id": "49", + "id": "50", "metadata": {}, "outputs": [], "source": [ @@ -697,7 +705,7 @@ { "cell_type": "code", "execution_count": null, - "id": "50", + "id": "51", "metadata": {}, "outputs": [], "source": [ @@ -709,7 +717,7 @@ }, { "cell_type": "markdown", - "id": "51", + "id": "52", "metadata": {}, "source": [ "In case we prefer to keep it more simple and calculate the intensity values with `xarray_sentinel` we can just uncomment the following cell and run it." @@ -718,7 +726,7 @@ { "cell_type": "code", "execution_count": null, - "id": "52", + "id": "53", "metadata": {}, "outputs": [], "source": [ @@ -734,7 +742,7 @@ { "cell_type": "code", "execution_count": null, - "id": "53", + "id": "54", "metadata": {}, "outputs": [], "source": [ @@ -746,7 +754,7 @@ }, { "cell_type": "markdown", - "id": "54", + "id": "55", "metadata": {}, "source": [ "### Create a datacube to prepare for time series analysis" @@ -754,7 +762,7 @@ }, { "cell_type": "markdown", - "id": "55", + "id": "56", "metadata": {}, "source": [ "Since we are performing a time series with `.zarr`, instead of analysing individual items stored in a list, we can create a combined dataset, containing all the data, stacked together by a new dimension `time`. Through the stacking, we are building a three-dimensional datacube.\n", @@ -765,7 +773,7 @@ { "cell_type": "code", "execution_count": null, - "id": "56", + "id": "57", "metadata": {}, "outputs": [], "source": [ @@ -778,7 +786,7 @@ }, { "cell_type": "markdown", - "id": "57", + "id": "58", "metadata": {}, "source": [ "### Geocoding onto regular grid" @@ -786,7 +794,7 @@ }, { "cell_type": "markdown", - "id": "58", + "id": "59", "metadata": {}, "source": [ "In order to stack data into a time series datacube, we need to ensure that pixels across different acquisition dates correspond to the exact same ground locations. Since each Sentinel-1 GRD product has lat/lon coordinates on an irregular grid (due to SAR geometry), we need to geocode all products onto a common regular grid.\n", @@ -802,7 +810,7 @@ { "cell_type": "code", "execution_count": null, - "id": "59", + "id": "60", "metadata": {}, "outputs": [], "source": [ @@ -907,7 +915,7 @@ }, { "cell_type": "markdown", - "id": "60", + "id": "61", "metadata": {}, "source": [ "Now we geocode all products onto the common regular grid and stack them into a time series datacube. Each pixel now corresponds to the exact same ground location across all acquisition dates, enabling accurate time series analysis." @@ -916,7 +924,7 @@ { "cell_type": "code", "execution_count": null, - "id": "61", + "id": "62", "metadata": {}, "outputs": [], "source": [ @@ -934,7 +942,7 @@ }, { "cell_type": "markdown", - "id": "62", + "id": "63", "metadata": { "vscode": { "languageId": "raw" @@ -946,7 +954,7 @@ }, { "cell_type": "markdown", - "id": "63", + "id": "64", "metadata": {}, "source": [ "The last step is to perform the time series and flood mapping analysis." @@ -954,7 +962,7 @@ }, { "cell_type": "markdown", - "id": "64", + "id": "65", "metadata": {}, "source": [ "### Simple visualisation of all datasets selected" @@ -962,7 +970,7 @@ }, { "cell_type": "markdown", - "id": "65", + "id": "66", "metadata": {}, "source": [ "First, we can plot all the datasets simply to create a visualisation of the flood. In addition to these plots, we are also plotting a chosen latitude and longitude point (as defined at beginning of this tutorial). The coordinate serves as a measure of comparison between all the datasets and from within different analysis methods.\n", @@ -975,7 +983,7 @@ { "cell_type": "code", "execution_count": null, - "id": "66", + "id": "67", "metadata": {}, "outputs": [], "source": [ @@ -1004,7 +1012,7 @@ }, { "cell_type": "markdown", - "id": "67", + "id": "68", "metadata": {}, "source": [ "### Create a flood map based on threshold values" @@ -1012,7 +1020,7 @@ }, { "cell_type": "markdown", - "id": "68", + "id": "69", "metadata": {}, "source": [ "It is known through [literature](https://www.researchgate.net/figure/VV-and-VH-threshold-statistics-1-obtained-via-graphical-interpretation-and-2_tbl4_360412209) and other [sources](https://mbonnema.github.io/GoogleEarthEngine/07-SAR-Water-Classification/?utm_source=chatgpt.com) that water appears as darker pixels, typically with values lower than **-15 dB**. This is a very good method for identifying water because separating the pixels within this threshold value will give us almost a `True` and `False` map for pixels which are greater or smaller than the defined threshold.\n", @@ -1025,7 +1033,7 @@ { "cell_type": "code", "execution_count": null, - "id": "69", + "id": "70", "metadata": {}, "outputs": [], "source": [ @@ -1052,7 +1060,7 @@ }, { "cell_type": "markdown", - "id": "70", + "id": "71", "metadata": {}, "source": [ "### Create a map showing differences between two images" @@ -1060,7 +1068,7 @@ }, { "cell_type": "markdown", - "id": "71", + "id": "72", "metadata": {}, "source": [ "Knowing the exact flood date, which we have, and from the images plotted previously, we can easily see that the second image is the one right before the flood event and that the third image is the one directly after it. These two images show significant differences in the flooded areas and backscatter values, ranging from **-5 dB** (in the image before the event) to **-20 dB** (in the image directly after the event).\n", @@ -1075,7 +1083,7 @@ { "cell_type": "code", "execution_count": null, - "id": "72", + "id": "73", "metadata": {}, "outputs": [], "source": [ @@ -1087,7 +1095,7 @@ }, { "cell_type": "markdown", - "id": "73", + "id": "74", "metadata": {}, "source": [ "### Create a time-series plot of one location within the flood" @@ -1095,7 +1103,7 @@ }, { "cell_type": "markdown", - "id": "74", + "id": "75", "metadata": {}, "source": [ "Taking advantage of the data cube we have created over a new `time` dimension, it is much easier to plot the data over this new dimension, as in a time series plot.
\n", @@ -1108,7 +1116,7 @@ { "cell_type": "code", "execution_count": null, - "id": "75", + "id": "76", "metadata": {}, "outputs": [], "source": [ @@ -1133,7 +1141,7 @@ }, { "cell_type": "markdown", - "id": "76", + "id": "77", "metadata": {}, "source": [ "Now we can plot the data cube, showing the backscatter intensity over the target point we defined earlier. Since the datasets are stacked along the time dimension, it becomes much easier to plot the evolution of water backscatter at a specific location. This provides an effective way to monitor the flooding status at that point. \n", @@ -1144,7 +1152,7 @@ { "cell_type": "code", "execution_count": null, - "id": "77", + "id": "78", "metadata": {}, "outputs": [], "source": [ @@ -1167,7 +1175,7 @@ }, { "cell_type": "markdown", - "id": "78", + "id": "79", "metadata": {}, "source": [ "
" @@ -1175,7 +1183,7 @@ }, { "cell_type": "markdown", - "id": "79", + "id": "80", "metadata": {}, "source": [ "## Challenges" @@ -1183,7 +1191,7 @@ }, { "cell_type": "markdown", - "id": "80", + "id": "81", "metadata": {}, "source": [ "While using the optimised `.zarr` format saves a lot of time and makes creating workflows relatively simple and achievable, there are still a few challenges to handle and to keep in mind:\n", @@ -1199,7 +1207,7 @@ }, { "cell_type": "markdown", - "id": "81", + "id": "82", "metadata": {}, "source": [ "
" @@ -1207,7 +1215,7 @@ }, { "cell_type": "markdown", - "id": "82", + "id": "83", "metadata": {}, "source": [ "## Conclusion" @@ -1215,7 +1223,7 @@ }, { "cell_type": "markdown", - "id": "83", + "id": "84", "metadata": {}, "source": [ "The `.zarr` format is particularly well suited for hazard analysis because it enables multiple datasets to be combined into a single structure, either as a data cube or as a list of datatrees. This makes it ideal for rapid, multi-temporal, and multi-spatial monitoring. Unlike the `.SAFE` format, which required downloading entire products, `.zarr` only loads the specific groups needed, while the rest is accessed on the fly. As a result, both data handling and subsequent operations are much faster and more efficient. The `.load()` helps a lot on these situations.\n", @@ -1225,7 +1233,7 @@ }, { "cell_type": "markdown", - "id": "84", + "id": "85", "metadata": {}, "source": [ "
" @@ -1233,7 +1241,7 @@ }, { "cell_type": "markdown", - "id": "85", + "id": "86", "metadata": {}, "source": [ "## What's next?" @@ -1241,7 +1249,7 @@ }, { "cell_type": "markdown", - "id": "86", + "id": "87", "metadata": {}, "source": [ "In the following [notebook](./65_create_overviews.ipynb) we will explore a new approach to the **EOPF Zarr** format.
\n", diff --git a/06_eopf_zarr_in_action/65_create_overviews.ipynb b/06_eopf_zarr_in_action/65_create_overviews.ipynb index 2ff7b00..fcb828a 100644 --- a/06_eopf_zarr_in_action/65_create_overviews.ipynb +++ b/06_eopf_zarr_in_action/65_create_overviews.ipynb @@ -33,6 +33,14 @@ "cell_type": "markdown", "id": "2", "metadata": {}, + "source": [ + "**By:** *[@christophenoel](https://github.com/christophenoel)*" + ] + }, + { + "cell_type": "markdown", + "id": "3", + "metadata": {}, "source": [ "### Introduction\n", "\n", @@ -65,7 +73,7 @@ }, { "cell_type": "markdown", - "id": "3", + "id": "4", "metadata": {}, "source": [ "#### Import libraries" @@ -74,7 +82,7 @@ { "cell_type": "code", "execution_count": null, - "id": "4", + "id": "5", "metadata": {}, "outputs": [], "source": [ @@ -85,7 +93,7 @@ }, { "cell_type": "markdown", - "id": "5", + "id": "6", "metadata": {}, "source": [ "We prepare our credentials for S3 access:" @@ -94,7 +102,7 @@ { "cell_type": "code", "execution_count": null, - "id": "6", + "id": "7", "metadata": {}, "outputs": [], "source": [ @@ -120,7 +128,7 @@ }, { "cell_type": "markdown", - "id": "7", + "id": "8", "metadata": {}, "source": [ "## Copy Remote Dataset to Local Storage\n", @@ -133,7 +141,7 @@ }, { "cell_type": "markdown", - "id": "8", + "id": "9", "metadata": {}, "source": [ "To make sure that we use a convenient scene, we select a source URL from the STAC catalogue." @@ -142,7 +150,7 @@ { "cell_type": "code", "execution_count": null, - "id": "9", + "id": "10", "metadata": {}, "outputs": [], "source": [ @@ -153,7 +161,7 @@ }, { "cell_type": "markdown", - "id": "10", + "id": "11", "metadata": {}, "source": [ "As a first step, we download the remote Zarr dataset and saves it as a Zarr copy ready on a S3 bucket for exploration.\n", @@ -163,7 +171,7 @@ { "cell_type": "code", "execution_count": null, - "id": "11", + "id": "12", "metadata": {}, "outputs": [], "source": [ @@ -184,7 +192,7 @@ }, { "cell_type": "markdown", - "id": "12", + "id": "13", "metadata": {}, "source": [ "Then, we can access the dataset from the S3 bucket and look inside the group that contains the 10-metre reflectance data to understand which variables, dimensions, and coordinates it contains." @@ -193,7 +201,7 @@ { "cell_type": "code", "execution_count": null, - "id": "13", + "id": "14", "metadata": {}, "outputs": [], "source": [ @@ -208,7 +216,7 @@ }, { "cell_type": "markdown", - "id": "14", + "id": "15", "metadata": {}, "source": [ "## Compute Overviews (In-Memory)\n", @@ -226,7 +234,7 @@ }, { "cell_type": "markdown", - "id": "15", + "id": "16", "metadata": {}, "source": [ "In this step, we identify the spatial dimensions and variables in the dataset and define the scale levels that will be used to generate lower-resolution overviews." @@ -235,7 +243,7 @@ { "cell_type": "code", "execution_count": null, - "id": "16", + "id": "17", "metadata": {}, "outputs": [], "source": [ @@ -249,7 +257,7 @@ }, { "cell_type": "markdown", - "id": "17", + "id": "18", "metadata": {}, "source": [ "Accessing such information allows us to generate a series of lower-resolution overview datasets directly in memory.\n", @@ -260,7 +268,7 @@ { "cell_type": "code", "execution_count": null, - "id": "18", + "id": "19", "metadata": {}, "outputs": [], "source": [ @@ -278,7 +286,7 @@ }, { "cell_type": "markdown", - "id": "19", + "id": "20", "metadata": {}, "source": [ "## Attach Multiscales Metadata\n", @@ -295,7 +303,7 @@ }, { "cell_type": "markdown", - "id": "20", + "id": "21", "metadata": {}, "source": [ "Here we prepare the information needed to describe the overview hierarchy in the GeoZarr metadata. We set `overview_path` to indicate where the overview groups will be stored, record the `resampling_method(\"average\")` used to create them, and compute the base spatial resolutions (`x_res` and `y_res`) from the coordinate spacing." @@ -304,7 +312,7 @@ { "cell_type": "code", "execution_count": null, - "id": "21", + "id": "22", "metadata": {}, "outputs": [], "source": [ @@ -316,7 +324,7 @@ }, { "cell_type": "markdown", - "id": "22", + "id": "23", "metadata": {}, "source": [ "Now, we build the multiscales layout metadata that describes how all overview levels relate to the base dataset.\n", @@ -329,7 +337,7 @@ { "cell_type": "code", "execution_count": null, - "id": "23", + "id": "24", "metadata": {}, "outputs": [], "source": [ @@ -347,7 +355,7 @@ { "cell_type": "code", "execution_count": null, - "id": "24", + "id": "25", "metadata": {}, "outputs": [], "source": [ @@ -359,7 +367,7 @@ }, { "cell_type": "markdown", - "id": "25", + "id": "26", "metadata": {}, "source": [ "### Write Overviews to Local Zarr Store\n", @@ -407,7 +415,7 @@ { "cell_type": "code", "execution_count": null, - "id": "26", + "id": "27", "metadata": {}, "outputs": [], "source": [ @@ -453,7 +461,7 @@ }, { "cell_type": "markdown", - "id": "27", + "id": "28", "metadata": {}, "source": [ "
" @@ -461,7 +469,7 @@ }, { "cell_type": "markdown", - "id": "28", + "id": "29", "metadata": {}, "source": [ "## 💪 Now it is your turn\n", diff --git a/06_eopf_zarr_in_action/66_use_overviews.ipynb b/06_eopf_zarr_in_action/66_use_overviews.ipynb index 880d952..bc1ec43 100644 --- a/06_eopf_zarr_in_action/66_use_overviews.ipynb +++ b/06_eopf_zarr_in_action/66_use_overviews.ipynb @@ -34,6 +34,18 @@ { "cell_type": "markdown", "id": "2", + "metadata": { + "vscode": { + "languageId": "raw" + } + }, + "source": [ + "**By:** *[@christophenoel](https://github.com/christophenoel)*" + ] + }, + { + "cell_type": "markdown", + "id": "3", "metadata": {}, "source": [ "### Introduction" @@ -41,7 +53,7 @@ }, { "cell_type": "markdown", - "id": "3", + "id": "4", "metadata": {}, "source": [ "Overviews enable efficient visualisation by providing progressively coarser representations of the data, allowing us to quickly navigate and explore large satellite images without loading the full-resolution dataset every time.\n", @@ -56,7 +68,7 @@ }, { "cell_type": "markdown", - "id": "4", + "id": "5", "metadata": {}, "source": [ "### What we will learn" @@ -64,7 +76,7 @@ }, { "cell_type": "markdown", - "id": "5", + "id": "6", "metadata": {}, "source": [ "* 📊 How to parse and inspect the multiscales metadata structure?\n", @@ -76,7 +88,7 @@ }, { "cell_type": "markdown", - "id": "6", + "id": "7", "metadata": {}, "source": [ "### Prerequisites" @@ -84,7 +96,7 @@ }, { "cell_type": "markdown", - "id": "7", + "id": "8", "metadata": {}, "source": [ "To successfully run this notebook, make sure to have:\n", @@ -102,7 +114,7 @@ }, { "cell_type": "markdown", - "id": "8", + "id": "9", "metadata": {}, "source": [ "
" @@ -110,7 +122,7 @@ }, { "cell_type": "markdown", - "id": "9", + "id": "10", "metadata": {}, "source": [ "#### Import libraries\n", @@ -121,7 +133,7 @@ { "cell_type": "code", "execution_count": null, - "id": "10", + "id": "11", "metadata": {}, "outputs": [], "source": [ @@ -131,7 +143,7 @@ }, { "cell_type": "markdown", - "id": "11", + "id": "12", "metadata": {}, "source": [ "We prepare our credentials for S3 access:" @@ -140,7 +152,7 @@ { "cell_type": "code", "execution_count": null, - "id": "12", + "id": "13", "metadata": {}, "outputs": [], "source": [ @@ -160,7 +172,7 @@ }, { "cell_type": "markdown", - "id": "13", + "id": "14", "metadata": {}, "source": [ "## Understanding the Overview Hierarchy" @@ -168,7 +180,7 @@ }, { "cell_type": "markdown", - "id": "14", + "id": "15", "metadata": {}, "source": [ "The first step in working with overviews is to understand their structure. In the previous tutorial, we created a multiscale pyramid with several levels (L0 through L7), where each level represents the data at a different resolution.\n", @@ -182,7 +194,7 @@ }, { "cell_type": "markdown", - "id": "15", + "id": "16", "metadata": {}, "source": [ "### Opening the dataset" @@ -190,7 +202,7 @@ }, { "cell_type": "markdown", - "id": "16", + "id": "17", "metadata": {}, "source": [ "We start by opening the dataset at the group level where our overviews are stored on S3. This is the same path we used in the previous tutorial when creating the overviews (`measurements/reflectance/r10m`)." @@ -199,7 +211,7 @@ { "cell_type": "code", "execution_count": null, - "id": "17", + "id": "18", "metadata": {}, "outputs": [], "source": [ @@ -214,7 +226,7 @@ }, { "cell_type": "markdown", - "id": "18", + "id": "19", "metadata": {}, "source": [ "The output confirms that our dataset has dimensions of 10,980 × 10,980 pixels and contains four spectral bands (b02, b03, b04, b08), along with the multiscales metadata we created in the previous tutorial." @@ -222,7 +234,7 @@ }, { "cell_type": "markdown", - "id": "19", + "id": "20", "metadata": {}, "source": [ "### Parsing the multiscales metadata" @@ -230,7 +242,7 @@ }, { "cell_type": "markdown", - "id": "20", + "id": "21", "metadata": {}, "source": [ "The `multiscales` attribute contains all the information about our overview pyramid. Let's parse it into a table format using pandas to make it easier to understand.\n", @@ -248,7 +260,7 @@ { "cell_type": "code", "execution_count": null, - "id": "21", + "id": "22", "metadata": {}, "outputs": [], "source": [ @@ -261,7 +273,7 @@ }, { "cell_type": "markdown", - "id": "22", + "id": "23", "metadata": {}, "source": [ "Notice the hierarchical structure:\n", @@ -275,7 +287,7 @@ }, { "cell_type": "markdown", - "id": "23", + "id": "24", "metadata": {}, "source": [ "## Loading Overview Levels" @@ -283,7 +295,7 @@ }, { "cell_type": "markdown", - "id": "24", + "id": "25", "metadata": {}, "source": [ "Now that we understand the structure from the metadata, let's load each overview level into memory. We'll iterate through the layout array from the multiscales metadata and open each level as a separate xarray Dataset.\n", @@ -300,7 +312,7 @@ { "cell_type": "code", "execution_count": null, - "id": "25", + "id": "26", "metadata": {}, "outputs": [], "source": [ @@ -321,7 +333,7 @@ }, { "cell_type": "markdown", - "id": "26", + "id": "27", "metadata": {}, "source": [ "Observe how the dimensions decrease at each level:\n", @@ -336,7 +348,7 @@ }, { "cell_type": "markdown", - "id": "27", + "id": "28", "metadata": {}, "source": [ "## Visualization of Overviews" @@ -344,7 +356,7 @@ }, { "cell_type": "markdown", - "id": "28", + "id": "29", "metadata": {}, "source": [ "### Creating RGB Composites\n", @@ -360,7 +372,7 @@ }, { "cell_type": "markdown", - "id": "29", + "id": "30", "metadata": {}, "source": [ "### Percentile stretching for contrast enhancement" @@ -368,7 +380,7 @@ }, { "cell_type": "markdown", - "id": "30", + "id": "31", "metadata": {}, "source": [ "The `normalise()` helper function below performs **percentile-based contrast stretching**, a standard technique in remote sensing visualisation. It works by:\n", @@ -384,7 +396,7 @@ { "cell_type": "code", "execution_count": null, - "id": "31", + "id": "32", "metadata": {}, "outputs": [], "source": [ @@ -402,7 +414,7 @@ }, { "cell_type": "markdown", - "id": "32", + "id": "33", "metadata": {}, "source": [ "### Interactive visualization with zoom levels" @@ -410,7 +422,7 @@ }, { "cell_type": "markdown", - "id": "33", + "id": "34", "metadata": {}, "source": [ "Now let's create an **interactive visualisation** that demonstrates the power of overviews. The widget below allows you to:\n", @@ -425,7 +437,7 @@ { "cell_type": "code", "execution_count": null, - "id": "34", + "id": "35", "metadata": {}, "outputs": [], "source": [ @@ -445,7 +457,7 @@ }, { "cell_type": "markdown", - "id": "35", + "id": "36", "metadata": {}, "source": [ "## Comparing Multiple Resolution Levels" @@ -453,7 +465,7 @@ }, { "cell_type": "markdown", - "id": "36", + "id": "37", "metadata": {}, "source": [ "To better understand the effect of downsampling, let's compare **multiple overview levels side by side**. This visual comparison helps us see:\n", @@ -468,7 +480,7 @@ { "cell_type": "code", "execution_count": null, - "id": "37", + "id": "38", "metadata": {}, "outputs": [], "source": [ @@ -488,7 +500,7 @@ }, { "cell_type": "markdown", - "id": "38", + "id": "39", "metadata": {}, "source": [ "Notice the progressive loss of detail:\n", @@ -502,7 +514,7 @@ }, { "cell_type": "markdown", - "id": "39", + "id": "40", "metadata": {}, "source": [ "## Interactive Web Map with Automatic Level Selection" @@ -510,7 +522,7 @@ }, { "cell_type": "markdown", - "id": "40", + "id": "41", "metadata": {}, "source": [ "Now let's put everything together by creating a **professional-like interactive web map** using ipyleaflet. This demonstrates a real-world application of overviews for geospatial visualisation.\n", @@ -534,7 +546,7 @@ { "cell_type": "code", "execution_count": null, - "id": "41", + "id": "42", "metadata": {}, "outputs": [], "source": [ @@ -567,7 +579,7 @@ }, { "cell_type": "markdown", - "id": "42", + "id": "43", "metadata": {}, "source": [ "**How to use the interactive map:**\n", @@ -588,7 +600,7 @@ }, { "cell_type": "markdown", - "id": "43", + "id": "44", "metadata": {}, "source": [ "
" @@ -596,7 +608,7 @@ }, { "cell_type": "markdown", - "id": "44", + "id": "45", "metadata": {}, "source": [ "## 💪 Now it is your turn" @@ -604,7 +616,7 @@ }, { "cell_type": "markdown", - "id": "45", + "id": "46", "metadata": {}, "source": [ "Now that you've learned how to visualise and interact with multiscale overviews, try these exercises to deepen your understanding:\n", @@ -643,7 +655,7 @@ }, { "cell_type": "markdown", - "id": "46", + "id": "47", "metadata": {}, "source": [ "Now that you've mastered creating and visualising GeoZarr overviews through tutorials 65 and 66, you can:\n", @@ -661,7 +673,7 @@ }, { "cell_type": "markdown", - "id": "47", + "id": "48", "metadata": {}, "source": [ "
" @@ -669,7 +681,7 @@ }, { "cell_type": "markdown", - "id": "48", + "id": "49", "metadata": {}, "source": [ "## Conclusion" @@ -677,7 +689,7 @@ }, { "cell_type": "markdown", - "id": "49", + "id": "50", "metadata": {}, "source": [ "In this tutorial, we've learned how to **visualise and interact with multiscale overview pyramids** for GeoZarr datasets. We covered:\n", @@ -700,7 +712,7 @@ }, { "cell_type": "markdown", - "id": "50", + "id": "51", "metadata": {}, "source": [ "
" @@ -708,7 +720,7 @@ }, { "cell_type": "markdown", - "id": "51", + "id": "52", "metadata": {}, "source": [ "## What's next?" @@ -716,7 +728,7 @@ }, { "cell_type": "markdown", - "id": "52", + "id": "53", "metadata": {}, "source": [ "In the following [notebook](./67_reservoir_surface_monitoring.ipynb), we will go South and showcase the application of Sentinel 2 L1C data, focusing on Water Reservoirs.
\n", diff --git a/06_eopf_zarr_in_action/67_reservoir_surface_monitoring.ipynb b/06_eopf_zarr_in_action/67_reservoir_surface_monitoring.ipynb index 62d325c..6e4c3ca 100644 --- a/06_eopf_zarr_in_action/67_reservoir_surface_monitoring.ipynb +++ b/06_eopf_zarr_in_action/67_reservoir_surface_monitoring.ipynb @@ -38,12 +38,20 @@ "id": "2", "metadata": {}, "source": [ - "### Introduction" + "**By:** *[@atsiokanos](https://github.com/atsiokanos)*" ] }, { "cell_type": "markdown", "id": "3", + "metadata": {}, + "source": [ + "### Introduction" + ] + }, + { + "cell_type": "markdown", + "id": "4", "metadata": { "vscode": { "languageId": "raw" @@ -61,7 +69,7 @@ }, { "cell_type": "markdown", - "id": "4", + "id": "5", "metadata": { "vscode": { "languageId": "raw" @@ -73,7 +81,7 @@ }, { "cell_type": "markdown", - "id": "5", + "id": "6", "metadata": {}, "source": [ "- How to retrieve Sentinel-2 L1C data from the EOPF STAC Catalog?\n", @@ -91,7 +99,7 @@ }, { "cell_type": "markdown", - "id": "6", + "id": "7", "metadata": {}, "source": [ "
" @@ -99,7 +107,7 @@ }, { "cell_type": "markdown", - "id": "7", + "id": "8", "metadata": {}, "source": [ "#### Import libraries" @@ -108,7 +116,7 @@ { "cell_type": "code", "execution_count": null, - "id": "8", + "id": "9", "metadata": {}, "outputs": [], "source": [ @@ -135,7 +143,7 @@ }, { "cell_type": "markdown", - "id": "9", + "id": "10", "metadata": {}, "source": [ "#### Helper functions" @@ -143,7 +151,7 @@ }, { "cell_type": "markdown", - "id": "10", + "id": "11", "metadata": {}, "source": [ "This notebook utilizes a set of helper functions from the Python file [reservoir_surface_monitoring_utils.py](./reservoir_surface_monitoring_utils.py) to handle key tasks, including loading and preprocessing the datasets, computing water masks using the GWW algorithm, and calculating reservoir areas. The main logic of each function is outlined in the notebook prior to its use, and the full code can be inspected in `reservoir_surface_monitoring_utils.py`.\n" @@ -152,7 +160,7 @@ { "cell_type": "code", "execution_count": null, - "id": "11", + "id": "12", "metadata": {}, "outputs": [], "source": [ @@ -168,7 +176,7 @@ }, { "cell_type": "markdown", - "id": "12", + "id": "13", "metadata": {}, "source": [ "
" @@ -176,7 +184,7 @@ }, { "cell_type": "markdown", - "id": "13", + "id": "14", "metadata": {}, "source": [ "## Section 1 - Data Preparation\n" @@ -184,7 +192,7 @@ }, { "cell_type": "markdown", - "id": "14", + "id": "15", "metadata": { "vscode": { "languageId": "raw" @@ -200,7 +208,7 @@ }, { "cell_type": "markdown", - "id": "15", + "id": "16", "metadata": {}, "source": [ "### 1.1 - Initial settings and data connection" @@ -208,7 +216,7 @@ }, { "cell_type": "markdown", - "id": "16", + "id": "17", "metadata": {}, "source": [ "Initiate dask client for parallel processing" @@ -217,7 +225,7 @@ { "cell_type": "code", "execution_count": null, - "id": "17", + "id": "18", "metadata": {}, "outputs": [], "source": [ @@ -229,7 +237,7 @@ }, { "cell_type": "markdown", - "id": "18", + "id": "19", "metadata": {}, "source": [ "Define the settings and retrieve Sentinel-2 L1C data from the EOPF STAC Catalog. In this example, we will use the bounding box covering Mita Hills, apply a 30% cloud cover cutoff for the query, and select a period from August 2022 to December 2024. This relatively long period ensures a sufficient number of scenes, covering both dry and wet conditions." @@ -238,7 +246,7 @@ { "cell_type": "code", "execution_count": null, - "id": "19", + "id": "20", "metadata": {}, "outputs": [], "source": [ @@ -287,7 +295,7 @@ }, { "cell_type": "markdown", - "id": "20", + "id": "21", "metadata": {}, "source": [ "### 1.2 - Load and preprocess data" @@ -295,7 +303,7 @@ }, { "cell_type": "markdown", - "id": "21", + "id": "22", "metadata": {}, "source": [ "First we load the datasets using the `load_datatrees` function. Note that in the following cell loading is done lazily using `dask.delayed`, which builds a Dask task graph. Execution of this graph triggers **parallel processing** of all dataset loading tasks, improving efficiency and reducing memory usage.\n" @@ -304,7 +312,7 @@ { "cell_type": "code", "execution_count": null, - "id": "22", + "id": "23", "metadata": {}, "outputs": [], "source": [ @@ -315,7 +323,7 @@ }, { "cell_type": "markdown", - "id": "23", + "id": "24", "metadata": {}, "source": [ "The obtained datatrees contain the full set of source bands (at all available resolutions) as well as useful metadata for every retrieved scene. Some preprocessing is necessary. This is handled by the `preprocess_datatree` function, which extracts only the bands of interest, i.e. green (b03) and NIR (b08), clips the tile to the AOI, merges the two bands into a single dataset, and finally adds a **time dimension** derived from the datatree attributes (\"stac_discovery\" / \"properties\" / \"start_datetime\").\n", @@ -326,7 +334,7 @@ { "cell_type": "code", "execution_count": null, - "id": "24", + "id": "25", "metadata": {}, "outputs": [], "source": [ @@ -345,7 +353,7 @@ }, { "cell_type": "markdown", - "id": "25", + "id": "26", "metadata": {}, "source": [ "Apply the preprocessing and compute the datasets" @@ -354,7 +362,7 @@ { "cell_type": "code", "execution_count": null, - "id": "26", + "id": "27", "metadata": {}, "outputs": [], "source": [ @@ -375,7 +383,7 @@ }, { "cell_type": "markdown", - "id": "27", + "id": "28", "metadata": {}, "source": [ "Create a datacube with all datasets along time dimension" @@ -384,7 +392,7 @@ { "cell_type": "code", "execution_count": null, - "id": "28", + "id": "29", "metadata": {}, "outputs": [], "source": [ @@ -399,7 +407,7 @@ }, { "cell_type": "markdown", - "id": "29", + "id": "30", "metadata": {}, "source": [ "### 1.3 - Filtering data" @@ -407,7 +415,7 @@ }, { "cell_type": "markdown", - "id": "30", + "id": "31", "metadata": {}, "source": [ "Although we selected scenes with less than 30% cloud coverage in our query, this threshold applies to the entire tile, meaning some clouds may still be present in the clipped `data_cube`. \n", @@ -430,7 +438,7 @@ { "cell_type": "code", "execution_count": null, - "id": "31", + "id": "32", "metadata": {}, "outputs": [], "source": [ @@ -447,7 +455,7 @@ { "cell_type": "code", "execution_count": null, - "id": "32", + "id": "33", "metadata": {}, "outputs": [], "source": [ @@ -467,7 +475,7 @@ }, { "cell_type": "markdown", - "id": "33", + "id": "34", "metadata": {}, "source": [ "Finally, we have a clipped and cleaned `data_cube` containing all scenes, including a **time dimension** for the relevant bands as data variables (`time`, `lat`, `lon`). We will use this dataset in **Section 2** to perform time series analysis, namely to derive water extents and calculate the area for each scene." @@ -475,7 +483,7 @@ }, { "cell_type": "markdown", - "id": "34", + "id": "35", "metadata": {}, "source": [ "
" @@ -483,7 +491,7 @@ }, { "cell_type": "markdown", - "id": "35", + "id": "36", "metadata": {}, "source": [ "## Section 2 - Reservoir Water Surface Estimation" @@ -491,7 +499,7 @@ }, { "cell_type": "markdown", - "id": "36", + "id": "37", "metadata": {}, "source": [ "**Overview of Section 2:**\n", @@ -506,7 +514,7 @@ }, { "cell_type": "markdown", - "id": "37", + "id": "38", "metadata": {}, "source": [ "### 2.1 - Compute Normalized Difference Water Index (NDWI)" @@ -514,7 +522,7 @@ }, { "cell_type": "markdown", - "id": "38", + "id": "39", "metadata": {}, "source": [ "Once we have preprocessed and prepared the data for our AOI, we can start by computing the Normalized Difference Water Index (NDWI) using the green (b03) and NIR (b08) bands. The NDWI highlights water features by producing high positive values for water pixels and lower or negative values for land and vegetation. This provides an initial indication of where water is located, although simple thresholding at this stage can be unreliable, particularly near shorelines or in areas with shallow water or vegetation." @@ -523,7 +531,7 @@ { "cell_type": "code", "execution_count": null, - "id": "39", + "id": "40", "metadata": {}, "outputs": [], "source": [ @@ -535,7 +543,7 @@ }, { "cell_type": "markdown", - "id": "40", + "id": "41", "metadata": {}, "source": [ "Visualize example output from Subsection 2.1" @@ -544,7 +552,7 @@ { "cell_type": "code", "execution_count": null, - "id": "41", + "id": "42", "metadata": {}, "outputs": [], "source": [ @@ -557,7 +565,7 @@ }, { "cell_type": "markdown", - "id": "42", + "id": "43", "metadata": {}, "source": [ "As shown in the plot, the water body area is visible, with high values represented by the deep sky-blue color. However, clouds are present within the water body area.\n" @@ -565,7 +573,7 @@ }, { "cell_type": "markdown", - "id": "43", + "id": "44", "metadata": {}, "source": [ "### 2.2 - Integrate Water Occurrence (WO) data" @@ -573,7 +581,7 @@ }, { "cell_type": "markdown", - "id": "44", + "id": "45", "metadata": {}, "source": [ "The Global Surface Water Explorer (GSWE) dataset from the European Commission's Joint Research Centre is also used here. This dataset maps the location and temporal distribution of water surfaces at the global scale over the past 32 years, providing statistics on the extent and change of those water surfaces, namely water occurrence (WO) probabilities per pixel. WO represents the fraction of time a given pixel was observed as water over the study period. The table below shows the range of values and their interpretation.\n", @@ -598,7 +606,7 @@ { "cell_type": "code", "execution_count": null, - "id": "45", + "id": "46", "metadata": {}, "outputs": [], "source": [ @@ -615,7 +623,7 @@ }, { "cell_type": "markdown", - "id": "46", + "id": "47", "metadata": {}, "source": [ "Visualize example output from Subsection 2.2" @@ -624,7 +632,7 @@ { "cell_type": "code", "execution_count": null, - "id": "47", + "id": "48", "metadata": {}, "outputs": [], "source": [ @@ -637,7 +645,7 @@ }, { "cell_type": "markdown", - "id": "48", + "id": "49", "metadata": {}, "source": [ "As shown in the plot, the main water body has very high probability values, close to one, indicating that these pixels were almost always water during the observation period, representing the permanent water areas of the reservoir. Lower values appear near the edges of the reservoir, which are the most challenging areas to detect. These lower probabilities typically indicate that water was present frequently but not permanently (values around 0.5–0.9) or only occasionally (values around 0.1–0.5). In the next subsection, we will focus on detecting these variable water areas in each image.\n" @@ -645,7 +653,7 @@ }, { "cell_type": "markdown", - "id": "49", + "id": "50", "metadata": {}, "source": [ "### 2.3 - Generate water extents (core GWW algorithm)" @@ -653,7 +661,7 @@ }, { "cell_type": "markdown", - "id": "50", + "id": "51", "metadata": {}, "source": [ "To detect water more reliably, we combine the following two datasets: the NDWI index and the Water Occurrence (WO).\n", @@ -670,7 +678,7 @@ { "cell_type": "code", "execution_count": null, - "id": "51", + "id": "52", "metadata": {}, "outputs": [], "source": [ @@ -695,7 +703,7 @@ }, { "cell_type": "markdown", - "id": "52", + "id": "53", "metadata": {}, "source": [ "Visualize example output from Subsection 2.3" @@ -704,7 +712,7 @@ { "cell_type": "code", "execution_count": null, - "id": "53", + "id": "54", "metadata": {}, "outputs": [], "source": [ @@ -731,7 +739,7 @@ }, { "cell_type": "markdown", - "id": "54", + "id": "55", "metadata": {}, "source": [ "In the example scene from 2024-05-23 shown in the figure above, the left panel displays the initial water detected using Otsu, the middle panel shows the water filling based on the WO dataset, and the right panel presents the total detected water produced by our algorithm." @@ -739,7 +747,7 @@ }, { "cell_type": "markdown", - "id": "55", + "id": "56", "metadata": {}, "source": [ "### 2.4 - Extract Largest Connected Component" @@ -747,7 +755,7 @@ }, { "cell_type": "markdown", - "id": "56", + "id": "57", "metadata": {}, "source": [ "Even after thresholding, small water patches or noise may appear in the mask. Since we are interested in the main reservoir, we extract the **largest connected component** from the binary water mask. This isolates the primary water body while ignoring smaller, isolated water regions, giving a clean representation of the reservoir. For this we are using the `largest_connected_component` function. (Note that the largest connected component is not extracted in the original GWW algorithm)" @@ -756,7 +764,7 @@ { "cell_type": "code", "execution_count": null, - "id": "57", + "id": "58", "metadata": {}, "outputs": [], "source": [ @@ -774,7 +782,7 @@ }, { "cell_type": "markdown", - "id": "58", + "id": "59", "metadata": {}, "source": [ "### 2.5 - Compute reservoir area\n" @@ -782,7 +790,7 @@ }, { "cell_type": "markdown", - "id": "59", + "id": "60", "metadata": {}, "source": [ "As a last step, we can calculate the reservoir area by counting the number of pixels in the largest water body and converting this count to square kilometers using the pixel size (10 m for our case; see the `compute_area_km2` function)." @@ -791,7 +799,7 @@ { "cell_type": "code", "execution_count": null, - "id": "60", + "id": "61", "metadata": {}, "outputs": [], "source": [ @@ -810,7 +818,7 @@ }, { "cell_type": "markdown", - "id": "61", + "id": "62", "metadata": {}, "source": [ "Visualize example output from Subsections 2.4 and 2.5" @@ -819,7 +827,7 @@ { "cell_type": "code", "execution_count": null, - "id": "62", + "id": "63", "metadata": {}, "outputs": [], "source": [ @@ -830,7 +838,7 @@ }, { "cell_type": "markdown", - "id": "63", + "id": "64", "metadata": {}, "source": [ "Compute and visualize the estimated areas over time. (Note that the final surface water area time series in the raw GWW algorithm are post-processed with a temporal quantile-based filter)" @@ -839,7 +847,7 @@ { "cell_type": "code", "execution_count": null, - "id": "64", + "id": "65", "metadata": {}, "outputs": [], "source": [ @@ -849,7 +857,7 @@ }, { "cell_type": "markdown", - "id": "65", + "id": "66", "metadata": {}, "source": [ "Plotting the entire time series" @@ -858,7 +866,7 @@ { "cell_type": "code", "execution_count": null, - "id": "66", + "id": "67", "metadata": {}, "outputs": [], "source": [ @@ -880,7 +888,7 @@ }, { "cell_type": "markdown", - "id": "67", + "id": "68", "metadata": {}, "source": [ "
" @@ -888,7 +896,7 @@ }, { "cell_type": "markdown", - "id": "68", + "id": "69", "metadata": {}, "source": [ "## Conclusion" @@ -896,7 +904,7 @@ }, { "cell_type": "markdown", - "id": "69", + "id": "70", "metadata": {}, "source": [ "In this notebook, we demonstrated how Sentinel-2 L1C data can be accessed and processed to monitor reservoir water surface area over a historical time series using Earth Observation techniques. To achieve this, we applied parts of the [Global Water Watch (GWW)](https://www.globalwaterwatch.earth/) algorithms (detailed description [here](https://www.nature.com/articles/s41598-022-17074-6)) to filter images, derive water masks, fill gaps and estimate reservoir surface.\n", @@ -908,7 +916,7 @@ }, { "cell_type": "markdown", - "id": "70", + "id": "71", "metadata": {}, "source": [ "
" @@ -916,7 +924,7 @@ }, { "cell_type": "markdown", - "id": "71", + "id": "72", "metadata": {}, "source": [ "## 💪 Now it is your turn\n" @@ -924,7 +932,7 @@ }, { "cell_type": "markdown", - "id": "72", + "id": "73", "metadata": {}, "source": [ "Now that you’re familiar our reservoir detection workflow, based on the zarr format provided by the EOPF Sentinel STAC Catalog, it’s time to put your skills into practice! Below are suggested exercises that scale from replication to comparison and finally to an advanced extension using a different sensor.\n", @@ -937,7 +945,7 @@ }, { "cell_type": "markdown", - "id": "73", + "id": "74", "metadata": {}, "source": [ "### Task 1: Try Another Reservoir\n", @@ -1038,7 +1046,7 @@ }, { "cell_type": "markdown", - "id": "74", + "id": "75", "metadata": {}, "source": [ "
" @@ -1046,7 +1054,7 @@ }, { "cell_type": "markdown", - "id": "75", + "id": "76", "metadata": {}, "source": [ "## What's next?" @@ -1054,7 +1062,7 @@ }, { "cell_type": "markdown", - "id": "76", + "id": "77", "metadata": {}, "source": [ "In the following [notebook](./68_vegetation_anomalies.ipynb), we will explore the application of Sentinel 2 L2A data for analysing vegetation anomalies in German forests.
" diff --git a/06_eopf_zarr_in_action/68_vegetation_anomalies.ipynb b/06_eopf_zarr_in_action/68_vegetation_anomalies.ipynb index 6dada7a..b52d449 100644 --- a/06_eopf_zarr_in_action/68_vegetation_anomalies.ipynb +++ b/06_eopf_zarr_in_action/68_vegetation_anomalies.ipynb @@ -36,6 +36,14 @@ { "cell_type": "markdown", "id": "2", + "metadata": {}, + "source": [ + "**By:** *[@davemlz](https://github.com/davemlz)*" + ] + }, + { + "cell_type": "markdown", + "id": "3", "metadata": { "vscode": { "languageId": "raw" @@ -62,7 +70,7 @@ }, { "cell_type": "markdown", - "id": "3", + "id": "4", "metadata": {}, "source": [ "### What we will learn\n", @@ -78,7 +86,7 @@ }, { "cell_type": "markdown", - "id": "4", + "id": "5", "metadata": {}, "source": [ "### Prerequisites\n", @@ -90,7 +98,7 @@ }, { "cell_type": "markdown", - "id": "5", + "id": "6", "metadata": {}, "source": [ "
" @@ -98,7 +106,7 @@ }, { "cell_type": "markdown", - "id": "6", + "id": "7", "metadata": {}, "source": [ "#### Import libraries" @@ -107,7 +115,7 @@ { "cell_type": "code", "execution_count": null, - "id": "7", + "id": "8", "metadata": {}, "outputs": [], "source": [ @@ -125,7 +133,7 @@ }, { "cell_type": "markdown", - "id": "8", + "id": "9", "metadata": {}, "source": [ "#### Helper functions" @@ -133,7 +141,7 @@ }, { "cell_type": "markdown", - "id": "9", + "id": "10", "metadata": {}, "source": [ "Helper functions were defined in the `vegetation_anomalies_utils.py` file. Here we import all of the functions and explain their role." @@ -142,7 +150,7 @@ { "cell_type": "code", "execution_count": null, - "id": "10", + "id": "11", "metadata": {}, "outputs": [], "source": [ @@ -151,7 +159,7 @@ }, { "cell_type": "markdown", - "id": "11", + "id": "12", "metadata": {}, "source": [ "##### `get_items`\n", @@ -161,7 +169,7 @@ }, { "cell_type": "markdown", - "id": "12", + "id": "13", "metadata": {}, "source": [ "##### `latlon_to_buffer_bbox`\n", @@ -173,7 +181,7 @@ }, { "cell_type": "markdown", - "id": "13", + "id": "14", "metadata": {}, "source": [ "##### `open_and_curate_data`\n", @@ -185,7 +193,7 @@ }, { "cell_type": "markdown", - "id": "14", + "id": "15", "metadata": {}, "source": [ "##### `curate_gpp`\n", @@ -195,7 +203,7 @@ }, { "cell_type": "markdown", - "id": "15", + "id": "16", "metadata": {}, "source": [ "
" @@ -203,7 +211,7 @@ }, { "cell_type": "markdown", - "id": "16", + "id": "17", "metadata": { "vscode": { "languageId": "raw" @@ -215,7 +223,7 @@ }, { "cell_type": "markdown", - "id": "17", + "id": "18", "metadata": {}, "source": [ "We will start the notebook with a forest ecosystem that was severely affacted by the drought of 2018: DE-Hai." @@ -224,7 +232,7 @@ { "cell_type": "code", "execution_count": null, - "id": "18", + "id": "19", "metadata": {}, "outputs": [], "source": [ @@ -235,7 +243,7 @@ }, { "cell_type": "markdown", - "id": "19", + "id": "20", "metadata": {}, "source": [ "Initialize a **Dask distributed client** to enable parallel and delayed computation. This will manage the execution of tasks, such as loading and processing large Sentinel-2 Zarr datasets, efficiently." @@ -244,7 +252,7 @@ { "cell_type": "code", "execution_count": null, - "id": "20", + "id": "21", "metadata": {}, "outputs": [], "source": [ @@ -254,7 +262,7 @@ }, { "cell_type": "markdown", - "id": "21", + "id": "22", "metadata": {}, "source": [ "### Load the GPP data" @@ -262,7 +270,7 @@ }, { "cell_type": "markdown", - "id": "22", + "id": "23", "metadata": {}, "source": [ "Load the **GPP time series** for the DE-Hai site and compute **weekly anomalies**, identifying extreme low-GPP events.\n", @@ -280,7 +288,7 @@ { "cell_type": "code", "execution_count": null, - "id": "23", + "id": "24", "metadata": {}, "outputs": [], "source": [ @@ -289,7 +297,7 @@ }, { "cell_type": "markdown", - "id": "24", + "id": "25", "metadata": {}, "source": [ "### Create the Sentinel-2 L2A Data Cube\n", @@ -302,7 +310,7 @@ { "cell_type": "code", "execution_count": null, - "id": "25", + "id": "26", "metadata": {}, "outputs": [], "source": [ @@ -312,7 +320,7 @@ }, { "cell_type": "markdown", - "id": "26", + "id": "27", "metadata": {}, "source": [ "For each item, **open and curate** the data by subsetting around the site coordinates and selecting the relevant bands:\n", @@ -330,7 +338,7 @@ { "cell_type": "code", "execution_count": null, - "id": "27", + "id": "28", "metadata": {}, "outputs": [], "source": [ @@ -343,7 +351,7 @@ }, { "cell_type": "markdown", - "id": "28", + "id": "29", "metadata": {}, "source": [ "After computed, datasets are concatenated along the **time dimension**, sorted by time, and finally **loaded into memory** as a single `xarray.Dataset`." @@ -352,7 +360,7 @@ { "cell_type": "code", "execution_count": null, - "id": "29", + "id": "30", "metadata": {}, "outputs": [], "source": [ @@ -365,7 +373,7 @@ }, { "cell_type": "markdown", - "id": "30", + "id": "31", "metadata": {}, "source": [ "### Compute Vegetation Indices\n", @@ -400,7 +408,7 @@ { "cell_type": "code", "execution_count": null, - "id": "31", + "id": "32", "metadata": {}, "outputs": [], "source": [ @@ -420,7 +428,7 @@ }, { "cell_type": "markdown", - "id": "32", + "id": "33", "metadata": {}, "source": [ "Add the name and units of each index to the attributes according to the CF conventions." @@ -429,7 +437,7 @@ { "cell_type": "code", "execution_count": null, - "id": "33", + "id": "34", "metadata": {}, "outputs": [], "source": [ @@ -442,7 +450,7 @@ }, { "cell_type": "markdown", - "id": "34", + "id": "35", "metadata": {}, "source": [ "Resample the NDVI and kNDVI time series to **weekly frequency**, taking the **median** within each week. After resampling, fill temporal gaps by applying **cubic interpolation** along the time dimension. This produces smooth, continuous weekly index time series suitable for anomaly computation." @@ -451,7 +459,7 @@ { "cell_type": "code", "execution_count": null, - "id": "35", + "id": "36", "metadata": {}, "outputs": [], "source": [ @@ -460,7 +468,7 @@ }, { "cell_type": "markdown", - "id": "36", + "id": "37", "metadata": {}, "source": [ "### Calculate Vegetation Anomalies\n", @@ -475,7 +483,7 @@ { "cell_type": "code", "execution_count": null, - "id": "37", + "id": "38", "metadata": {}, "outputs": [], "source": [ @@ -484,7 +492,7 @@ }, { "cell_type": "markdown", - "id": "38", + "id": "39", "metadata": {}, "source": [ "Plot the MSC of the NDVI." @@ -493,7 +501,7 @@ { "cell_type": "code", "execution_count": null, - "id": "39", + "id": "40", "metadata": {}, "outputs": [], "source": [ @@ -502,7 +510,7 @@ }, { "cell_type": "markdown", - "id": "40", + "id": "41", "metadata": {}, "source": [ "Plot the MSC of the kNDVI." @@ -511,7 +519,7 @@ { "cell_type": "code", "execution_count": null, - "id": "41", + "id": "42", "metadata": {}, "outputs": [], "source": [ @@ -520,7 +528,7 @@ }, { "cell_type": "markdown", - "id": "42", + "id": "43", "metadata": {}, "source": [ "Compute **vegetation anomalies** by subtracting the **median seasonal cycle (MSC)** from the weekly NDVI and kNDVI values. This step isolates deviations from the expected seasonal pattern, allowing us to identify abnormal vegetation conditions potentially linked to stress or extreme events." @@ -529,7 +537,7 @@ { "cell_type": "code", "execution_count": null, - "id": "43", + "id": "44", "metadata": {}, "outputs": [], "source": [ @@ -538,7 +546,7 @@ }, { "cell_type": "markdown", - "id": "44", + "id": "45", "metadata": {}, "source": [ "Add the name and units of each index anomaly to the attributes according to the CF conventions." @@ -547,7 +555,7 @@ { "cell_type": "code", "execution_count": null, - "id": "45", + "id": "46", "metadata": {}, "outputs": [], "source": [ @@ -560,7 +568,7 @@ }, { "cell_type": "markdown", - "id": "46", + "id": "47", "metadata": {}, "source": [ "### Visualize Time Series\n", @@ -571,7 +579,7 @@ { "cell_type": "code", "execution_count": null, - "id": "47", + "id": "48", "metadata": {}, "outputs": [], "source": [ @@ -580,7 +588,7 @@ }, { "cell_type": "markdown", - "id": "48", + "id": "49", "metadata": {}, "source": [ "Here, we defined the colors to use for our indices." @@ -589,7 +597,7 @@ { "cell_type": "code", "execution_count": null, - "id": "49", + "id": "50", "metadata": {}, "outputs": [], "source": [ @@ -603,7 +611,7 @@ }, { "cell_type": "markdown", - "id": "50", + "id": "51", "metadata": {}, "source": [ "Now, we will plot the NDVI and kNDVI time series together with the GPP measurements for the DE-Hai site. A secondary axis will be used to display GPP, allowing direct visual comparison between vegetation dynamics and ecosystem productivity. Extreme low-GPP events are highlighted as shaded red intervals: These events are defined as **periods of at least two consecutive days** in which GPP anomalies fall **below the 10th percentile** of the lower tail of the distribution. This information is contained in the `HAI_df` dataframe created via `curate_gpp` helper function.\n", @@ -614,7 +622,7 @@ { "cell_type": "code", "execution_count": null, - "id": "51", + "id": "52", "metadata": {}, "outputs": [], "source": [ @@ -648,7 +656,7 @@ }, { "cell_type": "markdown", - "id": "52", + "id": "53", "metadata": {}, "source": [ "Now, let's do the same for the anomalies by aggregating the anomalies of the indices in space using the median to produce a time series." @@ -657,7 +665,7 @@ { "cell_type": "code", "execution_count": null, - "id": "53", + "id": "54", "metadata": {}, "outputs": [], "source": [ @@ -666,7 +674,7 @@ }, { "cell_type": "markdown", - "id": "54", + "id": "55", "metadata": {}, "source": [ "Now we can visualize the **anomaly time series** of NDVI, kNDVI, and GPP for the DE-Hai site. Here, both vegetation indices and GPP have been transformed into **weekly anomalies**, representing deviations from their typical seasonal cycles. A horizontal line at zero indicates the expected baseline. A secondary axis displays **GPP anomalies**, allowing direct comparison between canopy-level spectral responses and ecosystem-level carbon uptake changes. Extreme low-GPP events are shown as shaded red intervals. \n", @@ -677,7 +685,7 @@ { "cell_type": "code", "execution_count": null, - "id": "55", + "id": "56", "metadata": {}, "outputs": [], "source": [ @@ -712,7 +720,7 @@ }, { "cell_type": "markdown", - "id": "56", + "id": "57", "metadata": {}, "source": [ "
" @@ -720,7 +728,7 @@ }, { "cell_type": "markdown", - "id": "57", + "id": "58", "metadata": {}, "source": [ "## 💪 Now it is your turn" @@ -728,7 +736,7 @@ }, { "cell_type": "markdown", - "id": "58", + "id": "59", "metadata": {}, "source": [ "The following exercises will help you reproduce the previous workflow for another dataset.\n", @@ -742,7 +750,7 @@ { "cell_type": "code", "execution_count": null, - "id": "59", + "id": "60", "metadata": {}, "outputs": [], "source": [ @@ -761,7 +769,7 @@ }, { "cell_type": "markdown", - "id": "60", + "id": "61", "metadata": {}, "source": [ "### Task 2: Compute Vegetation Indices\n", @@ -772,7 +780,7 @@ { "cell_type": "code", "execution_count": null, - "id": "61", + "id": "62", "metadata": {}, "outputs": [], "source": [ @@ -784,7 +792,7 @@ }, { "cell_type": "markdown", - "id": "62", + "id": "63", "metadata": {}, "source": [ "### Task 3: Calculate Vegetation Anomalies\n", @@ -794,7 +802,7 @@ { "cell_type": "code", "execution_count": null, - "id": "63", + "id": "64", "metadata": {}, "outputs": [], "source": [ @@ -808,7 +816,7 @@ }, { "cell_type": "markdown", - "id": "64", + "id": "65", "metadata": {}, "source": [ "## Conclusion" @@ -816,7 +824,7 @@ }, { "cell_type": "markdown", - "id": "65", + "id": "66", "metadata": {}, "source": [ "In this notebook, we explored how **Sentinel-2 L2A Zarr data cubes** can be used to monitor forest vegetation dynamics and detect anomalous behavior linked to ecosystem stress. By leveraging Zarr, STAC-based discovery, and `xarray`/`Dask` for scalable computation, we built an end-to-end workflow that included:\n", @@ -833,7 +841,7 @@ }, { "cell_type": "markdown", - "id": "66", + "id": "67", "metadata": {}, "source": [ "### Acknowledgements\n", @@ -843,7 +851,7 @@ }, { "cell_type": "markdown", - "id": "67", + "id": "68", "metadata": {}, "source": [ "### References\n", @@ -855,7 +863,7 @@ }, { "cell_type": "markdown", - "id": "68", + "id": "69", "metadata": {}, "source": [ "## What's next?\n", diff --git a/06_eopf_zarr_in_action/69_coastal_water_dynamics_s1.ipynb b/06_eopf_zarr_in_action/69_coastal_water_dynamics_s1.ipynb index 7f58d53..2859941 100644 --- a/06_eopf_zarr_in_action/69_coastal_water_dynamics_s1.ipynb +++ b/06_eopf_zarr_in_action/69_coastal_water_dynamics_s1.ipynb @@ -22,12 +22,7 @@ "id": "1", "metadata": {}, "source": [ - "### Author\n", - "\n", - "- **Walid Ghariani** - [GitHub Profile](https://github.com/WalidGharianiEAGLE)\n", - "\n", - "### Affiliation\n", - "- **DHI** - https://www.dhigroup.com/" + "**By:** *[@WalidGharianiEAGLE](https://github.com/WalidGharianiEAGLE)*" ] }, { @@ -1156,7 +1151,7 @@ "source": [ "## What's next?\n", "\n", - "This resource is constantly updated!. Stay Tuned for new chapters 🛰️ !" + "This resource is constantly updated! Stay Tuned for new chapters 🛰️ !" ] } ],