diff --git a/doc/big_datasets.rst b/doc/big_datasets.rst index bac60e83..26851034 100644 --- a/doc/big_datasets.rst +++ b/doc/big_datasets.rst @@ -1,11 +1,55 @@ Handling Large Datasets ------------------------------------- -Often, one desires to use *tobac* to identify and track features in large datasets ("big data"). This documentation strives to suggest various methods for doing so efficiently. Current versions of *tobac* do not allow for out-of-memory computation, meaning that these strategies may need to be employed for both computational and memory reasons. +Often, one desires to use *tobac* to identify and track features in large datasets ("big data"). This documentation strives to suggest various methods for doing so efficiently. Current versions of *tobac* do not support out-of-core (e.g., :code:`dask`) computation, meaning that these strategies may need to be employed for both computational and memory reasons. .. _Split Feature Detection: ======================= -Split Feature Detection +Split Feature Detection and Run in Parallel ======================= -Current versions of threshold feature detection (see :doc:`feature_detection_overview`) are time independent, meaning that one can parallelize feature detection across all times (although not across space). *tobac* provides the :py:meth:`tobac.utils.combine_tobac_feats` function to combine a list of dataframes produced by a parallelization method (such as :code:`jug` or :code:`multiprocessing.pool`) into a single combined dataframe suitable to perform tracking with. +Current versions of threshold feature detection (see :doc:`feature_detection_overview`) are time independent, meaning that one can easily parallelize feature detection across all times (although not across space). *tobac* provides the :py:meth:`tobac.utils.combine_feature_dataframes` function to combine a list of dataframes produced by a parallelization method (such as :code:`jug`, :code:`multiprocessing.pool`, or :code:`dask.bag`) into a single combined dataframe suitable to perform tracking with. + +Below is a snippet from a larger notebook demonstrating how to run feature detection in parallel ( :doc:`big_datasets_examples/notebooks/parallel_processing_tobac`): + +:: + + # build list of tracked variables using Dask.Bag + + b = db.from_sequence( + [ + combined_ds["data"][x : x + 1] + for x in range(len(combined_ds["time"])) + ], + npartitions=1, + ) + out_feature_dfs = db.map( + lambda x: tobac.feature_detection_multithreshold( + x.to_iris(), 4000, **parameters_features + ), + b, + ).compute() + + combined_dataframes = tobac.utils.general.combine_feature_dataframes(out_feature_dfs) + + +.. _Split Segmentation: + +====================================== +Split Segmentation and Run in Parallel +====================================== +Recall that the segmentation mask (see :doc:`segmentation_output`) is the same size as the input grid, which results in large files when handling large input datasets. The following strategies can help reduce the output size and make segmentation masks more useful for the analysis. + +The first strategy is to only segment on features *after tracking and quality control*. While this will not directly impact performance, waiting to run segmentation on the final set of features (after discarding, e.g., non-tracked cells) can make analysis of the output segmentation dataset easier. + +To enhance the speed at which segmentation runs, one can process multiple segmentation times in parallel independently, similar to feature detection. Unlike feature detection, however, there is currently no built-in *tobac* method to combine multiple segmentation times into a single file. While one can do this using typical NetCDF tools such as :code:`nccat` or with xarray utilities such as :code:`xr.concat`, you can also leave the segmentation mask output as separate files, opening them later with multiple file retrievals such as :code:`xr.open_mfdataset`. + + +.. _Tracking Hanging: + +===================================== +Tracking Hangs with too many Features +===================================== + +When tracking on a large dataset, :code:`tobac.tracking.linking_trackpy` can hang using the default parameters. This is due to the tracking library :code:`trackpy` searching for the next timestep's feature in too large of an area. This can be solved *without impact to scientific output* by lowering the :code:`subnetwork_size` parameter in :code:`tobac.tracking.linking_trackpy`. + diff --git a/doc/big_datasets_examples/notebooks/parallel_processing_tobac.ipynb b/doc/big_datasets_examples/notebooks/parallel_processing_tobac.ipynb new file mode 100644 index 00000000..c65f698a --- /dev/null +++ b/doc/big_datasets_examples/notebooks/parallel_processing_tobac.ipynb @@ -0,0 +1,1160 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Parallel Processing of Feature Detection with `dask`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This notebook demonstrates how to run *tobac* feature detection in parallel using the `dask` library as the parallel processor." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Imports and Dask Cluster Setup" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/seanfreeman/mambaforge/envs/tobac_installed/lib/python3.11/site-packages/tobac/utils/decorators.py:366: UserWarning: Numba not able to be imported; periodic boundary calculations will be slower.Exception raised: ModuleNotFoundError(\"No module named 'numba'\")\n", + " warnings.warn(\n", + "/Users/seanfreeman/mambaforge/envs/tobac_installed/lib/python3.11/site-packages/tobac/utils/decorators.py:366: UserWarning: Numba not able to be imported; periodic boundary calculations will be slower.Exception raised: ModuleNotFoundError(\"No module named 'numba'\")\n", + " warnings.warn(\n" + ] + } + ], + "source": [ + "import tobac\n", + "import dask.bag as db\n", + "import xarray as xr\n", + "import s3fs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are many different ways to initialize a dask cluster. This is just one example, running two workers on a single local machine. " + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "
\n", + "
\n", + "

Client

\n", + "

Client-5a603f42-cd1a-11ee-a16f-06803d36ca4b

\n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + "
Connection method: Cluster objectCluster type: distributed.LocalCluster
\n", + " Dashboard: http://127.0.0.1:8787/status\n", + "
\n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "

Cluster Info

\n", + "
\n", + "
\n", + "
\n", + "
\n", + "

LocalCluster

\n", + "

e2f9b06d

\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + "\n", + " \n", + "
\n", + " Dashboard: http://127.0.0.1:8787/status\n", + " \n", + " Workers: 2\n", + "
\n", + " Total threads: 2\n", + " \n", + " Total memory: 64.00 GiB\n", + "
Status: runningUsing processes: True
\n", + "\n", + "
\n", + " \n", + "

Scheduler Info

\n", + " \n", + "\n", + "
\n", + "
\n", + "
\n", + "
\n", + "

Scheduler

\n", + "

Scheduler-b2b92b52-530a-4c41-b0b5-5da2c4cfcafb

\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " Comm: tcp://127.0.0.1:51973\n", + " \n", + " Workers: 2\n", + "
\n", + " Dashboard: http://127.0.0.1:8787/status\n", + " \n", + " Total threads: 2\n", + "
\n", + " Started: Just now\n", + " \n", + " Total memory: 64.00 GiB\n", + "
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "

Workers

\n", + " \n", + "\n", + " \n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "

Worker: 0

\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + "\n", + " \n", + "\n", + "
\n", + " Comm: tcp://127.0.0.1:51981\n", + " \n", + " Total threads: 1\n", + "
\n", + " Dashboard: http://127.0.0.1:51982/status\n", + " \n", + " Memory: 32.00 GiB\n", + "
\n", + " Nanny: tcp://127.0.0.1:51976\n", + "
\n", + " Local directory: /var/folders/bj/m6g82c6n41g83y3_dx02y7ch0000gp/T/dask-scratch-space/worker-pou0krcp\n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "

Worker: 1

\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + "\n", + " \n", + "\n", + "
\n", + " Comm: tcp://127.0.0.1:51980\n", + " \n", + " Total threads: 1\n", + "
\n", + " Dashboard: http://127.0.0.1:51983/status\n", + " \n", + " Memory: 32.00 GiB\n", + "
\n", + " Nanny: tcp://127.0.0.1:51978\n", + "
\n", + " Local directory: /var/folders/bj/m6g82c6n41g83y3_dx02y7ch0000gp/T/dask-scratch-space/worker-cygts80d\n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "\n", + "
\n", + "
\n", + "\n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "\n", + "
\n", + "
" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from dask.distributed import Client, progress\n", + "\n", + "client = Client(n_workers=2, threads_per_worker=1)\n", + "client" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Read in Data" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here, we are using the NOAA Global Mosaic of Geostationary Satellite Imagery (GMGSI) as our input data source from AWS s3." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "fs = s3fs.S3FileSystem(anon=True)\n", + "aws_urls = [\n", + " \"s3://noaa-gmgsi-pds/GMGSI_LW/2024/01/01/00/GLOBCOMPLIR_nc.2024010100\",\n", + " \"s3://noaa-gmgsi-pds/GMGSI_LW/2024/01/01/01/GLOBCOMPLIR_nc.2024010101\",\n", + "]\n", + "\n", + "all_ds = list()\n", + "for aws_url in aws_urls:\n", + " fileObj = fs.open(aws_url)\n", + " all_ds.append(xr.open_dataset(fileObj, engine=\"h5netcdf\"))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We loaded in two files and we will use xarray to concatenate them." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "combined_ds = xr.concat(all_ds, dim=\"time\")" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
<xarray.Dataset>\n",
+       "Dimensions:  (time: 2, yc: 3000, xc: 4999)\n",
+       "Coordinates:\n",
+       "    lat      (yc, xc) float32 72.72 72.72 72.72 72.72 ... -72.74 -72.74 -72.74\n",
+       "    lon      (yc, xc) float32 180.0 -179.9 -179.9 -179.8 ... 179.8 179.8 179.9\n",
+       "  * time     (time) datetime64[ns] 2024-01-01 2024-01-01T01:00:00\n",
+       "Dimensions without coordinates: yc, xc\n",
+       "Data variables:\n",
+       "    data     (time, yc, xc) float32 206.0 204.0 204.0 ... 187.0 188.0 182.0\n",
+       "Attributes:\n",
+       "    Conventions:          CF-1.4\n",
+       "    Source:               McIDAS Area File\n",
+       "    Satellite Sensor:     DERIVED DATA\n",
+       "    time_coverage_start:  2024-01-01T00:00:00\n",
+       "    instrument_name:      GLOBCOMPLIR\n",
+       "    history:              Mon Jan  1 00:38:21 2024: ncks -d xc,0,4998 templir...\n",
+       "    NCO:                  netCDF Operators version 4.7.5 (Homepage = http://n...
" + ], + "text/plain": [ + "\n", + "Dimensions: (time: 2, yc: 3000, xc: 4999)\n", + "Coordinates:\n", + " lat (yc, xc) float32 72.72 72.72 72.72 72.72 ... -72.74 -72.74 -72.74\n", + " lon (yc, xc) float32 180.0 -179.9 -179.9 -179.8 ... 179.8 179.8 179.9\n", + " * time (time) datetime64[ns] 2024-01-01 2024-01-01T01:00:00\n", + "Dimensions without coordinates: yc, xc\n", + "Data variables:\n", + " data (time, yc, xc) float32 206.0 204.0 204.0 ... 187.0 188.0 182.0\n", + "Attributes:\n", + " Conventions: CF-1.4\n", + " Source: McIDAS Area File\n", + " Satellite Sensor: DERIVED DATA\n", + " time_coverage_start: 2024-01-01T00:00:00\n", + " instrument_name: GLOBCOMPLIR\n", + " history: Mon Jan 1 00:38:21 2024: ncks -d xc,0,4998 templir...\n", + " NCO: netCDF Operators version 4.7.5 (Homepage = http://n..." + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "combined_ds" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "These feature detection parameters are just examples." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## *tobac* Feature Detection" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "parameters_features = {}\n", + "parameters_features[\"position_threshold\"] = \"weighted_diff\"\n", + "parameters_features[\"sigma_threshold\"] = 0.5\n", + "parameters_features[\"n_min_threshold\"] = 4\n", + "parameters_features[\"target\"] = \"minimum\"\n", + "parameters_features[\"threshold\"] = [180, 170]\n", + "parameters_features[\"PBC_flag\"] = \"hdim_2\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "While future versions (1.6 and greater) of *tobac* will support xarray natively in feature detection and segmentation, current versions of *tobac* rely on Iris for gridded data. Because of this, we have to make some conversions to have this data be compatible with iris. " + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "# iris issues\n", + "combined_ds[\"data\"].attrs[\"units\"] = \"kelvin\"\n", + "combined_ds[\"data\"][\"time\"].attrs[\"long_name\"] = \"time\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we will use a *dask bag* to parallelize our feature detection over time. " + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/seanfreeman/mambaforge/envs/tobac_installed/lib/python3.11/site-packages/distributed/client.py:3162: UserWarning: Sending large graph of size 57.21 MiB.\n", + "This may cause some slowdown.\n", + "Consider scattering data ahead of time and using futures.\n", + " warnings.warn(\n", + "/Users/seanfreeman/mambaforge/envs/tobac_installed/lib/python3.11/site-packages/tobac/utils/decorators.py:366: UserWarning: Numba not able to be imported; periodic boundary calculations will be slower.Exception raised: ModuleNotFoundError(\"No module named 'numba'\")\n", + " warnings.warn(\n", + "/Users/seanfreeman/mambaforge/envs/tobac_installed/lib/python3.11/site-packages/tobac/utils/decorators.py:366: UserWarning: Numba not able to be imported; periodic boundary calculations will be slower.Exception raised: ModuleNotFoundError(\"No module named 'numba'\")\n", + " warnings.warn(\n" + ] + } + ], + "source": [ + "b = db.from_sequence(\n", + " [\n", + " combined_ds[\"data\"][x : x + 1][0:500, 0:500]\n", + " for x in range(len(combined_ds[\"time\"]))\n", + " ],\n", + " npartitions=1,\n", + ")\n", + "out_feature_dfs = db.map(\n", + " lambda x: tobac.feature_detection_multithreshold(\n", + " x.to_iris(), 4000, **parameters_features\n", + " ),\n", + " b,\n", + ").compute()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Combining parallel-detected features into one coherent DataFrame" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
frameidxhdim_1hdim_2numthreshold_valuefeaturetimetimestrlatitudelongitude
0010.97532560.7181322618012024-01-01 00:00:002024-01-01 00:00:0072.694528-175.628134
1020.67001879.0748181618022024-01-01 00:00:002024-01-01 00:00:0072.701065-174.306289
2050.3581151492.9797781518032024-01-01 00:00:002024-01-01 00:00:0072.707742-72.492469
3060.4825791531.5202152618042024-01-01 00:00:002024-01-01 00:00:0072.705077-69.717214
4073.4098962113.18577028518052024-01-01 00:00:002024-01-01 00:00:0072.642292-27.832080
....................................
156511283498.18090630.0050311617015662024-01-01 01:00:002024-01-01 01:00:0058.262049-177.839756
156611284497.817803715.879216717015672024-01-01 01:00:002024-01-01 01:00:0058.275803-128.450668
156711285498.1582323295.1861716317015682024-01-01 01:00:002024-01-01 01:00:0058.26290857.282537
156811288498.4544043793.095530517015692024-01-01 01:00:002024-01-01 01:00:0058.25168293.136465
156911293498.6489113317.641704617015702024-01-01 01:00:002024-01-01 01:00:0058.24431058.899535
\n", + "

1570 rows × 11 columns

\n", + "
" + ], + "text/plain": [ + " frame idx hdim_1 hdim_2 num threshold_value feature \\\n", + "0 0 1 0.975325 60.718132 26 180 1 \n", + "1 0 2 0.670018 79.074818 16 180 2 \n", + "2 0 5 0.358115 1492.979778 15 180 3 \n", + "3 0 6 0.482579 1531.520215 26 180 4 \n", + "4 0 7 3.409896 2113.185770 285 180 5 \n", + "... ... ... ... ... ... ... ... \n", + "1565 1 1283 498.180906 30.005031 16 170 1566 \n", + "1566 1 1284 497.817803 715.879216 7 170 1567 \n", + "1567 1 1285 498.158232 3295.186171 63 170 1568 \n", + "1568 1 1288 498.454404 3793.095530 5 170 1569 \n", + "1569 1 1293 498.648911 3317.641704 6 170 1570 \n", + "\n", + " time timestr latitude longitude \n", + "0 2024-01-01 00:00:00 2024-01-01 00:00:00 72.694528 -175.628134 \n", + "1 2024-01-01 00:00:00 2024-01-01 00:00:00 72.701065 -174.306289 \n", + "2 2024-01-01 00:00:00 2024-01-01 00:00:00 72.707742 -72.492469 \n", + "3 2024-01-01 00:00:00 2024-01-01 00:00:00 72.705077 -69.717214 \n", + "4 2024-01-01 00:00:00 2024-01-01 00:00:00 72.642292 -27.832080 \n", + "... ... ... ... ... \n", + "1565 2024-01-01 01:00:00 2024-01-01 01:00:00 58.262049 -177.839756 \n", + "1566 2024-01-01 01:00:00 2024-01-01 01:00:00 58.275803 -128.450668 \n", + "1567 2024-01-01 01:00:00 2024-01-01 01:00:00 58.262908 57.282537 \n", + "1568 2024-01-01 01:00:00 2024-01-01 01:00:00 58.251682 93.136465 \n", + "1569 2024-01-01 01:00:00 2024-01-01 01:00:00 58.244310 58.899535 \n", + "\n", + "[1570 rows x 11 columns]" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tobac.utils.general.combine_feature_dataframes(out_feature_dfs)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "tobac_installed", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.0" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/doc/bulk_statistics/notebooks/compute_statistics_during_feature_detection.ipynb b/doc/bulk_statistics/notebooks/compute_statistics_during_feature_detection.ipynb index cb55918f..e9eb2a73 100644 --- a/doc/bulk_statistics/notebooks/compute_statistics_during_feature_detection.ipynb +++ b/doc/bulk_statistics/notebooks/compute_statistics_during_feature_detection.ipynb @@ -34,6 +34,7 @@ "import shutil\n", "from six.moves import urllib\n", "from pathlib import Path\n", + "\n", "%matplotlib inline" ] }, @@ -60,7 +61,8 @@ "source": [ "# Import tobac itself\n", "import tobac\n", - "print('using tobac version', str(tobac.__version__))" + "\n", + "print(\"using tobac version\", str(tobac.__version__))" ] }, { @@ -78,10 +80,11 @@ "source": [ "# Disable a few warnings:\n", "import warnings\n", - "warnings.filterwarnings('ignore', category=UserWarning, append=True)\n", - "warnings.filterwarnings('ignore', category=RuntimeWarning, append=True)\n", - "warnings.filterwarnings('ignore', category=FutureWarning, append=True)\n", - "warnings.filterwarnings('ignore',category=pd.io.pytables.PerformanceWarning)" + "\n", + "warnings.filterwarnings(\"ignore\", category=UserWarning, append=True)\n", + "warnings.filterwarnings(\"ignore\", category=RuntimeWarning, append=True)\n", + "warnings.filterwarnings(\"ignore\", category=FutureWarning, append=True)\n", + "warnings.filterwarnings(\"ignore\", category=pd.io.pytables.PerformanceWarning)" ] }, { @@ -106,7 +109,7 @@ }, "outputs": [], "source": [ - "data_out=Path('../')" + "data_out = Path(\"../\")" ] }, { @@ -123,18 +126,18 @@ "outputs": [], "source": [ "# Download the data: This only has to be done once for all tobac examples and can take a while\n", - "data_file = list(data_out.rglob('data/Example_input_Precip.nc'))\n", + "data_file = list(data_out.rglob(\"data/Example_input_Precip.nc\"))\n", "if len(data_file) == 0:\n", - " file_path='https://zenodo.org/records/3195910/files/climate-processes/tobac_example_data-v1.0.1.zip'\n", - " #file_path='http://zenodo..'\n", - " tempfile=Path('temp.zip')\n", - " print('start downloading data')\n", - " request=urllib.request.urlretrieve(file_path, tempfile)\n", - " print('start extracting data')\n", + " file_path = \"https://zenodo.org/records/3195910/files/climate-processes/tobac_example_data-v1.0.1.zip\"\n", + " # file_path='http://zenodo..'\n", + " tempfile = Path(\"temp.zip\")\n", + " print(\"start downloading data\")\n", + " request = urllib.request.urlretrieve(file_path, tempfile)\n", + " print(\"start extracting data\")\n", " shutil.unpack_archive(tempfile, data_out)\n", " tempfile.unlink()\n", - " print('data extracted')\n", - " data_file = list(data_out.rglob('data/Example_input_Precip.nc'))" + " print(\"data extracted\")\n", + " data_file = list(data_out.rglob(\"data/Example_input_Precip.nc\"))" ] }, { @@ -157,7 +160,7 @@ }, "outputs": [], "source": [ - "Precip=iris.load_cube(str(data_file[0]),'surface_precipitation_average')" + "Precip = iris.load_cube(str(data_file[0]), \"surface_precipitation_average\")" ] }, { @@ -324,7 +327,7 @@ } ], "source": [ - "#display information about the iris cube containing the surface precipitation data:\n", + "# display information about the iris cube containing the surface precipitation data:\n", "display(Precip)" ] }, @@ -341,11 +344,11 @@ }, "outputs": [], "source": [ - "#Set up directory to save output:\n", - "savedir=Path(\"Save\")\n", + "# Set up directory to save output:\n", + "savedir = Path(\"Save\")\n", "if not savedir.is_dir():\n", " savedir.mkdir()\n", - "plot_dir=Path(\"Plot\")\n", + "plot_dir = Path(\"Plot\")\n", "if not plot_dir.is_dir():\n", " plot_dir.mkdir()" ] @@ -380,17 +383,17 @@ }, "outputs": [], "source": [ - "parameters_features={}\n", - "parameters_features['position_threshold']='weighted_diff'\n", - "parameters_features['sigma_threshold']=0.5\n", - "parameters_features['min_distance']=0\n", - "parameters_features['sigma_threshold']=1\n", - "parameters_features['threshold']=[1,2,3,4,5,10,15] #mm/h\n", - "parameters_features['n_erosion_threshold']=0\n", - "parameters_features['n_min_threshold']=3\n", + "parameters_features = {}\n", + "parameters_features[\"position_threshold\"] = \"weighted_diff\"\n", + "parameters_features[\"sigma_threshold\"] = 0.5\n", + "parameters_features[\"min_distance\"] = 0\n", + "parameters_features[\"sigma_threshold\"] = 1\n", + "parameters_features[\"threshold\"] = [1, 2, 3, 4, 5, 10, 15] # mm/h\n", + "parameters_features[\"n_erosion_threshold\"] = 0\n", + "parameters_features[\"n_min_threshold\"] = 3\n", "\n", "# get temporal and spation resolution of the data\n", - "dxy,dt=tobac.get_spacings(Precip)" + "dxy, dt = tobac.get_spacings(Precip)" ] }, { @@ -421,9 +424,9 @@ "outputs": [], "source": [ "statistics = {}\n", - "statistics['mean_precip'] = np.mean\n", - "statistics['total_precip'] = np.sum\n", - "statistics['max_precip'] = np.max" + "statistics[\"mean_precip\"] = np.mean\n", + "statistics[\"total_precip\"] = np.sum\n", + "statistics[\"max_precip\"] = np.max" ] }, { @@ -446,7 +449,7 @@ }, "outputs": [], "source": [ - "statistics['percentiles'] = (np.percentile, {'q': [95,99]})" + "statistics[\"percentiles\"] = (np.percentile, {\"q\": [95, 99]})" ] }, { @@ -485,11 +488,13 @@ ], "source": [ "# Feature detection based on based on surface precipitation field and a range of thresholds\n", - "print('starting feature detection based on multiple thresholds')\n", - "Features= tobac.feature_detection_multithreshold(Precip,dxy,**parameters_features, statistic=statistics) \n", - "print('feature detection done')\n", - "Features.to_hdf(savedir / 'Features.h5','table')\n", - "print('features saved')" + "print(\"starting feature detection based on multiple thresholds\")\n", + "Features = tobac.feature_detection_multithreshold(\n", + " Precip, dxy, **parameters_features, statistic=statistics\n", + ")\n", + "print(\"feature detection done\")\n", + "Features.to_hdf(savedir / \"Features.h5\", \"table\")\n", + "print(\"features saved\")" ] }, { diff --git a/doc/bulk_statistics/notebooks/compute_statistics_during_segmentation.ipynb b/doc/bulk_statistics/notebooks/compute_statistics_during_segmentation.ipynb index 21f51511..0ee1b71a 100644 --- a/doc/bulk_statistics/notebooks/compute_statistics_during_segmentation.ipynb +++ b/doc/bulk_statistics/notebooks/compute_statistics_during_segmentation.ipynb @@ -32,6 +32,7 @@ "import shutil\n", "from six.moves import urllib\n", "from pathlib import Path\n", + "\n", "%matplotlib inline" ] }, @@ -50,10 +51,11 @@ "source": [ "# Disable a few warnings:\n", "import warnings\n", - "warnings.filterwarnings('ignore', category=UserWarning, append=True)\n", - "warnings.filterwarnings('ignore', category=RuntimeWarning, append=True)\n", - "warnings.filterwarnings('ignore', category=FutureWarning, append=True)\n", - "warnings.filterwarnings('ignore',category=pd.io.pytables.PerformanceWarning)" + "\n", + "warnings.filterwarnings(\"ignore\", category=UserWarning, append=True)\n", + "warnings.filterwarnings(\"ignore\", category=RuntimeWarning, append=True)\n", + "warnings.filterwarnings(\"ignore\", category=FutureWarning, append=True)\n", + "warnings.filterwarnings(\"ignore\", category=pd.io.pytables.PerformanceWarning)" ] }, { @@ -79,7 +81,8 @@ "source": [ "# Import tobac itself\n", "import tobac\n", - "print('using tobac version', str(tobac.__version__))" + "\n", + "print(\"using tobac version\", str(tobac.__version__))" ] }, { @@ -95,20 +98,20 @@ }, "outputs": [], "source": [ - "data_out=Path('../')\n", + "data_out = Path(\"../\")\n", "# Download the data: This only has to be done once for all tobac examples and can take a while\n", - "data_file = list(data_out.rglob('data/Example_input_Precip.nc'))\n", + "data_file = list(data_out.rglob(\"data/Example_input_Precip.nc\"))\n", "if len(data_file) == 0:\n", - " file_path='https://zenodo.org/records/3195910/files/climate-processes/tobac_example_data-v1.0.1.zip'\n", - " #file_path='http://zenodo..'\n", - " tempfile=Path('temp.zip')\n", - " print('start downloading data')\n", - " request=urllib.request.urlretrieve(file_path, tempfile)\n", - " print('start extracting data')\n", + " file_path = \"https://zenodo.org/records/3195910/files/climate-processes/tobac_example_data-v1.0.1.zip\"\n", + " # file_path='http://zenodo..'\n", + " tempfile = Path(\"temp.zip\")\n", + " print(\"start downloading data\")\n", + " request = urllib.request.urlretrieve(file_path, tempfile)\n", + " print(\"start extracting data\")\n", " shutil.unpack_archive(tempfile, data_out)\n", " tempfile.unlink()\n", - " print('data extracted')\n", - " data_file = list(data_out.rglob('data/Example_input_Precip.nc'))" + " print(\"data extracted\")\n", + " data_file = list(data_out.rglob(\"data/Example_input_Precip.nc\"))" ] }, { @@ -124,11 +127,11 @@ }, "outputs": [], "source": [ - "#Set up directory to save output:\n", - "savedir=Path(\"Save\")\n", + "# Set up directory to save output:\n", + "savedir = Path(\"Save\")\n", "if not savedir.is_dir():\n", " savedir.mkdir()\n", - "plot_dir=Path(\"Plot\")\n", + "plot_dir = Path(\"Plot\")\n", "if not plot_dir.is_dir():\n", " plot_dir.mkdir()" ] @@ -146,7 +149,7 @@ }, "outputs": [], "source": [ - "Precip=iris.load_cube(str(data_file[0]),'surface_precipitation_average')" + "Precip = iris.load_cube(str(data_file[0]), \"surface_precipitation_average\")" ] }, { @@ -169,17 +172,17 @@ }, "outputs": [], "source": [ - "parameters_features={}\n", - "parameters_features['position_threshold']='weighted_diff'\n", - "parameters_features['sigma_threshold']=0.5\n", - "parameters_features['min_distance']=0\n", - "parameters_features['sigma_threshold']=1\n", - "parameters_features['threshold']=[1,2,3,4,5,10,15] #mm/h\n", - "parameters_features['n_erosion_threshold']=0\n", - "parameters_features['n_min_threshold']=3\n", + "parameters_features = {}\n", + "parameters_features[\"position_threshold\"] = \"weighted_diff\"\n", + "parameters_features[\"sigma_threshold\"] = 0.5\n", + "parameters_features[\"min_distance\"] = 0\n", + "parameters_features[\"sigma_threshold\"] = 1\n", + "parameters_features[\"threshold\"] = [1, 2, 3, 4, 5, 10, 15] # mm/h\n", + "parameters_features[\"n_erosion_threshold\"] = 0\n", + "parameters_features[\"n_min_threshold\"] = 3\n", "\n", "# get temporal and spation resolution of the data\n", - "dxy,dt=tobac.get_spacings(Precip)" + "dxy, dt = tobac.get_spacings(Precip)" ] }, { @@ -218,11 +221,11 @@ ], "source": [ "# Feature detection based on based on surface precipitation field and a range of thresholds\n", - "print('starting feature detection based on multiple thresholds')\n", - "Features= tobac.feature_detection_multithreshold(Precip,dxy,**parameters_features) \n", - "print('feature detection done')\n", - "Features.to_hdf(savedir / 'Features.h5','table')\n", - "print('features saved')" + "print(\"starting feature detection based on multiple thresholds\")\n", + "Features = tobac.feature_detection_multithreshold(Precip, dxy, **parameters_features)\n", + "print(\"feature detection done\")\n", + "Features.to_hdf(savedir / \"Features.h5\", \"table\")\n", + "print(\"features saved\")" ] }, { @@ -255,12 +258,12 @@ "outputs": [], "source": [ "# Dictionary containing keyword arguments for segmentation step:\n", - "parameters_segmentation={}\n", - "parameters_segmentation['method']='watershed'\n", - "parameters_segmentation['threshold']=1 # mm/h mixing ratio\n", + "parameters_segmentation = {}\n", + "parameters_segmentation[\"method\"] = \"watershed\"\n", + "parameters_segmentation[\"threshold\"] = 1 # mm/h mixing ratio\n", "\n", "# get temporal and spation resolution of the data\n", - "dxy,dt=tobac.get_spacings(Precip)" + "dxy, dt = tobac.get_spacings(Precip)" ] }, { @@ -291,9 +294,9 @@ "outputs": [], "source": [ "statistics = {}\n", - "statistics['mean_precip'] = np.mean\n", - "statistics['total_precip'] = np.sum\n", - "statistics['max_precip'] = np.max" + "statistics[\"mean_precip\"] = np.mean\n", + "statistics[\"total_precip\"] = np.sum\n", + "statistics[\"max_precip\"] = np.max" ] }, { @@ -316,7 +319,7 @@ }, "outputs": [], "source": [ - "statistics['percentiles'] = (np.percentile, {'q': [95,99]})" + "statistics[\"percentiles\"] = (np.percentile, {\"q\": [95, 99]})" ] }, { @@ -355,12 +358,16 @@ ], "source": [ "# Perform Segmentation and save resulting mask to NetCDF file:\n", - "print('Starting segmentation based on surface precipitation')\n", - "Mask,Features_Precip=tobac.segmentation_2D(Features,Precip,dxy,**parameters_segmentation, statistic=statistics)\n", - "print('segmentation based on surface precipitation performed, start saving results to files')\n", - "iris.save([Mask], savedir / 'Mask_Segmentation_precip.nc', zlib=True, complevel=4) \n", - "Features_Precip.to_hdf(savedir / 'Features_Precip.h5', 'table')\n", - "print('segmentation surface precipitation performed and saved')" + "print(\"Starting segmentation based on surface precipitation\")\n", + "Mask, Features_Precip = tobac.segmentation_2D(\n", + " Features, Precip, dxy, **parameters_segmentation, statistic=statistics\n", + ")\n", + "print(\n", + " \"segmentation based on surface precipitation performed, start saving results to files\"\n", + ")\n", + "iris.save([Mask], savedir / \"Mask_Segmentation_precip.nc\", zlib=True, complevel=4)\n", + "Features_Precip.to_hdf(savedir / \"Features_Precip.h5\", \"table\")\n", + "print(\"segmentation surface precipitation performed and saved\")" ] }, { diff --git a/doc/bulk_statistics/notebooks/compute_statistics_postprocessing_example.ipynb b/doc/bulk_statistics/notebooks/compute_statistics_postprocessing_example.ipynb index 112f07f2..3ed2bb69 100644 --- a/doc/bulk_statistics/notebooks/compute_statistics_postprocessing_example.ipynb +++ b/doc/bulk_statistics/notebooks/compute_statistics_postprocessing_example.ipynb @@ -29,12 +29,13 @@ "import iris\n", "import numpy as np\n", "import pandas as pd\n", - "import xarray as xr \n", + "import xarray as xr\n", "import matplotlib.pyplot as plt\n", "import datetime\n", "import shutil\n", "from six.moves import urllib\n", "from pathlib import Path\n", + "\n", "%matplotlib inline" ] }, @@ -61,7 +62,8 @@ "source": [ "# Import tobac itself\n", "import tobac\n", - "print('using tobac version', str(tobac.__version__))" + "\n", + "print(\"using tobac version\", str(tobac.__version__))" ] }, { @@ -79,10 +81,11 @@ "source": [ "# Disable a few warnings:\n", "import warnings\n", - "warnings.filterwarnings('ignore', category=UserWarning, append=True)\n", - "warnings.filterwarnings('ignore', category=RuntimeWarning, append=True)\n", - "warnings.filterwarnings('ignore', category=FutureWarning, append=True)\n", - "warnings.filterwarnings('ignore',category=pd.io.pytables.PerformanceWarning)" + "\n", + "warnings.filterwarnings(\"ignore\", category=UserWarning, append=True)\n", + "warnings.filterwarnings(\"ignore\", category=RuntimeWarning, append=True)\n", + "warnings.filterwarnings(\"ignore\", category=FutureWarning, append=True)\n", + "warnings.filterwarnings(\"ignore\", category=pd.io.pytables.PerformanceWarning)" ] }, { @@ -105,11 +108,11 @@ }, "outputs": [], "source": [ - "#Set up directory to save output:\n", - "savedir=Path(\"Save\")\n", + "# Set up directory to save output:\n", + "savedir = Path(\"Save\")\n", "if not savedir.is_dir():\n", " savedir.mkdir()\n", - "plot_dir=Path(\"Plot\")\n", + "plot_dir = Path(\"Plot\")\n", "if not plot_dir.is_dir():\n", " plot_dir.mkdir()" ] @@ -127,20 +130,20 @@ }, "outputs": [], "source": [ - "data_out=Path('../')\n", + "data_out = Path(\"../\")\n", "# Download the data: This only has to be done once for all tobac examples and can take a while\n", - "data_file = list(data_out.rglob('data/Example_input_Precip.nc'))\n", + "data_file = list(data_out.rglob(\"data/Example_input_Precip.nc\"))\n", "if len(data_file) == 0:\n", - " file_path='https://zenodo.org/records/3195910/files/climate-processes/tobac_example_data-v1.0.1.zip'\n", - " #file_path='http://zenodo..'\n", - " tempfile=Path('temp.zip')\n", - " print('start downloading data')\n", - " request=urllib.request.urlretrieve(file_path, tempfile)\n", - " print('start extracting data')\n", + " file_path = \"https://zenodo.org/records/3195910/files/climate-processes/tobac_example_data-v1.0.1.zip\"\n", + " # file_path='http://zenodo..'\n", + " tempfile = Path(\"temp.zip\")\n", + " print(\"start downloading data\")\n", + " request = urllib.request.urlretrieve(file_path, tempfile)\n", + " print(\"start extracting data\")\n", " shutil.unpack_archive(tempfile, data_out)\n", " tempfile.unlink()\n", - " print('data extracted')\n", - " data_file = list(data_out.rglob('data/Example_input_Precip.nc'))" + " print(\"data extracted\")\n", + " data_file = list(data_out.rglob(\"data/Example_input_Precip.nc\"))" ] }, { @@ -156,7 +159,7 @@ }, "outputs": [], "source": [ - "Precip=iris.load_cube(str(data_file[0]),'surface_precipitation_average')" + "Precip = iris.load_cube(str(data_file[0]), \"surface_precipitation_average\")" ] }, { @@ -172,17 +175,17 @@ }, "outputs": [], "source": [ - "parameters_features={}\n", - "parameters_features['position_threshold']='weighted_diff'\n", - "parameters_features['sigma_threshold']=0.5\n", - "parameters_features['min_distance']=0\n", - "parameters_features['sigma_threshold']=1\n", - "parameters_features['threshold']=[1,2,3,4,5,10,15] #mm/h\n", - "parameters_features['n_erosion_threshold']=0\n", - "parameters_features['n_min_threshold']=3\n", + "parameters_features = {}\n", + "parameters_features[\"position_threshold\"] = \"weighted_diff\"\n", + "parameters_features[\"sigma_threshold\"] = 0.5\n", + "parameters_features[\"min_distance\"] = 0\n", + "parameters_features[\"sigma_threshold\"] = 1\n", + "parameters_features[\"threshold\"] = [1, 2, 3, 4, 5, 10, 15] # mm/h\n", + "parameters_features[\"n_erosion_threshold\"] = 0\n", + "parameters_features[\"n_min_threshold\"] = 3\n", "\n", "# get temporal and spation resolution of the data\n", - "dxy,dt=tobac.get_spacings(Precip)" + "dxy, dt = tobac.get_spacings(Precip)" ] }, { @@ -221,11 +224,11 @@ ], "source": [ "# Feature detection based on based on surface precipitation field and a range of thresholds\n", - "print('starting feature detection based on multiple thresholds')\n", - "Features= tobac.feature_detection_multithreshold(Precip,dxy,**parameters_features) \n", - "print('feature detection done')\n", - "Features.to_hdf(savedir / 'Features.h5','table')\n", - "print('features saved')" + "print(\"starting feature detection based on multiple thresholds\")\n", + "Features = tobac.feature_detection_multithreshold(Precip, dxy, **parameters_features)\n", + "print(\"feature detection done\")\n", + "Features.to_hdf(savedir / \"Features.h5\", \"table\")\n", + "print(\"features saved\")" ] }, { @@ -249,12 +252,12 @@ "outputs": [], "source": [ "# Dictionary containing keyword arguments for segmentation step:\n", - "parameters_segmentation={}\n", - "parameters_segmentation['method']='watershed'\n", - "parameters_segmentation['threshold']=1 # mm/h mixing ratio\n", + "parameters_segmentation = {}\n", + "parameters_segmentation[\"method\"] = \"watershed\"\n", + "parameters_segmentation[\"threshold\"] = 1 # mm/h mixing ratio\n", "\n", "# get temporal and spation resolution of the data\n", - "dxy,dt=tobac.get_spacings(Precip)" + "dxy, dt = tobac.get_spacings(Precip)" ] }, { @@ -293,12 +296,18 @@ ], "source": [ "# Perform Segmentation and save resulting mask to NetCDF file:\n", - "print('Starting segmentation based on surface precipitation')\n", - "Mask_Precip,Features_Precip=tobac.segmentation_2D(Features,Precip,dxy,**parameters_segmentation)\n", - "print('segmentation based on surface precipitation performed, start saving results to files')\n", - "iris.save([Mask_Precip], savedir / 'Mask_segmentation_precip.nc', zlib=True, complevel=4) \n", - "Features_Precip.to_hdf(savedir / 'Features_Precip.h5', 'table')\n", - "print('segmentation surface precipitation performed and saved')" + "print(\"Starting segmentation based on surface precipitation\")\n", + "Mask_Precip, Features_Precip = tobac.segmentation_2D(\n", + " Features, Precip, dxy, **parameters_segmentation\n", + ")\n", + "print(\n", + " \"segmentation based on surface precipitation performed, start saving results to files\"\n", + ")\n", + "iris.save(\n", + " [Mask_Precip], savedir / \"Mask_segmentation_precip.nc\", zlib=True, complevel=4\n", + ")\n", + "Features_Precip.to_hdf(savedir / \"Features_Precip.h5\", \"table\")\n", + "print(\"segmentation surface precipitation performed and saved\")" ] }, { @@ -354,9 +363,9 @@ "outputs": [], "source": [ "statistics = {}\n", - "statistics['mean_precip'] = np.mean\n", - "statistics['total_precip'] = np.sum\n", - "statistics['max_precip'] = np.max" + "statistics[\"mean_precip\"] = np.mean\n", + "statistics[\"total_precip\"] = np.sum\n", + "statistics[\"max_precip\"] = np.max" ] }, { @@ -379,7 +388,7 @@ }, "outputs": [], "source": [ - "statistics['percentiles'] = (np.percentile, {'q': [95,99]})" + "statistics[\"percentiles\"] = (np.percentile, {\"q\": [95, 99]})" ] }, { @@ -395,7 +404,9 @@ }, "outputs": [], "source": [ - "features_with_stats = get_statistics_from_mask(Features_Precip, Mask_Precip, Precip, statistic=statistics)" + "features_with_stats = get_statistics_from_mask(\n", + " Features_Precip, Mask_Precip, Precip, statistic=statistics\n", + ")" ] }, { diff --git a/doc/conf.py b/doc/conf.py index 0b8acaa2..02a7c5b6 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -20,6 +20,7 @@ "sphinx.ext.napoleon", "nbsphinx", "sphinx_gallery.load_style", + "sphinx_toolbox.code", ] @@ -63,6 +64,8 @@ def setup(app): sys.path.insert(0, os.path.abspath("../")) +pygments_style = "sphinx" + # Napoleon settings for configuring the Napoleon extension # See documentation here: # https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html diff --git a/doc/examples.rst b/doc/examples.rst index 85160f30..34aac497 100644 --- a/doc/examples.rst +++ b/doc/examples.rst @@ -32,4 +32,19 @@ tobac is provided with a set of Jupyter notebooks that show examples of the appl WRF Updrafts WRF Mesoscale Vorticity + +.. nbgallery:: + :caption: Calculating Bulk Statistics + + Calculating Bulk Statistics during Feature Detection + Calculating Bulk Statistics during Segmentation + Calculating Bulk Statistics as a Postprocessing Step + + +.. nbgallery:: + :caption: Examples of Using *tobac* with Large Datasets and in Parallel + + Simple Dask Parallel Tutorial + + The notebooks can be found in the **examples** folder as part of the python package. The necessary input data for these examples is avaliable on zenodo and can be downloaded automatically by the Jupyter notebooks. \ No newline at end of file diff --git a/doc/feature_detection/notebooks/feature_detection_filtering.ipynb b/doc/feature_detection/notebooks/feature_detection_filtering.ipynb index 1dcc998d..44c3d3d4 100644 --- a/doc/feature_detection/notebooks/feature_detection_filtering.ipynb +++ b/doc/feature_detection/notebooks/feature_detection_filtering.ipynb @@ -82,13 +82,13 @@ ], "source": [ "# Dimensions here are time, y, x.\n", - "input_field_arr = np.zeros((1,100,200))\n", - "input_field_arr[0, 15:85, 10:185]=50\n", - "input_field_arr[0, 20:80, 20:80]=100\n", + "input_field_arr = np.zeros((1, 100, 200))\n", + "input_field_arr[0, 15:85, 10:185] = 50\n", + "input_field_arr[0, 20:80, 20:80] = 100\n", "input_field_arr[0, 40:60, 125:170] = 100\n", - "input_field_arr[0, 30:40, 30:40]=200\n", - "input_field_arr[0, 50:75, 50:75]=200\n", - "input_field_arr[0, 55:70, 55:70]=300\n", + "input_field_arr[0, 30:40, 30:40] = 200\n", + "input_field_arr[0, 50:75, 50:75] = 200\n", + "input_field_arr[0, 55:70, 55:70] = 300\n", "\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", @@ -118,10 +118,14 @@ } ], "source": [ - "# We now need to generate an Iris DataCube out of this dataset to run tobac feature detection. \n", - "# One can use xarray to generate a DataArray and then convert it to Iris, as done here. \n", - "input_field_iris = xr.DataArray(input_field_arr, dims=['time', 'Y', 'X'], coords={'time': [np.datetime64('2019-01-01T00:00:00')]}).to_iris()\n", - "# Version 2.0 of tobac (currently in development) will allow the use of xarray directly with tobac. " + "# We now need to generate an Iris DataCube out of this dataset to run tobac feature detection.\n", + "# One can use xarray to generate a DataArray and then convert it to Iris, as done here.\n", + "input_field_iris = xr.DataArray(\n", + " input_field_arr,\n", + " dims=[\"time\", \"Y\", \"X\"],\n", + " coords={\"time\": [np.datetime64(\"2019-01-01T00:00:00\")]},\n", + ").to_iris()\n", + "# Version 2.0 of tobac (currently in development) will allow the use of xarray directly with tobac." ] }, { @@ -164,23 +168,36 @@ ], "source": [ "thresholds = [50, 100, 150, 200]\n", - "fig, axarr = plt.subplots(2,2, figsize=(10,6))\n", + "fig, axarr = plt.subplots(2, 2, figsize=(10, 6))\n", "sigma_values = [0, 1, 2, 5]\n", "for sigma_value, ax in zip(sigma_values, axarr.flatten()):\n", - " single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', sigma_threshold=sigma_value)\n", - " \n", + " single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " sigma_threshold=sigma_value,\n", + " )\n", + "\n", " # This is what tobac sees\n", - " filtered_field = scipy.ndimage.gaussian_filter(input_field_arr[0], sigma=sigma_value)\n", + " filtered_field = scipy.ndimage.gaussian_filter(\n", + " input_field_arr[0], sigma=sigma_value\n", + " )\n", " color_mesh = ax.pcolormesh(filtered_field)\n", " plt.colorbar(color_mesh, ax=ax)\n", " # Plot all features detected\n", - " ax.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + " ax.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + " )\n", " ax.legend()\n", " if sigma_value == 0:\n", " sigma_val_str = \"0 (off)\"\n", " else:\n", " sigma_val_str = \"{0}\".format(sigma_value)\n", - " ax.set_title(\"sigma_threshold= \"+ sigma_val_str)\n", + " ax.set_title(\"sigma_threshold= \" + sigma_val_str)\n", "plt.tight_layout()\n", "plt.show()" ] @@ -225,31 +242,44 @@ ], "source": [ "thresholds = [100]\n", - "fig, axarr = plt.subplots(2,2, figsize=(10,6))\n", + "fig, axarr = plt.subplots(2, 2, figsize=(10, 6))\n", "erosion_values = [0, 5, 10, 15]\n", "for erosion, ax in zip(erosion_values, axarr.flatten()):\n", - " single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', n_erosion_threshold=erosion)\n", - " \n", + " single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " n_erosion_threshold=erosion,\n", + " )\n", + "\n", " # Create our mask- this is what tobac does internally for each threshold.\n", - " tobac_mask = 1*(input_field_arr[0] >= thresholds[0])\n", - " \n", + " tobac_mask = 1 * (input_field_arr[0] >= thresholds[0])\n", + "\n", " if erosion > 0:\n", - " # This is the parameter for erosion that gets passed to the scikit-image library. \n", + " # This is the parameter for erosion that gets passed to the scikit-image library.\n", " footprint = np.ones((erosion, erosion))\n", - " # This is what tobac sees after erosion. \n", - " filtered_mask = skimage.morphology.binary_erosion(tobac_mask, footprint).astype(np.int64)\n", + " # This is what tobac sees after erosion.\n", + " filtered_mask = skimage.morphology.binary_erosion(tobac_mask, footprint).astype(\n", + " np.int64\n", + " )\n", " else:\n", " filtered_mask = tobac_mask\n", "\n", " color_mesh = ax.pcolormesh(filtered_mask)\n", " # Plot all features detected\n", - " ax.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + " ax.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + " )\n", " ax.legend()\n", " if erosion == 0:\n", " sigma_val_str = \"0 (off)\"\n", " else:\n", " sigma_val_str = \"{0}\".format(erosion)\n", - " ax.set_title(\"n_erosion_threshold= \"+ sigma_val_str)\n", + " ax.set_title(\"n_erosion_threshold= \" + sigma_val_str)\n", "plt.tight_layout()\n", "plt.show()" ] diff --git a/doc/feature_detection/notebooks/multiple_thresholds_example.ipynb b/doc/feature_detection/notebooks/multiple_thresholds_example.ipynb index 64f427a6..39bd33ed 100644 --- a/doc/feature_detection/notebooks/multiple_thresholds_example.ipynb +++ b/doc/feature_detection/notebooks/multiple_thresholds_example.ipynb @@ -73,13 +73,13 @@ ], "source": [ "# Dimensions here are time, y, x.\n", - "input_field_arr = np.zeros((1,100,200))\n", - "input_field_arr[0, 15:85, 10:185]=50\n", - "input_field_arr[0, 20:80, 20:80]=100\n", + "input_field_arr = np.zeros((1, 100, 200))\n", + "input_field_arr[0, 15:85, 10:185] = 50\n", + "input_field_arr[0, 20:80, 20:80] = 100\n", "input_field_arr[0, 40:60, 125:170] = 100\n", - "input_field_arr[0, 30:40, 30:40]=200\n", - "input_field_arr[0, 50:75, 50:75]=200\n", - "input_field_arr[0, 55:70, 55:70]=300\n", + "input_field_arr[0, 30:40, 30:40] = 200\n", + "input_field_arr[0, 50:75, 50:75] = 200\n", + "input_field_arr[0, 55:70, 55:70] = 300\n", "\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", @@ -118,7 +118,11 @@ } ], "source": [ - "input_field_iris = xr.DataArray(input_field_arr, dims=['time', 'Y', 'X'], coords={'time': [np.datetime64('2019-01-01T00:00:00')]}).to_iris()" + "input_field_iris = xr.DataArray(\n", + " input_field_arr,\n", + " dims=[\"time\", \"Y\", \"X\"],\n", + " coords={\"time\": [np.datetime64(\"2019-01-01T00:00:00\")]},\n", + ").to_iris()" ] }, { @@ -160,13 +164,26 @@ } ], "source": [ - "thresholds = [50,]\n", + "thresholds = [\n", + " 50,\n", + "]\n", "# Using 'center' here outputs the feature location as the arithmetic center of the detected feature\n", - "single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', position_threshold='center')\n", + "single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " position_threshold=\"center\",\n", + ")\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", "# Plot all features detected\n", - "plt.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + "plt.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + ")\n", "plt.legend()\n", "plt.title(\"Single Threshold of 50\")\n", "plt.show()" @@ -203,13 +220,26 @@ } ], "source": [ - "thresholds = [150,]\n", + "thresholds = [\n", + " 150,\n", + "]\n", "# Using 'center' here outputs the feature location as the arithmetic center of the detected feature\n", - "single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', position_threshold='center')\n", + "single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " position_threshold=\"center\",\n", + ")\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", "# Plot all features detected\n", - "plt.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + "plt.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + ")\n", "plt.legend()\n", "plt.title(\"Single Threshold of 150\")\n", "plt.show()" @@ -261,14 +291,27 @@ } ], "source": [ - "thresholds = [100, ]\n", + "thresholds = [\n", + " 100,\n", + "]\n", "# Using 'center' here outputs the feature location as the arithmetic center of the detected feature\n", - "single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', position_threshold='center')\n", + "single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " position_threshold=\"center\",\n", + ")\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", "\n", "# Plot all features detected\n", - "plt.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + "plt.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + ")\n", "plt.legend()\n", "plt.title(\"Single Threshold of 100\")\n", "plt.show()" @@ -307,12 +350,23 @@ "source": [ "thresholds = [50, 100, 150, 200]\n", "# Using 'center' here outputs the feature location as the arithmetic center of the detected feature\n", - "single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', position_threshold='center')\n", + "single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " position_threshold=\"center\",\n", + ")\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", "\n", "# Plot all features detected\n", - "plt.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + "plt.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + ")\n", "plt.legend()\n", "plt.title(\"Thresholds: [50, 100, 150, 200]\")\n", "plt.show()" @@ -344,17 +398,27 @@ "source": [ "thresholds = [50, 100, 150, 200, 250]\n", "# Using 'center' here outputs the feature location as the arithmetic center of the detected feature\n", - "single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', position_threshold='center')\n", + "single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " position_threshold=\"center\",\n", + ")\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", "\n", "# Plot all features detected\n", "for i, threshold in enumerate(thresholds):\n", - " thresholded_points = single_threshold_features[single_threshold_features['threshold_value'] == threshold]\n", - " plt.scatter(x=thresholded_points['hdim_2'].values, \n", - " y=thresholded_points['hdim_1'].values, \n", - " color='C'+str(i),\n", - " label=\"Threshold: \"+str(threshold))\n", + " thresholded_points = single_threshold_features[\n", + " single_threshold_features[\"threshold_value\"] == threshold\n", + " ]\n", + " plt.scatter(\n", + " x=thresholded_points[\"hdim_2\"].values,\n", + " y=thresholded_points[\"hdim_1\"].values,\n", + " color=\"C\" + str(i),\n", + " label=\"Threshold: \" + str(threshold),\n", + " )\n", "plt.legend()\n", "plt.title(\"Thresholds: [50, 100, 150, 200]\")\n", "plt.show()" diff --git a/doc/feature_detection/notebooks/position_threshold_example.ipynb b/doc/feature_detection/notebooks/position_threshold_example.ipynb index 37a1231b..016fb245 100644 --- a/doc/feature_detection/notebooks/position_threshold_example.ipynb +++ b/doc/feature_detection/notebooks/position_threshold_example.ipynb @@ -73,13 +73,13 @@ ], "source": [ "# Dimensions here are time, y, x.\n", - "input_field_arr = np.zeros((1,100,200))\n", - "input_field_arr[0, 15:85, 10:185]=50\n", - "input_field_arr[0, 20:80, 20:80]=100\n", + "input_field_arr = np.zeros((1, 100, 200))\n", + "input_field_arr[0, 15:85, 10:185] = 50\n", + "input_field_arr[0, 20:80, 20:80] = 100\n", "input_field_arr[0, 40:60, 125:170] = 100\n", - "input_field_arr[0, 30:40, 30:40]=200\n", - "input_field_arr[0, 50:75, 50:75]=200\n", - "input_field_arr[0, 55:70, 55:70]=300\n", + "input_field_arr[0, 30:40, 30:40] = 200\n", + "input_field_arr[0, 50:75, 50:75] = 200\n", + "input_field_arr[0, 55:70, 55:70] = 300\n", "\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", @@ -109,10 +109,14 @@ } ], "source": [ - "# We now need to generate an Iris DataCube out of this dataset to run tobac feature detection. \n", - "# One can use xarray to generate a DataArray and then convert it to Iris, as done here. \n", - "input_field_iris = xr.DataArray(input_field_arr, dims=['time', 'Y', 'X'], coords={'time': [np.datetime64('2019-01-01T00:00:00')]}).to_iris()\n", - "# Version 2.0 of tobac (currently in development) will allow the use of xarray directly with tobac. " + "# We now need to generate an Iris DataCube out of this dataset to run tobac feature detection.\n", + "# One can use xarray to generate a DataArray and then convert it to Iris, as done here.\n", + "input_field_iris = xr.DataArray(\n", + " input_field_arr,\n", + " dims=[\"time\", \"Y\", \"X\"],\n", + " coords={\"time\": [np.datetime64(\"2019-01-01T00:00:00\")]},\n", + ").to_iris()\n", + "# Version 2.0 of tobac (currently in development) will allow the use of xarray directly with tobac." ] }, { @@ -154,15 +158,28 @@ } ], "source": [ - "thresholds = [50,]\n", - "position_threshold = 'center'\n", - "single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', position_threshold=position_threshold)\n", + "thresholds = [\n", + " 50,\n", + "]\n", + "position_threshold = \"center\"\n", + "single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " position_threshold=position_threshold,\n", + ")\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", "# Plot all features detected\n", - "plt.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + "plt.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + ")\n", "plt.legend()\n", - "plt.title(\"position_threshold \"+ position_threshold)\n", + "plt.title(\"position_threshold \" + position_threshold)\n", "plt.show()" ] }, @@ -205,15 +222,28 @@ } ], "source": [ - "thresholds = [50,]\n", - "position_threshold = 'extreme'\n", - "single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', position_threshold=position_threshold)\n", + "thresholds = [\n", + " 50,\n", + "]\n", + "position_threshold = \"extreme\"\n", + "single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " position_threshold=position_threshold,\n", + ")\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", "# Plot all features detected\n", - "plt.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + "plt.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + ")\n", "plt.legend()\n", - "plt.title(\"position_threshold \"+ position_threshold)\n", + "plt.title(\"position_threshold \" + position_threshold)\n", "plt.show()" ] }, @@ -256,15 +286,28 @@ } ], "source": [ - "thresholds = [50,]\n", - "position_threshold = 'weighted_diff'\n", - "single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', position_threshold=position_threshold)\n", + "thresholds = [\n", + " 50,\n", + "]\n", + "position_threshold = \"weighted_diff\"\n", + "single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " position_threshold=position_threshold,\n", + ")\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", "# Plot all features detected\n", - "plt.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + "plt.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + ")\n", "plt.legend()\n", - "plt.title(\"position_threshold \"+ position_threshold)\n", + "plt.title(\"position_threshold \" + position_threshold)\n", "plt.show()" ] }, @@ -307,15 +350,28 @@ } ], "source": [ - "thresholds = [50,]\n", - "position_threshold = 'weighted_abs'\n", - "single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', position_threshold=position_threshold)\n", + "thresholds = [\n", + " 50,\n", + "]\n", + "position_threshold = \"weighted_abs\"\n", + "single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " position_threshold=position_threshold,\n", + ")\n", "plt.pcolormesh(input_field_arr[0])\n", "plt.colorbar()\n", "# Plot all features detected\n", - "plt.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + "plt.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + ")\n", "plt.legend()\n", - "plt.title(\"position_threshold \"+ position_threshold)\n", + "plt.title(\"position_threshold \" + position_threshold)\n", "plt.show()" ] }, @@ -357,18 +413,30 @@ } ], "source": [ - "thresholds = [50,]\n", - "fig, axarr = plt.subplots(2,2, figsize=(10,6))\n", - "testing_thresholds = ['center', 'extreme', 'weighted_diff', 'weighted_abs']\n", + "thresholds = [\n", + " 50,\n", + "]\n", + "fig, axarr = plt.subplots(2, 2, figsize=(10, 6))\n", + "testing_thresholds = [\"center\", \"extreme\", \"weighted_diff\", \"weighted_abs\"]\n", "for position_threshold, ax in zip(testing_thresholds, axarr.flatten()):\n", - "\n", - " single_threshold_features = tobac.feature_detection_multithreshold(field_in = input_field_iris, dxy = 1000, threshold=thresholds, target='maximum', position_threshold=position_threshold)\n", + " single_threshold_features = tobac.feature_detection_multithreshold(\n", + " field_in=input_field_iris,\n", + " dxy=1000,\n", + " threshold=thresholds,\n", + " target=\"maximum\",\n", + " position_threshold=position_threshold,\n", + " )\n", " color_mesh = ax.pcolormesh(input_field_arr[0])\n", " plt.colorbar(color_mesh, ax=ax)\n", " # Plot all features detected\n", - " ax.scatter(x=single_threshold_features['hdim_2'].values, y=single_threshold_features['hdim_1'].values, color='r', label=\"Detected Features\")\n", + " ax.scatter(\n", + " x=single_threshold_features[\"hdim_2\"].values,\n", + " y=single_threshold_features[\"hdim_1\"].values,\n", + " color=\"r\",\n", + " label=\"Detected Features\",\n", + " )\n", " ax.legend()\n", - " ax.set_title(\"position_threshold \"+ position_threshold)\n", + " ax.set_title(\"position_threshold \" + position_threshold)\n", "plt.tight_layout()\n", "plt.show()" ] diff --git a/doc/requirements.txt b/doc/requirements.txt index 550ba3d6..99bb7e46 100644 --- a/doc/requirements.txt +++ b/doc/requirements.txt @@ -3,3 +3,4 @@ nbsphinx numpy sphinx_rtd_theme sphinx-gallery +sphinx-toolbox