Deep-learning-powered data analysis in plankton ecology

By Harshith Bachimanchi, Matthew I. M. Pinder, Chloé Robert, Pierre De Wit, Jonathan Havenhand, Alexandra Kinnby, Daniel Midtvedt, Erik Selander and Giovanni Volpe.

The repository contains source code and data for the article, Deep-learning-powered data analysis in plankton ecology.

Description

We provide examples of how to use deep learning for plankton data analysis. The examples are provided as Jupyter notebooks and can be run on Google Colab. The examples are divided into three categories:

• detection-tutorials: For plankton detection from microscopy videos
• segmentation-tutorials: For plankton segmentation and classification from microscopy images
• trajectory-tutorials: For plankton trajectory linking from microscopy videos

Usage

Detection-tutorials

We provide three examples of plankton detection from microscopy videos using deep learning. The examples are provided as Jupyter notebooks and can be found in detection-tutorials folder. The pdf versions of the notebooks are included for quick preview. Alternatively, the notebooks can be run on Google Colab by clicking on the links below:

1. 1-detection_plankton1.ipynb demonstrates the detection of plankton species Oxyrrhis marina from microscopy videos.

2. 2-detection_plankton2.ipynb demonstrates the detection of plankton species Dunaliella tertiolecta from microscopy videos.

3. 3-detection_plankton3.ipynb demonstrates the detection of plankton species Isochrysis galbana from microscopy videos.

Segmentation-tutorials

In these set of tutorials, we will see how to generate simulated datasets to train a deep learning model for plankton segmentation and classification. The examples are provided as Jupyter notebooks and can be found in segmentation-tutorials. The pdf versions of the notebooks are included for quick preview. Alternatively, the notebooks can be run on Google Colab by clicking on the links below:

1. 4-simulation_plankton.ipynb demonstrates how to simulate images that look closer to experimental images using 'DeepTrack' software package. In this tutorial, we simulate images of plankton species Noctiluca scintillans and Dunaliella tertiolecta and use the simulated images to train a deep learning model for segmentation and classification. The training code is provided in the next tutorial notebook.

2. 5-training_UNet_segmentation.ipynb demonstrates how to train a U-Net model for segmentation and classification of plankton species Noctiluca scintillans and Dunaliella tertiolecta using the simulated images generated in the previous tutorial. The trained model is then used to segment and classify the experimental images of the same species.

Citation

If you use this code for your research, please consider citing our papers:

1. Deep-learning-powered data analysis in plankton ecology.

Harshith Bachimanchi, Matthew I. M. Pinder, Chloé Robert, Pierre De Wit, Jonathan Havenhand, Alexandra Kinnby, Daniel Midtvedt, Erik Selander and Giovanni Volpe.
"Deep-learning-powered data analysis in plankton ecology."
bioRxiv (2021).
https://doi.org/

2. Microplankton life histories revealed by holographic microscopy and deep learning.

Harshith Bachimanchi, Benjamin Midtvedt, Daniel Midtvedt, Erik Selander, Giovanni Volpe (2022).
"Microplankton life histories revealed by holographic microscopy and deep learning."
eLife 11:e79760.
https://doi.org/10.7554/eLife.79760

3. Single-shot self-supervised object detection in microscopy.

Midtvedt, B., Pineda, J., Skärberg, F. et al.
"Single-shot self-supervised object detection in microscopy."
Nat Commun 13, 7492 (2022).

4. Geometric deep learning reveals the spatiotemporal features of microscopic motion.

Jesús Pineda, Benjamin Midtvedt, Harshith Bachimanchi, Sergio Noé, Daniel  Midtvedt, Giovanni Volpe,1 and  Carlo  Manzo
"Geometric deep learning reveals the spatiotemporal fingerprint ofmicroscopic motion."
arXiv 2202.06355 (2022).

See also DeepTrack 2.1 GitHub repository for other useful examples on using deep learning for microscopy data analysis: DeepTrack2.1:

Benjamin Midtvedt, Saga Helgadottir, Aykut Argun, Jesús Pineda, Daniel Midtvedt, Giovanni Volpe.
"Quantitative Digital Microscopy with Deep Learning."
Applied Physics Reviews 8 (2021), 011310.
https://doi.org/10.1063/5.0034891