Image segmentation is a process that breaks down an image into smaller segments according to some criteria. For semantic segmentation this means that all pixels of one segment represent an object (or objects) of a single class; in this case a class has been predefined to represent a type of object of interest. For example in the slide image shown below, the objects can be classified as being a specimen (in the centre), regular labels (either side of the specimen), type labels (red circle), barcode labels, or otherwise as being part of the 'background'.
Segments can be broken down further into instances, which represent distinct objects separately - even if they have the same class. For example, the labels on either side of the slide image would be part of the same 'segment' (even though it is not contiguous), but are treated as separate instances. A corresponding representation of instances for the slide image can be seen below, where each unique colour maps pixels to a specific instance.
In situations where labelled example data is hard to come by, as is the case for manually generated example segmentations for slides, it is desirable to use methods that can perform well with small datasets. Semi-supervised learning covers a number of techniques to leverage large datasets of unlabelled data to enhance the capability of models otherwise learned on small amounts of data.
This project combines approaches from each of these problems, primarily using the following prior works as reference:
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Semantic Instance Segmentation with a Discriminative Loss Function
- Semi-Supervised Learning with Ladder Networks
The implementation is entirely in Python and all dependencies can be installed most straightforwardly by using the Anaconda package manager. Creating a new conda environment is done using the command:
conda env create -f environment.yml
which will create a new environment with the name segmentation. It is assumed that a PyTorch-supported GPU is available; notices of dropping support for GPU models can be found in the PyTorch release notes. For older GPUs it may still be possible to install PyTorch from source although behaviour may not be guaranteed. Adapting the code to work without a GPU should be straightforward but could be prohibitively slow - especially for training.
Notes on usage can be found in the wiki