0.4.2

🧰 Maintenance

Bump version to 0.4.2 @elaubsch (#90)

Bump PyPI release version to 0.4.2.

0.4.1

🚀 Features

Update spot detection model to SpotDetection-8 @elaubsch (#84)

This PR updates the default model for the SpotDetection application to SpotDetection-8. The model hash and model metadata have also been updated accordingly. The training data used to train SpotDetection-8(SpotNet v1.1) adds Airlocalize to the set of spot detection methods used to create the consensus spot labels.

🐛 Bug Fixes

Update model name for SpotDetection-8 @elaubsch (#85)

This PR updates the model name for SpotDetection-8.

Remove deprecated parameter from training script @elaubsch (#83)

This PR removes the deprecated alpha parameter from a call of DotNetLosses in training.py.

0.4.0

Improve print statements for Polaris application @elaubsch (#82)

This PR makes the print statements for prediction progress more informative. It add a progress bar for the spot detection prediction and removes a less useful progress bar for gene assignment.

Add skip round functionality to spot prediction in Polaris @elaubsch (#68)

This PR adds a check for rounds with no labeling in defined codebook. If detected, these rounds will be skipped during spot detection to prevent hallucination.

Update copyright to 2023 @elaubsch (#48)

This PR updates the copyright to 2023.

Tighten CI configuration @rossbar (#46)

A couple minor tweaks to the CI configuration to (hopefully) prevent duplication and unnecessary runs on experimental branches. Specifically:

• 0ce6c0c limits CI so that runs are only triggered when either 1) a PR to master is opened, or 2) there is a new push to a branch from which a PR already originates. This should prevent CI from running when commits are pushed up to non-master branches.
• 48f4bc7 adds a check which will cancel any in-progress jobs when new changes are pushed up. This can help e.g. when you push two commits up in rapid succession. By default, GH will queue up the 2nd set of jobs and wait for the first set to finish. With this option, the first set of jobs will be cancelled and the 2nd job will start immediately.

The motivation for these changes is to reduce the CI load in private repositories.

Add back deleted function and bug fix @elaubsch (#35)

This PR adds back the ca_to_adjacency_matrix which was removed in a previous PR. It also fixes a bug in the graph visualization functions.

🚀 Features

Move masking function outside of Polaris application @elaubsch (#81)

This PR moves the abstracted _mask_spots to be outside of the application. The function is now available in results_utils.py. The unit tests and example notebook for Polaris have been updated.

Add authentication to SpotDetection @elaubsch (#75)

This PR adds authentication to SpotDetection by using fetch_data to download the spot detection model upon instantiation. fetch_data requires a DeepCell API key.

For this reason, a large number of the tests in polaris_tests.py have been temporarily removed, because models cannot be downloaded in the GitHub actions test environment without an API key.

Pixel-wise decoding for Polaris @elaubsch (#73)

This PR introduces an algorithmic change to Polaris that increases the number of pixels sent through the SpotDecoding application. Previously, we performed peak finding to determine which pixels were decoded. Now with this change, we will threshold the spot probability image and decode all pixels above a certain spot probability. Then, we create a mask for all of the pixels decoded to genes and apply this mask to the spot probability image. We then perform peak finding on this masked image to call the gene locations. This change is like a compromise between Polaris' original method and pixel-wise decoding methods that are common in MERFISH analysis pipelines. We found that this method increases the number of spots decoded to genes by Polaris, while yielding results with a better correlation to bulk sequencing data.

This change redefines of the threshold parameter for the predict method. Instead of being used in peak finding, this parameter is used to create a mask for tissue area. Therefore, the default value has been changed.

This PR also includes changes the output of Polaris that are unrelated to the algorithmic change. Polaris previously returned df_spots and df_intensities, but now these two outputs have been concatenated column-wise to yield a single DataFrame.

Add function for barcode assignment to cells for optical pooled screens @elaubsch (#70)

This PR adds a function that processes Polaris predictions for barcode assignment to cells for optical pooled screens. Unit tests have been added for this function.

Add get\_cell\_counts to results utils @elaubsch (#67)

This PR adds the function get_cell_counts to results_utils. This utility function converts the Polaris output format to a gene expression per cell table for compatibility with downstream analysis packages like scanpy and Seurat. Unit tests for this function have been added.

This PR also adds an example notebook to demonstrate how to use get_cell_counts to generate an input for scanpy.

Add gene scatter plot to results utils @elaubsch (#65)

This PR adds an additional function gene_scatter to results_utils. This function creates a scatter plot with Plotly to visualize the location of decoded genes. It also adds some additional arguments to expression_correlation to offer more control over the plot appearance and input.

Add arguments to expression\_correlation @elaubsch (#64)

This PR adds arguments to expression_correlation:

1. log: Boolean that determines whether to create the scatter plot in log space.
2. exclude_genes: List of outlier genes excluded from the plot.
3. exclude_zeros: Boolean that determines whether zero counts from control and experimental sets are excluded.
4. eps: A small epsilon value added to the counts to avoid errors taking the log of zero counts.

This function yields a figure and is not tested, so no additional testing has been added to cover the code added for these arguments.

This PR also includes minor changes to docstrings in results_utils.py.

Add results\_utils functions @elaubsch (#63)

This PR adds a few utility functions for processing and visualizing the results output by Polaris. Many of these functions require plotly which has been added to the requirements file. Unit tests have been added for the functions added in this PR that don't output a figure. This PR include also includes a minor change to a Polaris application docstring.

Add masking of bright background objects @elaubsch (#57)

This PR adds a function, _mask_spots, to the Polaris application, which creates a mask for bright fluorescent objects in the background image of a FISH sample. It then marks all detected spots inside this mask as 'masked', so they can be exclude from downstream analysis. This information is added to the decoding_result and appears as an additional column in the main Polaris output.

Because of the increasing complexity of the Polaris prediction inputs, a function , _validate_prediction_input was added to Polaris. This method checks the shapes of the inputs spots_image, segmentation_image, and background_image. It also checks the values of threshold and mask_threshold.

Test cases have been added to cover these two new functions.

Add mixed barcode rescue to SpotDecoding @elaubsch (#56)

This PR adds a function _rescue_mixed_spots to the SpotDecoding application. This function addresses the case of mixed barcodes caused by spatial crowding of spots. An argument rescue_mixed has been added to the predict method of SpotDecoding to toggle this function. A test case has been added to cover this function. Print statements have been added to make the prediction more verbose to make the amount of error correction more obvious.

The function _rescue_spots has been refactored to _rescue_errors, because there are now two methods for rescuing spots. The exposed argument rescue_spots has also been changed to rescue_errors.

Regardless of error correction, two items have been added to the dictionary returned by SpotDetection.predict.

1. spot_index indexes the spots, because rescue_mixed_spots introduces the case that two gene assignments can be made for the same spot. In that case, a new entry is added to the output with the same index as the original spot.

2. source details the origin of a prediction. Its values can be:

   • 'prediction' from SpotDetection.predict
   • 'error rescue' from rescue_errors
   • 'mixed rescue' from _rescue_mixed_spots

Add Bernoulli to decoding distributions @elaubsch (#54)

This PR adds Bernoulli as an option for decoding distributions. Bernoulli has the same options for numbers of parameters as Relaxed Bernoulli, so the arguments for defining the model distribution have changed. There is a new argument distribution which has valid values ['Gaussian', 'Bernoulli', 'Relaxed Bernoulli']. This is a departure from the previous logic where Relaxed Bernoulli was implied unless params_mode was set to 'Gaussian'. The argument params_mode has the same valid values, except 'Gaussian'.

This PR also adds a _validate_spots_intensities function which will verify the spots_intensities_vec input into the SpotDecoding application, because the different distribution options have different requirements for input values. This function aims to return an error message that will be more interpretable to the user than the PyTorch message. Tests have been added for invalid examples of spots_intensities_vec. Logic has been added to the Polaris application to input the correctly pre-processed spot intensities into the SpotDecoding application. The singleplex version of the app now returns the original pixel values at the spot locatio...

0.3.2

🐛 Bug Fixes

Bug fix for model output format @elaubsch (#29)

This PR fixes a bug in assign_gene_identities introduced by changes to the output of the SpotDetection application.

Update image alignment functions and tests @elaubsch (#28)

This PR fixes a bug in align_images and adds tests for this function.

🧰 Maintenance

Add unit tests for model and loss scripts @elaubsch (#32)

This PR adds unit tests for dotnet.py, dotnet_losses.py, and utils.py. It also adds the "docs" badge to the README.

Update model version @elaubsch (#30)

This PR updates the version of the model used by the SpotDetection application.

📚️ Documentation

Add Read the Docs documentation @elaubsch (#31)

This PR adds Read the Docs documentation. This includes big changes to the docstrings for the majority of the functions in the package.

0.3.1

🐛 Bug Fixes

Update supported Python versions @elaubsch (#27)

This PR changes the supported Python versions as defined in setup.py.

0.3.0

🚀 Features

Refactor EM functions to use DataFrames @elaubsch (#24)

This PR introduces large changes to the functions used to handle coordinates data during EM for training data creation. The new functions use Pandas DataFrames and dictionaries instead of nested lists of lists, which makes them much less fragile and opaque. No changes were made to the functions used to perform EM, so no changes to the EM output are expected.

Upgrade to TensorFlow 2.8 @elaubsch (#25)

This PR upgrades the Tensorflow version from 2.5.1 to 2.8.0 and drops support for Python 3.6. All tensorflow.python.keras imports were changed to tensorflow.keras. The base Docker image and required DeepCell version have been upgraded to 0.12.0, which also requires Tensorflow 2.8.0.

A new version of the spots model has been trained with Tensorflow 2.8 and results are comparable to previous models. During this process, changes were made to the functions used to assemble the training data set without changing underlying functionality (see #24).

0.2.1

🧰 Maintenance

Change model output to dictionary @elaubsch (#22)

Update PyPI version @elaubsch (#23)

0.2.0

Remove postcode import from setup.py @elaubsch (#21)

🚀 Features

Update example notebook for singleplex app @elaubsch (#18)

Add Mesmer segmentation to Polaris app @elaubsch (#17)

Add Mesmer to options of segmentation applications for Polaris

Singleplex Polaris app @elaubsch (#15)

Adding the singleplex application, which will now be referred to as the Polaris application. The previous iteration of the Polaris application (without cell segmentation) has been refactored to the SpotDetection application.

🧰 Maintenance

Remove PoSTcode import @elaubsch (#20)

PoSTcode repo is no longer cloned as a requirement for this repository, necessary functions have been copied to local scripts

Update copyright to 2022 @elaubsch (#14)

Update copyright year to 2022 and some minor dependency fixes

0.1.0

🚀 Features

Rename SpotDetection application to Polaris @elaubsch (#13)

Repo management @elaubsch (#8)

Added Dataset objects to load data needed to run example notebooks

Model dev @elaubsch (#6)

Adding singleplex code and notebook

Model dev @elaubsch (#3)

Update Dockerfile @elaubsch (#2)

🐛 Bug Fixes

Update Dockerfile, fix imports, and lint docstrings. @willgraf (#9)

• Update Dockerfile to use published deepcell-tf images.
• Change opencv-python to opencv-python-headless for a smaller, lighter installation (Fixes #4)
• Lint a bunch of docstrings
• Fix import ordering with isort
• Ignore training.py from coverage reports.
• Add .idea directory to .gitignore and remove it from the project.

---

Existing issues:

• rn_classification_head uses imports that don't exist
• gene_counts_DBSCAN: DBSCAN is never imported

🧰 Maintenance

Update README badges and other minor touchups for first release. @willgraf (#12)

• update link for coverage badge
• add license badge
• updates .gitattributes to ignore the Jupyter notebooks from code statistics.
• replace space with - in package name
• fix python_requires definition in setup.py
• publish docker images on release.

Remove unused code and files @elaubsch (#11)

Fixes #10 and removes unnecessary data directory.

Repo management @elaubsch (#7)

📚️ Documentation

Update Dockerfile, fix imports, and lint docstrings. @willgraf (#9)

• Update Dockerfile to use published deepcell-tf images.
• Change opencv-python to opencv-python-headless for a smaller, lighter installation (Fixes #4)
• Lint a bunch of docstrings
• Fix import ordering with isort
• Ignore training.py from coverage reports.
• Add .idea directory to .gitignore and remove it from the project.

---

Existing issues:

• rn_classification_head uses imports that don't exist
• gene_counts_DBSCAN: DBSCAN is never imported