An elegant data analysis tool for CyTOF.
This package is an all-in-one CyTOF data analysis package for your experiments. From loading datasets to DR and evaluation, you have a consistent interface and readable code every step along the way. There is also support for some of HDCytoData's benchmark datasets as originally implemented in R by Weber & Soneson (2019) in this repository. Why wait? Start your PyCytoData journey right here, right now!
You can install PyCytoData easily from pip:
pip install PyCytoData
or from conda:
conda install pycytodata -c kevin931 -c bioconda
If you wish to use CytofDR along with PyCytoData, use can optionally install it as well:
pip install CytofDR
For more information on optional dependencies or installation details, look here.
You can load the data easily with the following python snippet:
>>> from PyCytoData import DataLoader
>>> exprs = DataLoader.load_dataset(dataset = "levine13")
>>> exprs.expression_matrix # Expression matrix
>>> exprs.cell_types # Cell types
>>> exprs.sample_index # Sample index
>>> exprs.features # The feature/marker namesThe resulting exprs is a PyCytoData object, which is easy to use. The expression matrix, cell types (if available), and sample index are directly accessible with attributes, and they are all stored as numpy.array. You can also access some metadata of the object with the following attributes:
>>> exprs.n_cells
>>> exprs.n_cell_types
>>> exprs.n_samples
>>> exprs.n_featuresAll these metadata is automatically set, and there is protection in place for unintended changes. You can also add a sample with the following:
>>> exprs.add_sample(expression_matrix, cell_types, sample_index) # All inputs should be ArrayLikeNote: The data are downloaded from a server instead of being shipped with this package. Each dataset only needs to be downloaded once, which is automatically managed. During the first-time download of the data, a command-line confirmation is needed.
Yes, you read it right! You can load your own datasets. Currently, we only support reading in plain text files with saved with delimiters. The data need to have cells as rows and features as columns. To do load them in as a PyCytoData object, you can simply do the following:
>>> from PyCytoData import FileIO
>>> FileIO.load_delim(files="/path", # Path to file
... col_names=True, # Whether the first row is feature (column) names
... delim="\t" # Delimiter
... ) If your experiment has multiple samples, you can simply import them together:
>>> from PyCytoData import FileIO
>>> expression_paths = ["path1", "path2", "path3"]
>>> FileIO.load_delim(files=expression_paths, # Path to file
... col_names=True, # Whether the first row is feature (column) names
... delim="\t" # Delimiter
... ) In this case, the expression matrices are concatenated automatically without any normalization. To access particular samples, you can access the sample_index of the attribute and use the standard numpy indexing techniques.
Note: This technique does not automatically load cell types. In fact, it does not not mixed-datatype array, except for column names. You will need to read in cell types and set them using the cell_types attribute of the object.
Currently, levine13, levine32, and samusik have all been mostly preprocessed. All you need to do is to perform aecsinh transformaion. You can simply do this:
>>> from PyCytoData import DataLoader
>>> exprs = DataLoader.load_dataset(dataset = "levine13", preprocess=True)When you perform BYOD, you can have much more flexibility:
>>> from PyCytoData import FileIO
>>> byod = FileIO.load_delim(files="/path", # Path to file
... col_names=True, # Whether the first row is feature (column) names
... delim="\t" # Delimiter
... )
>>> byod.lineage_channels = ["CD4", "CD8", "FoxP3", "CD15"]
>>> byod.preprocess(arcsinh=True,
... gate_debris_removal=True,
... gate_intact_cells=True,
... gate_live_cells=True,
... gate_center_offset_residual=True,
... bead_normalization=True)
>>> byod.expression_matrix # This is preprocessedAs the example shows, we support five unique preprocessing steps! And of course, you can use a subset of these to suit your own needs! By default, we automatically detect the necessary channels, such as "Bead1" or "Center". However, if your dataset is unconventionally named, our auto-detect algorithm may fail. Thus, we can perform a manual override:
>>> byod.preprocess(arcsinh=True,
... gate_debris_removal=True,
... gate_intact_cells=True,
... gate_live_cells=True,
... gate_center_offset_residual=True,
... bead_normalization=True,
... bead_channels = ["1bead", "2bead"],
... time_channel = ["clock"])If you wish to run DR on your dataset, you can easily do so as well if you have CytofDR installed (assume you have loaded the dataset and preprocessed it accordingly):
>>> exprs.run_dr_methods(methods = ["PCA", "UMAP", "ICA"])
Running PCA
Running ICA
Running UMAP
>>> type(exprs.reductions)
<class 'CytofDR.dr.Reductions'>The reductions attribute is a Reductions object from CytofDR. You can perform all downstream DR workflows as usual.
We only support the following datasets as of now. The Literal is the string literal used in this package to refer to the datasets whereas the Dataset Name is what these datasets are more commonly known for.
| Dataset Name | Literal |
|---|---|
| Levine-13dim | levine13 |
| Levine-32dim | levine32 |
| Samusik | samusik |
More datasets will be added in the future to be fully compatible with HDCytoData and to potentially incorporate other databases.
For detailed documentation along with tutorials and API Reference, please visit our Official Documentation. This is automatically updated with each update.
If you prefer to build documentation on your own, refer to this guide for more details.
This is a minor release that fixes a critical bug that affects all previous releases. Update is strongly recommended. A few quality-of-life improvements are included as well.
- Fixed a critical issue with subsetting channels not updating internal indices lineage channels.
- Fixed a verbiage error for subsetting error messages. Now, it is explicitly stated that integer indexing is not supported.
- Updated documentation to fix typos.
- Updated CI pipeline to include newest Python releases.
- Added our logo usage policy.
- Clarified the python version needed to run
PyCytoData. - No new software feature added.
If you used PyCytoData in your research or with Cytomulate as part of the pipeline, please cite our paper here:
Yang, Y., Wang, K., Lu, Z. et al. Cytomulate: accurate and efficient simulation of CyTOF data. Genome Biol 24, 262 (2023). https://doi.org/10.1186/s13059-023-03099-1
or with our BibTex:
@article {Yang2023,
author = {Yang, Yuqiu and Wang, Kaiwen and Lu, Zeyu and Wang, Tao and Wang, Xinlei},
title = {Cytomulate: accurate and efficient simulation of CyTOF data},
journal={Genome biology},
volume={24},
number={262},
year={2023},
publisher={Springer}
}
If you use PyCytoData to perform DR, citing the our DR Review paper is highly appreciated:
Wang, K., Yang, Y., Wu, F. et al. Comparative analysis of dimension reduction methods for cytometry by time-of-flight data. Nat Commun 14, 1836 (2023). https://doi.org/10.1038/s41467-023-37478-w
or
@article{wang2023comparative,
title={Comparative analysis of dimension reduction methods for cytometry by time-of-flight data},
author={Wang, Kaiwen and Yang, Yuqiu and Wu, Fangjiang and Song, Bing and Wang, Xinlei and Wang, Tao},
journal={Nature communications},
volume={14},
number={1},
pages={1--18},
year={2023},
publisher={Nature Publishing Group UK London}
}
If you use the builtin datasets, please visit our Reference Page and cite the papers accordingly.