Merge pull request #128 from fmi-basel/add_docs_info

Add task description to all tasks

Author: nrepina

.github/workflows/ci.yml

@@ -40,7 +40,7 @@ jobs:
       - name: Install Dependencies
         run: |
           python -m pip install -U pip
-          python -m pip install .[test,fractal-tasks]
+          python -m pip install .[test,fractal-tasks,spherical-harmonics]
 
       - name: 🧪 Run Tests
         run: pytest --color=yes --cov --cov-report=xml --cov-report=term-missing

setup.cfg

@@ -29,6 +29,7 @@ project_urls =
 
 [options]
 # TODO: fix issue 104 and remove numpy version requirement
+# TODO: fix pyshtools dependency issue on UZH cluster
 packages = find:
 install_requires =
     SimpleITK
@@ -38,7 +39,6 @@ install_requires =
     numcodecs
     numpy<2
     pandas
-    pyshtools
     scikit-image>=0.21.0
     scikit-learn
     scipy
@@ -56,7 +56,9 @@ where = src
 [options.extras_require]
 fractal-tasks =
     anndata
-    fractal-tasks-core==1.3.3
+    fractal-tasks-core==1.3.4
+spherical-harmonics =
+    pyshtools
 plotting =
     anndata
     dask

src/scmultiplex/__FRACTAL_MANIFEST__.json

@@ -0,0 +1,14 @@
+### Purpose
+- Calculates a **consensus linking table** across all multiplexing rounds in an HCS OME-Zarr dataset.
+- Aligns object labels from all acquisitions to a reference acquisition, ensuring consistent object identities across rounds.
+- Stores the resulting consensus table in the reference acquisition directory.
+
+### Outputs
+- A **consensus linking table** that maps object labels from all rounds to a single, aligned consensus.
+- The table includes:
+  - Original object labels from each round (e.g., `R0_label`, `R1_label`, ...).
+  - A new consensus label (`consensus_label`) and index (`consensus_index`) for aligned objects.
+
+### Limitations
+- Requires pre-existing linking tables generated by a previous linking task (e.g., `Calculate Object Linking`).
+- Assumes that the input linking tables follow a consistent structure across rounds.

src/scmultiplex/dev/task_info/calculate_linking_consensus.md

@@ -0,0 +1,14 @@
+### Purpose
+- Links segmented objects between a reference and an alignment acquisition within a single well of an HCS OME-Zarr dataset.
+- Calculates object shifts using segmentation label maps, aligns objects, and identifies matching labels based on an Intersection over Union (IoU) cutoff threshold.
+- Generates a linking table that maps object labels from the reference acquisition to those in the alignment acquisition.
+
+### Outputs
+- A linking table stored in the alignment acquisition directory.
+- The table includes matched object pairs and their IoU scores.
+
+### Limitations
+- Only works for HCS OME-Zarr datasets where a **single well ROI** is used for linking. Multi-ROI processing (e.g., for FOV ROI tables) is not yet supported.
+- Requires segmentation label maps to be provided for both the reference and alignment acquisitions.
+- Matching is performed using an IoU threshold; objects below the threshold are not linked.
+- Pixel sizes must match between the reference and alignment acquisitions for accurate registration.

src/scmultiplex/dev/task_info/calculate_object_linking.md

@@ -0,0 +1,15 @@
+### Purpose
+- Calculates point-cloud-based registration between segmentation images using **PlatyMatch**.
+- Works well for **complex 3D registration**.
+- Aligns sub-objects (e.g., nuclei) that belong to parent objects (e.g., organoids) by calculating **affine** and optionally **free-form deformation** transformations.
+- Outputs linking tables of matched sub-objects and optionally saves transformation matrices to disk.
+
+### Outputs
+- A **linking table** that maps sub-objects between reference and alignment rounds using affine and/or free-form deformation (FFD) transformations.
+- Transformation matrices (optional), saved on disk for each object pair.
+
+### Limitations
+- Only supports **single well ROI tables**; multi-ROI processing (e.g., FOV ROIs) is not yet implemented.
+- Requires parent objects to be linked in a prior step using a **consensus linking table**.
+- Assumes consistent pixel sizes between reference and alignment rounds for accurate registration.
+- Relies on sufficient sub-object counts for alignment; regions with fewer than 3 sub-objects are skipped.

src/scmultiplex/dev/task_info/calculate_platymatch_registration.md

@@ -0,0 +1,12 @@
+"""Generate JSON schemas for tasks and write them to the Fractal manifest."""
+
+from fractal_tasks_core.dev.create_manifest import create_manifest
+
+if __name__ == "__main__":
+    PACKAGE = "scmultiplex"
+    AUTHORS = "Nicole Repina, Enrico Tagliavini, Tim-Oliver Buchholz, Joel Luethi"
+    docs_link = "https://github.com/fmi-basel/gliberal-scMultipleX"
+    if docs_link:
+        create_manifest(package=PACKAGE, authors=AUTHORS, docs_link=docs_link)
+    else:
+        create_manifest(package=PACKAGE, authors=AUTHORS)

src/scmultiplex/dev/task_info/create_manifest.py

@@ -0,0 +1,12 @@
+### Purpose
+- Expands segmented **labels** in 2D or 3D images **without overlap**.
+- Supports expansion by a fixed pixel distance or dynamically based on **label size**.
+- Optionally masks expanded labels using **parent objects** to prevent spillover.
+- Outputs an expanded label image and preserves non-overlapping object boundaries.
+
+### Outputs
+- A new **expanded label image** saved with an `_expanded` suffix.
+
+### Limitations
+- If masking by parent is enabled, the parent object label image must be provided.
+- Expansion beyond object boundaries may be clipped, depending on the surrounding labels and image dimensions.

src/scmultiplex/dev/task_info/expand_labels.md

@@ -0,0 +1,12 @@
+### Purpose
+- **Measures intensity and morphology features** from existing segmentation images in an OME-Zarr dataset.
+- Computes advanced 3D morphology metrics, including surface area, using extended `regionprops` measurements.
+- Supports both intensity-based and morphology-only measurements:
+   - If no input intensity channels are provided, the task calculates morphology features only.
+   - For intensity measurements, channels can be specified individually, allowing flexibility across different image inputs.
+- Enables **measurements within masked objects** (e.g., measuring nuclei properties within organoids) by specifying an `input_ROI_table` that defines parent regions, such as organoid ROIs.
+
+### Limitations
+- Currently tested only on image data in the **CZYX** format.
+- Measurement accuracy and performance may depend on the spacing and resolution of input images.
+- Does not support measurements at lower resolutions (e.g., beyond level 0).

src/scmultiplex/dev/task_info/feature_measurements.md

@@ -0,0 +1,18 @@
+### Purpose
+- Extracts detailed **shape features** from 3D meshes, including volume, surface area, solidity, concavity, and aspect ratio.
+- Optionally calculates **Gaussian curvature** for mesh surfaces and **spherical harmonics** to decompose mesh geometry.
+- Outputs feature tables, curvature meshes, and optional harmonic-reconstructed meshes.
+
+### Outputs
+1. A **feature table** with extracted mesh measurements, including:
+   - Volume, surface area, aspect ratio, concavity, normalized surface area-to-volume ratio, and more.
+2. Optional **spherical harmonic coefficients** and reconstruction error (MSE), saved as a separate table.
+3. Optional **reconstructed meshes** from spherical harmonics saved as `.stl`.
+4. Optional **Gaussian curvature meshes** saved as `.vtp` files in a new `curvature` folder.
+5. Optional **convex hull** and **bounding box** meshes saved as `.vtp` files.
+
+### Limitations
+- Requires pre-existing **.stl meshes** within the Zarr structure (e.g., generated by the **Surface Mesh Multiscale** task).
+- Mesh files must match the **object labels** specified in the ROI table.
+- Computational cost increases when enabling **Gaussian curvature** or **spherical harmonics** calculations.
+- Mesh quality and accuracy depend on the input segmentation and mesh generation parameters.

src/scmultiplex/dev/task_info/mesh_measurements.md

@@ -0,0 +1,14 @@
+### Purpose
+- Relabels segmentation images and ROI tables in an OME-Zarr dataset based on a pre-calculated **consensus linking table**.
+- Aligns object labels across multiplexing rounds to ensure consistency with the consensus object identities.
+- Outputs new segmentation images and ROI tables with updated labels.
+- Requires a valid **consensus linking table** as input, typically generated by the `Calculate Linking Consensus` task.
+- Non-consensus objects are set to background and excluded from the outputs.
+
+### Outputs
+- A **new segmentation image** where object labels match the consensus labels.
+- A **relabelled ROI table** corresponding to the updated segmentation image, saved with a `_linked` suffix.
+
+### Limitations
+- Only supports **single well ROI tables**; multi-ROI processing (e.g., field of view ROIs) is not yet implemented.
+- Input segmentation images and ROI tables must match in terms of object labels; inconsistencies will cause errors.

src/scmultiplex/dev/task_info/relabel_by_linking_consensus.md

@@ -0,0 +1,15 @@
+### Purpose
+- Performs **full 3D object segmentation** of raw intensity images using intensity thresholding.
+- Combines two intensity channels, applies **Gaussian smoothing** and **Canny edge detection** for refined masks.
+- Filters out debris and neighboring objects by selecting the **largest connected component** within a masked region.
+- Outputs a new 3D segmentation label image and an updated masking ROI table.
+
+### Outputs
+- A **new 3D label image** stored in the dataset, with refined object segmentation.
+- A corresponding **bounding-box ROI table** saved as `{output_label_name}_ROI_table`.
+
+### Limitations
+- Requires pre-segmented 2D MIP-based ROI regions as input for masking.
+- Supports intensity thresholding with either **Otsu's method** or a user-defined threshold.
+- Assumes consistent image resolution and pixel intensities across channels.
+- Regions with extreme intensity variations or overlapping objects may require manual parameter tuning for optimal results.

src/scmultiplex/dev/task_info/segment_by_intensity_threshold.md

@@ -0,0 +1,13 @@
+### Purpose
+- Computes **spherical harmonics** for 3D segmented objects in a label image using the **aics_shparam** library.
+- Calculates and analyzes the **shape features** of objects, including reconstruction error.
+- Outputs spherical harmonic coefficients and optionally saves reconstructed surface meshes.
+
+### Outputs
+- A **feature table** containing spherical harmonic coefficients and reconstruction error (**MSE**) per object.
+- Optionally, the **computed surface mesh** and the **reconstructed mesh** (from harmonics), saved as `.stl` files in a new `meshes` folder.
+
+### Limitations
+- Input label image must contain 3D segmented objects; neighboring objects are removed by masking.
+- The accuracy of spherical harmonics depends on the chosen **maximum degree (`lmax`)** and input label quality.
+- Mesh reconstruction might smooth out fine details in highly complex shapes.

src/scmultiplex/dev/task_info/spherical_harmonics_from_label_image.md

@@ -0,0 +1,16 @@
+### Purpose
+- Calculates **3D surface meshes** for parent objects (e.g., tissues, organoids) based on 3D child-level segmentation (e.g., nuclei).
+- Optionally applies **multiscale label fusion** to estimate a smooth parent shape by merging child objects.
+- Generates smoothened surface meshes using **VTK algorithms**, with optional mesh decimation for reduced complexity.
+- Outputs 3D meshes in `.stl` or `.vtp` format and a new well label map in the dataset.
+
+### Outputs
+- **Surface meshes** of parent objects, saved as `.stl` (single object) or `.vtp` (multi-object) files in the dataset’s `meshes` folder.
+- A **new label map** containing fused child-level objects, saved in the OME-Zarr dataset (only if multiscale processing is enabled).
+- A **bounding-box ROI table** corresponding to the new label map.
+
+### Limitations
+- Requires pre-segmented child objects and a parent object ROI table.
+- Multiscale processing requires a **parent label** for accurate object grouping and fusion.
+- Label map outputs may have **overlaps clipped**, where higher-label IDs take precedence in dense regions.
+- Mesh quality can vary with complex geometries; manual tuning of smoothing parameters may be needed for optimal results.

src/scmultiplex/dev/task_info/surface_mesh_multiscale.md

@@ -26,6 +26,7 @@
         category="Registration",
         modality="HCS",
         tags=["multiplexing", "2D"],
+        docs_info="file:task_info/calculate_object_linking.md",
     ),
     CompoundTask(
         name="scMultiplex Calculate Linking Consensus",
@@ -36,6 +37,7 @@
         category="Registration",
         modality="HCS",
         tags=["multiplexing", "2D", "3D"],
+        docs_info="file:task_info/calculate_linking_consensus.md",
     ),
     CompoundTask(
         name="scMultiplex Relabel by Linking Consensus",
@@ -46,6 +48,7 @@
         category="Registration",
         modality="HCS",
         tags=["multiplexing", "2D", "3D"],
+        docs_info="file:task_info/relabel_by_linking_consensus.md",
     ),
     CompoundTask(
         name="scMultiplex Calculate Platymatch Registration",
@@ -56,6 +59,7 @@
         category="Registration",
         modality="HCS",
         tags=["multiplexing", "3D"],
+        docs_info="file:task_info/calculate_platymatch_registration.md",
     ),
     CompoundTask(
         name="scMultiplex Surface Mesh Multiscale",
@@ -66,6 +70,7 @@
         category="Image Processing",
         modality="HCS",
         tags=["3D", "mesh"],
+        docs_info="file:task_info/surface_mesh_multiscale.md",
     ),
     CompoundTask(
         name="scMultiplex Segment by Intensity Threshold",
@@ -76,6 +81,7 @@
         category="Segmentation",
         modality="HCS",
         tags=["Classical segmentation", "3D"],
+        docs_info="file:task_info/segment_by_intensity_threshold.md",
     ),
     CompoundTask(
         name="scMultiplex Spherical Harmonics from Label Image",
@@ -86,6 +92,7 @@
         category="Measurement",
         modality="HCS",
         tags=["3D"],
+        docs_info="file:task_info/spherical_harmonics_from_label_image.md",
     ),
     CompoundTask(
         name="scMultiplex Mesh Measurements",
@@ -96,18 +103,21 @@
         category="Measurement",
         modality="HCS",
         tags=["3D", "mesh", "morphology"],
+        docs_info="file:task_info/mesh_measurements.md",
     ),
     ParallelTask(
         name="scMultiplex Feature Measurements",
         executable="fractal/scmultiplex_feature_measurements.py",
         meta={"cpus_per_task": 4, "mem": 16000},
         category="Measurement",
         tags=["regionprops", "morphology", "intensity"],
+        docs_info="file:task_info/feature_measurements.md",
     ),
     ParallelTask(
         name="scMultiplex Expand Labels",
         executable="fractal/expand_labels.py",
         meta={"cpus_per_task": 4, "mem": 16000},
         tags=["2D", "3D"],
+        docs_info="file:task_info/expand_labels.md",
     ),
 ]