Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error when running steinbock classify ilastik run #195

Open
nilseling opened this issue Jun 12, 2023 · 4 comments
Open

Error when running steinbock classify ilastik run #195

nilseling opened this issue Jun 12, 2023 · 4 comments

Comments

@nilseling
Copy link
Contributor

I re-ran the bash script shown here:

#!/usr/bin/env bash
BASEDIR=$(cd -- "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)
cd "${BASEDIR}"

# raw data collection
mkdir raw
wget https://zenodo.org/record/7997296/files/IMCWorkflow.ilp
wget https://zenodo.org/record/7997296/files/analysis.zip
unzip analysis.zip
rm analysis.zip
rm -r analysis/cpinp
rm -r analysis/cpout
rm -r analysis/histocat
rm -r analysis/ilastik
rm -r analysis/ometiff
cd raw
wget https://zenodo.org/record/7575859/files/panel.csv
wget https://zenodo.org/record/7575859/files/Patient1.zip
wget https://zenodo.org/record/7575859/files/Patient2.zip
wget https://zenodo.org/record/7575859/files/Patient3.zip
wget https://zenodo.org/record/7575859/files/Patient4.zip
cd ${BASEDIR}

# steinbock alias setup
shopt -s expand_aliases
alias steinbock="docker run -v ${BASEDIR}:/data -u $(id -u):$(id -g) ghcr.io/bodenmillergroup/steinbock:0.16.0"

# raw data preprocessing
steinbock preprocess imc panel --namecol Clean_Target
steinbock preprocess imc images --hpf 50

# random forest-based segmentation using Ilastik/CellProfiler
steinbock classify ilastik prepare --cropsize 500 --seed 123
rm pixel_classifier.ilp && mv IMCWorkflow.ilp pixel_classifier.ilp
rm -r ilastik_crops && mv analysis/crops ilastik_crops
steinbock classify ilastik fix --no-backup
steinbock classify ilastik run
steinbock segment cellprofiler prepare
steinbock segment cellprofiler run -o masks_ilastik

In the past (using an older version of the ImcSegmentationPipeline output) worked fine but now I'm getting the following error:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/app/steinbock/steinbock/__main__.py", line 17, in <module>
    steinbock_cmd_group(prog_name="steinbock")
  File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/app/steinbock/steinbock/_cli/utils.py", line 34, in wrapper
    return func(*args, **kwargs)
  File "/app/steinbock/steinbock/classification/_cli/ilastik.py", line 336, in fix_cmd
    ilastik.fix_ilastik_project_file_inplace(
  File "/app/steinbock/steinbock/classification/ilastik/_ilastik.py", line 390, in fix_ilastik_project_file_inplace
    _fix_pixel_classification_group_inplace(
  File "/app/steinbock/steinbock/classification/ilastik/_ilastik.py", line 438, in _fix_pixel_classification_group_inplace
    block = np.transpose(block, axes=transpose_axes)
  File "<__array_function__ internals>", line 180, in transpose
  File "/opt/steinbock-venv/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 660, in transpose
    return _wrapfunc(a, 'transpose', axes)
  File "/opt/steinbock-venv/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc
    return bound(*args, **kwds)
ValueError: axes don't match array
/opt/ilastik/ilastik-meta/lazyflow/lazyflow/classifiers/__init__.py:20: UserWarning: init: Could not import tiktorch classifier
  warnings.warn("init: Could not import tiktorch classifier")
INFO ilastik_main: Starting ilastik from "/opt/ilastik".
Starting ilastik from "/opt/ilastik".
WARNING 2023-06-12 13:35:28,748 opConservationTracking 31 139775893624640 Could not find any ILP solver
WARNING 2023-06-12 13:35:28,756 opStructuredTracking 31 139775893624640 Could not find any ILP solver
WARNING 2023-06-12 13:35:28,758 structuredTrackingWorkflow 31 139775893624640 Could not find any learning solver. Tracking will use flow-based solver (DPCT). Learning for tracking will be disabled!
INFO ilastik.shell.projectManager: Opening Project: pixel_classifier.ilp
WARNING dataset.py(313): H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead.
WARNING opSimpleBlockedArrayCache.py(43): FutureWarning: Conversion of the second argument of issubdtype from `dtype` to `np.generic` is deprecated. In future, it will be treated as `np.object_ == np.dtype(dtype).type`.
WARNING opSlicedBlockedArrayCache.py(141): FutureWarning: Conversion of the second argument of issubdtype from `dtype` to `np.generic` is deprecated. In future, it will be treated as `np.object_ == np.dtype(dtype).type`.
ERROR 2023-06-12 13:35:31,985 log_exception 31 139775893624640 Traceback (most recent call last):
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/shell/projectManager.py", line 460, in _loadProject
    serializer.deserializeFromHdf5(self.currentProjectFile, projectFilePath, self._headless)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 1151, in deserializeFromHdf5
    ss.deserialize(topGroup)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/pixelClassification/pixelClassificationSerializer.py", line 100, in deserialize
    super().deserialize(group)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 264, in deserialize
    self._deserialize(group[self.name], self.inslot)
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/utility/timer.py", line 153, in wrapper
    return func(*args, **kwargs)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 622, in _deserialize
    self.inslot[index][slicing] = blockArray
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
    p[key] = value
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
    p[key] = value
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
    p[key] = value
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 988, in __setitem__
    self.operator.setInSlot(self.top_level_slot, self.subindex, roi, value)
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 394, in setInSlot
    self._setInSlotInput(slot, subindex, roi, new_pixels)
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 416, in _setInSlotInput
    block_rois = getIntersectingRois(self.Output.meta.shape, self._blockshape, (roi.start, roi.stop))
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 639, in getIntersectingRois
    block_rois = list(map(partial(getBlockBounds, dataset_shape, blockshape), block_starts))
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 667, in getBlockBounds
    block_bounds = getIntersection(block_bounds, entire_dataset_roi)
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 402, in getIntersection
    assert ((stop - start) > 0).all(), "Rois do not intersect: {} and {}".format(roiA, roiB)
AssertionError: Rois do not intersect: (array([0, 0, 2]), array([253, 252,   3])) and ([0, 0, 0], [500, 500, 1])

ERROR 2023-06-12 13:35:31,985 log_exception 31 139775893624640 Project could not be loaded due to the exception shown above.
Aborting Project Open Action
ERROR 2023-06-12 13:35:31,986 excepthooks 31 139775893624640 Unhandled exception in thread: 'MainThread'
ERROR 2023-06-12 13:35:31,988 excepthooks 31 139775893624640 Traceback (most recent call last):
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik.py", line 134, in <module>
    main()
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik.py", line 128, in main
    hShell = ilastik_main.main(parsed_args, workflow_cmdline_args)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik_main.py", line 170, in main
    f(shell)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik_main.py", line 347, in loadProject
    shell.openProjectFile(path, parsed_args.readonly)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/shell/headless/headlessShell.py", line 125, in openProjectFile
    self.projectManager._loadProject(hdf5File, projectFilePath, readOnly)
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/utility/timer.py", line 153, in wrapper
    return func(*args, **kwargs)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/shell/projectManager.py", line 460, in _loadProject
    serializer.deserializeFromHdf5(self.currentProjectFile, projectFilePath, self._headless)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 1151, in deserializeFromHdf5
    ss.deserialize(topGroup)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/pixelClassification/pixelClassificationSerializer.py", line 100, in deserialize
    super().deserialize(group)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 264, in deserialize
    self._deserialize(group[self.name], self.inslot)
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/utility/timer.py", line 153, in wrapper
    return func(*args, **kwargs)
  File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 622, in _deserialize
    self.inslot[index][slicing] = blockArray
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
    p[key] = value
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
    p[key] = value
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
    p[key] = value
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 988, in __setitem__
    self.operator.setInSlot(self.top_level_slot, self.subindex, roi, value)
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 394, in setInSlot
    self._setInSlotInput(slot, subindex, roi, new_pixels)
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 416, in _setInSlotInput
    block_rois = getIntersectingRois(self.Output.meta.shape, self._blockshape, (roi.start, roi.stop))
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 639, in getIntersectingRois
    block_rois = list(map(partial(getBlockBounds, dataset_shape, blockshape), block_starts))
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 667, in getBlockBounds
    block_bounds = getIntersection(block_bounds, entire_dataset_roi)
  File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 402, in getIntersection
    assert ((stop - start) > 0).all(), "Rois do not intersect: {} and {}".format(roiA, roiB)
AssertionError: Rois do not intersect: (array([0, 0, 2]), array([253, 252,   3])) and ([0, 0, 0], [500, 500, 1])

I'm not quite sure where this is coming from. What I changed was the version of the ImcSegmentationPipeline that I used for generating the ilastik crops and I upgraded from Ilastik v1.3.3 to v1.4.0. A similar issue was reported in #188 so it would be good to investigate where this is coming from.
@nilseling nilseling changed the title Adjustments to steinbock classify ilastik fix Error when running steinbock classify ilastik run Jun 12, 2023
@nilseling
Copy link
Contributor Author

This can be addressed together with #191

@Milad4849
Copy link
Contributor

Steinbock docker container is fixed to ilastik-1.3.3, it appears that classifiers from ilastik 14.0 and above are not compatible with the older versions. That is the cause of this and #188.

@Milad4849
Copy link
Contributor

Given above is this actually a bug? Shall we update ilastik with the next release? @jwindhager

@nilseling
Copy link
Contributor Author

Did you check if updating to 1.4.0 fixes the issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nilseling @Milad4849