Can I inverse image after using 'tio.SubjectsDataset' and 'torch.utils.data.DataLoader'?

[Bigsealion]

Hi
When I use Torchio, I have a new question.
I'm using torch.utils.data.DataLoader to load a dataset from tio.SubjectsDataset, and apply some augment (Specifically, tio.CropOrPad), but I can't inverse this image to raw shape.

My test code as follow:

# load subj
subject_a = tio.Subject(
    img=tio.ScalarImage(nii_path),
    mask=tio.LabelMap(mask_path),
)
subjects_list = [subject_a]

# transform
onehot = tio.OneHot()
pad = tio.CropOrPad((96, 112, 96))  # raw is (91, 109, 91)
transforms = [onehot, pad]
transform = tio.Compose(transforms)

# load data by torch
subjects_dataset = tio.SubjectsDataset(subjects_list, transform=transform)
training_loader = DataLoader(subjects_dataset, batch_size=1, num_workers=4)

for subjects_batch in training_loader:
    img = subjects_batch['img'][tio.DATA]
    mask = subjects_batch['mask'][tio.DATA]

tio_subj = tio.utils.get_subjects_from_batch(subjects_batch)[0] 
tio_subj = tio_subj.apply_inverse_transform()
print('raw shape: {}'.format(subject_a.img.shape))
print('new shape: {}'.format(tio_subj.img.shape))

And I get a warning as follow:

/python3.6/site-packages/torchio/transforms/augmentation/composition.py:69: RuntimeWarning: No invertible transforms found
  warnings.warn('No invertible transforms found', RuntimeWarning)
raw shape: (1, 91, 109, 91)
new shape: (1, 96, 112, 96)

I have read the doc about invertibility, but it's using transform(subject) but not DataLoader() (I think which is a convenient function).

I want to know, is any method to invert images after DataLoader(), or have to refactor my program and give up DataLoader() and tio.SubjectsDataset, and use transform() to each subject manually?

Thanks!

[romainVala]

Hi
not sure about your specific question. But be aware that neither CropOrPad nor OneHot are invertible.
this is the warning you get ...
To make CorOrPad invertible, we should store in the transform history the image original shape, which is kind of weird ...
Any way if you want to go back to your original space, just do an other CropOrPad(91,109,91) no ?

[fepegar]

CropOrPad is invertible, but lossy. This happens because the transforms history is being ignored. I will commit a patch and suggest a solution in a minute.

[romainVala]

oups, my bad, I search for invert_transform (and did not found it) but ok, CropOrPad is usign Crop and Pad that are invertible !
But note that is not a strict inversion since you will not recover the 'loss' data (if you first crop, and then pad to to the inverse, )
What about OneHot it is not invertible right?

[fepegar]

oups, my bad, I search for invert_transform (and did not found it) but ok, CropOrPad is usign Crop and Pad that are invertible !

Exactly! The same happens with e.g. RandomAnisotropy, which just uses Resample.

But note that is not a strict inversion since you will not recover the 'loss' data (if you first crop, and then pad to to the inverse, )

That's what I meant by "lossy". See Invertibility.

What about OneHot it is not invertible right?

At the moment it's not implemented, but I guess we can do it easily. It's just argmax, I guess.

[fepegar]

Hi, @Bigsealion. That's a good question.

That feature is currently not implemented, but I will commit a patch in a second. After that, your code should work with a slight modification: use collate_fn=tio.utils.history_collate for the DataLoader.

import torchio as tio
from torch.utils.data import DataLoader

subject_a = tio.datasets.Colin27()
subjects_list = [subject_a]

# transform
onehot = tio.OneHot()
pad = tio.CropOrPad((96, 112, 96))
transforms = [onehot, pad]
transform = tio.Compose(transforms)

# load data by torch
subjects_dataset = tio.SubjectsDataset(subjects_list, transform=transform)
training_loader = DataLoader(
    subjects_dataset,
    collate_fn=tio.utils.history_collate,
)

for subjects_batch in training_loader:
    tio_subj = tio.utils.get_subjects_from_batch(subjects_batch)[0]
    tio_subj = tio_subj.apply_inverse_transform()
    print('raw shape: {}'.format(subject_a.t1.shape))
    print('new shape: {}'.format(tio_subj.brain.shape))

[Bigsealion]

Thanks for your enthusiastic reply!
This new patch is very helpful!

Now, the tio.CropOrPad() (if first pad and then inverse) is invertible, but the tio.OneHot() is not invertible, right?

This is my test code (Follow the above code):

mask_label = np.argmax(tio_subj.mask.numpy(), axis=0)

print('raw image shape: {}'.format(subject_a.img.shape))
print('new image shape: {}'.format(tio_subj.img.shape))
print('raw image is exactly the same as new image: {}\n'.format(np.all(tio_subj.img.numpy() == subject_a.img.numpy())))

print('raw mask shape: {}'.format(subject_a.mask.shape))
print('new mask shape: {}'.format(tio_subj.mask.shape))
print('raw mask is exactly the same as new mask: {}'.format(np.all(tio_subj.mask.numpy() == subject_a.mask.numpy())))
print('raw mask is exactly the same as argmax new mask: {}'.format(np.all(mask_label == subject_a.mask.numpy())))

and the out is:

raw image shape: (1, 91, 109, 91)
new image shape: (1, 91, 109, 91)
raw image is exactly the same as new image: True

raw mask shape: (1, 91, 109, 91)
new mask shape: (2, 91, 109, 91)
raw mask is exactly the same as new mask: False
raw mask is exactly the same as argmax new mask: True

Thanks!

[fepegar]

It will be! See

• Add logic to invert OneHot transform #748

[Bigsealion]

Thanks! This will make Torchio more convenient!
Please tell me if you finish this update, and I can remove np.argmax in my code!

[fepegar]

This is now available in v0.18.67.

[Bigsealion]

I'm so sorry but I still can't inverse tio.OneHot in v0.18.67, and I want to know what's wrong with my code.

This is test code:

subject_a = tio.datasets.Colin27()
subjects_list = [subject_a]

# transform
onehot = tio.OneHot()
pad = tio.CropOrPad((96, 112, 96))
transforms = [onehot, pad]
transform = tio.Compose(transforms)

# load data by torch
subjects_dataset = tio.SubjectsDataset(subjects_list, transform=transform)
training_loader = DataLoader(
    subjects_dataset,
    collate_fn=tio.utils.history_collate,
)

for subjects_batch in training_loader:
    tio_subj = tio.utils.get_subjects_from_batch(subjects_batch)[0]
    tio_subj = tio_subj.apply_inverse_transform()

    print('Torchio version: {}'.format(tio.__version__))
    print('raw shape: {}'.format(subject_a.t1.shape))
    print('new shape: {}'.format(tio_subj.brain.shape))

and the out as follow:

Torchio version: 0.18.67
raw shape: (1, 181, 217, 181)
new shape: (2, 181, 217, 181)

Thanks!

[Bigsealion]

I guess I know why the apply_inverse_transform doesn't work on tio.OneHot() data after tio.utils.get_subjects_from_batch()

@fepegar
The type of new mask (created by tio.utils.get_subjects_from_batch()) was set to torchio.data.image.ScalarImage, but in raw mask, which is torchio.data.image.LabelMap.

The test code is as follow:

subject_a = tio.datasets.Colin27()
subjects_list = [subject_a]

# transform
onehot = tio.OneHot()
pad = tio.CropOrPad((96, 112, 96))
transforms = [onehot, pad]
transform = tio.Compose(transforms)

# load data by torch
subjects_dataset = tio.SubjectsDataset(subjects_list, transform=transform)
training_loader = DataLoader(
    subjects_dataset,
    collate_fn=tio.utils.history_collate,
)

for subjects_batch in training_loader:
    tio_subj = tio.utils.get_subjects_from_batch(subjects_batch)[0]
    tio_subj = tio_subj.apply_inverse_transform()

    print('tio version: {}'.format(tio.__version__))
    print('raw mask shape: {}'.format(subject_a.brain.shape))
    print('new mask shape: {}'.format(tio_subj.brain.shape))
    print('raw mask type: {}'.format(type(subject_a.brain)))
    print('new mask type: {}'.format(type(tio_subj.brain)))

out:

tio version: 0.18.67
raw mask shape: (1, 181, 217, 181)
new mask shape: (2, 181, 217, 181)
raw mask type: <class 'torchio.data.image.LabelMap'>
new mask type: <class 'torchio.data.image.ScalarImage'>

[fepegar]

This is a bug. Thanks again for reporting!

[fepegar]

The fix will be available in v0.18.68.

[Bigsealion]

Thanks, it's helpful!