Interpolation of Labels

[justusschock]

Hi, your docs state that

Instances of LabelMap are always resampled using nearest neighbor interpolation, independently of the interpolation type specified at transform instantiation, which will be used for instances of ScalarImage.

I was wondering if this really is the best approach to do so. Wouldn't it make more sense for LabelMaps to be converted to one-hot, interpolated with the same type as ScalarImage and then be argmaxed again?

I could imagine the interpolation to be inconsistent w.r.t images (though I do understand why interpolation on labels without the one-hot approach has to be do interpolated this way).

[fepegar]

Hi, @justusschock. That's a good point. This was shortly discussed with @romainVala in #342. I believe the conclusion was that it's not really worth it. Especially when there is a large number of labels, like in torchio.datasets.FPG. If you disagree, we can do some tests and implement this type of interpolation.

[justusschock]

Hi @fepegar,

Thanks for pointing me there (and sorry for not seeing this before). I think the tests conducted by @romainVala show that even though the differences may be small there are some. I am thinking of using this to interpolate based on a registration and therefore I would like my mask transformation to follow the image transformation as closely as possible.

[fepegar]

@romainVala I believe you said the tests were actually not reliable. Could you please confirm whether they made a significant difference for you?

[romainVala]

Hi there
yes there are some difference, but are they important enough to matter, is a difficult question, which will depend on the exact application ... so unless one try it, one could not know. (but if it is not implemented, one could not test ...!)

I check the dice score, and the differences was very small, but this is dependent of the label size you check ... (Gray matter in my case)

Actually I have an other use case: I do work with Partial Volume label, (similar to one-hot encoded label, but with continuous values in the [0 1] interval, (and the sum over channel ==1)
So in this case, it make more sens to test other interpolation,

So I vote +1 to add this functionality may be an easy way to do is to separate the problem
1 the user must add the one-hot transform first
2 then resample with the argument, like label_interpolation=...
3 convert back from one-hot to label, (is there a transform to do this ?)

the point 2 is then very easy to add, with few changes ... no ?

[romainVala]

may be it is not that easy, as pointed here
#342 (comment)

[justusschock]

@romainVala I think it is hard to check if the one-hot transform was added beforehand. Also I think a transform should be kind of self-contained. I think it should be something like

1.) Check if current format is one-hot (or at least the sum over channels == 1 and the min-max per channel between 0 and 1) and store previous format
2.) If 1.) evaluates to False, convert to onehot
3.) do interpolation
4.) if it was categorical before (see stored previous format from 1.): Argmax it again.

3 convert back from one-hot to label, (is there a transform to do this ?)

I don't think there is one, but that's pretty easy to add.

[romainVala]

ok, it may be more convinient to have it all in one

[fepegar]

Is this to apply an affine transformation? If yes, how would you handle the missing values? I guess you would need to treat the background and the rest of the labels differently. Although sometimes there is no background...

[romainVala]

yes, exactly, this was were I was stopped in my first test,
I did not find out a simple way to associate missing value to the background,
because you need to do the argmax for all voxel expect those with missing value

Any way it may be interesting to specify it with argument
add_missing_value_to = labe_id

doing so we can even add a new id in the label, (which is an interesting options since missing values does not look as background ... so may be one want to treat it separately ...