Clarification on the density scaling

[LukasNickel]

Hi!

After having a deeper look into the code while debugging problems related with #346,
I came across the HU->density transformation.
Since I do not have a strong medical background, it is a bit hard for me to make sense of the code here. I would have expected a simple linear transformation to get into the 0,1 range (potentially with special consideration regarding the -1024 fill values).

If you don't mind clarifying:

1. Why would you set the data in the "soft tissue range" to the lowest observed value? I guess it increases the bone contrast a bit, but that is not how a real x-ray would work, right? If the improved contrast is the goal that's fine, but I would have assumed you want to keep it as close to realistic as possible?
2. The .min() and .max() scalings do seem to be suboptimal in general. Isn't one huge advantage of the hounsfield scale that you can define reasonable ranges without looking at the particular data?

Granted, the second point is not really an issue with full CT volumes, BUT I ran into somewhat weird behaviour using (again) masks: The ranges are calculated on the full volume and only later the mask is applied. If, on the other hand, I use a masked volume directly (meaning my volume data only contains the region of interest), the scaling might be wildly different.
Maybe that is a super niche edge-case as the real CT volumes observed in the wild will not be "pre-segmented", but I still don't think it is intentional.

[eigenvivek]

I think there's a slight clarification for point 1: data in the air range is set to the min of soft_tissue. By default (i.e., bone_attenuation_multiplier = 1.0), soft_tissue and bone still obey the same linear relationship. Increasing bone_attenuation_multiplier just serves to make bones brighter, which while not exactly how X-ray works, provides a useful data augmentation mechanism for when training neural networks (see here). Please let me know if I misunderstood your question.

For point 2, you're right that that minmax scaling is a hacky way of implementing this conversion. There are some more principled ways of doing this (e.g., see #200), but haven't had time to change them. But ofc, happy to accept PRs to change this behavior!

[LukasNickel]

Apologies, you are of course right: It is the air range, that is set to the minimum of the soft tissue range. The way you do it, air and the low end of the soft tissue range both end up at 0.
That will probably affect the visual output way less than I thought (if at all)...
I was just worried about the resulting nonlinearities in the data scaling, but that mostly originated from my misunderstanding.

I am fine with minmax scaling, but I argue that the range should not be calculated on the specific sample (at least once masks are introduced). Instead I would personally hardcode the range (as the CT values are always on the Hounsfield scale) to make sure the scaling does not depend on the input.
That way different CT files with and without masks would be scaled in the same way and one can more easily compare the image values of different x-ray images.
That is debatable though, there are advantages to both approaches.

[eigenvivek]

If I'm not mistaken, I think a given CT scan with and without a mask is scaled by the same way.

Before masking the volume, the density is computed using the global min/max of the entire CT:

DiffDRR/diffdrr/data.py

Lines 80 to 82 in 6a486b6

	# Convert the volume to density
	density = transform_hu_to_density(volume.data, bone_attenuation_multiplier)
	density = ScalarImage(tensor=density, affine=volume.affine)

Only after this is the mask applied:

DiffDRR/diffdrr/data.py

Lines 134 to 141 in 6a486b6

	# Apply mask
	if labels is not None:
	if isinstance(labels, int):
	labels = [labels]
	mask = torch.any(
	torch.stack([subject.mask.data.squeeze() == idx for idx in labels]), dim=0
	)
	subject.density.data = subject.density.data * mask

[LukasNickel]

You are right again (and I need to be precise with what I write...).
Using the labelmap argument to read a mask alongside the full volume, everything is fine.

My issue arises only if you do not use a full CT + mask, but instead create a new file, that only contains the masked region or whatever your model predicts as the bone of interest and everything else is zero / -1024.
In that case, the values might not reflect what is inside a full CT. For example, there might be almost no soft tissue present.
Let's say you had a model predicting CT volumina, but it only predicts the bone structure, because all of the soft tissue was removed prior to training.

Like I said: Might be niche use and I am totally fine with closing this after it turned out I simply got confused about Point 1.