Adapting 3D Unet segmentation network for image regression

[hasansari]

Hello,

Thank you all your hard work developing and maintaining MONAI. I have been enjoying it very much for my image segmentation work.

I also created a similar discussion a few months ago, regarding the use of MONAI libraries to perform MR to CT synthesis, I have adapted my segmentation code, which uses MONAI's UNET archiecture with my own L1 loss function to perform this task. I have got some initial results which were not too bad but were in different intensities.

My question is what would be the easiest way to adapt MONAI's UNET architecture to perform 3D image synthesis tasks? Is it possible to have a tutorial i.e. adapting 3D spleen segmentation jupyter notebook for MR-CT regression?

Many thanks

[Nic-Ma]

Hi @wyli @ericspod ,

Could you please help share some help info here?

Thanks in advance.

[ericspod]

With the base UNet class in MONAI you can set the dimensionality to be 2D or 3D with the dimensions argument. With that done the class can be used like an autoencoder in your task, though whether it works better than using the AutoEncoder class itself is up to you to discover. I'm not sure what more to advise from your description, no more adaptation seems needed. If you want to synthesize CT from MR in a supervised setup then you should be able to use the UNet or AutoEncoder classes, with a GAN/CycleGAN approach they could be used in conjunction with our Discriminator class.

[hasansari]

Thank you! I actually already tried to use UNet class with L1 loss function. Everything seems normal during the training but when I run inference on some test data, I see that the output is low quality with a grid appearance and with lots of voxels alternating between negative and positive values.

It looks like others are having similar problems: Project-MONAI/tutorials#100 (comment)

I will try the AutoEncoder as well!

Many thanks

Hasan