Understanding the output space of Seg3D Segmentation Masks

[adam2392]

I was hoping to get some clarification that the user Manual doesn't have.

My Data Setup
I am working with a post-surgical T1 MRI. The post T1 contains a region that is ablated which I would like to create a segmentation for. I noticed that all the segmentations are saved as the .nrrd files, which contains a binary mask volume and then a corresponding vox2ras transform matrix.

However, the vox2ras transform matrix is now identity, suggesting that seg3d saved the output in a different space then the input files.

What I "need"
What I am trying to get is a .nrrd mask that when loaded in say... python, or matlab and overlays on the corresponding .nifti file it works. If I need to apply any transformation then that's fine, but I'm not exactly sure how seg3d transforms the image when it is read in, such that I can reproduce the same mapping.

Are you able to elaborate how seg3d writes out the final .nrrd mask segmentation?

Example
I ran seg3D on an image and I got this output:

('space', '3D-right-handed'), 

('sizes', array([256, 160, 256])),

 ('space origin', array([-113.76199603, -102.14299099,  -52.30242157])), 

('measurement frame', array([[1., 0., 0.],
       [0., 1., 0.],
       [0., 0., 1.]]))])

while the output of mri_info for my nifti image that was fed in is:

          type: nii
    dimensions: 256 x 160 x 256
   voxel sizes: 0.898000, 1.500003, 0.898000
          type: FLOAT (3)
           fov: 229.888
           dof: 0
        xstart: -114.9, xend: 114.9
        ystart: -120.0, yend: 120.0
        zstart: -114.9, zend: 114.9
            TR: 0.00 msec, TE: 0.00 msec, TI: 0.00 msec, flip angle: 0.00 degrees
       nframes: 1
       PhEncDir: UNKNOWN
       FieldStrength: 0.000000
ras xform present
    xform info: x_r =   1.0000, y_r =   0.0000, z_r =   0.0000, c_r =    -0.2840
              : x_a =   0.0000, y_a =   0.9697, z_a =   0.2443, c_a =     8.0902
              : x_s =   0.0000, y_s =  -0.2443, z_s =   0.9697, c_s =    29.8379
Orientation   : RAS
Primary Slice Direction: axial

voxel to ras transform:
                0.8980   0.0000   0.0000  -115.2280
                0.0000   1.4545   0.2194  -136.3575
                0.0000  -0.3665   0.8708   -52.3024
                0.0000   0.0000   0.0000     1.0000

voxel-to-ras determinant 1.20961

ras to voxel transform:
                1.1136   0.0000   0.0000   128.3163
                0.0000   0.6465  -0.1629    79.6302
                0.0000   0.2721   1.0798    93.5787
                0.0000   0.0000   0.0000     1.0000

So, while the dimensions of the image and mask line up (e.g. 256 x160 x256), the vox2ras and ras2vox and "centers" seem off, and I'm not exactly sure if/how seg3d does some recentering and/or rotations and/or some flips?

Let me know if I can clarify.

[jessdtate]

Seg3D is converting the data from the native axes into its coordinate system, with is axes aligned to RAS. In this example it would move the origin (not the center) to a different corner of the volume. For RAS with identity axes, this would be the most left, posterior, and inferior point and the axes would be along right, anterior, superior directions. Does that help?

There is an open issue to handle these off-axis data better. #365

[adam2392]

Hi @jessdtate thanks for the response. Just wondering, is there a principled way then for... "remapping" this back to the space of the image that was fed into Seg3D?

This seems like it could be very error prone (for me) and wanted to make sure I'm handling the way that Seg3D is transforming correctly, so I can remap everything back to where my analysis takes place.

[jessdtate]

We are working to make this a more robust and integrated part of the Seg3D, but there are a couple workarounds. one is to transform before importing into seg3D. The other is to try to reapply the transform that you pulled the nfti header to the seg3D output. @ChantelC @Sumientra have a way that uses Slicer 3D, but it could be similarly done in python or matlab.

Here is a description from @Sumientra:

Open Slicer
Load reference and mask files:
In Finder or File Explorer, navigate to
Drag file T1fs_conform.nii onto the Slicer window -> OK
Navigate to m2m/mask_prep, select all .nrrd files and drop them onto the Slicer window -> OK
Open the dropdown next to Modules -> Registration -> Resample Image (BRAINS)
Under Inputs, select T1fs_conform as the Reference Image
For each mask:
Under Inputs, for Image To Warp select the mask
Under Outputs, for Output Image select Create new Volume as…
Write the appropriate mask name: MASK_BONE, MASK_AIR, MASK_CSF, MASK_EYES, MASK_CAT_GM, MASK_SKIN, MASK_CAT_VENTRICLES, MASK_CAT_WM
OK -> Apply
Save files:
File -> Save
Uncheck all boxes except the newly created masks named MASK
For each mask, set File Format to NifTI (.nii.gz)
Change directory for selected files
Set the directory to /m2m_/mask_prep -> Save -> Yes
Close Slicer (exit and discard modifications)