Subjects Dataset with DICOM files

[gkrisp98]

Hi, I am trying to use a subjects dataset to perform classification on DICOM files. The problem is that in the training loop it loads the data very slow and I was wondering if I am messing up in my code
The code for the subjects list:

> 
> def makeSubjects(root_dir = "/content/drive/MyDrive/subjects"):
>   dataframe = pd.read_csv("/content/drive/MyDrive/KL_grade.csv", sep = ';')
>   labels = {}
>   id = list(dataframe["id"])
>   grades = list(dataframe["grade"])
>   for i,g in zip(id, grades):
>     labels[str(i).zfill(5)] = g
> 
>   subjects_list = []
> 
>   for i, idx in enumerate(ids):
>     sub = tio.Subject(
>         load_mri(ids[i]),
>         label = labels[first[i]],
>     )
>     subjects_list.append(sub)
>   return subjects_list

The load mri function:

> def load_mri(mri_path):
>   root = "/content/drive/MyDrive/subjects"
>   full_path = os.path.join(root, mri_path)
>   subject = tio.Subject(t1 = tio.ScalarImage(full_path))
>   return subject

The transforms and the line for the creation of the dataset:

transform = tio.Compose([
                         tio.CropOrPad((128,128,128)),
                         tio.RescaleIntensity((0,1)),
                         tio.RandomAffine(p=0.3),
                         tio.RandomNoise(p=0.2),
                         tio.RandomFlip(p=0.2)
])

val_test_transform = tio.Compose([
                                  tio.CropOrPad((128,128,128)),
                                  tio.RescaleIntensity((0,1))
])
train_set = tio.SubjectsDataset(train_data, transform = transform)
valid_set = tio.SubjectsDataset(valid_data, transform = val_test_transform)
test_set = tio.SubjectsDataset(test_data, transform = val_test_transform)`

[fepegar]

Hi, @gkrisp98. It seems that you are passing an instance of Subject to the Subject class.

[gkrisp98]

Couldn't that be working ? The classifier I trained using those datasets had good results

[fepegar]

I would need to think about how it works, but conceptually it doesn't really make sense, and it might be the reason of the slow loading.

[gkrisp98]

I should have mentioned that I transformed the dicom files to nifti and the loading was pretty fast. (With similar code for dataset)
Do you think that there is a case that my results are meaningless ?

Thanks for spending time trying to figure this out!

[gkrisp98]

So, I wrote the code for creating subjects list without this issue:

def makeSubjects():
  dataframe = pd.read_csv("/content/drive/MyDrive/nifti-grades.csv", sep = ';')
  labels = {}
  id = list(dataframe["id"])
  grades = list(dataframe["grade"])
  for i,g in zip(id, grades):
    labels[str(i).zfill(5)] = g

  ids = [a.split("/")[-1] for a in sorted(glob.glob(f"/content/drive/MyDrive/images_nifti/" + "/*"))]
  root = "/content/drive/MyDrive/images_nifti"
  subjects_list = []

  for i, idx in enumerate(ids):
    sub = tio.Subject(
        t1 = tio.ScalarImage(os.path.join(root, ids[i])),
        label = labels[ids[i]],
    )
    subjects_list.append(sub)
  return subjects_list

I think that this code is correct. I trained the classifier and I got similar results so I guess that passing a subject object to the subject class didn't have any affect on the model performance.