KeyError: Caught KeyError in DataLoader worker process 0 when training imvoxelnet on ScanNet Dataset

[Cindy0725]

Hi, I am trying to reproduce the result on ScanNet. I built the docker file and run the image. I followed your instruction on GitHub to process ScanNet. In the container, I run "python tools/train.py configs/imvoxelnet/imvoxelnet_scannet.py", but got the following error:

Is this due to the processing of ScanNet dataset? I followed exactly the instruction , I checked the "scannet_infos_train.pkl" and for each scene, I have the following annotation:

Looking forward to your kind reply. Thank you very much!

[Cindy0725]

Hi, I tried to debug the code, and I found that the error comes from

imvoxelnet/mmdet3d/datasets/scannet_monocular_dataset.py

Line 17 in 655094d

info = self.data_infos[index]

According to Custom3DDataset class, self.data_infos is dictionary type loaded from pickle file, the iterated calling process is:
self.data_infos = self.load_annotations(self.ann_file) -> mmcv.load() -> PickleHandler() -> load_from_fileobj -> pickle.load(ann_file, **kwargs)
And when calling get_item() function in Custom3DDataset class, the iterated calling is:
Custom3DDataset.__get_item__() -> Custom3DDataset.preprare_train_data(self, index) -> ScanNetMultiViewDataset.get_data_info() -> self.data_infos[index] (line17 of scannet_monocular_dataset.py where error occurs)

It seems that it throws error since self.data_infos is in dictionary type and can't be accessed by [index]. I think there is something wrong with the annotation file, e.g. "scannet_infos_train.pkl" or "scannet_infos_val.pkl"

I am wondering if I preprocess the ScanNet wrongly, could you please help check your ScanNet annotation file and data format?
Thank you very much! Looking forward to your kind reply.

[filaPro]

Have you preprocessed scannet with this repo or with master branch of mmdetection3d? It should be this repo, as preprocessing has changed in mmdetection3d since our release.

[Cindy0725]

Hi, sorry for this stupid mistake... I thought the files should be the same since they have the same name and structure... I have reprocessed the data according to this repo, and the error disappears. But the docker image can't be run on 6000ada or 4090 since the cuda101 + pytorch 1.6 don't support sm89 of 6000ada. I will try to install newer version of torch + cuda and try to modify the code. Thank you very much for your kind reply! Have a nice day.