Time and Memory performances

For each problem sequence, we also benchmark the time and GPU memory usage.

Time

We measure the total time elapsed for each problem sequence, using the linux command time

It can be installed using apt install time

Usage: command time --verbose -o output_file_time.txt python3 benchmark.py

Example of the time output:

Command being timed: "python3 geo_benchmark.py --weights=/home/docker/GeoTransformer_benchmark/weights/geotransformer-kitti.pth.tar --input_txt=/benchmark/point_clouds_registration_benchmark/dataset_voxelgrid_0.3//eth/apartment_global.txt --input_pcd_dir=/benchmark/point_clouds_registration_benchmark/dataset_voxelgrid_0.3//eth/apartment/ --output_dir=/benchmark/experiments/geotransformer/kitti"
	User time (seconds): 25.46
	System time (seconds): 2.77
	Percent of CPU this job got: 159%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:17.64

We consider the Elapsed (wall clock) time.

Memory

We use the nvidia-smi util to monitor the GPU memory usage for each problem sequence.

Example command `nvidia-smi --query-gpu=timestamp,memory.used -i GPU_NUMBER --format=csv -lms 1 > output_file_memory.txt')

To run nvidia-smi together with the benchmark and to stop it once the benchmark finishes, we use the parallel GNU util.

parallel -j2 --halt now,success=1 ::: {time_command} {nvidia_command}

The output files will be a csv file, containing the GPU memory usage monitored every 1 ms:

timestamp, memory.used [MiB]
2023/10/18 21:40:32.489, 16 MiB
2023/10/18 21:40:32.491, 16 MiB
2023/10/18 21:40:32.492, 16 MiB

Note: the memory files are not uploaded to the repo because they are too big, they are stored in the gdrive folder.