ODM Benchmarks

This repository is a part of the ODM project and is dedicated to benchmark sample datasets found in the ODMData repository and a few other thrid party sources, to better understand the behavioural characterists of ODM.

OpenDroneMap benchmarking data is summarized in the following pages:

• View benchmark data organized by dataset (Markdown)
• View benchmark data organized by ODM version (Markdown)
• View benchmark raw data (CSV)

Datasets for Benchmarking

Preferred datasets for benchmarking at this time are:

• Brighton Beach (18 photos)
• Toledo (87 photos)
• Wietrznia (225 photos)
• Shitan (493 photos)

These and many other sample datasets are indexed at the ODMData repository.

How to Contribute Benchmarks

This section provides instructions for contributing to the ODM Benchmarks project.

• Review Previous Benchmarks - You can have a look at previous benchmarks which is found in the benchmarks.csv file, located in the data directory to understand various things such as for example, figuring out which dataset has been benchmarked lesser than in comparision to other datasets.

• Select a desired Dataset - Choose one of the datasets of your choice which is listed in the table seen above. Links to download the dataset is provided in the same.

• Run the Dataset - Create a new task in ODM and process the images.

• Verify the results - Ensure that the task completed successfully by viewing the maps and models generated

• Submit Your Results - Open the benchmarks.csv file and look at the header for each of the column and note the desired information required before submitting and edit the csv file. Submissions are accepted Via a Pull Request or you can choose to post your results on the ODM community forum. A table is given below to explain the structure of the CSV.

Attribute	Explanation	Example
ID	Benchmark Number	1
DATASET	Dataset Name	Toledo
PROCESSING_TIME	Time taken to Process The dataset	1h 9m
PROCESSING_SUCCESS	Confirm If process was successful	Y
ERROR_TYPE	Mention error if any occured	-
RAM_SIZE	Size of RAM allocated	16 GB
RAM_CLOCK_SPEED	RAM Frequency of the Machine	2133 MT/s
CPU_TYPE	Make and model of the CPU	Intel i5
CPU_CLOCK_SPEED	Clock Speed of the CPU	2.3 Ghz
CPU_NUM_CORES	Number of Cores of the CPU	4
STORAGE_TYPE	Storage Type of the system. HDD/SSD/NVME	SSD
OS	Operating System of the Machine	Ubuntu 18.04
VM_TYPE	Virtual Machine on which ODM is running on	Docker
ODM_VERSION	Version of ODM Used to Benchmark	1.3.1
ODM_CLUSTER	Delcaration of usage of Cluster ODM	N
CONFIG_NAME	Mention of the Configuration in which the dataset is being processed	Default
CONFIG_RESIZE	mention of image size if images are being resized	2048 px
CONFIG_OTHER	Other Configurations made that are worth mentioning	-
TEST_DATE	Date of Test	2020-03-07
TEST_BY	Information of the individual who tests the data	Corey Snipes
INCLUDE_IN_SUMMARY	Check if Data has been included in summary	Y
NOTES	Additional Notes	-

Queries and Contributions

• Questions and Doubts - Any queries you may have can be posted in the ODM community forum.

• Contributing new Benchmark Data - If you have benchmark data to share, See this page for details on contributing.

License

MIT License