Evaluating Deep Learning based Building Damage Assessment Methods in earthquake-affected, densely built-up urban areas: The case of Kahramanmaraş
Repository for the master thesis for the degree of Master of Data Science for Public Policy
In post-disaster settings, damage assessments need to be conducted fast and reliably. To this end, deep learning approaches for building damage assessment have been researched and various models have been developed. However, the real-world performance on off-nadir post-event imagery of earthquakes in densely built-up urban areas still remains underexplored. In this analysis, a dataset for Kahramanmaraş, a Turkish city affected by the 2023 earthquake in the East Anatolian Fault Zone is created by combining open-source building footprints, emergency mapping information, and high-resolution open satellite imagery. Three different approaches are tested against the dataset: the xView2-baseline damage classification model component, combined with open-source building footprints as localization, the Multitask-Based Semi-Supervised Semantic Segmentation Framework MS4D-Net, and the deep object-based semantic change detection framework ChangeOS. The findings suggest that earthquake building damage in densely built-up urban setting poses significant challenges for model performance. The ChangeOS framework outperforms the other approaches, although robustness checks indicate that the model does not reliably predict the same damage scene on different imagery.
The project relies on one main data preparation pipeline that was used to create the main dataset for Kahramanmaraş. The data pipeline brings together OpenStreetMap building footprints, EU Copernicus EMS building damage information and satellite data from the Maxar Open Data Program. Next to this, a subset of the xBD dataset is used to train the supervised component of the MS4D-Net.
I used three separate environments for the experiments, which was neccessary because of the various dependencies, CUDA version, etc of the different models. The data preparation was conducted with a pyhon 3.11 environment. For the MS4D-Net experiments, I used a python 3.8 environment. Finally, the xView2-baseline scripts were run on a Google colab notebook which was a python 3.10 environment.
The pipeline starts with the retrieval of the satellite data from the Maxar Open Data Program. First, the download links are retrieved, then starts the retrieval of the iamgery. Then, Building footprints are triangulated with the building damage information form the Copernicus EMS for Kahramanmaraş, see this script for reference. To perform these steps, you can simply run the setup.sh script from the command line.
The final step in the processing pipeline prepares the satellite imadery (unifying the coordinate reference system, cropping out of irrelevant image parts, etc.), and overlays the pre and post-event satellite imagery, and the building footprints to create 512x512 image tiles for three combinations of pre and post-event satellite imagery. The segmentaiton masks are created in the same step. You can access the data from my Google Drive:
- 512x512 pixel image tile dataset used for ChangeOS & MS4D-Net: https://drive.google.com/drive/folders/1RoWwbzjkRmsUkKwUFCjgKLgoUN5VaP4b?usp=sharing
- Building patches for xView2-baseline: https://drive.google.com/drive/folders/1mTVJpXeXlqTFbfqJpa-GfrS9Q5hLLzKp?usp=sharing
- Prepared Mexico-earthquake xBD subset: https://drive.google.com/drive/folders/1adr9LhlkgTe39QfG_IIqLjkxJ8uMAYdC?usp=sharing
Finally, the xBD subset (Gupta et al. 2019) on the Mexcio City earthquake in 2017 is prepared. This data can be downloaded from the official xView2 Challenge website. The data is not directly compatible with the MS4D-Net model that is used to assess it. Therefore, a slightly adapted version of a script from the Microsoft Damage Assessment Model is used to convert the json masks to png files that can be used as segmenation masks. Another short script is then used to split up the images and masks into 512x512 files.
Among the models that are used to for the analysis is the xView2-baseline model. The training notebook relies in large parts on the training code from the original repository.
Another model that was tested is the ChangeOS model. The official implementation from the original paper (Zhen et al. 2020) was used.
Finally, I use the MS4D-Net by He et al. (2023) which is a semi-supervised learning framework. Here, I reused large parts of the original codebase.