Robust Image Segmentation

Overview of the project

As part of the CentraleSupélec Data Science Project Group, I collaborated with three peers under the esteemed guidance of Prof. Jean-Christophe Pesquet and Matthieu Terris from the Centre de Vision Numérique, OPIS Inria group at Université Paris-Saclay. We tackled a challenge crucial for both healthcare and technology sectors: making computer-driven image segmentation more reliable and robust, especially in medical diagnostics. This initiative was inspired by the critical need to bolster neural network resilience against adversarial perturbations, a challenge highlighted in foundational research. From February 2023 to January 2024, we hence embarked on a pioneering project focused on implementing Lipschitz regularization in neural networks for segmentation tasks to achieve stable firm non-expansiveness.

Lipschitz regularization to achieve firm non-expansiveness

The point of image segmentation is to not only classify images (like distinguishing cats from dogs) but to also pinpoint and segment specific areas within an image, a process vital for tasks such as identifying tumors in medical scans. We have implemented a supervised image classification model, called mResNet-18, which stems from ResNet-18, which is a neural network built for image classification. We have modified ResNet-18 to implement a visualization technique allowing for self-supervised segmentation, by essentially upsampling the output to input dimensions. In more prosaic terms, this means that we do not directly learn or supervise the image segmentation, we only supervise the classification to which we add a visualization method. To ensure the robustness of the model, we opted to constrain the network to be firmly non-expansive. To achieve firm non-expansiveness for a network $A_\theta$, it is sufficient to impose non-expansiveness on the reflection of the network $A_\theta^R$, e.g. being $1$-Lipschitz or $L(A_\theta^R)\leqslant 1$, meaning that it limits the error in its output to be at most equal to the error introduced in its input, hence is robust to adverse perturbations. We therefore added a Lipschitz regularization term to the loss function.

Results and applications

Leveraging the Data Centre d'Enseignement Metz's computational resources, our computations on the CIFAR-10 dataset unveiled a quantifiable relationship between the Lipschitz constant and regularization weight. CIFAR-10 is a classical dataset with real-life applications, for example in autonomous driving, as it is practical to train object detection and image segmentation. The relationship, validated by a comprehensive evaluation using the U-Net model on the COCO dataset, provides valuable insights into optimizing neural network performance and robustness. Our findings, encapsulated in the slides, reveal the nuanced impact of regularization weight on model efficacy, offering a framework for future research in neural network optimization.

This project not only underscores our team's capability to tackle complex challenges in data science and analytics but also contributes meaningful advancements to the field of image segmentation. By meticulously evaluating and enhancing our model's performance, we have not only established a vital step towards engineering more resilient neural networks but also unlocked new avenues for practical applications in various industries. Our project significantly enhances neural network precision and robustness, achieving up to 90% precision with weak regularization ($\lambda = 10^{-6}$) and markedly increasing system robustness by reducing the Lipschitz constant by a factor of 100,000. This breakthrough has profound implications for medical diagnostics, leading to more accurate diagnoses, fewer misdiagnoses, and improved treatment plans, thereby directly benefiting patient care. Additionally, the improved robustness of our model extends its applicability to various sectors, including autonomous driving and surveillance, where it ensures higher reliability and reduced error rates under diverse conditions.

References

J. Fix, S. Vialle, R. Hellequin, C. Mercier, P. Mercier, J.B. Tavernier (2022). Feedback from a data center for education at CentraleSupélec engineering school. EduPar-22: 12th NSF/TCPP Workshop on Parallel and Distributed Computing Education.