MoANA: Module of Axis-based Nexus Attention for Weakly Supervised Object Localization and Semantic Segmentation

This repository provides the official PyTorch implementation of the following paper:

MoANA: Module of Axis-based Nexus Attention for Weakly Supervised Object Localization and Semantic Segmentation
Junghyo Sohn¹, Eunjin Jeon², Wonsik Jung², Eunsong Kang², Heung-Il Suk^1,2
1 Department of Artificial Intelligence, Korea University, Seoul 02841, Republic of Korea
² Department of Brain and Cognitive Engineering, Korea University, Seoul 02841, Republic of Korea

Under review, IEEE Transactions on Image Processing

Abstract: Although recent advances in deep learning have led to an acceleration in the improvement in weakly supervised object localization (WSOL) tasks, it remains challenging to identify and segment an entire object rather than only discriminative parts of the object. To tackle this problem, corruption-based approaches have been devised, which involve the training of non-discriminative regions by corrupting (erasing) the input images or intermediate feature maps. However, this approach requires an additional hyperparameter, the corrupting threshold, to determine the degree of corruption and can unfavorably disrupt training. It also tends to localize object regions coarsely. In this paper, we propose a novel approach, Module of Axis-based Nexus Attention (MoANA), which helps to adaptively activate less discriminative regions along with the class-discriminative regions without an additional hyperparameter, and elaborately localizes an entire object by utilizing information distributed over widths, heights, and channels with an attention mechanism for calibrating features. Specifically, MoANA consists of three mechanisms (1) triple-view attentions representation, (2) attentions expansion, and (3) features calibration mechanism. Unlike other attention-based methods that train a coarse attention map with the same values across elements in feature maps, our proposed MoANA trains fine-grained values in an attention map by assigning different attention values to each element in a cost-efficient manner. We validated our proposed method by comparing it with recent WSOL and weakly supervised semantic segmentation (WSSS) methods over various datasets. We also analyzed the effect of each component in our MoANA and visualized attention maps to provide insights into the calibration.

Dependencies

• Python 3.6+
• PyTorch 1.6.0+

Requirements

conda env create -name moana -f environment.yaml
conda activate moana

Dataset

• ILSVRC: 11,788 images from 200 bird categories, divided into 5,994 images for training and 5,794 images for evaluation.
• CUB-200-2011: 1.2 million images in about 1,000 categories for training and 50,000 images for a validation.
• Pascal VOC 2012: 21 classes, composed of 1,464 training images, 1,449 validation images and 1,456 test images.

Code Reference

• Choe et al., Evaluating Weakly Supervised Object Localization Methods Right.
• Zhang et al., Causal Intervention for Weakly-Supervised Semantic Segmentation.
• DeepLab v2, PyTorch implementation of DeepLab v2 with a ResNet 101 backbone.

Acknowledge

This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No. 2019-0-00079, Artificial Intelligence Graduate School Program(Korea University))