CNSPN: A Few-Shot Remote Sensing Scene Classification Method

This is code of this IEEE TGRS paper "Improving Few-Shot Remote Sensing Scene Classification with Class Name Semantics"

Few-shot remote sensing scene classification (FSRSSC) has been used for new class recognition in the presence of a limited number of labeled samples. The representation vector (prototype) of categories obtained using images only confronts some challenges, such as insufficient generalization when the number of samples is too small. To address this problem, we propose a new FSRSSC method based on prototype networks, named improved prototype network with class name semantics (CNSPN), which combines semantic information of class names (name of the scene categories, such as aircraft, harbor, and bridge). First, CNSPN extracts semantics for class names using a pretrained word-embedding model, which enriches the feature representation ability of the category at the source. Then, an enhanced fusion prototype is generated by fusing the semantic information of text and visual information in the image through a multimodal prototype fusion module (MPFM). Finally, the query image is classified by measuring the distance between the query sample and the visual prototype, and between the query sample and the fusion prototype. Comparative experiments on the Northwestern Polytechnical University (NWPU)-remote sensing image scene classification (RESISC)45 and remote sensing dataset (RSD)46-Wuhan University (WHU) datasets show that the proposed method significantly improves FSRSSC performance.

paper

The paper link is CNSPN

With ResNet-18 backbone on NWPU RESISC45 and RSD46 WHU dataset.

For other backbones (Conv4), Please implement it yourself.

Results

1-shot on the NWPU RESISC45 dataset: 70.35% 1-shot on the RSD46 WHU dataset: 59.73%

How to train and test CNSPN model

Environment installation

1. Please install Python 3.6 and Python 1.8

data preparation

3. Download dataset
4. Put it in the data folder

train and test

5. You can start training from scratch or use the trained weights for testing.

for NWPU RESISC45 dataset

python train.py python test.py

for RSD46 WHU dataset

python train.py --shot 5 --train-way 20 --save-path ./save/proto-5

python test.py --load ./save/proto-5/max-acc.pth --shot 5

If it is helpful, please cite our paper

J. Chen et al., "Improving Few-Shot Remote Sensing Scene Classification With Class Name Semantics," in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-12, 2022, Art no. 5633712, doi: 10.1109/TGRS.2022.3219726.

Note

• 10/10/2022: code released
• 18/05/2023: add the URL link of our paper