Skip to content

Latest commit

 

History

History
151 lines (79 loc) · 7.33 KB

README.md

File metadata and controls

151 lines (79 loc) · 7.33 KB
Raw

A Closer Look at Fourier Spectrum Discrepancies for
CNN-generated Images Detection

Keshigeyan Chandrasegaran /  Ngoc‑Trung Tran /  Ngai‑Man Cheung
Singapore University of Technology and Design (SUTD)
CVPR 2021 (Oral)
 Project | CVPR Paper | GAN Models

Abstract

CNN-based generative modelling has evolved to produce synthetic images indistinguishable from real images in the RGB pixel space. Recent works have observed that CNN-generated images share a systematic shortcoming in replicating high frequency Fourier spectrum decay attributes. Furthermore, these works have successfully exploited this systematic shortcoming to detect CNN-generated images reporting up to 99% accuracy across multiple state-of-the-art GAN models.


In this work, we investigate the validity of assertions claiming that CNN-generated images are unable to achieve high frequency spectral decay consistency. We meticulously construct a counterexample space of high frequency spectral decay consistent CNN-generated images emerging from our handcrafted experiments using DCGAN, LSGAN, WGAN-GP and StarGAN, where we empirically show that this frequency discrepancy can be avoided by a minor architecture change in the last upsampling operation. We subsequently use images from this counterexample space to successfully bypass the recently proposed forensics detector which leverages on high frequency Fourier spectrum decay attributes for CNN-generated image detection.


Through this study, we show that high frequency Fourier spectrum decay discrepancies are not inherent characteristics for existing CNN-based generative models--contrary to the belief of some existing work--, and such features are not robust to perform synthetic image detection. Our results prompt re-thinking of using high frequency Fourier spectrum decay attributes for CNN-generated image detection.

About the code

GAN : This is written in Pytorch. This codebase leverages on Pytorch Lightning [1] module to efficiently scale our GAN training to multi-GPU infrastructure. We also use Webdataset [2] (Pytorch Iterable Dataset implementation) that allows efficient access to datasets stored in POSIX tar archives using only sequential disk access. We also provide the Docker file to run our code. The codebase is clearly documented with bash file execution points exposing all required arguments and hyper-parameters.

✅ Pytorch

✅ Pytorch Lightning [1]

✅ ​WebDataset [2]

✅ ​Multi-GPU training

✅ ​DockerFile

Fourier Synthetic Image classifier : The code uses high frequency features extracted using the official matlab implementation by Dzanic et al. [3]. The matlab code can be found [4]. The code is clearly documented.

Running the code

GAN : Clear steps on how to run and reproduce our results for DCGAN, LSGAN, WGAN-GP and StarGAN experiments can be found at src/gans/README.md.

Fourier Synthetic Image classifier : Clear steps on how to run and reproduce our results for KNN, SVM and MLP classfier experiments using high frequency features extracted from Dzanic et al work can be found at src/classifiers/README.md

GAN Samples

GAN Baseline N.1.5 B.1.5 Z.1.5
DCGAN
LSGAN
WGAN-GP

FID Scores for CelebA experiments

Setup Code DCGAN LSGAN WGAN-GP
Baseline 88.60 73.26 60.60
N.1.5 87.52 70.69 48.69
Z.1.5 69.14 60.29 47.73
B.1.5 84.65 78.66 52.18
N.1.7 90.80 73.09 60.11
Z.1.7 71.45 59.55 43.10
B.1.7 79.92 76.33 55.28
N.1.3 93.54 74.07 58.35
Z.1.3 65.46 61.45 56.91
B.1.3 76.04 81.97 58.55
N.3.5 73.63 78.31 55.47
Z.3.5 68.41 66.27 57.59
B.3.5 80.89 72.29 54.84

Pretrained Models

All pretrained DCGAN, LSGAN and WGAN-GP CelebA models are available here

Citation

@InProceedings{Chandrasegaran_2021_CVPR,
    author    = {Chandrasegaran, Keshigeyan and Tran, Ngoc-Trung and Cheung, Ngai-Man},
    title     = {A Closer Look at Fourier Spectrum Discrepancies for CNN-Generated Images Detection},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {7200-7209}
}

Acknowledgements

We gratefully acknowledge the following works:

References

[1] https://www.pytorchlightning.ai/

[2] https://github.com/tmbdev/webdataset

[3] Fourier Spectrum Discrepancies in Deep Network Generated Images, Tarik Dzanic, Karan Shah, and Freddie D. Witherden, In NeurIPS, 2020

[4] https://github.com/tarikdzanic/FourierSpectrumDiscrepancies)