DerainZoo (Single Image vs. Video Based)

Youzhao Yang, Hong Lu in Fudan Machine Vision Lab

1 Description

• DerainZoo: A list of deraining methods. Papers, codes and datasets are maintained. Other sources about deraining can be observed in web1 and web2.

• Datasets for single image deraining are available at the website.

• More datasets about other image processing task (brightening, HDR, color enhancement, and inpainting) are available here.

2 Image Quality Metrics

• PSNR (Peak Signal-to-Noise Ratio) [paper] [matlab code] [python code]
• SSIM (Structural Similarity) [paper] [matlab code] [python code]
• VIF (Visual Quality) [paper] [matlab code]
• FSIM (Feature Similarity) [paper] [matlab code]
• NIQE (Naturalness Image Quality Evaluator) [paper][matlab code][python code]

Image & Video Quality Assessment Algorithms [software release][Texas Lab]

3 Single Image Deraining

3.1 Datasets

---

3.1.1 Synthetic Datasets

• Rain12 [paper] [dataset] (2016 CVPR)
• Rain100L_old_version [paper][dataset](2017 CVPR)
  • Rain100L_new_version [paper][dataset]
• Rain100H_old_version [paper][dataset](2017 CVPR)
  • Rain100H_new_version [paper][dataset]
• Rain800 [paper][dataset] (2017 Arxiv)
• Rain1200 [paper][dataset] (2018 CVPR)
• Rain1400 [paper][dataset] (2017 CVPR)
• Heavy Rain Dataset [paper][dataset] (2019 CVPR)

3.1.2 Real-world Datasets

• Practical_by_Yang [paper][dataset] (2017 CVPR)
• Practica_by_Zhang [paper][dataset] (2017 Arxiv)
• Real-world Paired Rain Dataset [paper][dataset] (2019 CVPR)

3.2 Papers

---

2021

• NR [paper][code][web]

  • Fu, Xueyang etc. Improving De-raining Generalization via Neural Reorganization. (ICCV 2021)

• DerainRLNet [paper][code][web]

  • Chen, Chenghao etc. Robust Representation Learning with Feedback for Single Image Deraining. (CVPR 2021)

• VRGNet [paper][code][web]

  • Wang, Hong etc. From Rain Generation to Rain Removal. (CVPR 2021)

• IDCL [paper][code][web]

  • Zhou, Man etc. Image De-raining via Continual Learning. (CVPR 2021)

• DRG [paper][code][web]

  • Yue, Zongsheng etc. Semi-Supervised Video Deraining with Dynamical Rain Generator. (CVPR 2021)

• MPRNet [paper][code][web]

  • Zamir et al. Multi-Stage Progressive Image Restoration. (CVPR 2021)

• ADN [paper][code][web]

  • Yang, Youzhao etc. A Fast And Eefficient Network for Single Image Deraining. (ICASSP 2021)

• DualGCN [paper][code][web]

  • Fu, Xueyang etc. Rain Streak Removal via Dual Graph Convolutional Network. (AAAI 2021)

• IPT [paper][code][web]

  • Chen, Hanting etc. Pre-Trained Image Processing Transformer. (Arxiv 2020)

2020

• WDNet [paper][code][web]

  • Liu, Lin etc. Wavelet-Based Dual-Branch Network for Image Demoir´eing. (2020 ECCV)

• Rethinking Image Deraining [paper][code][web]

  • Wang, Yinglong etc. Rethinking Image Deraining via Rain Streaks and Vapors. (2020 ECCV)

• JDNet [paper][code][web]

  • Wang, Cong etc. Joint Self-Attention and Scale-Aggregation for Self-Calibrated Deraining Network. (2020 ACMMM)

• DCSFN [paper][code][web]

  • Wang, Cong etc. DCSFN: Deep Cross-scale Fusion Network for Single Image Rain Removal. (2020 ACMMM)

• CVID [paper][code][web]

  • Du, Yingjun etc. Conditional Variational Image Deraining. (2020 TIP)

• DRD-Net [paper][web][code]

  • Deng, Sen etc. Detail-recovery Image Deraining via Context Aggregation Networks. (2020 CVPR)

• RCDNet [paper][web][code]

  • Wang, Hong etc. A Model-driven Deep Neural Network for Single Image Rain Removal. (2020 CVPR)

• Syn2Rel [paper][web][code]

  • Rajeev Yasarla et al. Syn2Real Transfer Learning for Image Deraining using Gaussian Processes. (2020 CVPR)

• MSPFN [paper][code][web]

  • Jiang Kui et al. Multi-Scale Progressive Fusion Network for Single Image Deraining. (2020 CVPR)

• Physical Model Guided ID [paper][code][web]

  • Cong Wang et al. Physical Model Guided Deep Image Deraining. (2020 ICME)

• RDDAN [paper][code][website]

  • Yang, Youzhao et al. RDDAN: A Residual Dense Dilated Aggregated Network for Single Image Deraining. (2020 ICME)

• DiG-CoM [paper][code][website]

  • Ran, Wu; Yang, Youzhao et al. Single Image Rain Removal Boosting via Directional Gradient. (2020 ICME)

• VID [paper][code][web]

  • Xu, Jun et al. Variational Image Deraining. (2020 WACV)

• CMGD [paper][code][web]

  • Rajeev Yasarla et al. Confidence Measure Guided Single Image De-Raining. (2020 TIP)

2019

• Survey [paper][code][web]

  • Yang, Wenhan et al. Single Image Deraining: From Model-Based to Data-Driven and Beyond. (2019 TPAMI)

• RWL [paper][code][web]

  • Yang, Wenhan et al. Scale-Free Single Image Deraining Via VisibilityEnhanced Recurrent Wavelet Learning. (2019 TIP)

• DPRDN [paper][code][web]

  • Wei, Yanyan et al. A Coarse-to-Fine Multi-stream Hybrid Deraining Network for Single Image Deraining. (2019 ICDM)

• Survey [paper][code][web]

  • Wang, Hong et al. A Survey on Rain Removal from Video and Single Image. (2019 Arxiv)

• ERL-Net [paper][code][web]

  • Wang, Guoqing et al. ERL-Net: Entangled Representation Learning for Single Image De-Raining. (2019 ICCV)

• ReHEN [paper][code][web]

  • Yang, Youzhao et al. Single Image Deraining via Recurrent Hierarchy and Enhancement Network. (2019 ACM'MM)

• DTDN [paper][code][web]

  • Wang, Zheng et al. DTDN: Dual-task De-raining Network. (2019 ACM'MM)

• GraNet [paper][code][web]

  • Yu, Weijiang et al. Gradual Network for Single Image De-raining. (2019 ACM'MM)

• AMPE-Net [paper][code][web]

  • Wang, Yinglong et al. An Effective Two-Branch Model-Based Deep Network for Single Image Deraining. (2019 Arxiv)

• ReMAEN [paper][code][web]

  • Yang, Youzhao el al. Single Image Deraining using a Recurrent Multi-scale Aggregation and Enhancement Network. (2019 ICME)

• Rain Wiper [paper][code][web]

  • Liang, Xiwen et al. Rain Wiper: An Incremental Randomly Wired Network for Single Image Deraining. (2019 PG)

• Dual-ResNet [paper][code][web]

  • Liu, Xing et al. Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restoration. (2019 CVPR)

• Heavy Rain Image Restoration [paper][code][dataset][web]

  • Li, Ruoteng et al. Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning. (2019 CVPR)

• SPANet [paper][code][web][dataset]

  • Wang, Tianyu et al. Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset. (2019 CVPR)

• Comprehensive Benchmark Analysis [paper][code][dataset]

  • Li, Siyuan et al. Single Image Deraining: A Comprehensive Benchmark Analysis. (2019 CVPR)

• DAF-Net [paper][code][web]

  • Hu, Xiaowei et al. Depth-attentional Features for Single-image Rain Removal. (2019 CVPR)

• Semi-supervised Transfer Learning [paper][code][web]

  • Wei, Wei et al. Semi-supervised Transfer Learning for Image Rain Removal. (2019 CVPR)

• PReNet [paper][code][web]

  • Ren, Dongwei et al. Progressive Image Deraining Networks: A Better and Simpler Baseline. (2019 CVPR)

• UMRL-using-Cycle-Spinning [paper][code][web]

  • Rajeev Yasarla et al. Uncertainty Guided Multi-Scale Residual Learning-using a Cycle Spinning CNN for Single Image De-Raining. (2019 CVPR)

• RR-GAN [paper][code][web]

  • Zhu, Hongyuan et al. RR-GAN: Single Image Rain Removal Without Paired Information. (2019 AAAI)

• LPNet [paper][code][web]

  • Fu, Xueyang et al. Lightweight Pyramid Networks for Image Deraining. (2019 TNNLS)

• Morphological Networks [paper][code][web]

  • Mondal et al. Morphological Networks for Image De-raining. (2019 Arxiv)

2018

• QS Priors [paper][code][web]

  • Wang et al. Rain Removal By Image Quasi-Sparsity Priors. (2018 Arxiv)

• Linear model [paper][code][web]

  • Wang et al. Removing rain streaks by a linear model. (2018 Arxiv)

• Kernel Guided CNN [paper][code][web]

  • Deng et al. Rain Streak Removal for Single Image via Kernel Guided CNN. (2018 Arxiv)

• Physics-Based GAM [paper][code][web]

  • Pan, Jinshan et al. Physics-Based Generative Adversarial Models for Image Restoration and Beyond. (2018 Arxiv)

• Self-supervised Constraints [paper][code][paper]

  • Jin et al. Unsupervised Single Image Deraining with Self-supervised Constraints. (2018 Arxiv)

• SRSE-Net [paper][code][web]

  • Ye et al. Self-Refining Deep Symmetry Enhanced Network for Rain Removal. (2018 Arxiv)

• Tree-Structured Fusion Model [paper][code][web]

  • Fu, Xueyang et. al. A Deep Tree-Structured Fusion Model for Single Image Deraining. (2018 Arxiv)

• Deep DCNet [paper][code] [web1] [web2]

  • Li, Siyuan et al. Fast Single Image Rain Removal via a Deep Decomposition-Composition Network. (ArXiv2018)

• SFARL Model [paper][code][web]

  • Ren, Dongwei et al. Simultaneous Fidelity and Regularization Learning for Image Restoration. (ArXiv2018)

• GCAN [paper][code][web]

  • Chen et. al. Gated Context Aggregation Network for Image Dehazing and Deraining. (2018 WACV)

• Cycle-GAN [paper][code][web]

  • Pu, Jinchuan et al. Removing rain based on a Cycle Generative Adversarial Network. (2018 ICIEA)

• RESCAN [paper][code][web]

  • Li, Xia et al. Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining. (2018 ECCV)

• RGFFN [paper][code][web]

  • Fan, Zhiwen et al. Residual-Guide Feature Fusion Network for Single Image Deraining. (2018 ACM'MM)

• NLEDN [paper][code][web]

  • Li, Guanbin et al. Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining. (2018 ACM'MM)

• DualCNN [paper][code][web]

  • Pan, Jinshan et al. Learning Dual Convolutional Neural Networks for Low-Level Vision. (2018 CVPR)

• Attentive GAN [paper][code][web][project] [reimplement code]

  • Qian, Rui et al. Attentive Generative Adversarial Network for Raindrop Removal from a Single Image. (2018 CVPR) (tips: this research focuses on reducing the effets form the adherent rain drops instead of rain streaks removal)

• DID-MDN [paper][code][web]

  • Zhang, He et al. Density-aware Single Image De-raining using a Multi-stream Dense Network. (2018 CVPR)

• Directional global sparse model [paper] [code][web]

  • Deng, Liangjian et al. A directional global sparse model for single image rain removal. (2018 AMM)

• Gradient domain [paper][code][web]

  • Du, Shuangli et al. Single image deraining via decorrelating the rain streaks and background scene in gradient domain. (2018 PR)

2017

• ID_CGAN [paper][code] [web]

  • Zhang, He et al. Image De-raining Using a Conditional Generative Adversarial Network. (2017 Arxiv)

• Transformed Low-Rank Model [paper][code][web]

  • Chang, Yi et al. Transformed Low-Rank Model for Line Pattern Noise Removal. (2017 ICCV)

• JBO [paper][code][web]

  • Wei, Wei et al. Joint Bi-layer Optimization for Single-image Rain Streak Removal. (2017 ICCV)

• JCAS [paper][code][web]

  • Gu, Shuhang et al. Joint Convolutional Analysis and Synthesis Sparse Representation for Single Image Layer Separation. (2017 ICCV)

• DDN [paper] [code][web]

  • Fu, Xueyang et al. Removing rain from single images via a deep detail network. (2017 CVPR)

• JORDER [paper] [code][web]

  • Yang, Wenhan et al. Deep joint rain detection and removal from a single image. (2017 CVPR)

• Hierarchical Approach [paper][code][web]

  • Wang, Yinglong et al. A Hierarchical Approach for Rain or Snow Removing in a Single Color Image. (2017 TIP)

• Clearing The Skies [paper][code][web]

  • Fu, Xueyang et al. Clearing the skies: A deep network architecture for single-image rain removal. (2017 TIP)

• Error-optimized Sparse Representation [paper][code][web]

  • Chen, Bohao et al. Error-optimized sparse representation for single image rain removal. (2017 TIE)

2015-2016

• LP(GMM) (2016 CVPR, 2017 TIP)

  • Li, Yu et al. Rain streak removal using layer priors. [paper][code][web]
  • Li, Yu et al. Single Image Rain Streak Decomposition Using Layer Priors. [paper] [dataset][web]

• DSC [paper][code][web]

  • Luo, Yu et al. Removing rain from a single image via discriminative sparse coding. (2015 ICCV)

• Window Covered [paper][code][web]

  • David, Eigen et al. Restoring An Image Taken Through a Window Covered with Dirt or Rain. (2013 ICCV)

• Image Decomposition paper][code][web]

  • Kang, Liwei et al. Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition. (2012 TIP)

4 Video Based Deraining

2019

• D3R-Net [paper][code][web]
  • Yang, Wenhan et al. D3R-Net: Dynamic Routing Residue Recurrent Network for Video Rain Removal. (2019 TIP)

2018

• MSCSC [paper][code] [web][video]

  • Li, Minghan et al. Video Rain Streak Removal By Multiscale ConvolutionalSparse Coding. (2018 CVPR)

• CNN Framework [paper][code][web Chen] [web Chau]

  • Chen, Jie et al. Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework. (2018 CVPR)
  • Chen, Jie et al. Robust Video Content Alignment and Compensation for Clear Vision Through the Rain [paper][code][web](tips: I guess this is the extended journal version)

• Erase or Fill [paper][code][web Liu] [web Yang]

  • Liu, Jiaying et al. Erase or Fill? Deep Joint Recurrent Rain Removal and Reconstruction in Videos. (2018 CVPR)

2017

• MoG [paper] [code][web]

  • Wei, Wei et al. Should We Encode Rain Streaks in Video as Deterministic or Stochastic? (2017 ICCV)

• FastDeRain [paper][code]

  • Jiang, Taixiang et al. A novel tensor-based video rain streaks removal approach via utilizing discriminatively intrinsic priors. (2017 CVPR)

• Matrix Decomposition [paper][code][web]

  • Ren, Weilong et al. Video Desnowing and Deraining Based on Matrix Decomposition. (2017 CVPR)

2015-2016

• Adherent Raindrop Modeling [paper][code][web]

  • You, Shaodi et al. Adherent raindrop modeling, detectionand removal in video. (2016 TPAMI)

• Low-rank Matrix Completion [paper][code][web]

  • Kim, JH et al. Video deraining and desnowing using temporal correlation and low-rank matrix completion. (2015 TIP)

• Utilizing Local Phase Information [paper][code][web]

  • Santhaseelan et al. Utilizing local phase information to remove rain from video. (2015 IJCV)

5 Acknowledgement

• Thanks for the sharing of codes of image quality metrics by Wang, Hong.

6 Contact

• e-mail: yzyang17@fudan.edu.cn