Learning Deep Representations for Photo Retouching

By Di Li, and Susanto Rahardja

Introduction

The codebase provides the official PyTorch implementation for the paper " Learning Deep Representations for Photo Retouching" (accepted by IEEE Transactions on Multimedia).

In this project, we present a a novel framework to retouch the degraded photos towards a specific photographic style in an unsupervised fashion. To be specific, we unify the design philosophy of the generator and the discriminator into a multi-scale form and reuse these powerful networks as feature extractors to obtain deep latent representations of the photos with varying scales. Then, we employed projection heads to map these deep representations to a neater loss space for evaluation. In particular, we utilized a contrastive scheme for generator to keep the content consistency and a cross entropy scheme for discriminator to improve the aesthetic quality. Powered by this simple yet efficient design philosophy, our algorithm surpasses previous algorithms on two well known datasets with photo enhancement task. As a learning framework compliable with various enhancement networks, our algorithm shows great potential to be a practical tool to efficiently and effectively build photo retouching systems with different needs.

Dependencies

• Python 3 (Recommend to use Anaconda)
• PyTorch >= 1.0
• Opencv
• Imageio
• visdom

Datasets

The paper use the FiveK and HDR+ datasets for experiments.

• FiveK : You can download the original FiveK dataset from the dataset homepage and then process images using Adobe Lightroom.

  • To generate the input images, in the Collections list, select the collection Input with Daylight WhiteBalance minus 1.5.
  • To generate the target images, in the Collections list, select the collection Experts/C.
  • All the images are converted to .PNG format.

• HDR+ : You can download the original HDR+ dataset from the dataset homepage and then process images using rawpy.

  • To generate the input images, using raw.postprocess() function, set use_camera_wb=True and fbdd_noise_reduction=rawpy.FBDDNoiseReductionMode.Full.

The final directory structure is as follows.

./data/FiveK
    trainA/         # 8-bit sRGB train inputs
    trainB/         # 8-bit sRGB train groundtruth
    testA/          # 8-bit sRGB test inputs
    testB/          # 8-bit sRGB test groundtruth

Train

• run visdom to monitor status

visdom

• run

python train.py --name DRN --dataroot ./data/FiveK --batch_size 2 --gpu_ids 0 --netG rdnccut --model cut --lambda_NCE 10 --nce_includes_all_negatives_from_minibatch --ndf 32 --netD fe --niter 20 --niter_decay 80 --spectral_norm

Test

• run

python test.py --dataroot ./data/FiveK/testA --name DRN --gpu_ids 0 --netG rdnccut 

Citation

If you find this repository useful, please kindly consider citing the following paper:

@article{li2023learning,
title={Learning Deep Representations for Photo Retouching},
author={Li, Di and Rahardja, Susanto},
journal={IEEE Transactions on Multimedia},
year={2023},
publisher={IEEE}
}

License

Our project is licensed under a MIT License.