When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images

Mahmoud Afifi¹, Brian Price², Scott Cohen², and Michael S. Brown^1
1York University    ²Adobe Research

Reference code for the paper When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images. Mahmoud Afifi, Brian Price, Scott Cohen, and Michael S. Brown, CVPR 2019. If you use this code or our dataset, please cite our paper:

@inproceedings{afifi2019color,
  title={When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images},
  author={Afifi, Mahmoud and Price, Brian and Cohen, Scott and Brown, Michael S},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={1535--1544},
  year={2019}
}

The original source code of our paper was written in Matlab. We also provide a Python version of our code. We tried to make both versions identical. However, there is no guarantee that the Python version will give exactly the same results. The differences should be due to rounding errors when we converted our model to Python or differences between Matlab and OpenCV in reading compressed images.

Quick start

1. Matlab:

1. Run install_.m
2. Run demo.m to process a single image or demo_images.m to process all images in a directory.
3. Check evaluation_examples.m for examples of reporting errors using different evaluation metrics. Also, this code includes an example of how to hide the color chart for Set1 images.

2. Python:

1. Requirements: numpy, opencv-python, and skimage (skimage is required for evaluation code only).
2. Run demo.py to process a single image or demo_images.py to process all images in a directory.
3. Check evaluation_examples.py for examples of reporting errors using different evaluation metrics. Also, this code includes an example of how to hide the color chart for Set1 images.

Graphical user interface

We provide a Matlab GUI to help tuning our parameters in an interactive way. Please, check demo_GPU.m.

Code/GUI parameters and options

1. K: Number of nearest neighbors in the KNN search (Sec. 3.4 in the paper) -- change its value to enhance the results.
2. sigma: The fall-off factor for KNN blending (Eq. 8 in the paper) -- change its value to enhance the results.
3. device: GPU or CPU (provided for Matlab version only).
4. gamut_mapping: Mapping pixels in-gamut either using scaling (gamut_mapping= 1) or clipping (gamut_mapping= 2). In the paper, we used the clipping options to report our results, but the scaling option gives compelling results in some cases (esp., with high-saturated/vivid images).
5. upgraded_model and upgraded: To load our upgraded model, use upgraded_model=1 in Matlab or upgraded=1 in Python. The upgraded model has new training examples. In our paper results, we did not use this model.

Dataset

In the paper, we mentioned that our dataset contains over 65,000 images. We further added two additional sets of rendered images, for a total of 105,638 rendered images. You can download our dataset from here. You can also download the dataset from the following links:

Input images: Part1 | Part2 | Part3 | Part4 | Part5 | Part6 | Part7 | Part8 | Part9 | Part10

Input images [a single ZIP file]: Download (PNG lossless compression) | Download (JPEG) | Google Drive Mirror (JPEG)

Input images (without color chart pixels): Part1 | Part2 | Part3 | Part4 | Part5 | Part6 | Part7 | Part8 | Part9 | Part10

Input images (without color chart pixels) [a single ZIP file]: Download (PNG lossless compression) | Download (JPEG) | Google Drive Mirror (JPEG)

Augmented images (without color chart pixels): Download (rendered with additional/rare color temperatures)

Ground-truth images: Download

Ground-truth images (without color chart pixels): Download

Metadata files: Input images | Ground-truth images

Folds: Download

Online demo

Try the interactive demo by uploading your photo or paste a URL for a photo from the web.

Project page

For more information, please visit our project page

Related Research Projects

• sRGB Image White Balancing:
  • White-Balance Augmenter: Emulating white-balance effects for color augmentation; it improves the accuracy of image classification and image semantic segmentation methods (ICCV 2019).
  • Color Temperature Tuning: A camera pipeline that allows accurate post-capture white-balance editing (CIC best paper award, 2019).
  • Interactive White Balancing: Interactive sRGB image white balancing using polynomial correction mapping (CIC 2020).
  • Deep White-Balance Editing: A multi-task deep learning model for post-capture white-balance editing (CVPR 2020).
• Raw Image White Balancing:
  • APAP Bias Correction: A locally adaptive bias correction technique for illuminant estimation (JOSA A 2019).
  • SIIE: A sensor-independent deep learning framework for illumination estimation (BMVC 2019).
  • C5: A self-calibration method for cross-camera illuminant estimation (arXiv 2020).
• Image Enhancement:
  • CIE XYZ Net: Image linearization for low-level computer vision tasks; e.g., denoising, deblurring, and image enhancement (arXiv 2020).
  • Exposure Correction: A coarse-to-fine deep learning model with adversarial training to correct badly-exposed photographs (CVPR 2021).
• Image Manipulation:
  • MPB: Image blending using a two-stage Poisson blending (CVM 2016).
  • Image Recoloring: A fully automated image recoloring with no target/reference images (Eurographics 2019).
  • Image Relighting: Relighting using a uniformly-lit white-balanced version of input images (Runner-Up Award overall tracks of AIM 2020 challenge for image relighting, ECCV Workshops 2020).
  • HistoGAN: Controlling colors of GAN-generated images based on features derived directly from color histograms (CVPR 2021).

Commercial Use

This software and the dataset are provided for research purposes only. A license must be obtained for any commercial application.