Neural network predicts which colours to apply to black and white images
Fri 1 Apr 2016
Researchers have developed new software which is able to translate black and white images to colour, using a neural network to predict what a convincing photograph would look like.
The computer scientists, based at the University of California, Berkeley, taught the system to take a grayscale image and add colour using data compiled from more than a million colour images.
Lead researcher Richard Zhang explained that, learning from large-scale data sets, the neural system is able judge how different aspects in a picture should be coloured. While previous approaches to colourisation have necessitated user interaction or have produced desaturated results, the proposed implementation is fully automated and creates vibrant and realistic translations.
The software technique is described by Zhang as ‘hallucination.’ ‘Our task is to model enough of the statistical dependencies between the semantics and the textures of grayscale images and their colour versions in order to produce visually compelling results,’ he explained in the paper, Colorful Image Colorization [PDF].
In testing, the system was applied to a number of legacy images from renowned photographers and produced the following impressive results:
The following examples of water bottles and military uniform, however, demonstrate the lingering inaccuracy:
Following a ‘colourisation Turing test’, in which human subjects were asked to distinguish between two colour versions of a monochrome picture – the original and the neural network-produced image – the team found that the software managed to produce a convincing colour recreation 20.4% of the time.
Zhang added that this was a significantly higher ‘fool rate’ than previous colourisation models. ‘If our algorithm exactly reproduced the ground truth colours, the forced choice would be between two identical images and participants would be fooled 50% on expectation,’ he noted.
The volunteer survey also found that in some cases participants were deceived more often than 50% of the time, showing that the recreated images were deemed more realistic than the original. The scientists put this down to the original picture featuring poor white balancing and unusual colours, while the system delivered a more ‘prototypical’ result.