Evaluation of the the CIE Color Difference Formulas

Abstract

On-line evaluation of the CIE dE*, dE94(1,1) and dE00(1,1,1) and British Standards CMC(1,1) and CMC(2,1) color difference formulas are provided at dE=8, dE=5, dE=3, dE=2 and dE=1, the Photoshop Lab mode original is also available for download for more accurate off-line evaluation. Each evaluation table has 72 reference colors, and around them 10 different colors, at the specified dE. A major systematic error is found in the behavior of all the five color difference formulas and it is easily seen that none of the evaluated color difference formulas are usable for real life situations, the dE00 and CMC(2,1) being the worst of them.

A short history of the evaluated color difference formulas

The CIE dE*

In the year 1976 the CIE published the CIELAB (L*a*b*) color space specification, it was said to be perceptually uniform color-space. In a perceptually uniform color-space the color difference is easy to calculate, it is the distance between the two colors, mathematical expression for it is the Euclidean distance, the length of the straight line that connects the two points in a three dimensional space. So, along with the CIELAB the CIE dE* Color Difference Formula was published also [it is referred in this document as dE(1976)]. The dE(1976) Euclidean distance is calculated as:

The CMC

CMC was developed in 1984 by the Colour Measurement Committee of the Society of Dyes and Colourists of Great Britain. Published by British Standards BS:6923 "Method for calculation of small colour differences".

The CIE dE94

Gradually it became clear that the dE(1976) was not at all a perceptual color difference quantity (note that this is the same as to say that the CIELAB is not a perceptually uniform color space) therefore in the year 1995 the CIE decided to publish another color difference formula, namely the CIE94 [it is referred in this document as dE(1994)]. Of course the right thing to do at that time would have been to publish a new truly perceptually uniform color space. The CIE dE(1994) formula is available on the Web e.g. on Bruce Lindbloom's excellent site, in the math section.

The CIE dE00

But neither was the CIE dE(1994) a perceptual color difference quantity therefore in the year 2000 the CIE decided to publish yet another color difference formula, namely the CIE00 [it is referred in this document as dE(2000)]. Again the right thing to do at that time would have been to publish a new truly perceptually uniform color space. The CIE dE(2000) formula is available on the Web, again e.g. on Bruce Lindbloom's.

The evaluations

Five large on-top-of-each-other comparisons are shown below on this page, one comparison for dE=8, one comparison for dE=5, one comparison for dE=3, one comparison for dE=2 and one comparison for dE=1. In each of the five comparisons the five Color Difference Formulas dE(1976), dE(1994), dE(2000), CMC(1,1) and CMC(2,1) can be selected using the Option Buttons that are found on top of each comparison.

Each of the five comparison have 72 segments, these reference colors are the same in all the five comparisons. The whole comparison is designed in such way that all the color patches are inside the CRT color gamut. NOTE, in the CMC(2,1) some colors are our of gamut, they are replaced with back batches)

The comparison segment

In each of the segments the reference color is compared against 10 nearby colors that are at the specified dE distance, the specified dE distance is generated in the following way:

Please note the arrangement, the comparisons reveal a major systematic error in all the CIE Color Difference formulas.

About Accuracy of Color Evaluations Using Computer Monitors

The implementation of ICC color management in current applications gives one major trouble for color evaluations, they do not manage the blackpoint, no matter how accurately the display is calibrated and profiled.

When an RGB=0,0,0 or L*a*b*=0,0,0 color code is shown on the monitor screen that resulting color is not anywhere near to absolute black. A typical situation is that the black of the screen has something around 0.3cd/m^2 luminance, it is mainly reflected ambient illumination. At RGB=255,255,255 or L*a*b*=100,0,0 a typical CRT outputs about 100cd/m^2 luminance. The result of this is that what is color code L*=0 in the image data is about L*=2.7 on the screen.

This amount of blackpoint error is very significant for colors at the deep dark end of the range. For the colors that are at around L*=25 this error affects 0.91 L* units, for the colors that are at around L*=50 this error affects only 0.36 L* units and for colors that are at around L*=75 this error affects only 0.19 L* units. Nearly all of the color patches in the below comparisons are above L*=50 and this is a comparison so all patches are affected towards the same direction and in each comparison segment at about the same amount.

How to evaluate

In each comparison segment the visual difference between the reference color and the 10 comparison colors should be perceptually very similar/equal. Do not compare between any two comparison colors, compare only between the reference color and the 10 comparison colors.

Change the Color Difference model by clicking the Option Buttons that are found on top of each five comparisons.

What are the two major findings?

1. Always when the color difference is due to a change in L* channel alone the perceptual difference is huge compared to any of the 10 cases where the color difference is due to a change in a* and/or b*. In other words the transfer function of the CIELAB is faulty. The dE(2000) formula increase this error the most.
   
   
2. None of the evaluated color difference models are in good agreement with our vision, or, more accurately, they are completely useless measures for color difference. E.g. dE=8 can mean that two colors are perceptually extremely different from each other but it can also mean that two colors are perceptually extremely similar.

Why do the CIE Color Difference Formulas fail so enormously

Color scientists evaluate the color difference in a totally different viewing situation than what is encountered in the normal life (paint, textile, graphic art, digital imaging etc.) where we evaluate color differences. They stare at one comparison pair at a time, with dark surroundings, and they allow the vision to fully adapt according to the light from those two color patches, two minute adaptation time is very common.

In the normal life we always see a complex scene and the vision will adapt according to the light from that whole scene. In normal life the vision can never adapt to just two small objects that are somewhere in the field of the view, no matter how long time we stare at them. The state of the adaptation of the vision is always dictated by the whole scene.

So, we desperately need a Color Difference Formula for the real life use.

Plenty of conclusions, that are universally considered to be the absolute truth, have been derived from these very incorrect color difference functions, and from the equally incorrect CIELAB color-space specification. In the application area of digital imaging the extremely nonlinear RGB working-spaces (gamma 2.2 and gamma 1.8) are one example. Another example is the ICC profile specification, the only way to create printer profiles is to use the 3-dimensional CIE Lab look-up-tables, therefore the profiling criterions as well as the interpolation criterions are based on these incorrect color difference formulas.

The Comparisons:

Download the Original

These comparisons were generated using the AIM-XLA and exported to Photoshop layer by layer. For more accurate evaluation please download the original as a layered 16-Bit/channel Lab mode image. A layered 8-bit/Channel version is also provided.