Photoshop v 5.0 - v 5.5 Induced Noise

Please note that from version 6.0 onwards this noise can be disable in the Edit/ColorSettings/Advanced dialog.

Findings, experiments and other info is in chronological order.

Photoshop 16-bit to 8-bit conversion adds noise and/or is not repeatable

I noticed that the conversion from 16-bit mode to 8-bit mode both in RGB (using Photoshop 5.0.2 Windows version) seems to add noise, so I did an experiment. 

To repeat my test please download the 16bit_noise.zip (1kB)

This zip contains the file: 16bit_noise.raw.  It is in raw mode so one can more easily verify the byte values in it using e.g. the UltraEdit shareware that has the Hex mode. 

The file has 20 RGB patches (19-steps) that are all gray (R=G=B) and the hex values of the steps are: 0, D79, 1AF2, 286C, 35E5, 435E, 50D7, 5E50, 6BCA, 7943, 86BC, 9435, A1AF, AF28, BCA1, CA1A, D793, E50D, F286, FFFF.  So it is equally spaced 19-step wedge from level 0 to level 65535 in decimal values.

To open the 16bit_noise.raw use the following values in the Raw Options dialog: width=40, height=800, count=3, interleaved=yes, depth=16, byte order=IBM PC, header size=0.
Firstly examine the patches visually at 800% zoom or with the eye-dropper, there is no noise in any of the patches.

Now do Image/Duplicate, then convert the original into 8-bit and convert the copy to 8-bit too.

Examine the patches visually at 800% zoom or with the eye-dropper, you will see that they now do contain noise.

Now select the original and do Image/Apply_Image. Set the following into the Apply Image dialog:

• source=16bit_noise copy (select this from the dropdown)
• layer=backgound
• channel=RGB
• blending=difference
• opacity=100
• scale=1
• offset=0

This operation will turn those pixels that are the same between the images into level 0 and other pixels turn into that level that is the difference. Obviously the original should turn into pure black (level 0) completely. 

Now inspect the histogram of the resulting image or do Image/Adjust/Levels and drag the white-point slider downwards until you can see the noise. Histogram shows that some pixels are at level 1 the rest are at level 0. 

Conclusion, there is some _random_ generator included in Photoshop 16-bit to 8-bit conversion.

Now, at first glance this may seem to be a good idea since it seems to use a sort of dithering to convey the extra info from the 16-bit space into 8-bit space, _but_ it is far away from being good.  Further image enhancement such as sharpening or USM will boost up this noise very inconveniently.

Because it is random function it just represents noise. In fact the conversion should be called 16-bit to 7-bit conversion. 

Hopefully there is some ini setting for disabling the random generator?

An Another Test

• Create a RGB new image of 400 x 400 pixels in RGB.
• fill it with mid gray RGB level (128,128,128).
• convert to 16-bit mode.
• convert to 8-bit mode.
• view the image at 200%.
• make the noise visible by using Levels. Enter the value 126 into the left input box and enter the value 130 into the right input box.

Now repeat the above 3 times so that you have four images. You will find out that the noise pattern in the images is actually quite similar but their location changes from image to image. What could be the reason for such behavior?

Improved test showing repeating pattern among the noise

I have now experimented with two PC systems (all my systems have legal, licensed installation). The following experiment will give greatly different looking "noise" pattern between these two systems:

1. Create a new RGB image of 1000x1000 pixels.
2. Fill it completely with R,G,B=240,240,240 
3. Convert to 16-bit mode.
4. Convert to 8-bit mode. Now there is a "noise" pattern in the image that differs by one level from level 240.
5. View the image at 200% zoom.
6. Use Image/Adjust/Threshold to make the noise clearly visible, set it to 240 and press OK.

On both systems the "noise" pattern will be in different location in consecutive trials. 

When I stack two consecutive trials on the same system to layers and then move the topmost layer (using normal mode, 50% opacity)  I can detect a certain repeating pattern among the "noise". It takes a little time to recognize the repeating pattern but it is there in every trial, only in different location. I can nudge the top layer so that the repeating pattern match accurately with the layer beneath and then when the top layer is changed to difference (still at 50% opacity) the repeating pattern can be differentiated easily from the rest of the noise.

Here is an example, 16-bit_noise_example.zip It contains the file "16-bit_noise_example.psd" with two layers matched for the repeating pattern, dark pixels match, light pixels are the rest of the noise. View it at 200% zoom.

This repeating pattern among the noise is only something below 10% of the total "noise" and the repeating pattern too is different between the two installations of Photoshop 5.0.2. The digimark plug-in was deleted from both of the system at the time Photoshop was installed.

I have also tested this with my home machine that has AMD K6 processor, the machines at work have Intel Pentiums. The installation at home (the digimark plug-in is removed also from this system) is a copy of one of the installations at work (this is according to license agreement) and in my AMD K6 machine Photoshop gives the same noise pattern as it does in the machine at work, also the repeating pattern is same.

This feature has to be removed since it makes the Photoshop 16-bit mode pretty much useless since one can only recover 7-bit real data from the 16-bit mode.

Comparison with Corel Photo-Paint version 8

I have now tested this with Corel Photo-Paint v.8, it also has the 16-bit mode (called 48-bit RGB in Photo-Paint).  I did the same spot color tests as I explain above and Photo-Paint does not create any noise in the conversion.

In addition I opened the 16-bit step wedge file "16bit_noise.raw" (see above) into the 16-bit mode of Photoshop 5.0.2 and saved it as TIF and PSD, both in 16-bit mode. I then opened both files into Corel Photo-Paint v.8 and converted them to RGB 24-bit (the same as the RGB 8-bit mode in Photoshop) and again, there is absolutely no noise, conversion is completely clean for both of the files.

Demonstration I

This demo was done completely in gamma space 2.2 in order not to rise questions about the gamma issue with this bug. It really does not matter in what gamma space the Photoshop Indused Noise is demonstrated since Photoshop applies 1 RGB level noise all over the range, independent of the working space gamma.

1. The face of the lady from Kodak Q-60 was scanned with Umax PowerLook III at 300 DPI using the 14-bit/color mode and acquired to Photoshop 5.0.2 16-bit mode. This image was saved as "ps5-pp8_photoshop5.psd" and "ps5-pp8_photo-paint8.psd". 
2. "ps5-pp8_photo-paint8.psd" was opened to Corel Photo-Paint v8 and the file was converted into 8-bit per color and saved.
3. "ps5-pp8_photo-paint8.psd" was opened into Photoshop 5.0.2.
4. Both images were sharpened in Photoshop using USM at v=350%, r=0.5, t=0
5. Both images were saved from Photoshop as JPG at q=6. Yes, the Photoshop Induced noise propagates even in to JPEG. Errors are much  larger between the uncompressed originals. 

Converted in Photoshop

Converted in Photo-Paint

View the images in gamma space 2.2 in a dim room.

• view the image at normal reading distance. Compare e.g. the large dark area at lower right corner. The Photo-Paint converted image has larger tonal range, black is better, deeper black. This is is fact, it can be seen numerically using Histogram. But it is obvious since Photoshop adds random noise into the image, all over it's range. 

 
• view the images at some 2 meter distance (6 feet). Photo-Paint converted image has more details. Again obvious, the more there is noise the less there is details.

 
• Assess the color quality, chroma. Yes, there is richer chroma in Photo-Paint converted image. This is fact, it can be seen numerically using Histogram.

 
• Assess the subtle shading in highlights, e.g. her  left cheek (on the right on the monitor of course). Again Corel Photo-Paint converted image performs better.

Here is the difference map, taken in Photoshop using Apply/image, difference mode, o=100%. Histogram is stretched in order to show show that: The noise does not carry any recognizable info. It is just random noise that makes nothing else than damages the image quality. Maximum difference was 8 RGB level in all the color channel. In Luminosity channel: Mean=1.14, StdDev=1.27 In Red channel: Mean=1.08, StdDev=1.74 In Green channel: Mean=1.20, StdDev=1.81 In Blue Channel: Mean=1.29, StdDev=1.85 In plain English Photoshop converted image is more than one bit lower quality than the Photo-Paint converted image.

Note that any subsequent image manipulation operation after the conversion is adversely affected by the Photoshop Indused Noise and can multifold the effective value of this noise.

Note also that every 16-bit to 8-bit conversion will add the noise.
 

Demonstration II

Below demonstration shows the noise pixels closer, they appear in clusters or groups of pixels. Original was the 16bit_noise.raw. The left, noisy step-wedge was converted to 8-bit mode in Photoshop, step-wedge on the right was converted in Photo-Paint. UnsharpMask was applied at a=200%, r=1, t=0 over both images in Photoshop after the conversion.

Other information

Usenet post where Mr. Cox (who is a codewriter at Adobe) explain the Photoshop Induced Noise.

I have received Adobe Tech Support answer to my complaint. They suggested me to acquire the images in 8-bit instead of the higher bit-depth mode. 

It does not make sense to buy very expensive acquire devices that are capable to provide higher bit-depth, high quality images and then to just acquire in 8-bit. 

This bug is particularly difficult for owners of Kodak (with Canon and Nikon SLR body) high-end digital cameras since they all use the same Kodak DSC Twain Driver that does not work with any other program than Photoshop.

How this problem relates to the gamma or other calibration settings

This problem has nothing to do with the gamma or RGB setup in general. The "noise" pixels are always 1 RGB level off from what they should be and this "noise" appears all over the range. I have been testing this with different calibration and gamma settings and the result is always the same.

Photoshop has only 15-bit mode, not 16-bit mode

Please see this Usenet thread where Mr. Chris Russ explains his finding. 
From my follow-up:

Yes, Photoshop has only 15-bit space, not 16-bit mode !

I created a gradation from level 0 to level 255 with one level steps in double byte hex (16-bit space) using VisualBasic for application, saved this as gradation.raw. Opened it into Photoshop, saved it as gradation.psd, closed this and re-opened it, then saved it as gradation_ps.raw.  Now the gradation.raw and gradation_ps.raw should be the exactly the same, but there are only 128 different values in the gradation_ps.raw, stepping is by two levels. So one bit is missing, it is only 15-bit space.

This means that when the acquire device is 12/36 bit or better and it's Twain module applies calibration or other operations over the image data in a true 16/48 bit space then some image information will be lost when Photoshop convert that image into the 15-bit space. Addmittedly this loss is not large. This is propably the reason why some Twain acquire modules only work with Photoshop, they know that Photoshop wants 15-bit data instead 16-bit.

The mode conversion noise of Photoshop is much worse since the 1 bit noise is applied over the 8/24 bit data. One way to get around the conversion noise is to save the image as tiff and use Photo-Paint for the mode conversion, it has true 16-bit mode. However that is an expensive and timeconsuming method, fortunately the Photoshop induced noise can be safely removed:

The Healing Knife

The Photoshop Induced Noise can be removed by Hex editing the file Photoshp.exe (the same works for versions 5.0.2 and 5.5).  Edit the three hex locations as follows:

This removes the random noise from RGB, Gray and CMYK 15-bit modes and does proper rounding.