"Dynamic range should decrease with ISO Sensitivity."
Overall, this statement is correct, but for some fairly complex reasons, there are some exceptions to the rule.
Dynamic range (DR) is the ratio between the highest and the lowest luminance that the sensor can handle, or equivalently, the highest and lowest gray levels. The lowest gray level considered is that for which SNR=0dB.
Why can DR be stationary with ISO at low values? For some sensors, the noise in dark shadows is dominated by conversion noise and is mostly independent from low ISO settings. Therefore, the lowest gray level such that SNR=0dB is also independent from ISO. That is why, for instance for Canon cameras, DR at ISO 100 and 200 are often pretty close. For higher ISO values, photonic noise takes the lead again, and predictably, DR loses 1bit.
Why can DR even increase from ISO 100 to ISO 200? If we examine the other end of the scale, highlights, we see that the upper bound is fixed by sensor saturation. There are two possible limiting factors:
1. The capacity of the pixel well (the number of electrons)
2. The digital A/D conversion (12- or 14-bit).
Usually saturation is attained via digital A/D conversion, and the saturation value on the sensor is 4095 or 16383 (or close). However, for a few sensors at very low ISO, the electron well saturates before the digital converter. Therefore, for low ISO, the highest gray level is lower than can be expected. If this effect cumulates with the noise in the shadows as described previously, then for these sensors, DR can actually be lower for lower ISOs!
"SNR curves should not cross!"
False: SNR curves are normalized with respect to the saturation value. Since this value can change depending on the ISO, curves can indeed cross. (Read the above question).
"Optics can bias your measurements."
Very true, so we were very careful about removing this bias from our protocol! Noise depends only on the quantity of signal on the sensor. It does not depend on the optics. However, the optics could indeed introduce a bias in measuring ISO sensitivity.
For each D-SLR, we measured ISO with no optics at all, using a controlled light source far away from the sensor.
Obviously, we could not do so with bridge cameras and DSCs. For these models, we referred to ISO 12232 standard and applied a correcting factor to ISO sensitivity measurements to account for the transmission loss due to optics.
"Why are noise measurements in RAW relevant for evaluating the sensor, and not for JPEG?"
Noise in JPEG has gone through a lot of processing (demosaicing, noise reduction, sharpening, etc.). This increases the Signal-to-Noise Ratio (SNR) in homogeneous zones, but also very often removes details. The performance variability of RAW converters is huge, so measuring noise in JPEG mixes the performance of the camera and the performance of the RAW converter. Our aim is to provide measurements that are independent of the RAW converter in order to establish a baseline for camera performance potential.
If you now compare the rating of two cameras in RAW, the baseline is that if you use the same RAW converter for both cameras with equivalent settings, the comparison after RAW conversion will be more or less equivalent to the results in RAW.
"Some cameras already process their RAW images. In particular, they apply noise reduction, which leads to unfair comparison."
This is correct. In order to be completely fair, we also measure the amount of filtering applied and then retrieve the value of noise before filtering. Until now, no camera in the dxomark.com database generates noise-reduced RAW images that impacts its DxOMark ranking. In particular, the Sony Alpha 900 does not filter the green channel, but only red and blue. We will publish some explanations about the Sony Alpha 700’s “pre-RAW” processing very soon.
"Some dxomark.com datasheets provide an ISO latitude that differs from the manufacturer’s datasheet."
Many cameras have “special sensitivity modes,” often named “H” or “L,” which in fact are parameters for auto-exposure and RAW conversion but have no bearing on the sensor’s sensitivity itself. The definition of these special sensitivity modes is not standardized. In dxomark.com, we analyze all settings and report only on those that influence the RAW output. Thus the ISO latitude of the dxomark.com report may differ from the manufacturer’s ISO latitude.
"It looks like some manufacturers cheat with ISO sensitivity."
First, ISO Sensitivity values as determined by ISO 12232 standard are quantized (with a rounding method that is precisely described in the standard itself), therefore there is always a difference between the manufacturer’s published value and actual measurement data. Also, RAW conversion typically increases ISO sensitivity (by applying a digital gain), so when a manufacturer reports an ISO value that takes into account these two factors, it may very well differ from the actual ISO Sensitivity measurement.
"Canon 40D has an option called “Highlight Tone Priority,” and other manufacturers offer similar features. Were the dxomark tests performed with this feature on or off?"
Digital cameras commonly offer numerous shooting options and settings (portrait, landscape, sport, night mode…) that may impact the sensor signal processing. In order to measure only a camera’s RAW sensor quality, all measurements on the dxomark website are then made with camera’s image processing options turned off, without exception.
"Why RAW-based ISO sensitivity values reported on dxomark.com sometime differ from manufacturer’s ISO sensitivity values?"
All ISO sensitivity measurements available on dxomark.com are measured in RAW while manufacturers provide an ISO value ("ISO Manufacturer" on the dxomark website) that actually reflects the final ISO sensitivity on the finished JPEG image. And the RAW to Jpeg conversion processing, that involves applying a digital gain and a tone curve, generally increases the ISO sensitivity and then enables manufacturers to apply lower RAW-based sensitivity values versus marked ISO settings.
"How is ISO Sensitivity measured for DSC cameras (cameras with non-removable optics)?"
DxOMark Sensor measures ISO sensitivity without optics whenever possible. As DSC/Bridge cameras have non-removable optics, however, DxOMark Sensor measures their ISO sensitivity using a modified protocol based on the scene luminance and the camera lens aperture. For more details, see the "Measurement definitions" section