Measuring sensors using RAW and testing lenses on cameras

News
Introduction | Why use RAW for measurements? | Why measure lenses on cameras?
Measuring sensors using RAW and testing lenses on cameras

DxOMark has many unique features, two of which are that (1) we perform sensor measurements on RAW images and (2) we test each lens on every camera on which it can be mounted. Why do we do this?

  • We deliberately chose to perform all DxOMark measurements on RAW images in order to evaluate the intrinsic image quality of lenses and camera sensors.
    This is particularly relevant for photographers who shoot in RAW and then need a reliable way to evaluate and compare the quality of the RAW signal output of their equipment.
    Only RAW-based measurements report on the image quality of the photographic hardware irrespective of the RAW converter. Evaluating RAW converters is a significant task in itself that we currently do not address on DxOMark.
    Read more about the reasons for this choice.
  • As optical performance factors such as resolution or vignetting not only depend on lens design and quality but also on sensor characteristics, it makes complete sense to provide separate measurements and scores for each lens for every different camera body on which it can be mounted. So a lens that can be mounted on 10 different camera bodies, for example, is measured and scored 10 times.
    When you browse through DxOMark, make sure you select the camera or cameras of interest to you when looking to evaluate lenses.
    Read more about the reasons for this feature.

What is a RAW image?

A RAW image is the picture that is directly delivered by the sensor. It is mostly unprocessed, and cannot be visualized on directly on a screen. It has to be transformed by a fairly complex signal processing known as RAW conversion.

Why is there a RAW image?

To answer this question, consider the image as viewed by the end-user of a camera. A color image is an array of pixels, each characterized by its spatial position and three integer values that convey intensities in three color channels—red, green, and blue.

The image "seen" by the sensor is completely different. Each photosite, corresponding to a pixel in the final image, can see only one type of color (either red, green, or blue). This means that the signal perceived by the sensor has a single channel at each pixel. The spatial distribution of the different color channels is up to the sensor maker, but the vast majority of sensors use a Bayer color filter array.

Moreover, the red, green, and blue that the sensor sees does not match the red, green, and blue seen by the human eye, for both physical and psychophysical reasons (meaning the color constancy that involves the brain and not just the eye).

Bayer color filter array.

A Bayer color filter array. Each photosite is either sensitive to red, or green, or blue. There are twice as many green photosites as red or blue photosites.

This image with interlaced channels is the RAW image. There is no way it can be avoided, since it is essentially what the sensor "sees." A RAW converter transforms it into an image that the user can see. Every camera has a RAW converter embedded on a chip. The RAW image is transferred to this image processing chip, converted to a RGB image, then transferred to the memory card. This process is generally sufficient for most users, who are usually interested only in the final image.

However, expert photographers want to control many parameters of the RAW conversion itself (such as white balance, exposure, and so on), a possibility now offered by camera manufacturers, who have made versions of their RAW converters available as PC software. In this case, the RAW image is directly transferred to the memory card before undergoing RAW conversion.

As we do not have access to intermediate outputs on the sensor, DxOMark measures RAW images — the very same images that can be accessed by photographers who use cameras that shoot in RAW.

"Pre-cooked" RAW?

Certain manufacturers embed a small part of the processing directly in the sensor, which means that some degree of processing occurs before the RAW image is sent to the RAW converter. In this case, measurements for these "pre-cooked" RAW images can be biased by this processing.

To avoid any potential impact on our measurements, DxOMark always tests all cameras to detect any pre-processing of RAW images. A processed or pre-cooked RAW image has different characteristics from a genuinely unprocessed image. To some extent, these characteristics enable us to walk back the processing and reconstruct the original image to perform unbiased measurements, and we always inform the user about models with embedded pre-processing.