|DxOMark unique features | Test result reliability|
When looking for digital camera image quality evaluation data and benchmarks, you want to be sure that the provided measurements have been openly and objectively performed by a completely independent testing operation that uses state-of-the-art equipment and follows industry-standard methods.
It is our constant focus at DxO Labs to ensure that dxomark.com is a source of information that photographers can fully trust for objective evaluations and comparisons of digital cameras and lenses.
All published measurements are RAW sensor-based and performed prior to any digital image processing (except processing that may happen on the sensor chip itself). RAW format is analogous to a film negative in traditional photography, and it is the format by which objective performance for digital cameras is most effectively evaluated, as it does not depend on the efficiency of the RAW conversion applied.
RAW is also intrinsically more relevant for measuring lens performance, given that no further processing was used that might have smoothed or hidden any intrinsic optical limitations or lens defects.
Overall, for both sensors and lenses, DxOMark RAW-based measurements and ranking scales provide an objective and essential evaluation and comparison tool. Read more about using RAW to measure image quality.
DxOMark lens measurements are unique in that they are performed on RAW and are specific to all camera bodies for which each lens is designed. The test results published on dxomark.com conclusively show that the same lens performs differently when mounted on different camera bodies. Many sensor characteristics, such as physical size, pixel pitch, and microlenses, influence the optical performance of lens/camera combinations (read more on how lens performance is affected by the sensor).
On DxOMark, the camera body is always referenced for lens measurements, and you can easily browse through the compatible bodies for a given lens to view how the different combinations perform.
We perform measurements for each lens-camera combination using a wide range of shooting parameters (aperture, ISO speed, object distance, focal length, etc). Up to a thousand shots of various targets under different exposure conditions may be required for each combination, generating hundreds of thousands of measurement data reflecting lens performance (read more about how we test).
Does manufacturing variability impact DxOMark lens measurements and scoring?
DxOMark contains hundreds of thousands of lens measurements. As with any measurements, it is not possible to achieve absolute precision. Although we follow very strict procedures, there will always be a small margin of error.
Another question users might raise is: "Are the results presented on DxOMark really representative of my own personal lens?" — In other words, individual lenses of the same model may vary because of manufacturing tolerances, and so we can expect some variability of performance.
What is the order of magnitude of these manufacturing variations? Is there a lottery with lucky winners having awesome lenses and unlucky users who might be better off using the bottom of a soda pop bottle?
Although we can never exclude outliers, our tests show that it is not very likely that variances in lens manufacturing will significantly change (for better or worse) the performance of a lens.
All measurements follow the methods and protocols described in ISO (International Standard Organization) standard documentation or developed by industry players and considered as industry standards. DxO Labs image scientists and engineers have studied, developed, and implemented such standard methods and protocols and then packaged these tools and know-how in a commercial solution, DxO Analyzer.
DxO Analyzer is now used in the digital camera and component industry worldwide by such manufacturers as Nikon, Olympus, Panasonic and Samsung, to name just a few. DxO Analyzer is also used to perform certain tests by many web and print photo magazines, including Popular Photography, Nippon Camera, Chasseur d’Images, DIWA Labs, Imaging Resource, Réponse Photo, Let’s Go Digital, Digital Camera Magazine Japan, digitalkamera.de, and more than 30 other press partners.
Further, DxO Labs has put in place a very thorough data validation process. Once data are produced, they are analyzed in detail and compared with those in our existing image quality measurements database (thousands of digital cameras, lenses and imaging systems have been tested and analyzed over time). This process has led us to develop specific protocols or tools to deal with difficult measurement issues such as best focus optical performance and sensor spectral sensitivity.
DxO Labs is an active participant in international imaging standardization activities. For example, we have been elected as Technical Editor of the International Imaging Industry Association (I3A) CPIQ standard for defining new methods to assess photographic quality from the consumer point of view, and DxO Labs image scientists regularly present their results at such renowned digital imaging conferences as the Electronic Imaging Conference, and Image Sensors (Europe and US).
To learn more, visit the official DxO Labs website.