"Medium-format cameras are dedicated to specific uses and DxOMark equally weighs three use cases. Why are medium-format cameras ranked on such a scale?”
This question has been partly addressed in our reviews of medium-format cameras
Medium-format cameras are dedicated to studio conditions and high-resolution applications. To summarize, the DxOMark Sensor scale is an average of three metrics which evaluate sensor behavior under three typical shooting conditions. If a photographer uses his or her camera for only one or two of these three scenarios, he or she should consult the specific rankings for the corresponding metrics.
"Why aren’t other parameters such as fps (frames per second) or tropicalization factored into dxomark?"
DxOMark Sensor scale and dxomark.com data evaluate only the sensor image quality. We do not claim to address all digital camera characteristics; rather, we provide a set of key metrics that are essential to assessing RAW sensor performance.
"Most medium-format cameras do not have an anti-aliasing filter, and DSLRs do. Is this taken into account in the DxOMark scale?”
When a sensor receives a scene pattern with higher spatial frequencies than its maximum resolution, the captured image will be affected by a moiré pattern (or alias) (see on Wikipedia). This typically occurs when the lens resolution is much higher than the sensor resolution. The purpose of the anti-aliasing filter is precisely to remove these image frequencies that are higher than the sensor resolution limit. DLSRs commonly feature anti-aliasing filters as their lenses commonly demonstrate higher resolution.
As the DxOMark Sensor scale does not directly factor in resolution, the presence or lack of an anti-aliasing filter has no bearing on a camera’s DxOMark ranking.
(It is worth noting that medium-format cameras are not the only cameras on dxomark.com without anti-aliasing filters: for example, the Nikon D70 and many Nikon cameras launched during the same period also do not have anti-aliasing filters and incidentally have many Moiré problems.)
"DSLRs have intrinsic RAW processing capabilities. Doesn’t this mean that their DxOMark ranking is biased in their favor, to the detriment of medium-format cameras (which do not do RAW processing)?"
DxO Labs does, in fact, check for RAW processing prior to testing any camera. Until now, no cameras in the dxomark.com database show any spatial noise-correlation (indicative of digital noise filtering on RAW images) that would significantly impact their SNR measurements and therefore their DxOMark scores. So MF cameras are not placed at a disadvantage in this respect. Read more
"Isn’t high ISO over-weighted on DxOMark to the disadvantage of medium-format cameras?"
High ISO represents 1/3 of the DxOMark Sensor scale. If high ISO is not a priority, photographers should consult the separate Color Depth and/or Dynamic Range scales that correspond to the specific photographic usages for medium-format cameras. It is useful to photographers to have objective measurements of medium-format cameras’ ISO performance to factor in with all the other considerations involved in choosing a camera.
"How can DxOMark rank medium-format sensors below DSLRs, given that MF camera sensors are known to be far superior? "
First of all, the first 4 medium-formats available in the dxomark.com database are not the latest generation MF available on the market. Their launch dates were between 2006 and the beginning of 2007. Their non-top rankings are mostly due to their noise levels, which are not as good as those of the latest high-end DSLRs.
It is important to note that medium-format cameras utilize CCD-based sensor technology, while all high-end DSLRs now use CMOS sensors. It is possible that stronger competition among DSLR manufacturers has helped to advance CMOS pixel technology more quickly than has been the case for CCD technology.
"Is it logical and fair to normalize based on an 8Mpix A3 print when ranking medium-format cameras are primarily designed for very high resolution applications and large-size prints?"
Using a higher-resolution print format would not change the camera ranking; it would only exclude any camera that does not have a high enough resolution to produce the chosen print. For further information about normalization, click here, and for further information about using DxOMark Sensor scale, click here.
Overall, objective versus subjective evaluation is being questioned: the end-product of a camera is a photo looked at by a human. How can its quality be assessed by numbers?
While assessing a camera as a whole photographic system requires a visual and subjective evaluation, particularly of its output (photos), the role of the sensor by itself is to capture the scene lighting with the best possible accuracy and the least possible noise. Its performance can be measured objectively, which is what dxomark.com is all about.
It is the RAW converter’s role to render a subjectively pleasing image from the pixel values delivered by the sensor. It is the RAW converter that embeds into the image all the subjectivity related to contrast, colors, etc., not the sensor itself. Anyhow, everything else held equal (optics and RAW converter), the better the sensor, the better the output image.
This said, even though dxomark.com only provides objective RAW sensor evaluation, we at DxO Labs have a keen interest in subjective image quality analysis. To this end, we participate in scientific and industrial initiatives which aim to develop objective measurements that correlate well with visual perception. For example, DxO Labs is actively providing testing methodologies and serving as a Technical Editor for the I3A-CPIQ initiative, a consortium of sensor manufacturers, lens manufacturers, camera and cameraphone manufacturers, image processing companies, and other members of the imaging industry.
"How is it possible for dxomark to assess the quality of a camera from just few shots of very simple color and density pattern charts?"
dxomark.com provides measurements only for the image sensor, and the primary function of a sensor is to capture the light level or count the photons on all the sensitive areas (pixels) across its entire surface. Simply put, a sensor is a photon counter. To measure the performance of a counter, there is no need for highly structured patterns; rather, what is needed is a set of accurately calibrated charts that provide a large variation of light levels, and a set of very controlled, uniform patches of distinct colors to measure color signal sensitivity.