Once again we hear a DxO proclamation that this or that provides the best image quality over this or that.. yawn..
Yet little has changed over the years, photographers continue to purchase, in greater numbers, the cameras that are no where near the top of the grand list.. Why you ask? Because the testing methods DxO uses do not mean anything to the majority of the public, they do not take into account the range of operation, the range of output, the range of use of said product.
To give an example, the current publicly ranked 'king' D800e, can only produce its 'advertised' results at ~100-200 ISO (plenty of studio work in that range) BUT only in the form of a ~75% downsampled ~8mp image. While that may be fine for a percentage of D800/e users, the reality (and reason for its corporate revenue killing sales) is that the majority of users are wanting to output full resolution 36mp images in some form. Everything from full rez prints (landscape, macro, sports) to the 'croppers' (wildlife, sports, daily, peepers, journalists, etc) where the IQ performance at 1:1 is what matters. It is in this area DxO is apparently under the impression that no one uses a digital camera for anything but 100iso 8mp 8x10 output, so there is no need to evaluate that area of performance in our ranking... and even that thinking would be fine if said camera produced its ranked results through its operating range.
**It is here we find the crux of the story**
It does not take much effort to click a few options in the comparison tool to see that the D800/e are the worst performing Nikon FF camera when using native resolution in any form, and at base ISO! Even if one were to speculate that the performance curve from native to 75% downsample was in its favor, one would be wrong. It also does not take much effort to see that of the cameras that outperform the D800/e (D4, D4s, D600, D610, D3s, D700) their performance at their native resolutions surpasses the D800/e even when downsampled to those resolutions. Meaning that even when downsampled to 24mp, the D800/e cannot match the output of the D610 at its native 24mp... it cannot match the 16mp performance of the D4 even when downsampled to 16mp.
Understand this is not a bash on Nikon, or the D800/e, merely the most identifiable examples of the two primary questions DxO fails to satisfactorily comment on for unknown reasons.
1: What possible reason could there be that DxO does not 'rank' sensors based on their entire operating range? We all understand that a certain area of photography world rarely has to go above 100iso, but these tools are not designed solely for 100iso. In actuality the rest of the photography world shoots considerably higher than base ISO. Certainly that would mean that the sensors testing, measurement, and ranking should reflect its performance across its ISO range and not just one setting...
2: What possible reason could there be that DxO does not 'rank' sensors based on their entire output range? We all understand that there are certain areas of photography that operate in the 8x10 world, but there are exponentially more images printed in magazines and other forms of media that are much smaller. Therefore allowing all equipment being tested to benefit accurately from the same percentage of downsampling. So instead of focusing on what one camera can achieve at some arbitrary output setting, DxO could certainly test, measure, and rank the performance of the device across all possible output ranges.
Then rank the device based on an averaged scope of its performance.
It has been mentioned by DxO that the ~8mp base ISO output fills some esoteric demand in the photography industry, and therefore represents an established criteria for measurement. However that niche was established back in the film days where an 8x10 or 8x12 was essentially the max output of 35mm film - not to mention the largest size the average person could reliably get printed. One could print larger, but if limiting the print to the highest amount of detail, sharpness, and color whether scrutinized by eye or loupe, that was around 8x11. Today we have magazines that are A3/B3 in size, so focusing on the size of the print is not really the optimal measurement criteria. We are also in a time where digital media is larger than print media by a quantum measure, so an 8mp image sample becomes even more arbitrary when the typical 1080p monitor only needs 2mp of data to fill it. We understand that there is a threshold, a point of diminishing return, in downsampling so why not decide on a percentage as opposed to output size. If a digital image of something like a D800/e no longer returns appreciable benefits beyond say a 75% downsample, then pick that as the ratio that all images are measured at. For a 16mp sensor a 75% downsample is not going to result in a 240-300dpi 8x10, but it would still produce an image size used significantly across other printed media - but more importantly it represents the beneficial result of a specific downsample that any sensor cam achieve and be fairly measured on. The higher the mp of the sensor the more knowledge and assurance that there is no more to be gained and most importantly nothing lost, so no brand or model is disadvantaged due to mp count versus downsample ratio.
The most significant discrepancy however is in the sensitivity range of the equipment being tested, as it clearly does include any calculation of the poor performance curve in some equipment in the 'scoring'.. Here again we have a situation where DxO has commented that there would be no clear range in which to measure and rank equally all devices given the advancements of technology. That is simply a load of self-absorbed empirically asserted horse manure. Just pick a range, any range, and go with it.. If the majority of digital imaging devices today are designed to operate between 100iso to 6400iso, then pick that range, or 100iso-3200iso. For older devices that cannot achieve the chosen ISO range well they have already been surpassed by modern devices in every way, so they would continue to be ranked below newer devices regardless. As newer devices are released they should improve on that performance range as mere consequence of trying to reach for higher ISO ranges. So If XX camera today achieves an average ISO score of 2900 for a designed 100-6400iso operating range, and two years from now YY camera improves the operating range to 100-51200iso without improving, or even depreciating, their previous performance then they are not working in the right direction and would be deserved of a lower score - otherwise their score within the same range should increase, regardless of any higher operating ranges designed, and ranking would continue to reflect that. See, it isn't that hard to really 'measure' and 'rank' the equipment without giving any advantage to one or artificial disadvantage to another.
More than anything else it would accurately test, measure, and 'rank' the devices on what they were designed to do, for any user, under all design conditions, instead of some esoteric criteria. The measurement and ranking methods used by DxO are certainly unique to DxO, and perhaps that is why you feel so validated in your empirical proclamations. To give some perspective of the anomaly though, if your testing, measurement, and ranking methods were applied to anything else we would have the best performing automobile based on the speed it can achieve while coasting downhill during a tail-wind with two of its four windows rolled down and tires under inflated by 15psi...
Of course one could make those 'settings' on every product tested, and therefore feel quantified in the results, but there are many more variables of the products design and operation range that determine its performance that any badge of accomplishment not based/averaged on overall operation is meaningless.
| Read all replies for this comment