We evaluate the performance of the smartphone when taking still images in a large number of use cases such as friends & family moments, landscape, close ups, cityscape, architecture, etc.
Our tests are performed :
- Using the default camera app and its default mode, at 1x zoom mode
- In an extended range of lighting conditions from 1 lux for lowlight to 1000 lux for bright light conditions and even higher for actual outdoor scenes
- With a tripod or through real or simulated handheld conditions.
To evaluate photo, assessments are made on a total of six technical quality attributes :
How well the camera properly adjusts to and captures the brightness of the subject and the background. Face exposure and contrast are also evaluated.
How accurately the camera can reproduce color in a variety of lighting conditions and use cases, as well as how pleasing its color rendering is to its viewers. We particularly focus on white balance, color rendering, color shading, as well as skin tone rendering, which are tested on a large spectrum, from deep to fair to light.
How quickly and accurately the camera can focus on a subject in a variety of lighting conditions. Besides speed, we also evaluate the depth of field of the camera and its capability to capture the instant
How well the camera can preserve small and fine details such as those found on object surfaces. A large set of details type is evaluated in all lighting conditions. The level of sharpness is also measured on a moving object, evaluating if the camera freezes well the motion while capturing a high level of details.
How much noise is present in an image, which is particularly sensitive in low light captures
Any effect appearing in an image which is not present in the original imaged object (distortion, halo effect…)
Our range of acceptable renderings for all attributes in various light conditions are based on the findings gathered from our multi-cultural focus groups. We believe that within this inclusive scope, the risk of bias is eliminated when evaluating a smartphone camera for a particular attribute. In keeping with this approach to testing, we do not evaluate manufacturers’ signature renderings.