Apple iPhone Air Camera test

For scoring and analysis, DXOMARK engineers capture and evaluate more than 3,800 test images in controlled lab environments as well as outdoor, indoor and low-light natural scenes, using the camera’s default settings. The photo protocol is designed to take into account the main use cases and is based on typical shooting scenarios, such as portraits, landscape and zoom photography. The evaluation is performed by visually inspecting images against a reference of natural scenes, and by running objective measurements on images of charts captured in the lab under different lighting conditions from 0.1 to 10,000+ lux and color temperatures from 2,300K to 6,500K.

Exposure and color are the key attributes for technically good pictures. For exposure, the main attribute evaluated is the brightness level of the main subject through various use cases such as landscape, portrait, or still life. Other factors evaluated are the global contrast and the ability to render the dynamic range of the scene (ability to render visible details in both bright and dark areas). When the camera provides Photo HDR format, the images are analyzed with a visualization on an HDR reference monitor, under reference conditions specified in the ISO-22028-5 standard. Repeatability is also important because it demonstrates the camera's ability to provide the same rendering when shooting several images of the same scene.
For color, the image quality attributes analyzed are skin-tone rendering, white balance, color shading, and repeatability. For color and skin tone rendering, we penalize unnatural colors according to results gathered in various studies and consumer insights while respecting the manufacturer's choice of color signature.

Autofocus tests concentrate on focus accuracy, focus repeatability, shooting time delay, and depth of field. Shooting delay is the difference between the time the user presses the capture button and the time the image is actually taken. It includes focusing speed and the capability of the device to capture images at the right time, what is called 'zero shutter lag' capability. Even if a shallow depth of field can be pleasant for a single subject portrait or close-up shot, it can also be a problem in some specific conditions such as group portraits; Both situations are tested. Focus accuracy is also evaluated in all the real-life images taken, from infinity to close-up objects and in low light to outdoor conditions.

Texture tests analyze the level of details and the texture of subjects in the images taken in the lab as well as in real-life scenarios. For natural shots, particular attention is paid to the level of details in the bright and dark areas of the image. Objective measurements are performed on chart images taken in various lighting conditions from 0.1 to 10,000+ lux and different kinds of dynamic range conditions. The charts used are the proprietary DXOMARK chart (DMC), and the Dead Leaves chart. We also have an AI based metric for the level of details on our realistic mannequins Eugene and Diana.

Noise tests analyze various attributes of noise such as intensity, chromaticity, grain, structure on real-life images as well as images of charts taken in the lab. For natural images, particular attention is paid to the noise on faces, landscapes, but also on dark areas and high dynamic range conditions. Noise on moving objects is also evaluated on natural images. Objective measurements are performed on images of charts taken in various conditions from 0.1 to 10000 lux and different kinds of dynamic range conditions. The chart used is the Dead Leaves chart and the standardized measurement such as Visual Noise derived from ISO 15739.

The artifacts evaluation looks at flare, lens shading, chromatic aberrations, geometrical distortion, edges ringing, halos, ghosting, quantization, unexpected color hue shifts, among others type of possible unnatural effects on photos. The more severe and the more frequent the artifact, the higher the point deduction on the score. The main artifacts observed and corresponding point loss are listed below.

Bokeh is tested in one dedicated mode, usually portrait or aperture mode, and analyzed by visually inspecting all the images captured in the lab and in natural conditions. The goal is to reproduce portrait photography comparable to one taken with a DLSR and a wide aperture. The main image quality attributes paid attention to are depth estimation, artifacts, blur gradient, and the shape of the bokeh blur spotlights. Portrait image quality attributes (exposure, color, texture) are also taken into account.

All image quality attributes are evaluated at focal lengths from approximately 40 mm to 300 mm, with particular attention paid to texture and detail. The score is derived from a number of objective measurements in the lab and perceptual analysis of real-life images.

DXOMARK engineers capture and evaluate almost 3 hours of video in controlled lab environments and in natural low-light, indoor and outdoor scenes, using the camera’s default settings. The evaluation consists of visually inspecting natural videos taken in various conditions and running objective measurements on videos of charts recorded in the lab under different conditions from 0.1 to 10000+ lux and color temperatures from 2,300K to 6,500K.

Exposure tests evaluate the brightness level of the main subject, the global contrast and the ability to render the dynamic range of the scene (ability to render visible details in both bright and dark areas). When the camera provides Video HDR format, the videos are analyzed with visualization on an HDR reference monitor, under reference conditions specified in the metadata. Stability and temporal adaption of the exposure are also analyzed.
Image-quality color analysis looks at color rendering, skin-tone rendering, white balance, color shading, stability of the white balance and its adaption when light is changing.

Texture tests analyze the level of details and texture of the real-life videos as well as the videos of charts recorded in the lab. Natural videos recordings are visually evaluated, with particular attention paid to the level of details in the bright and areas as well as in the dark. Objective measurements are performed of images of charts taken in various conditions from 0.1 to 10000 lux. The charts used are the DXOMARK chart (DMC) and Dead Leaves chart.

Noise tests analyze various attributes of noise such as intensity, chromaticity, grain, structure, temporal aspects on real-life video recording as well as videos of charts taken in the lab. Natural videos are visually evaluated, with particular attention paid to the noise in the dark areas and high dynamic range conditions. Objective measurements are performed on the videos of charts recorded in various conditions from 0.1 to 10000 lux. The chart used is the DXOMARK visual noise chart.

Stabilization evaluation tests the ability of the device to stabilize footage thanks to software or hardware technologies such as OIS, EIS, or any others means. The evaluation looks at residual motion, smoothness, jello artifacts and residual motion blur on walk and run use cases in various lighting conditions. The video below is an extract from one of the tested scenes.

Artifacts are evaluated with MTF and ringing measurements on the SFR chart in the lab as well as frame-rate measurements using the LED Universal Timer. Natural videos are visually evaluated by paying particular attention to artifacts such as aliasing, quantization, blocking, and hue shift, among others. The more severe and the more frequent the artifact, the higher the point deduction from the score. The main artifacts and corresponding point loss are listed below.

All image quality attributes are evaluated at focal lengths from approximately 12 mm to 300 mm, with particular attention paid to texture and smoothness of the zooming effect. The score is derived from a number of objective measurements in the lab and perceptual analysis of real-life video recordings.