Introducing the DXOMARK Selfie test protocol

Reading Time: 5 min read

The Oxford English Dictionary defines a selfie as “a photograph that one has taken of oneself, typically one taken with a smartphone or webcam and shared via social media.” The fact that the best-known authoritative dictionary of the English language validated the term in 2013 shows just how quickly this form of photography has become very popular.

It will be obvious to anyone scrolling through the feeds of their social media apps that a huge proportion of photos these days are self-portraits of some kind that have been captured with the front camera of a smartphone. It is hard to deny that social media has fundamentally changed the way we use cameras and photography. On platforms such as Facebook, Instagram, Snapchat, and WhatsApp, sharing selfies is extremely popular, and many of those apps select the front camera by default. It is obvious that the rise of the selfie is strongly linked to the rapid growth of these social media applications and platforms over recent years.

Users capture and share group selfies with their friends at events; selfies that show cityscapes or landscapes in the background while traveling; and closeup selfies simply to document their current mood. Video calls with applications such as Skype or FaceTime are another important use case for smartphone front cameras.

Group selfie
Group selfie video

The phenomenon is truly global, encompassing all cultures and countries; but despite the significant increase in front camera use, there tends to be a lot more talk about the specifications and capabilities of the main camera than the front camera when new smartphone models are launched. Even so, in line with changing user behavior and increased front cam use, front camera performance has increased at a fast pace in the last few years.

Given their tiny sensors and fixed-focus systems, earlier front cameras used to capture blurry and grainy images, even in bright light, with downright unusable output when shooting in dimmer conditions. That’s a thing of the past, though: a number of advanced imaging technologies, such as High Dynamic Range (HDR) modes, phase-detection autofocus (PDAF), and time-of-flight (TOF) autofocus have migrated from the main camera to the front units, greatly improving both performance and image quality. These improvements to the front camera are joined by the increased processing power of mobile chipsets, which have expanded such processing-intensive tasks as multi-frame processing, object and scene identification, live beautification, and 4K video.

Modern high-end smartphones, such as the Huawei P20 Pro (left) and the Apple iPhone X (right),
come with sophisticated imaging technologies in the front camera.

This said, not all manufacturers deploy these latest front camera technologies, and there is a lot of variation between devices in terms of front camera image quality, image processing, and performance, as you can see in the samples below. All three images capture the same scene, but there are very visible differences in color response, in skin and detail rendition, and in dynamic range as well. (Note that some front-facing cameras take mirror images in default mode, which is also apparent in the sample photos below.)

Meitu V6
Apple iPhone X
Huawei P20 Pro

All of these factors are why we are introducing the DXOMARK Selfie protocol—to provide comprehensive, neutral, and reliable test data about smartphone front camera performance and image quality to consumers and other interested parties. Read on to find out how we use our newly-developed DXOMARK front camera protocol to test and score smartphone front cameras in a reliable and repeatable way.

Testing and scores

Front camera testing and scoring follows very similar methodologies to those in our DXOMARK Mobile test protocol for smartphone main cameras. The tests take place in both lab and real-world situations, using a wide variety of subjects and light conditions. Scores are generated both from purely objective tests for which the results are calculated directly by the test equipment, and from perceptual tests based on a sophisticated set of metrics that allow a panel of image experts to compare various aspects of image quality that require human judgment.

The testing process includes shooting approximately 1,500 images and dozens of clips totaling more than two hours of video. Every device is tested in exactly the same way, in identically-configured lab setups, using the same procedures, the same scenes, and the same software and evaluation methodologies. This means that the new front camera protocol is just as reliable and repeatable as our tried and trusted DXOMARK Mobile protocol.

Front camera testing in the street.
Front camera testing in the DxOMark imaging lab.

While the new front camera protocol and DXOMARK Mobile are based on the same procedures and methodologies, we have made some important modifications to the former in order to reflect the way front cameras are used. Front cameras are all about self-portraits or “selfies,” which means that by definition at least one human subject—the photographer herself or himself—appears in the image. Therefore a good front camera should be optimized for portraiture at relatively short shooting distances, between 30cm for a close-up portrait (one in which the subject’s face takes up the largest part of the frame), and 120cm for group portraits or images that include an important element in the background and are typically captured with the help of a selfie-stick.

Close-up portrait
Portrait
Portrait with selfie-stick

To cover as many use cases as possible, we test at three subject distances: 30, 55 and 120cm. It is a real challenge for front cameras to produce good image quality at all three of these subject distances; so if you capture most of your selfies at the same or similar subject distances, our test results will help you pick the best smartphone for your particular style of selfie shooting.

We also make sure our test results are valid across all regions and cultures. DXOMARK tests the default modes of international firmware versions that are designed for ethnically diverse markets, and we evaluate portrait image quality for a range of skin tones.

To evaluate image quality for these specific portrait use cases, we have developed a range of new dedicated portrait test equipment:

  • A Bokeh chart to evaluate portraits shot using a bokeh effect, specifically examining depth estimation and bokeh shape
  • A Close-up Dead Leaves chart for noise and detail analysis at close distance (45cm)
  • A realistic mannequin to test for repeatability and accuracy at a distance of 45cm
  • An HDR portrait setup to facilitate objective measurements of backlit portrait scenes

Just like the DXOMARK testing for main cameras, DXOMARK selfie testing is a very comprehensive and thorough process. Testing a smartphone’s front camera involves a team of several people for approximately a week. Once we have run all the tests and analyzed the results, we score the Photo and Video quality separately, and then combine them into an Overall score for quick comparison between different models.

As with DxOMark Mobile main camera scoring, a number of sub-scores feed into Selfie Photo and Video scores, which are then combined into an overall Selfie score for each device.

The new DXOMARK front camera test protocol was designed to help consumers make informed buying decisions by thoroughly evaluating the front camera performance of the latest mobile devices (sometimes challenging OEM marketing claims).

We hope you found this brief overview of our new protocol useful and that it has given you some understanding of how we test and score smartphone front cameras. For more details about testing and scoring of front cameras, please read the following articles:

Click through to see our DXOMARK Selfie front camera reviews and test results.

DXOMARK invites our readership (you) to post comments on the articles on this website. Read more about our Comment Policy.