Alchera Face Recognition Vendor Test 2018 Results

Alchera Inc. submitted two computer vision algorithms to the National Institute of Standards and Technology’s (NIST) Face Recognition Vendor Test (FVRT) in early 2019. The results show that Alchera face recognition technology stands out among some of the largest and brightest leaders in the AI field, particularly in areas that are highly relevant to commercial use.

The FRVT is an open and ongoing study with the goal of comparing submitted algorithms and stating their overall accuracy on several datasets. In this test, the datasets used were US visa images, mugshot images, and wild images.  NIST also uses child exploitation images in this test, however many company’s algorithms, including Alchera’s, were not used as “NIST executes those runs only infrequently.” Lastly, the primary result data given is false match rate (FMR), which allows NIST to set a threshold to target a particular false match rate appropriate to the security objectives of the application. The lower the FMR, the more difficult the test.

Alchera Face Recognition Results Overview

The graph below shows the overall ranking of Alchera face recognition algorithms over the 5 verification tests. On average, Alchera’s algorithm ranked 30/110 entrants with .0177% false match rate which includes various unicorn AI companies and global leaders. The following results detail the specific dataset types, test methods and parameters, and Alchera’s individual ranking and success rates.

Average Score Across All Tests


On average, Alchera algorithm ranked 30/110 entrants with .0177% false match rate.

Alchera Face Recognition Individual Results

Visa Images – Matched Covariates

The first test in the FRVT was for visa images with “matched-covariates” i.e. impostors of the same sex, age group, and country of birth. Alchera ranked 43/110 entrants with .0165% false match rate.

Visa Images – Matched Covariates (False Match Rate = .0001)

  • The number of images is on the order of 10^5.
  • The number of subjects is on the order of 10^5.
  • The comparisons are of visa photos against visa photos.
  • The number of genuine comparisons is on the order of 10^4.
  • The number of impostor comparisons is on the order of 10^10.
  • The number of subjects with two images on the order of 10^4.
  • The images are of subjects from greater than 100 countries, with significant imbalance due to visa issuance patterns.
  • The images are of subjects of all ages, including children, again with imbalance due to visa issuance demand.
  • Many of the images are live capture. A substantial number of the images are photographs of paper photographs.

Visa Images – False Match Rate at 1E-06

The second visa image comparison was “fully zero-effort,” meaning impostors are paired without attention to sex, age or other covariates. Additionally, NIST set a false match rate which is the proportion of impostor comparisons at or above that threshold. In this test, Alchera ranked 33/110 entrants with .0243% false match rate.

Visa Images (False Match Rate = 1E-06)

  • The number of images is on the order of 10^5.
  • The number of subjects is on the order of 10^5.
  • The comparisons are of visa photos against visa photos.
  • The number of genuine comparisons is on the order of 10^4.
  • The number of impostor comparisons is on the order of 10^10.
  • The number of subjects with two images on the order of 10^4.
  • The images are of subjects from greater than 100 countries, with significant imbalance due to visa issuance patterns.
  • The images are of subjects of all ages, including children, again with imbalance due to visa issuance demand.
  • Many of the images are live capture. A substantial number of the images are photographs of paper photographs.

Visa Images – False Match Rate at 0.0001

The third comparison was also fully zero-effort, though with an even lower FMR. In this test, Alchera ranked 50/110 entrants with just .0078% false match rate.

Visa Images (False Match Rate = 0.0001)

  • The number of images is on the order of 10^5.
  • The number of subjects is on the order of 10^5.
  • The comparisons are of visa photos against visa photos.
  • The number of genuine comparisons is on the order of 10^4.
  • The number of impostor comparisons is on the order of 10^10.
  • The number of subjects with two images on the order of 10^4.
  • The images are of subjects from greater than 100 countries, with significant imbalance due to visa issuance patterns.
  • The images are of subjects of all ages, including children, again with imbalance due to visa issuance demand.
  • Many of the images are live capture. A substantial number of the images are photographs of paper photographs.

Wild Images – False Match Rate at 0.0001

Similarly, Alchera face recognition algorithms were tested on wild images – images such as photojournalism images or others which were taken in unconstrained, widely variable conditions. This comparison was also fully zero-effort, with the impostors being paired without attention to sex, age or other covariates. In this test, Alchera ranked 29/110 entrants with .0370% false match rate.

Wild Images (False Match Rate = .0001)

  • The number of images is on the order of 10^5.
  • The number of subjects is on the order of 10^3.
  • The comparisons are of wild photos against wild photos.
  • The number of genuine comparisons is on the order of 10^6.
  • The number of impostor comparisons is on the order of 10^7.
  • The number of subjects with two images on the order of 10^3.
  • The images include many photojournalism-style images. Images are given to the algorithm using a variable but generally tight crop of the head. Resolution varies very widely. The images are very unconstrained, with wide yaw and pitch pose variation. Faces can be occluded, including hair and hands.
  • The images are of adults.
  • All of the images are live capture, none are scanned.

Mugshot Images – False Match Rate at 1E-05

Next, Alchera face recognition algorithms were tested on mugshot images taken from a dataset which includes images collected over a 17-year period, meaning that ageing has been much better characterized. As such, this test includes the effects of extended ageing, and is the “most important according” to NIST. Alchera ranked 32/110 entrants with just .0125% false match rate, specifically showing Alchera facial recognition strength for ageing, which is a proprietary technology from Alchera.

Mugshot Images (False Match Rate = 1E-05)

  • The number of images is on the order of 10^6.
  • The number of subjects is on the order of 10^6.
  • The number of subjects with two images on the order of 10^6.
  • The comparisons are of mugshot photos against mugshot photos.
  • The number of genuine comparisons is on the order of 106.
  • The number of impostor comparisons is on the order of 108.
  • The impostors are paired by sex, but not by age or other covariates.
  • The images have geometry in reasonable conformance with the ISO/IEC 19794-5 Full Frontal image type.
  • The images are of variable sizes. The median IOD is 104 pixels. The mean IOD is 123 pixels.
  • The images are of subjects from the United States.
  • The images are of adults.
  • The images are all live capture.

The full FRVT Verification Report 2019 can be found and downloaded on the NIST website for free. For more information on Alchera face recognition technology, please contact us to learn more.