How AI face analysis works: a glossary for people who want the honest version

You have just been handed a number by an app. You are staring at dots laid over your own face, wondering whether they explain the number or merely make it look scientific.

The dots are real computer vision. The number is different: a prediction about how some set of human ratings might respond to your image. This glossary explains the machinery, then names the point where measurement ends and taste begins.

Key numbers

• Algorithms tested: 189 algorithms from 99 developers were tested in NIST's face-recognition evaluation. Read the NIST report.
• Images tested: 18.27 million images from operational databases were used. Read the NIST report.
• People represented: Those images represented 8.49 million people. Read the NIST report.
• False-positive differential: One-to-one matching often showed differentials from 10 to 100 times across demographic comparisons. Read the NIST report.
• Equity and accuracy: The most equitable algorithms were among the most accurate overall. Read the NIST report.

Table of contents

• How a computer sees a face
• How a model gets an opinion
• What the camera did before the model saw anything
• Words the apps use that are not computer vision terms
• What can the model verify, and what is it asked to predict?
• What does Real World Appeal admit about its own model?
• The bottom line
• Sources

How a computer sees a face

A computer first locates a face, then builds a stable representation of what it sees. These operations can be checked against images and labels; they describe pixels and geometry, not your social future.

Face detection and bounding box. Face detection asks whether a face is present. A bounding box draws the rectangle around it. Both are image-location tasks, so an evaluator can compare output with an image and identify a miss.

Facial landmark detection and 68-point landmark model. Landmark detection places reference points on features. A 68-point model is one conventional arrangement around the eyes, brows, nose, mouth, and jaw, useful for alignment but not a theory of beauty.

Dense landmark model. A dense model uses many more reference points across the face than a sparse outline. It can preserve finer shape information around cheeks, eyelids, and jaw contours, while still saying nothing about preference.

Alignment / normalisation. Alignment rotates, scales, and crops faces into a comparable format. Normalisation reduces irrelevant variation before analysis. It improves consistency, but a cleaner input does not turn a subjective target into an objective fact.

Embedding (face vector). An embedding is a numerical representation of visual patterns, often called a face vector. Similar images may sit near each other in that representation. Nearness reflects training choices, not an intrinsic location on a human appeal scale.

Convolutional neural network. A convolutional neural network learns visual patterns through layered filters responding to edges, textures, and larger structures. It can learn useful signals without explicit rules, but its preferences remain tied to its training data.

Vision transformer. A vision transformer uses attention among image patches to capture broad relationships. Architecture changes how patterns are learned; it does not supply a missing definition of appeal or make the target more objective.

Vision-language model. A vision-language model connects images with text concepts learned from paired data. It can describe appearance fluently. Fluency is not validation, and a polished explanation can exceed what the image actually supports.

The caveat: a model can be excellent at locating and representing faces while being poorly suited to predict a lived first impression.

How a model gets an opinion

A model's “opinion” comes from examples, labels, and an optimization rule. The crucial question is who supplied the target, how disagreement was handled, and whether the output was checked outside training data.

Training set, label, and ground truth. A training set contains examples; a label names the target attached to each. Ground truth is the reference answer used for evaluation. For attractiveness, there is no ground truth in that sense, only ratings from a group of people.

Inter-rater agreement. Inter-rater agreement measures how similarly raters judge the same material. Langlois and colleagues' review contains eleven meta-analyses and found strong agreement within and across cultures, while raters judged the whole face rather than isolated features.

Regression versus classification. Regression predicts a continuous value; classification assigns a category. An app can use either. The format changes, but the deeper issue remains: the target reflects labels selected by people.

Loss function. A loss function tells training how wrong a prediction is and which errors deserve more penalty. Minimizing loss can match labels closely, but it cannot repair labels that are narrow, noisy, or culturally limited.

Confidence score and calibration. Confidence expresses how strongly a model favors an output. Calibration asks whether high confidence corresponds to reliable performance. A confident attractiveness prediction may simply recognize a familiar rating pattern, not measure beauty.

Overfitting and hallucination. Overfitting learns quirks of training examples instead of general patterns. Hallucination presents unsupported detail as input-based. A responsible report separates visible observations from interpretation and marks uncertainty rather than hiding it.

The caveat: agreement is meaningful evidence that people often perceive patterns similarly, but it does not create a universal ruler or prove that every observer responds alike.

What the camera did before the model saw anything

The model never receives “you in real life.” It receives one camera's projection under particular conditions. Inspect the capture first: perspective, distance, light, color, and compression can change the evidence.

Focal length and perspective distortion. Focal length affects how much scene fits the frame. Perspective distortion comes mainly from camera-to-subject distance, changing apparent relationships among near and far features. A close phone portrait can therefore exaggerate shape.

Lens compression and subject distance. “Lens compression” describes a flatter-looking perspective created when the camera is farther away and a longer focal length frames the subject. Distance is the key variable; focal length helps preserve that distance while framing.

Lighting direction and white balance. Lighting direction changes shadows, planes, and apparent texture. White balance shifts the color cast of skin, hair, and clothing. These are capture conditions, not permanent traits, yet a model may treat them as evidence.

Compression artefacts. Compression artefacts are blockiness, ringing, smearing, or lost detail created when an image is reduced or re-encoded. They blur edges and texture, making the model work from a damaged version of the face.

The caveat: better capture does not guarantee a favorable result. It gives the model a fairer representation, which is different from making the prediction true.

Words the apps use that are not computer vision terms

These terms are measurements or aesthetic conventions layered on top of landmark output. Some are real clinical measurements used in surgery. None is evidence that an app's number predicts anything about your life, dating outcome, or worth.

Facial thirds and midface ratio. Facial thirds divide the face into vertical regions; midface ratio compares distances around the central face. They provide consistent vocabulary, but a proportion is descriptive, not automatically a universal preference.

Canthal tilt and gonial angle. Canthal tilt describes the angle between inner and outer eye corners. Gonial angle describes an angle around the jaw. Both can be measured from landmarks, but measurement cannot predict how an observer reads the whole face.

Symmetry index, golden ratio, and phi mask. A symmetry index compares corresponding sides under a chosen method. Golden ratio and phi masks are aesthetic conventions overlaid on measurements. They can look authoritative without proving that any output predicts personal appeal.

Harmony score. Harmony score compresses several proportion judgments into one label. It may be shorthand for an aesthetic system, but it is not a computer-vision primitive and should not be mistaken for a verified fact about a person.

The caveat: clinical vocabulary can help describe anatomy and plan treatment. That legitimate use does not transfer automatically to predicting attraction in ordinary social life.

What can the model verify, and what is it asked to predict?

The clean distinction is simple: detection and identity tasks have external answers that can be checked. Attractiveness analysis predicts human response, so its truth is a pattern in ratings, not a hidden property waiting to be uncovered.

This is why NIST's identity work matters even though it was not an attractiveness study. Identity matching is easier to test, better funded, and more tightly defined. Yet NIST found false-positive differentials often ranging from 10 to 100 times across demographic comparisons in one-to-one matching. A taste-prediction model with no public audit deserves less trust, not more. Read the NIST findings.

What does Real World Appeal admit about its own model?

Real World Appeal is a perception model too, and it inherits these limits. We report on a 70-155 perception axis, describe it as a threshold model rather than a ladder, and do not claim to measure objective beauty, because there is nothing there to measure.

That disclosure is the reason to trust the rest of the page. Our output is a structured estimate of first-impression perception, useful for noticing presentation levers and blind spots, not a verdict about your face.

We keep the first result visible before you decide anything: there is no paywall on your result. You can begin with our free first-impression test. The caveat is simple: transparency makes a product easier to judge, not the underlying human judgment objective.

If an app gave you a low number and you have been carrying it around, put it down: it is a prediction of a rating pattern, not a verdict about your future. Use it, if at all, to examine photos, styling, and context. Read first impressions in real life for a healthier frame.

The bottom line

AI face analysis is a pipeline, not an oracle. Landmarks, embeddings, camera corrections, and neural networks can produce a consistent description; they cannot manufacture ground truth for attractiveness. “The model says X” honestly means “a model trained on some raters' opinions predicts those raters might have said X.”

That can be useful when limits are explicit and advice points toward changeable presentation choices. It becomes harmful when diagrams disguise taste as certainty. For the difference between an app result and a lived encounter, read AI face rating versus real life, then decide whether a perception test would help.

Sources

• National Institute of Standards and Technology. (2019, December 19). “NIST Study Evaluates Effects of Race, Age, Sex on Face Recognition Software.” Reporting NISTIR 8280, Face Recognition Vendor Test, Part 3: Demographic Effects. https://www.nist.gov/news-events/news/2019/12/nist-study-evaluates-effects-race-age-sex-face-recognition-software
• Langlois, J. H., Kalakanis, L., Rubenstein, A. J., Larson, A., Hallam, M., & Smoot, M. (2000). “Maxims or myths of beauty? A meta-analytic and theoretical review.” Psychological Bulletin, 126(3), 390-423. PMID 10825783.
• Willis, J., & Todorov, A. (2006). “First impressions: Making up your mind after a 100-ms exposure to a face.” Psychological Science, 17(7), 592-598.