How does facial recognition actually work?
By converting a face into a numerical vector and comparing distances between vectors — not by matching pictures, which is the misconception underlying most public discussion of it.
The pipeline:
Detection. Find the faces in an image — a separate and much older problem from identifying them.
Alignment. Locate landmarks (eyes, nose, mouth corners) and geometrically normalise the face to a standard orientation and scale.
Embedding. A neural network converts the normalised face into a vector of a few hundred numbers — the faceprint. The network is trained so that images of the same person produce nearby vectors and different people produce distant ones, regardless of lighting, expression, angle or age.
Comparison. Measure the distance between vectors and apply a threshold.
The two very different tasks, routinely conflated:
Verification (1:1) — is this the person whose face is on file? Unlocking a phone or passing a passport gate. Comparatively easy and highly accurate.
Identification (1:N) — who is this, out of a database of millions? Far harder, and the error mathematics are unforgiving: even a tiny false-match rate produces many false positives when searched against a large population, which is the heart of the civil liberties argument.
Where the threshold sits is a policy decision, not a technical one. Lower it and you catch more true matches and more false ones; raise it and you miss people. There is no neutral setting.
Accuracy and bias. Performance has improved enormously, and evaluations have repeatedly found higher error rates for some demographic groups, driven by training data composition, image quality and threshold effects. The gap has narrowed in leading systems and has not vanished.
Why phone unlocking is different. Systems projecting infrared dot patterns build a depth map, which resists photographs and screens and is far more robust than a camera-only system.
Regulation is moving fast, particularly on live use in public spaces and on scraping images to build databases — several enforcement actions have concerned exactly that.