How Does Facial Recognition Work?

Frame the puzzle: faces vary in lighting, angle, expression, and age — yet phones unlock in under a second. How is that even possible?

• Faces are not stable like fingerprints
• Speed and accuracy seem contradictory
• This sets up what we need to explain

Before any explanation, the learner picks what they think facial recognition actually does behind the scenes.

• Make one independent hypothesis
• Reveal whether you guessed pixel matching or something deeper

Try to 'recognize' faces two different ways and feel the difference: raw pixel comparison versus a similarity score in feature space.

• Compare the same face under two lighting conditions
• See that pixel difference is large but feature similarity stays high
• Notice when a different person scores close under pixel comparison

Before recognition can happen, the system has to find the face in the image. Explain bounding boxes and landmark localization.

• A detector scans the image for face-like regions
• Landmarks mark eyes, nose, mouth, jaw
• The face is cropped and aligned to a canonical pose

A manipulable diagram showing how a deep network turns the aligned face into a vector of numbers that encodes its geometry and texture.

• Each layer of the network extracts increasingly abstract features
• The final vector is the embedding — a compact numerical fingerprint
• Adjust input conditions to see which embedding values change and which stay stable

Show how two embeddings are compared: distance (or cosine similarity) decides whether the face matches a stored identity.

• Embeddings of the same person cluster together
• Embeddings of different people sit far apart
• A threshold turns distance into a yes/no decision

A visualization that pushes the system to its limits: extreme angles, masks, twins, and aging — see which conditions collapse the match.

• Heavy occlusion degrades the embedding
• Identical twins produce near-identical embeddings
• Lighting extremes shift the feature values noticeably

The same recipe — detect, embed, compare — powers voice ID, music recommendation, and anomaly detection. The principle generalizes.

• Voice assistants use audio embeddings
• Recommenders use preference embeddings
• Anomaly detection uses behavioral embeddings

Tie everything back: facial recognition is not pixel matching — it is geometry and texture compressed into a vector and compared by distance.

• Detect the face
• Extract a stable embedding
• Match by similarity in embedding space
• Threshold converts similarity into an identity decision