Face Recognition Patterns In Cross-Race Effect

Challenges in identifying people are commonplace. In that context, Cross-Race Effect suggests that we efficiently recognize faces from races we are familiar with. This project disputes widespread belief that the phenomenon results from innate racial bias.
Tarana Sharma
Grade 12


  1. Is our cognitive ability to recognize and differentiate between faces related to intelligence?
  2. Is Cross-Race Effect purely an outcome of racial preconception?
  3. Why is in-between category differentiation better than within-category differentiation?


•Initially, the project design intended to investigate foundational research in cognitive psychology and cognitive neuropsychology exploring intuitive reliance on patterns during face recognition through simplistic behavioural experiments. Critical analysis revealed that the investigation was eye-opening but minimal in scope, which led to an inclusion of other landmark studies and popular literature in machine learning to ensure a robust body of research evidence. 

•Through extensive reading and detailed analysis, a systematic understanding was developed which elucidated the basis of Face Recognition patterns in Cross-Race Effect through processes mainly including: 


ØFace Inversion Effect

ØComposite Effect

ØWhole-Part Effect

Norm-Based Coding


1. Face Recognition ≠ Object Recognition

•Face Inversion Effect: Robert Yin demonstrated that Face Recognition is different from object recognition by asking subjects to recognize inverted faces

•His experiments showed that there was a significantly greater decrease in memory and recall when people viewed faces in inverted position compared to when they viewed non-human objects in inverted manner

•I think perhaps this happens because the brain stores a more complex representation of faces than non-human objects causing less flexibility in pattern recognition

2. Face Recognition is Holistic

•Composite Effect: Andrew Young and his team’s research proposed that Face Recognition is holistic and not decomposed into individual parts! 

•Experiments showed that people find it difficult to distinguish pictures of faces if they are made by combining halves of two separate faces unless there is some misalignment in the two halves

3. Face Recognition ≠ Decomposable

•Whole-Part Effect: Tanaka and Farah’s studies highlighted a whole-part effect indicating that any part of a face – nose/eyes/mouth is more easily recognized on the whole face than by itself as an isolated item

•Subjects did not show any advantage for part identification in whole object recognition for scrambled faces, inverted faces and houses

4. Norm-Based Face Space Coding

•Face-Space Coding of Face Identity: Face recognition is based on a norm-based coding system in multi-dimensional face spaces

ØEach individual face is coded as a point in a multi-dimensional perceptual space that has dimensions corresponding to attributes that vary across faces and that has average face at the center. 

ØFace after-effects are commonly interpreted as arising from a shift in the location of this average which is caused by persistent varying exposure to external stimuli

ØThis framework is analogous to a lookup table or a data map which can be updated based on neural perception

5. Memory & Recall

•Machine Intelligence expert, Jeff Hawkins theorizes that the neocortex:

ØStores sequences of patterns

ØRecalls patterns auto associatively

ØStores patterns in an invariant form

ØStores patterns in a hierarchy

•The maps are conditioned based on norms or averages created in response to external stimuli the brain is exposed to over time

•This was experimentally validated by Mckone et al. using the distance between the eyes and the mouth of faces as a facial attribute used to recognize faces



•I think that these concepts imply that our brains memorize faces like a map where encoding involves the relative positioning of facial features combined with contextual or background information

•Essentially, lessons from these foundational experiments and studies in cognitive neuropsychology, neuroscience and other subsequent applied research indicate three aspects about our ability to recognize and differentiate between faces: 

ØFace Recognition is different from object recognition

ØFace Recognition takes into account holistic face representation

ØFace Recognition uses norm-based coding systems




•Findings of this study have direct implications for Face Recognition technologies.

•In fact, high level results from NISTIR 8280 - a landmark US Federal study released in December of 2019 showed that Facial Recognition systems misidentified people of color more often than white people. In a press release, Patrick Grother – primary author of the report was quoted “While it is usually incorrect to make statements across algorithms, we found empirical evidence for the existence of demographic differentials in the majority of the face recognition algorithms, we studied.” The shocking finding demands intensified focus on mitigation of inherent racial bias in surveillance tools.

• Systems can be improved by training algorithms on a larger and more diverse set of faces and by using additional facial parameters which are independent of skin color to compensate for the limitations of the human brain. Minimizing and eventually eliminating systemic bias in surveillance and security systems is crucial for equitable law enforcement.

•Comprehension of neural mechanisms for facial recognition can enable an alternate pathway to induce or enhance vision in visually impaired people

•The fascinating findings from this research review demonstrate that individual differences can be investigated to understand aspects of cognitive neuroscience because we can learn about the cognitive ability of face recognition using simple measurements of behavioral responses based on accuracy and time

•Interesting and pertinent questions identified for further exploration include:

ØWhat is the role of experience in Face Recognition?

ØAre there any age-effects that influence Face Recognition?

ØWhat are the neural responses of challenges on Face Recognition tasks?

•Although the research review methodology of this project limited the scope of exploration, the distinct findings are encouraging for experimental studies using sizeable sample sets and multiple trials






•The human brain uses patterns to memorize and identify faces

•Facial recognition differentiates people by spontaneously yet systematically capturing, analyzing, and comparing patterns based on facial details

ØThe face detection process detects and locates human faces

ØThe face capture process transforms analog information (a face) into a set of digital information (data) detailing the person's facial features

ØThe face analysis process analyses the recorded facial geometry

ØThe face match process verifies if two faces belong to the same person

•Neural conditioning uses a range of external stimuli for a flexible and non-linear face recognition mechanism spanning multi-dimensional face spaces. People have difficulty recognizing faces accurately when presented with facial features outside the pre-conditioned range, which forms the basis for Cross-Race Effect


Objectives Answered

•Face Recognition is not an exclusive outcome of intelligence. 

•Evidence of norm-based coding of faces contradicts innate racial bias as the reason underlying an inability to differentiate between faces 

•In-between category differentiation tends to be better than within-category differentiation because of limited range of exposure or what I’d like to call unfamiliarity quotient


Hypothesis Supported

Findings clearly supported the hypothesis through evidence that our intuitive ability to encode and decode faces draws on systematic pattern recognition


•Deen, B., Richardson, H., Dilks, D. et al. (2017). Organization of high-level visual cortex in human infants. Nat Commun 8, 13995.

•Deen, B., Saxe, R & Kanwisher, N. (2020) Processing communicative facial and vocal cues in the superior temporal sulcus. NeuroImage, Volume 221, 117191,

•Kanwisher, Nancy. 2020. Nancysbraintalks. MIT.

•Lebrecht S, Pierce LJ, Tarr MJ, Tanaka JW (2009) Perceptual Other-Race Training Reduces Implicit Racial Bias. PLoS ONE 4(1): e4215.

•McKone E, Crookes K, Jeffery L & Dilks D D. (2012) A critical review of the development of face recognition: Experience is less important than previously believed, Cognitive Neuropsychology, 29:1-2, 174-212, DOI: 10.1080/02643294.2012.660138

•Tanaka, J. W., & Farah, M. J. (1993). Parts and wholes in face recognition. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 46A(2), 225–245.

•Taylor, John. (2005). On Intelligence, Jeff Hawkins, Sandra Blakeslee Times Books (2004). Artificial Intelligence. 169. 192-195. 10.1016/j.artint.2005.10.011

•Yin, R. K. (1969). Looking at upside-down faces. Journal of Experimental Psychology, 81(1), 141–145.

•Young AW, Hellawell D, Hay DC. Configurational information in face perception. Perception. 1987;16(6):747-59. doi: 10.1068/p160747. PMID: 3454432

Intelligence. 169. 192-195. 10.1016/j.artint.2005.10.011

Yin, R. K. (1969). Looking at upside-down faces. Journal of Experimental Psychology, 81(1),

Young AW, Hellawell D, Hay DC. Configurational information in face perception. Perception.
1987;16(6):747-59. doi: 10.1068/p160747. PMID: 3454432


I would like to acknowledge my teachers, Ms. Sharissa Dyke, Ms. Allison Pinnock and Mr. Monaghan for the interesting discussions that shaped the completion of my project. I would also like to acknowledge my school science fair coordinators Ms. Lai, Ms. Cruickshank and Mr. Ferg for their support. Finally, I would like to thank my friends and family for their encouragement every step of the way.