A recent study published in Brain Research provides evidence that people recognize facial expressions faster and more accurately when viewing the entire face rather than just the eyes. The research suggests that while the eyes hold important social cues, the brain relies heavily on the rest of the face to process emotions efficiently.
The eyes tend to be a primary focus when people read the emotions of others. Furrowed eyebrows, wide-open eyelids, or small creases at the corners of the eyes communicate rich emotional information.
When people look at faces, they typically use configural processing, which means taking in the whole face at once and understanding how all the features relate to one another. When visual information is limited, the brain is forced to rely on featural processing, analyzing individual parts like the eyes or nose separately.
Katherine A. Billetdeaux, a doctoral student at Penn State University, began this research as an undergraduate at the College of Wooster. The idea came from a period when most of the people she talked to face-to-face were wearing face masks. "I noticed how much harder it was to read certain facial expressions when the lower half of someone's face was covered," Billetdeaux said. "That got me thinking about what facial information we actually rely on to recognize emotions."
To explore this question, the researchers asked 40 undergraduate students to view 480 photographs of faces. The photos showed 30 models expressing four emotions: anger, fear, happiness, and sadness.
The scientists manipulated the images to create four viewing conditions. The first showed the full, unaltered face. In the second, the face stayed whole but the eyes were blurred with a semi-transparent white rectangle. The third showed only the intact eyes, with the rest of the face hidden behind a solid white mask. The fourth showed only the eyes, and those eyes were blurred as well.
Participants identified each emotion as quickly as possible by pressing keyboard buttons. Meanwhile, the researchers recorded their brain activity using an electroencephalogram, which picks up tiny electrical signals through sensors on the scalp. They looked at an early brain wave, which appears roughly 170 milliseconds after a person sees a face and reflects the brain's initial structural processing of the image. They also looked at later brain waves linked to higher-level processes, such as paying close attention and evaluating emotional meaning.
The behavioral results showed that participants were most accurate and fastest when they could see the entire face. Hiding the rest of the face led to more mistakes and slower answers. Similarly, blurring the eyes led to slower and less accurate responses.
The brain waves supported these observations. When participants viewed a full face, the early brain wave was smaller and occurred faster, suggesting that the brain exerts less effort when all facial features are present together. When participants saw only the eyes, this wave grew larger and slower, a sign that the brain works harder when the overall layout is missing.
"The most surprising finding was that diminishing the eye region affected different stages of brain processing depending on whether the rest of the face was visible," Billetdeaux said. "In other words, the brain doesn't process emotional information from the eyes in isolation."
Blurring the eyes disrupted the early processing step only when the rest of the face was hidden. The later brain waves, by contrast, were larger when the eyes were blurred but the rest of the face was visible. This suggests that once the brain completes its initial structural scan, it looks to the details of the eyes to fully evaluate the emotion.
Different emotions also relied on different facial features. Anger was recognized easily whether the whole face was shown or just the eyes, which suggests that angry expressions are concentrated in the eye region. Sadness was also processed relatively easily, suggesting it too is strongly communicated through the eyes.
Fear was the hardest to recognize when the rest of the face was hidden. When participants viewed only the eyes of a fearful face, their brain activity spiked, indicating a high level of mental effort. A wide-open mouth or other lower facial features might be needed to easily recognize fear. Happiness, meanwhile, was identified faster and more accurately with the whole face in view, since the smile seems to give the strongest signal.
"The key takeaway is that both the eye region and the surrounding face are important for how we process facial expressions," Billetdeaux said. "The picture is more nuanced than a simple 'eyes matter most' conclusion."