When Your Eyes Hear Better Than Your Ears: The McGurk Effect

Lexicon Valley
A Blog About Language
June 27 2014 1:47 PM

When Your Eyes Hear Better Than Your Ears: The McGurk Effect

screen_shot_20140627_at_3.02.19_am
How does what you see affect what you hear?

PBS

The clip below from PBS demonstrates the McGurk Effect: When you hear a sound (like "ba") that conflicts with how you're seeing someone "produce" it (like "ga"), your mind tries to reconcile them by making you think you're hearing something more consistent with your visual input (like "da").

If you want to prove to yourself that both versions in the video above are the same sound, try rewatching the first part of the video with your eyes closed. Sure enough, you'll hear "ba ba ba" instead of "da da da."

Advertisement

For another example of the McGurk Effect, check out this clip from the BBC, which shows how "ba ba ba" can sound like "fa fa fa" when accompanied by the appropriate visuals, and includes a brief interview with the psychologist Lawrence Rosenblum, who's been studying the McGurk Effect for decades.

The McGurk Effect is surprisingly robust: According to this list from Rosenblum, it persists with babies as young as 4-5 months, with speakers of every language that's been tested, when the audio and video are from people of different genders, when viewers don't realize they're looking at a face, when viewers touch a face instead of looking at it, and when the audio and video aren't precisely synched. It does work better with certain consonant pairs than others, and less well with vowels or non-speech sounds, such as plucking versus bowing sounds on a cello. But it even happens when the viewer knows perfectly well to be expecting the McGurk Effect, such as Rosenblum himself!

Gretchen McCulloch is a linguist and the editor of Slate's Lexicon Valley blog. She has a master's in linguistics from McGill University and blogs daily at All Things Linguistic.

TODAY IN SLATE

Culturebox

The Ebola Story

How our minds build narratives out of disaster.

The Budget Disaster That Completely Sabotaged the WHO’s Response to Ebola

PowerPoint Is the Worst, and Now It’s the Latest Way to Hack Into Your Computer

The Shooting Tragedies That Forged Canada’s Gun Politics

A Highly Unscientific Ranking of Crazy-Old German Beers

Education

Welcome to 13th Grade!

Some high schools are offering a fifth year. That’s a great idea.

Culturebox

The Actual World

“Mount Thoreau” and the naming of things in the wilderness.

Want Kids to Delay Sex? Let Planned Parenthood Teach Them Sex Ed.

Would You Trust Walmart to Provide Your Health Care? (You Should.)

  News & Politics
The World
Oct. 22 2014 2:05 PM Paul Farmer Says Up to Ninety Percent of Those Infected Should Survive Ebola. Is He Right?
  Business
Business Insider
Oct. 22 2014 2:27 PM Facebook Made $595 Million in the U.K. Last Year. It Paid $0 in Taxes
  Life
The Eye
Oct. 22 2014 1:01 PM The Surprisingly Xenophobic Origins of Wonder Bread
  Double X
The XX Factor
Oct. 22 2014 10:00 AM On the Internet, Men Are Called Names. Women Are Stalked and Sexually Harassed.
  Slate Plus
Tv Club
Oct. 22 2014 5:27 PM The Slate Walking Dead Podcast A spoiler-filled discussion of Episodes 1 and 2.
  Arts
Brow Beat
Oct. 22 2014 10:39 PM Avengers: Age of Ultron Looks Like a Fun, Sprawling, and Extremely Satisfying Sequel
  Technology
Future Tense
Oct. 22 2014 2:59 PM Netizen Report: Twitter Users Under Fire in Mexico, Venezuela, Turkey
  Health & Science
Wild Things
Oct. 22 2014 2:42 PM Orcas, Via Drone, for the First Time Ever
  Sports
Sports Nut
Oct. 20 2014 5:09 PM Keepaway, on Three. Ready—Break! On his record-breaking touchdown pass, Peyton Manning couldn’t even leave the celebration to chance.