When Your Eyes Hear Better Than Your Ears: The McGurk Effect

A Blog About Language
June 27 2014 1:47 PM

When Your Eyes Hear Better Than Your Ears: The McGurk Effect

screen_shot_20140627_at_3.02.19_am
How does what you see affect what you hear?

PBS

The clip below from PBS demonstrates the McGurk Effect: When you hear a sound (like "ba") that conflicts with how you're seeing someone "produce" it (like "ga"), your mind tries to reconcile them by making you think you're hearing something more consistent with your visual input (like "da").

If you want to prove to yourself that both versions in the video above are the same sound, try rewatching the first part of the video with your eyes closed. Sure enough, you'll hear "ba ba ba" instead of "da da da."

Advertisement

For another example of the McGurk Effect, check out this clip from the BBC, which shows how "ba ba ba" can sound like "fa fa fa" when accompanied by the appropriate visuals, and includes a brief interview with the psychologist Lawrence Rosenblum, who's been studying the McGurk Effect for decades.

The McGurk Effect is surprisingly robust: According to this list from Rosenblum, it persists with babies as young as 4-5 months, with speakers of every language that's been tested, when the audio and video are from people of different genders, when viewers don't realize they're looking at a face, when viewers touch a face instead of looking at it, and when the audio and video aren't precisely synched. It does work better with certain consonant pairs than others, and less well with vowels or non-speech sounds, such as plucking versus bowing sounds on a cello. But it even happens when the viewer knows perfectly well to be expecting the McGurk Effect, such as Rosenblum himself!

Gretchen McCulloch is a linguist and the contributing editor of Slate's Lexicon Valley blog. She has a master's in linguistics from McGill University and blogs daily at All Things Linguistic.

TODAY IN SLATE

Politics

Don’t Worry, Obama Isn’t Sending U.S. Troops to Fight ISIS

But the next president might. 

IOS 8 Comes Out Today. Do Not Put It on Your iPhone 4S.

Why Greenland’s “Dark Snow” Should Worry You

How Much Should You Loathe NFL Commissioner Roger Goodell?

Here are the facts.

Three Talented Actresses in Three Terrible New Shows

Science

The Human Need to Find Connections in Everything

It’s the source of creativity and delusions. It can harm us more than it helps us.

Foreigners

More Than Scottish Pride

Scotland’s referendum isn’t about nationalism. It’s about a system that failed, and a new generation looking to take a chance on itself. 

The Ungodly Horror of Having a Bug Crawl Into Your Ear and Scratch Away at Your Eardrum

We Could Fix Climate Change for Free. What Exactly Is Holding Us Back?

  News & Politics
Weigel
Sept. 17 2014 6:09 PM Here's Who Voted to Fund Syrian Rebels and Get This ISIS Conflict Really Cooking
  Business
Business Insider
Sept. 17 2014 1:36 PM Nate Silver Versus Princeton Professor: Who Has the Right Models?
  Life
Gentleman Scholar
Sept. 17 2014 5:10 PM Should Men Still Open Doors for Women? Or is it ungentlemanly to do so at all?  
  Double X
The XX Factor
Sept. 17 2014 6:14 PM Today in Gender Gaps: Biking
  Slate Plus
Slate Fare
Sept. 17 2014 9:37 AM Is Slate Too Liberal?  A members-only open thread.
  Arts
Brow Beat
Sept. 17 2014 5:56 PM Watch Louis C.K., Dave Chappelle, Bill Hicks, Mitch Hedberg, and More on New YouTube Channel
  Technology
Future Tense
Sept. 17 2014 5:26 PM If Fixing Global Warming Is Free, What’s the Holdup?
  Health & Science
Jurisprudence
Sept. 17 2014 4:49 PM Schooling the Supreme Court on Rap Music Is it art or a true threat of violence?
  Sports
Sports Nut
Sept. 17 2014 3:51 PM NFL Jerk Watch: Roger Goodell How much should you loathe the pro football commissioner?