Hearing With Your Eyes: The McGurk Effect and Why Lip-Reading Matters

September 14, 2026

In 1976, Harry McGurk and John MacDonald, psychologists at the University of Surrey, published a short paper in Nature with a strange result. They dubbed the sound of a person saying “ba” onto video of the same person saying “ga”. Most adults who watched it reported hearing “da”, a syllable present in neither the audio nor the video. Close your eyes and the sound becomes “ba” again. Open them and it changes back, even when you know exactly what is going on.

Demonstrations are easy to find online and worth trying. The effect shows something that matters a great deal to people with hearing loss: the brain does not treat speech as sound alone. It combines what it hears with what it sees, automatically and before you are aware of it.

Where the brain combines sight and sound

Brain imaging points to a region called the superior temporal sulcus, which sits between auditory and visual processing areas and responds more strongly to speech that is both seen and heard. In a 2010 study, briefly disrupting the left superior temporal sulcus with transcranial magnetic stimulation reduced how often people reported the McGurk illusion. That suggests the region does the combining instead of simply watching it happen.

The combining has a time window. The brain will fuse sound and lip movement into one event if they arrive close enough together, and it is more tolerant of sound arriving late than early, probably because light outruns sound in the real world. Broadcast engineers rely on this. The international guideline for television, ITU-R BT.1359, puts the threshold for noticing lip-sync error at about 45 milliseconds when sound leads the picture and about 125 milliseconds when sound lags.

Susceptibility varies widely. Some people report the fused “da” almost every time and others rarely do, and studies have found the effect is weaker on average in native speakers of Japanese and Mandarin than in English speakers. Strong susceptibility to the illusion does not mean someone is a good lip-reader.

How much a face is worth

In 1954, William Sumby and Irwin Pollack measured how well listeners understood words in noise with and without a view of the talker. At poor signal-to-noise ratios, seeing the face improved scores dramatically. Later researchers have expressed the benefit as equivalent to improving the signal-to-noise ratio by around 10 to 15 dB. In a noisy café, a few decibels of signal-to-noise ratio decides whether you follow a conversation or lose it, so a gain of that size is large.

It also explains why many people with hearing loss do better in daily life than their test results predict. Most speech-in-noise tests are auditory only. When the visual cues disappear, performance drops back towards what the audiogram alone would suggest, and people notice for the first time how much they had been relying on faces.

What lip-reading can and cannot tell you

Vision and hearing work well together because they carry different information. Many sounds that are hard to hear with high-frequency hearing loss are easy to see, and the reverse.

  • “P”, “b” and “m” are made with the lips closed and are highly visible. Acoustically, “m” is easy to hear while “p” and “b” are easily confused.
  • “F” and “v” are made with the top teeth on the lower lip and are easy to see. “F” is a quiet, high-frequency sound that is often lost to hearing loss.
  • “K” and “g” are made at the back of the mouth and are almost invisible.
  • Voicing, the difference between “p” and “b” or “f” and “v”, can’t be seen at all and has to be heard.

Sounds that look identical on the lips are grouped into what researchers call visemes. English has around 40 to 45 phonemes but only a dozen or so visually distinct groups, so lip-reading alone leaves a lot of ambiguity. Estimates of how much of English can be identified by sight alone vary, but most are well under half. Skilled lip-readers fill the gaps with context. For most people with hearing loss, lip-reading works best as a partner to hearing.

Masks, lighting and video calls

The pandemic made the visual channel hard to ignore. Measurements published in 2020 found that surgical masks reduced high-frequency speech by roughly 3 to 4 dB and N95 masks by up to around 12 dB, which matters on its own for someone with high-frequency loss. The bigger loss was the mouth. Many hearing aid users struggled in shops and clinics during that period while their devices were working normally. Masks with a clear window restore the visual cue but tend to muffle sound more.

Lighting has a similar effect. A person standing with a bright window behind them has their face in shadow. Video calls add compression, low frame rates and sometimes audio and video that drift out of step. Once that drift passes the brain’s tolerance, the face can become a distraction, and switching on captions may work better.

How to help someone with hearing loss follow you

Most of what helps comes down to letting the listener see you clearly.

  • Face the person when you talk, and get their attention before you start. Talking from another room or while walking away removes the visual cue entirely.
  • Keep your hands, a cup or a phone away from your mouth, and avoid talking while chewing.
  • Have the light on your face. In a restaurant, offer the person the seat with its back to the window so they look towards you in good light.
  • Speak clearly at a natural pace. Exaggerated mouthing distorts lip patterns and makes lip-reading harder.
  • If a phrase isn’t understood, rephrase it. Different words bring different sounds and lip shapes, which gives the listener a genuinely new chance.
  • Reduce background noise where you can, and choose a table away from the kitchen and the speakers.

Where hearing aids fit

Hearing aids restore audibility, and directional microphones and noise reduction help in noise. None of that replaces the information a face provides. If conversation is still exhausting with well-fitted aids, look at the visual side as well as the settings. The cocktail party problem and better hearing in restaurants cover other strategies that work alongside lip-reading.

Written by Dr Signe Steers, Doctor of Audiology and principal audiologist at The Audiology Place in Epping and Forestville. This article is general information and does not replace an individual assessment.

author avatar
Dr Signe SteersAudiologist
Welcome to my clinic. With nearly 20 years of experience, I have dedicated my career to enhancing the hearing health of individuals across all stages of life, from infants to the elderly. My passion for Speech and Hearing Science was sparked early on, driven by the understanding that improved hearing significantly enhances education, behaviour, and overall well-being. My career has taken me from presenting research at the World Health Organization to working in rural communities in the Philippines, where I helped developed systems that improved health and educational outcomes for disadvantaged populations. Last year I completed a Doctorate in Audiology at A.T. Still University in Arizona. Dr Signe Steers (Peitersen) holds a Bachelor of Speech and Hearing science from Macquarie University, Sydney, A Masters in Clinical Audiology from Macquarie University Sydney, and a Doctor of Audiology from A.T. Still University Arizona. Signe is a full member of Audiology Australia and Independent Audiologists Australia.
September 29, 2025
Hearing Loss in Australia: Age, Impact, and Global Comparisons
Hearing loss is one of the most common health challenges facing Australians today. Around one in six Australians lives with some degree of hearing loss, and […]
CallBook