In 1976, Harry McGurk and John MacDonald, psychologists at the University of Surrey, published a short paper in Nature with a strange result. They dubbed the sound of a person saying “ba” onto video of the same person saying “ga”. Most adults who watched it reported hearing “da”, a syllable present in neither the audio nor the video. Close your eyes and the sound becomes “ba” again. Open them and it changes back, even when you know exactly what is going on.
Demonstrations are easy to find online and worth trying. The effect shows something that matters a great deal to people with hearing loss: the brain does not treat speech as sound alone. It combines what it hears with what it sees, automatically and before you are aware of it.
Where the brain combines sight and sound
Brain imaging points to a region called the superior temporal sulcus, which sits between auditory and visual processing areas and responds more strongly to speech that is both seen and heard. In a 2010 study, briefly disrupting the left superior temporal sulcus with transcranial magnetic stimulation reduced how often people reported the McGurk illusion. That suggests the region does the combining instead of simply watching it happen.
The combining has a time window. The brain will fuse sound and lip movement into one event if they arrive close enough together, and it is more tolerant of sound arriving late than early, probably because light outruns sound in the real world. Broadcast engineers rely on this. The international guideline for television, ITU-R BT.1359, puts the threshold for noticing lip-sync error at about 45 milliseconds when sound leads the picture and about 125 milliseconds when sound lags.
Susceptibility varies widely. Some people report the fused “da” almost every time and others rarely do, and studies have found the effect is weaker on average in native speakers of Japanese and Mandarin than in English speakers. Strong susceptibility to the illusion does not mean someone is a good lip-reader.
How much a face is worth
In 1954, William Sumby and Irwin Pollack measured how well listeners understood words in noise with and without a view of the talker. At poor signal-to-noise ratios, seeing the face improved scores dramatically. Later researchers have expressed the benefit as equivalent to improving the signal-to-noise ratio by around 10 to 15 dB. In a noisy café, a few decibels of signal-to-noise ratio decides whether you follow a conversation or lose it, so a gain of that size is large.
It also explains why many people with hearing loss do better in daily life than their test results predict. Most speech-in-noise tests are auditory only. When the visual cues disappear, performance drops back towards what the audiogram alone would suggest, and people notice for the first time how much they had been relying on faces.
What lip-reading can and cannot tell you
Vision and hearing work well together because they carry different information. Many sounds that are hard to hear with high-frequency hearing loss are easy to see, and the reverse.
- “P”, “b” and “m” are made with the lips closed and are highly visible. Acoustically, “m” is easy to hear while “p” and “b” are easily confused.
- “F” and “v” are made with the top teeth on the lower lip and are easy to see. “F” is a quiet, high-frequency sound that is often lost to hearing loss.
- “K” and “g” are made at the back of the mouth and are almost invisible.
- Voicing, the difference between “p” and “b” or “f” and “v”, can’t be seen at all and has to be heard.
Sounds that look identical on the lips are grouped into what researchers call visemes. English has around 40 to 45 phonemes but only a dozen or so visually distinct groups, so lip-reading alone leaves a lot of ambiguity. Estimates of how much of English can be identified by sight alone vary, but most are well under half. Skilled lip-readers fill the gaps with context. For most people with hearing loss, lip-reading works best as a partner to hearing.
Masks, lighting and video calls
The pandemic made the visual channel hard to ignore. Measurements published in 2020 found that surgical masks reduced high-frequency speech by roughly 3 to 4 dB and N95 masks by up to around 12 dB, which matters on its own for someone with high-frequency loss. The bigger loss was the mouth. Many hearing aid users struggled in shops and clinics during that period while their devices were working normally. Masks with a clear window restore the visual cue but tend to muffle sound more.
Lighting has a similar effect. A person standing with a bright window behind them has their face in shadow. Video calls add compression, low frame rates and sometimes audio and video that drift out of step. Once that drift passes the brain’s tolerance, the face can become a distraction, and switching on captions may work better.
How to help someone with hearing loss follow you
Most of what helps comes down to letting the listener see you clearly.
- Face the person when you talk, and get their attention before you start. Talking from another room or while walking away removes the visual cue entirely.
- Keep your hands, a cup or a phone away from your mouth, and avoid talking while chewing.
- Have the light on your face. In a restaurant, offer the person the seat with its back to the window so they look towards you in good light.
- Speak clearly at a natural pace. Exaggerated mouthing distorts lip patterns and makes lip-reading harder.
- If a phrase isn’t understood, rephrase it. Different words bring different sounds and lip shapes, which gives the listener a genuinely new chance.
- Reduce background noise where you can, and choose a table away from the kitchen and the speakers.
Where hearing aids fit
Hearing aids restore audibility, and directional microphones and noise reduction help in noise. None of that replaces the information a face provides. If conversation is still exhausting with well-fitted aids, look at the visual side as well as the settings. The cocktail party problem and better hearing in restaurants cover other strategies that work alongside lip-reading.




