Why can't some speakers hear the difference between certain sounds?
Because early in life the brain tunes itself to the sound categories that matter in the surrounding language, and distinctions that carry no meaning in that language stop being registered.
The key concept is the phoneme. A phoneme is a sound unit that distinguishes meaning in a given language. English pat and bat differ in one sound, so /p/ and /b/ are separate phonemes.
Crucially, sounds that are physically different can belong to the same phoneme. The p in pin is aspirated — a puff of air — while the p in spin is not. Physically distinct, and English speakers cannot easily hear the difference, because it never changes meaning. In Hindi and Thai, aspiration does distinguish words, and speakers hear it immediately.
What happens developmentally. Infants begin able to discriminate the sound contrasts of all languages. Research by Janet Werker and Richard Tees found that this ability narrows between roughly six and twelve months — infants lose the ability to discriminate contrasts absent from their environment while their ability to discriminate native contrasts sharpens.
This is perceptual narrowing, and it is not a loss so much as specialisation: the system becomes efficient at the distinctions that matter and stops spending resources on those that do not.
The familiar examples:
Japanese speakers and English /r/ and /l/. Japanese has a single liquid consonant covering that space, so the contrast is not phonemic, and adults find it genuinely difficult to hear — not merely to produce.
English speakers and Hindi dental versus retroflex stops, which sound identical to untrained ears.
Arabic emphatic consonants, Mandarin tones, and the vowel length distinctions of Finnish or Japanese.
It is not irreversible. Training studies show adults can improve substantially with intensive high-variability practice, though rarely reaching native performance.
This explains accents too — substituting the nearest native phoneme is the default when a sound is not in your inventory.