INQUIRING LINE

Do AI models that learn speech from raw audio end up hearing universal human speech, or just one language's accent?

Do self-supervised speech models learn dialect-agnostic representations across languages?

This explores whether self-supervised speech models, trained on raw audio without labels, end up with internal representations that ignore dialect and language differences. In other words: do they hear 'speech in general' rather than 'this accent of this language'?


This explores whether self-supervised speech models learn representations that look past dialect and language, and what that would mean. The short answer from the corpus is about languages, not dialects. The strongest evidence suggests these models learn something more basic than any one language's sound inventory: they appear to infer the physical, articulatory processes that produce speech, meaning how the tongue, lips and vocal tract shape sound Do speech models learn language-specific sounds or universal physics?. Every human shares that physics, so a model that learns it gets cross-language transfer almost for free. Measuring how well a model captures articulation also predicts its downstream performance better than checking whether it recognizes specific phonetic categories.

That finding reframes the question. If the model's base layer is articulation rather than phonemes, then 'dialect-agnostic' is partly the wrong lens. A dialect is largely a different habit of articulation: a vowel produced further forward, a consonant softened. A model that represents articulation faithfully wouldn't erase those differences. It would encode them precisely, as positions in a shared physical space. So the representation may be *universal* (one space for all speakers) without being *agnostic* (blind to accent). The corpus doesn't directly test dialect variation, though, so treat this as an inference from the articulatory result, not a measured finding.

Text models offer a useful warning. Interpretability work on text LLMs finds that cultures with little training data, such as Ethiopia or Algeria, get represented internally through high-resource proxies, even when the model's surface answers are correct Do LLMs represent low-resource cultures through dominant cultural proxies?. The parallel risk for speech is clear: a 'shared' representation built mostly from dominant accents could map minority dialects onto them rather than represent them on their own terms. Whether speech models avoid this because articulatory physics is a more neutral foundation than cultural text is an open question the corpus doesn't settle.

There is a hint of how you could check. Sparse-autoencoder work on production text models recovers features that are abstract and multilingual, firing for the same concept across languages and causally steering behavior Can dictionary learning scale to production language models?. Applying the same feature-level analysis to speech models would show whether there are 'dialect features' riding on top of shared articulatory ones, or whether some dialects quietly collapse into others. The collection doesn't yet hold that study. It's a gap worth noting: the material supports language-general representations in speech models, but says nothing direct about how well dialects are preserved.


Sources 3 notes

Do speech models learn language-specific sounds or universal physics?

Self-supervised speech models learn the language-agnostic physics of how the vocal tract produces acoustics, not language-specific phonetic categories. This explains their multilingual transfer and predicts their downstream task performance better than phonetic probing.

Do LLMs represent low-resource cultures through dominant cultural proxies?

Mechanistic interpretability analysis reveals that low-resource cultures like Ethiopia and Algeria are structurally represented through high-resource cultural proxies in internal model states, not just output. This architectural bias persists even when models can produce correct surface-level answers.

Can dictionary learning scale to production language models?

Sparse autoencoders extract high-quality, abstract, multilingual features from Claude 3 Sonnet that both respond to and causally influence model behavior. The work demonstrates interpretability is tractable at production scale, not limited to toy models.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.