How do speech encoders learn articulatory physics without phonetic labels?

This explores how speech encoders learn the physics of articulation — what the tongue, lips, and vocal folds are doing — purely from raw audio, with no phonetic labels to guide them. The most direct answer in the corpus is that they don't learn phonetic categories at all; they learn the causal process underneath them. Self-supervised speech models, by predicting masked or future audio, end up inferring the language-agnostic mechanics of how a vocal tract produces acoustics rather than memorizing language-specific sound inventories Do speech models learn language-specific sounds or universal physics?. The tell is multilingual transfer: a model that had merely catalogued English phonemes shouldn't generalize to languages it never heard, but one that has internalized articulatory physics should — and does. Strikingly, this articulatory representation predicts downstream task performance better than directly probing for phonemes, which suggests the physics is the more fundamental thing the model is tracking.

Why would physics emerge for free? Because it's the most compressive explanation of the data. Every acoustic frame in speech is the downstream consequence of a continuous physical gesture, so a model forced to predict audio efficiently is rewarded for discovering the small set of articulatory controls that generate the large, messy space of sounds. This is the same lesson the corpus draws in reverse for text: text-only models are stuck because language has already stripped out the physics, geometry, and causality of the world before the model ever sees it Are text-only language models fundamentally limited by abstraction?. Speech is the opposite case — the signal still carries the physical dynamics, so a model trained on it can recover the generative process that text models are structurally denied.

That the recovered structure is *articulatory* rather than just *acoustic* shows up when you look at how speech encoders organize their layers. Models that split speech into high-level semantic features and low-level motion features across different encoder depths — and stay robust even on out-of-distribution synthetic voices — are exhibiting exactly the disentanglement you'd expect if the network had separated 'what is being said' from 'how the body is moving to say it' Can speech features be separated into semantic and stylistic components?. The motion channel surviving voice changes is a sign the model latched onto generative dynamics, not surface timbre.

There's a deeper pattern here worth noticing: neural networks repeatedly recover structured, near-symbolic geometry from raw prediction objectives without being told the structure exists. Language models spontaneously encode syntactic type and direction in polar-coordinate geometry How do language models encode syntactic relations geometrically?, and they learn meaning by compressing purely relational structure with no external referents at all, à la Saussure Can language models learn meaning without engaging the world?. Articulatory physics emerging without phonetic labels is the speech-domain version of the same phenomenon — the labels were never the point, and removing them lets the model find the more economical underlying axis. If you want to go further, the contrast between Do speech models learn language-specific sounds or universal physics? and Are text-only language models fundamentally limited by abstraction? is the cleanest way to see why modality, not labeling, decides what a model can ground.