Can a single regularizer prevent JEPA representation collapse?

Joint-Embedding Predictive Architectures (JEPAs) learn world models in compact latent spaces, but existing methods are fragile — they rely on complex multi-term losses, exponential moving averages, pretrained encoders, or auxiliary supervision to avoid representation collapse (the degenerate solution where the encoder maps everything to a constant). The engineering needed to keep them stable is itself the barrier.

LeWorldModel (LeWM) is the first JEPA that trains stably end-to-end from raw pixels using only two loss terms: a next-embedding prediction loss and a regularizer enforcing Gaussian-distributed latent embeddings. That single regularizer does the anti-collapse work that the usual stack of tricks did, cutting tunable loss hyperparameters from six to one. The payoff is practical: 15M parameters trainable on one GPU in hours, planning up to 48× faster than foundation-model-based world models, competitive across 2D and 3D control. The latent space also encodes meaningful physical structure — probing recovers physical quantities, and the model reliably flags physically implausible events.

The general lesson is that simplicity in the self-supervised objective can replace brittle engineering: collapse is prevented by an explicit distributional constraint rather than by carefully balanced auxiliary terms. This is the practical face of Why is predicting latents more sample-efficient than tokens? — the theory says latent prediction is the efficient target; LeWM shows the missing piece was a principled way to keep those latents non-degenerate.