Can learnable spline activations beat fixed MLP designs?

MLPs — fixed activations on nodes, linear weights on edges — are the default nonlinear approximator and the bulk of a transformer's non-embedding parameters, yet they are hard to interpret. Inspired by the Kolmogorov-Arnold representation theorem, KANs invert the design: no linear weights at all — every weight is a learnable univariate function (a spline) on an edge, and there are no fixed node activations. This seemingly small change yields three claims: much smaller KANs match or beat much larger MLPs on data fitting and PDE solving; KANs obey faster neural scaling laws; and they are interpretable — visualizable and able to act as "collaborators" helping scientists rediscover mathematical and physical laws.

The keeper is the architectural bet: moving nonlinearity from nodes to learnable edge-functions trades the MLP's opacity for a structure you can inspect and that scales better — a genuine alternative to the MLP monoculture, at least in science-adjacent regimes (the paper is candid that deep-KAN theory is still thin).

This sits in the vault's architecture/interpretability thread as a structural alternative. It rhymes with the inductive-bias-over-capacity lesson of Why does dot product beat MLP-based similarity in practice? — the right structural prior beats raw MLP capacity — and offers an interpretability-by-construction contrast to post-hoc methods like Can dictionary learning scale to production language models?.