Can language models judge legal reasonableness like humans do?

The paper compares the answers of human participants with those of twenty-six LLMs on twenty-five legally relevant reasonableness judgments. Its conclusion is a qualified match. The authors write that "the models often provided answers that were statistically different from humans," yet their "total impression is one of overall coherence." For no question is the models' mean or median "wildly divergent" from the human response, and visual inspection of the violin plots shows that "both the central tendencies of the models and their overall distribution of responses tend to match" the human ones. So the finding is approximation, not equivalence: close in shape and location, but detectably different.

The paper stresses that this happens on "an inherently vague legal standard." Reasonableness "relies on variable context and implicit conceptual schemas," and lawyers, judges, and lay people all find it vexing. The authors add that the answers to these questions "are unlikely to exist in LLMs' latent training data" the way a factual answer such as the minimum age for a U.S. Senator does. The reasoning is that a model cannot look up a reasonableness judgment, so agreement with people must come from something closer to generalized judgment. That is the authors' stated expectation, not something the excerpt tests, and it is what gives the result its weight for the "silicon juror" idea, the legal counterpart of "silicon sampling" in social science.

This extends the social-norms result in Can AI systems learn social norms without embodied experience? into a legal register. There, models were scored against a collective human benchmark and beat most individuals. Here the excerpt reports only a looser fit and does not rank models against individual people. It also contrasts with Why do language models struggle with historical legal cases?, where legal performance depended on when the material was written. A vague standard with no fixed answer asks for a different competence than doctrinal recall, and the two results need not conflict. The survey-simulation work in Why do LLMs give unrealistic survey responses? bears on how such questions get asked. The excerpt does not say how this paper elicited answers.

The excerpt is silent on much that a reader would want. It does not identify the twenty-six models or the human sample, and it gives no effect sizes, no test statistics, and no account of how "statistically different" was assessed. The discussion passage also begins mid-sentence, so part of the argument is missing. The abstract notes that scholars caution reasonableness "may vary along demographic lines," but the excerpt does not report whether the models track any particular group or only a pooled average. Matching a distribution of responses is a claim about aggregates. It says little about whether one model would judge a given case as a given juror would, and Should we treat LLM outputs as real empirical data? is a reminder that closeness to human data is not the same as being human data. What the excerpt supports is narrower: on these questions, model answers are a reasonable first approximation of human ones.