AI Assertion

Source: Patrick Butlin, Emanuel Viebahn; Ergo vol. 12 · 2025

Modern generative AI systems have shown the capacity to produce remarkably ­fluent language, prompting debates both about their semantic understanding and, less prominently, about whether they can perform speech acts. This paper addresses the latter question, focusing on assertion. We argue that to be capable of assertion, an entity must meet two requirements: it must produce outputs with descriptive functions, and it must be capable of being sanctioned by agents with which it inter­ acts. The second requirement arises from the nature of assertion as a norm-governed social practice. Pre-trained large language models that have not been subject to finetuning fail to meet the first requirement. Language models that have been fine-tuned for “groundedness” or “correctness” may meet the first requirement, but fail the second. We also consider the significance of the point that AI systems can be used to generate proxy assertions on behalf of human agents.

Introduction. The development of Transformer-based large language models (LLMs) and LLM-driven conversational agents has put language use at the cutting edge of AI research. The measurable performance of these systems is human-like on many linguistic tasks (Mahowald & Ivanova et al. 2023), and their capacity to produce fluent and apt outputs is extraordinary. These systems include GPT-3 (Brown et al. 2020) and several successors, including ChatGPT (OpenAI 2022) and GPT-4 (OpenAI 2023). They also include LaMDA (Thoppilan et al. 2022), Sparrow (Glaese et al. 2022) and others. In this paper we use LaMDA and Spar­ row as our primary examples. The success of these systems raises several press­ ing questions about the nature of their linguistic and communicative capacities.

Some of these questions concern the semantic capabilities of AI systems such as LaMDA. LaMDA famously produced the following output in response to a question about its consciousness/sentience:

Does LaMDA understand the meanings of the words it uses? Does it have any grasp of what the world is, or what it is to feel happy or sad?1 Other questions fall within the realm of pragmatics: Does LaMDA have the ability to perform speech acts, such as assertions, commands or apologies? How similar are LaMDA’s outputs to the actions humans perform in using language? And, more generally, what requirements would an AI system have to meet so as to be capable of performing speech acts? The present paper is concerned with these latter questions about the pragmatics of AI systems and their outputs, with a particular focus on the central speech act of assertion.2 LaMDA and Sparrow are good examples for our purposes because they are among the first LLM-based systems to be fine-tuned for correctness. This makes them particularly promising cases of AI systems which might be able to make assertions. They are designed not only to converse naturally, but also to get things right. And they often do. For example, when asked who was the first to climb Mount Everest, LaMDA looks up the relevant information on Wikipedia and answers:

So many of these systems’ outputs not only look like assertions, they also resem­ ble human assertions in providing helpful and often correct information. But are (1) and (2) genuine assertions? Are AI systems capable of performing the same kinds of actions a human would perform in uttering (1) or (2)?

It is our aim to show that LaMDA, Sparrow and other current AI systems do not have the capability to assert, even as proxies for human agents. To bring this out, we will discuss two requirements for assertion. The first, basic requirement is that for an output of some system to be an assertion the output must have a descriptive function. The second—which is much more demanding—is that for a system to be capable of assertion it must be sanctionable by agents with which it interacts. This requirement arises from the status of assertion as a form of engagement with the human social practice of using language to share infor­ mation; our view is that to assert a system must be capable of being sanctioned for failing to meet the standards of this practice. We will consider different kinds of existing AI systems to show that they fail to fulfil at least one of these criteria. And we will indicate that turning to proxy assertion does not make it easier to argue for the possibility of AI assertion. We also hope to show that consider­ ing the possibility and conditions of AI assertion allows for insights into the nature of assertion, including the nature and significance of sanctionability and the mechanisms of proxy assertion.

Related work. Existing work on AI pragmatics suggests a positive answer to these ques­ tions. For example, in discussing which conversational norms are appropriate for AI systems, Kasirzadeh & Gabriel state that AI systems can perform speech acts such as assertions:

Similarly, Green & Michel (2022: 120) argue that there are technologically possi­ ble machines which “can perform speech acts such as assertions, directives, and warnings” as proxies for humans who are running them. This positive outlook fits with comments in Freiman & Miller (2020), Evans et al. (2021) and Glaese et al. (2022).3 Moreover, Kneer (2021) has shown that laypeople accept that advanced AI systems can lie. Confronted with a vignette in which either a human or “an advanced robot driven by artificial intelligence, which can take its own decision” (2021: 6) make a believed-false statement, participants were just as likely to judge the robot as lying as they were to judge the human agent as lying.

Method. A key idea in this section is that assertion is a species in the genus of descrip­ tive representation. Descriptive representations are those that say that things are a certain way, and are therefore apt to be true or false, or accurate or inac­ curate (as opposed to, for instance, directive representations, which tell consum­ ers what to do). The point that assertion is a species of this genus is important for two reasons. First, as we have just suggested, an output of an AI system must have a descriptive function—the kind of function characteristic of descrip­ tive representation—to be an assertion. Second, the category of assertion is only explanatorily useful or interesting if it is narrower than that of descriptive rep­ resentation. We therefore assume that at least some systems (AI or otherwise) produce descriptive representations that are not assertions.

Descriptive representations are distinguished by their function, or the pur­ pose for which they are produced. This claim is an element of a broader theory responding to Grice’s problem of the distinction between natural and non-nat­ ural meaning (Grice 1957; Armstrong 2023). Grice’s problem is to distinguish meaningful or communicative behaviours from other behaviours, and from other features of the natural world, given that information is ubiquitous. For instance, what is it about an animal’s alarm call that makes it a meaningful signal, when that same animal’s freeze response is not, given that both are correlated with We cannot give a complete account of the descriptive function here, but we take its essence to be as follows. For an output of some system to have a descrip­ tive function is for it to have the function of conveying information to an observer or consumer system, so as to cause the consumer to behave as though some con­ dition holds.4 We take the liberal view that such a function can be established, in principle, by natural selection, learning, training of machine learning models, design of artifacts, social convention, or the producer’s intentions.

A few examples will help to illustrate this idea. First, a male firefly’s flash has a descriptive function because it has been selected for conveying the information that the male is present to female fireflies, which then approach. In this case, the descriptive function of flashes has been established by natural selection. Flashes are produced because they tell female fireflies where to find males, with fitness benefits for both parties.

Second, the mercury level in a thermometer has a descriptive function because the thermometer has been designed to make information about the tem­ perature at some location available to users, through their observations of this output. The purpose of making this information available is to allow people to modify their behaviour in response.

Third, suppose Ingrid tells James that Kathy will be at the party, hoping that this will encourage him to come. In this case, Ingrid’s words have a descriptive function established by her intention, which is to convey to James the informa­ tion that Kathy will be at the party, and thus to cause him to take this to be the case when deciding whether to attend.

In each of these cases, the function of the producer’s output—the flash, the mercury level, or the words—is to convey a certain piece of information to a con­ sumer. Conveying this information is worthwhile because it may affect the con­ sumer’s behaviour. The outputs are assessable for truth or falsity because there is a specific condition under which they are supposed to be produced, which is also relevant to explaining how they affect the consumer’s behaviour.

With this account of descriptive functions in hand, we can see that some AI sys­ tems produce linguistic outputs that take the characteristic form of assertions, but lack a descriptive function. In their pre-trained form, LLMs are an important example of this phenomenon. LLMs are trained in two stages. First, they are trained by self-supervised learning on large corpuses of text to select the most likely next word (or sub-word token). Second, they can be fine-tuned to improve specific aspects of their performance.

Discussion. We think that there at least two further reasons why this result is important. For one thing, it suggests that verdicts on whether a certain system is capable of assertion or other speech acts cannot be made on the basis of considering the system’s outputs alone. Rather, it is essential to assess whether the system can have desires or interests of its own, and thus to assess its architecture and training as well as its outputs. In this respect, there is a parallel with the case of Blake Lemoine’s (2022) attribution of sentience to LaMDA: Lemoine focuses only on LaMDA’s outputs to argue that the system is sentient, without taking into account features of its architecture, for which he has been widely criticized (Chalmers 2023). And just as we cannot tell solely on the basis of its outputs whether an AI system is sentient, we cannot tell solely on the basis of its outputs whether an AI system can perform assertions.

Secondly, Sparrow and similar fine-tuned LLMs provide real-world counter­ examples to the claim that losing credibility (and finding out about it) is always an analogue of punishment. While fine-tuned LLMs can be sensitive to cred­ ibility scores and while this can even influence their outputs, they are not sanc­ tionable in a full sense, as low credibility scores are not bad or undesirable for them. This shows that the element of sensitivity to credibility and the element of undesirability of low credibility scores in the account of sanctioning motivated above can come apart, and it raises the question whether this might also happen in the case of human assertion.

Conclusion. Many current AI system produce outputs that have the appearance of assertions. We have tried to make plausible that such outputs do not amount to assertions.

Some of them fail to be assertions because they fail to have a descriptive func­ tion. Others cannot be assertions because asserting requires sanctionability. We have also argued against the view that AI systems can assert in their role as prox­ ies. In our view, optimism about AI assertion in current systems is misplaced.

However, we don’t take this to be a purely negative result. On the one hand, it highlights that even as AI systems become more sophisticated and indepen­ dent, it is the humans who employ such systems who are responsible and sanc­ tionable for the outputs and their effects. A view that sees AI systems as genuine producers of assertions might be tempting for those who want to shift respon­ sibility away from themselves and onto the systems they employ. We hope that our arguments show that such a move starts with a false presupposition.

On the other hand, our conclusion can be seen as a challenge for those hop­ ing to devise AI systems that are capable of assertion. We have tried to make plausible that the decisive missing ingredient is the feature which grounds interests, which may be consciousness, emotions, desires or higher-order pref­ erences. If we are right, that is where researchers should focus their attention on the path to AI assertion.