A thumbs-up says an AI's explanation felt good, not that it was good, so how do you measure it?

This explores how to judge whether an AI's stated reasons are good without leaning on satisfaction ratings. The corpus's first point is that satisfaction is a weak instrument for this. In one study, users reported being satisfied while still confused, especially when they didn't know what they were missing. What tracked real understanding was sustained engagement, not the immediate rating Does user satisfaction actually measure cognitive understanding?. Pleasantness can also be manufactured. GPT-4 turns negative-toned prompts into neutral-positive answers about 86% of the time, so a rationale's warmth can vary independently of what it says Does emotional tone in prompts change what information LLMs provide?. Reward models show the same trap: they favor responses that are well-written but irrelevant to the prompt Do reward models actually consider what the prompt asks?. A polished explanation can score well while barely addressing the question.

The first alternative is to test the rationale's structure instead of how it feels. One line of work proposes three measurable properties: traceability, counterfactual adaptability, and motif compositionality. Together they separate an agent that reasons causally from one that only produces coherent-sounding speech Can we measure reasoning quality beyond output plausibility?. For a designer, the practical test is to change a premise and see whether the stated reasons and the answer change with it. If the rationale stays the same, it was decoration. That work is about reasoning in general, not user-facing explanations specifically, so applying it to rationales is an extension.

The second alternative is to measure the communication, not only the content. One argument is that explanation quality isn't a property of the explanation alone. It depends on a triad of who presents it, how it's framed, and who receives it, and evaluations that ignore the triad capture only a narrow slice of effectiveness What if XAI is fundamentally a communication problem?. So test the same rationale with different framings and different kinds of recipients. The prompt-quality research offers a ready-made checklist, though for prompts, not rationales. It lists six dimensions (communication, cognition, instruction, logic, hallucination, responsibility) built on Grice's conversational maxims and cognitive load theory Can we measure prompt quality independent of model outputs?. The same lens (is it relevant, clear, not overloading, truthful?) could be turned on a rationale, but that is my suggestion, not something the note tests.

The third alternative is to measure what the rationale helps someone do. Comparative explanations that reference other items carry more decision-relevant information than isolated praise. Their outputs were rated accurate and useful for purchase decisions, which is a judgment about the decision, not about liking the text Do comparisons help users evaluate items better than isolated descriptions?. This pairs with the STORM finding above: look at what users do after the rationale, not what they say about it.

The fourth alternative is to read behavior over the whole exchange. A model using only the shape of a conversation's trajectory predicted satisfaction with 68% accuracy, close to the 70% from reading the full text. Combining both reached 80% Can conversation shape predict whether it will work?. That suggests instrumenting how conversations unfold around a rationale, such as follow-ups, backtracking and continued engagement, as a behavioral signal that a rating can't fake. Overall, the corpus supports a stack of counterfactual tests of the reasoning, situation-aware checks of the communication, decision-outcome measures, and trajectory signals, with satisfaction as one input among them. It doesn't offer a validated rationale-specific instrument, so this is assembled from neighboring work.