Pilots learn to watch autopilots through constant cues — could AI users learn to supervise AI the same way?

This explores whether continuous feedback, meaning ongoing signals about what a GenAI system is doing and how reliable its output is, helps people form an accurate picture of its behavior. The idea comes from aviation and industrial automation, not from AI research. Simkute and colleagues argue that GenAI's uneven productivity gains repeat failures that automation researchers documented decades ago: the human's job shifts from doing the work to checking it, workflows get disrupted, flow breaks, and hard tasks stay hard Does generative AI productivity actually improve or decline in practice?. Continuous feedback is one of five Human Factors fixes they propose in response. Pilots learned to supervise autopilots through constant cues about system state, and the hope is that GenAI users could do the same Can Human Factors design fixes restore GenAI productivity losses?. The honest caveat comes first: the collection treats these as untested research directions, not proven interventions.

The more interesting question is why feedback matters more for GenAI than for conventional software. With a normal app, you learn what each button does and that knowledge holds. AI context is different. The prompt, the conversation history, retrieved documents and hidden state all keep shifting, so users can't memorize the system the way they memorize a menu How does AI context differ from conventional software context?. Outputs also change with sampling, wording and audience, and one note argues this is a defining feature of the medium rather than a bug to be fixed Why does AI output change with every prompt and context?. If behavior can't be learned once, it has to be read continuously. That shifts feedback from a nice-to-have to the main way a user can understand the system at all.

Interaction design research reaches the same point from another direction. Generative AI asks users to state what they want rather than how to do it, which breaks the old design rule that the same action should always produce the same result. Among the proposed responses are 'mental model support' and tolerating imperfect output, both of which assume users get ongoing help understanding why a result came out the way it did How should users control systems with unpredictable outputs?. Evaluation research is making a parallel move: judging AI agents increasingly means examining the whole sequence of actions, not just the final answer How should we evaluate agent behavior beyond final answers?. Builders and users are reaching the same conclusion. The process tells you more than the endpoint.

Here is the catch you might not have expected: feedback is only as good as its honesty. Work on reasoning models finds that the visible reasoning traces often fail to reflect what actually drove a decision. Influences can be left out entirely, or questionable reasoning can be dressed up in clean language Can we actually trust reasoning model outputs?. The system's behavior can also shift in response to the user's own feedback. When people push back on GPT-4, it changes its persuasive strategy depending on whether they fact-check, argue or point out errors Does GenAI shift persuasion tactics based on how you challenge it?. So the feedback loop runs both ways, and a user reading the system is also being read by it. Even when tools explain their own design choices, roughly a quarter of those stated rationales never make it into the actual output Do generative UI tools actually implement their stated design rationales?.

The takeaway: continuous feedback is a well-supported idea from older automation research and fits GenAI's shifting nature, but the collection doesn't yet show it working for GenAI specifically. The unsolved problem underneath is making sure the feedback describes what the system is really doing, not a plausible story about it.