When testing AI tools, do you lock down the AI's behavior for clean data, or let it act unpredictably like it would for real users?

This explores how researchers handle the tension between keeping a GenAI study controlled and letting the system behave the way it really does. The corpus has one paper aimed squarely at this question, plus several neighbors that show the same problem from other directions. The direct one is a reflective look back at four lab studies How does GenAI's unpredictability break traditional lab study methods?. It names the 'fidelity-control trade-off' as one of five problems that appear once the system under test gives a different answer each time. You can fix the outputs (scripted responses, fixed seeds, canned examples) and get clean comparisons, but then participants aren't really using GenAI. Or you can let the model run live and get realistic behavior, but two participants in the 'same' condition may have had very different experiences. The authors offer eighteen recommendations, not a single fix. That alone tells you the field treats this as something to manage, not something to solve.

The less obvious part is that the variability isn't just noise. It's part of what's being studied. Interaction design work on GenAI argues that users now describe *what* they want rather than *how* to get it, so output that changes from run to run is a basic property of the medium, not a defect How should users control systems with unpredictable outputs?. If you control that variability away, you may be studying a different kind of system. The lab-study paper finds a related effect: participants fall back on habits from older, predictable software, and the feedback loops between user and system change. A tightly scripted study would hide exactly those behaviors.

The same trade-off shows up in AI safety under a different name. UK AISI tested whether frontier models would sabotage safety research. They built simulated lab scenarios with planted chances to do so and found none Do frontier AI models sabotage safety research tasks?. Building the scenarios gave them control, but the models' high refusal rates suggest the test setting itself shaped what the models did, which raises the realism question in reverse. Redwood Research's case for AI control takes a different route around the problem. Instead of trying to infer a model's intentions under realistic conditions, it tests only what the model is *capable* of doing against a red team, because capability is easier to evaluate in a controlled way Can AI control work even if models are actively scheming?. One way to handle the trade-off, then, is to change the question until control becomes possible.

Two more notes show why controlled conditions are harder to define than they look. Small edits to hidden system prompts can abruptly change model behavior Who actually controls what AI systems say?. If a vendor quietly updates its system prompt partway through your study, your 'controlled' condition has changed without you knowing. Post-trained models also behave measurably differently when they are reading their own generated text compared with text supplied by someone else Do models recognize their own outputs as actions shaping future inputs?. So feeding a model scripted transcripts, a common control technique, can put it into a different behavioral mode than live use does.

The corpus is thin here. Only one note tackles the methods question directly, and the Human Factors proposals for steadying GenAI tasks remain untested ideas Can Human Factors design fixes restore GenAI productivity losses?. What the collection does show is the outline of the problem. Every move toward control risks turning the thing you're studying into something else. HCI researchers, safety evaluators and interaction designers are each finding that out separately, often without using the same words for it.