Can declaring experiments in code make them reproducible?

The paper's central claim is that a live human–AI experiment becomes a shareable research object once the whole protocol is declared in a single configuration. With Pairit, "a single YAML configuration file" declares "an executable experiment graph" covering pages, routing, randomization, matchmaking, chat, shared workspaces, server-hosted agents, surveys, timers, and custom HTML components, and any number of humans and AI agents can be combined in live sessions. The introduction locates the gap: oTree and Empirica coordinate live human-to-human sessions, and Deliberate Lab supports real-time human–AI group chat, but researchers "cannot easily declare" protocols in which AI agents converse, co-edit shared documents, and take protocol-defined actions within one auditable configuration that also fixes team assignment, channels, and routing.

The mechanism the paper gives is build cost. Each new design "requires custom software," so live protocols "rarely become shareable research objects other scholars can inspect or run." The discussion lists what that custom work involved: synchronizing participants, routing conversation, managing shared work, and embedding agents that both speak and act. It calls this slow and design-intensive "even with AI-assisted coding." Pairit converts the build work into a declarative graph, so researchers can vary "who participates, what role AI plays, and when it intervenes." The closing sentence carries the argument: "Standard prose methods sections cannot fully specify live experiment protocols." On this reading, the configuration file is the specification of the method, not a convenience layered on top of it.

This sits beside How do communication modalities shape human-agent collaboration patterns?, which describes a different configurable platform and stresses which interaction controls to vary and what its validation study found. Pairit adds a layer that note does not address, which is how a manipulation gets recorded and handed to another lab. The two arguments compose, though this is my inference and not the paper's: theory-grounded controls only accumulate into shared evidence if the configuration that set them can be rerun. The paper also names delegation, facilitation, and bargaining assistance as things to test in "repeatable, customizable, and shareable" experiments, which connects to What makes delegation work beyond just splitting tasks?. That note's axes are candidate variables for such a graph, but the excerpt does not say any have been manipulated.

The excerpt is an infrastructure claim and reports no findings about human–AI team behavior. It cites "multiple live deployments, including peer-reviewed published studies" and "high-resolution process traces" of communication, negotiation, and collaborative work in live dyads, without naming the studies, sample sizes, or results. It gives no measurement of build time saved against oTree, Empirica, or custom code, and it does not show that another lab has run a protocol authored elsewhere. Sharing and reproducibility are stated as capabilities, not demonstrated outcomes. What follows at the strength the evidence allows is narrow: the format in which a live human–AI protocol is specified belongs to the method, and a prose description alone leaves interaction structure underdetermined.