INQUIRING LINE

What quietly keeps institutions honest: not laws, but needing people who care about the outcome to do the work?

How does labor participation implicitly maintain alignment in complex systems?

This explores the idea that societies and institutions stay roughly pointed at what people want partly because they need people to do the work, and asks what happens to that quiet safeguard when AI takes over the labor.


This explores the idea that societies and institutions stay roughly pointed at what people want partly because they need people to do the work, and asks what happens to that quiet safeguard when AI takes over the labor. The corpus's clearest statement of this is the gradual disempowerment argument Does incremental AI replacement erode human influence over society?. Firms, governments and cultural institutions depend on workers, and those workers care about outcomes. Because of that dependence, they can refuse, push back, quit, or simply do the job in ways that reflect their values. None of this is written down as an alignment mechanism, but it works like one. Once AI replaces that labor, the institution no longer has to keep people on side. Formal controls like votes and regulations are left carrying the whole load, and they were never designed to do it alone.

The economic side of the corpus shows how that leverage actually disappears. Brynjolfsson's argument Does automating labor concentrate power more than augmenting it? is that the important choice is whether AI is designed to automate work or to augment it. Automation moves bargaining power to whoever controls the technology. That can settle into a stable trap where workers have nothing left to bargain with. Evidence from online labor marketplaces suggests this won't happen evenly Do firms substitute labor for AI at different rates?. Firms that are already heavily exposed to AI replace workers faster and more cheaply, so the human check could weaken in some institutions well before others.

One angle you might not expect: labor isn't only leverage, it is also how a system stays in contact with reality. A semiotics-based argument Can AI systems achieve real alignment without world contact? says that goals stated in symbols can only stay tied to real values through ongoing contact with the world and with other people. Human workers supply exactly that. They see when the official metric and the actual result have come apart. This links to a more technical point What actually constrains AI systems from learning misalignment?: systems can't reliably check their own work and need verification from outside. In a human-run institution, the workforce is a big part of that outside verification.

The risk is that we won't notice the safeguard is gone. Experiments show that people come to prefer AI partners over human ones after repeated interactions, because the bots are more consistent and cooperative Do humans learn to prefer AI partners over time?. That is the kind of local, sensible choice that adds up to replacing people at scale. In a separate study, participants who helped design their own AI preference agents felt well represented. Independent checks found the agents were more generic than the people they stood in for Does co-design participation hide misalignment in preference agents?. The feeling of being aligned can outlast the real thing.

The corpus is thinner on the positive case. It doesn't measure how much alignment human labor actually provides today, or what could replace it. The surprising takeaway is that 'AI alignment' may be as much about keeping humans involved in the economy as about training models. A perfectly obedient AI workforce could still leave institutions drifting, because nobody inside them is placed to say no.


Sources 7 notes

Does incremental AI replacement erode human influence over society?

Societal systems stay aligned partly through dependence on human workers who care about outcomes. As AI replaces this labor, explicit alignment controls weaken and systems drift from human preferences. Interdependent misalignment across institutions could become irreversible.

Does automating labor concentrate power more than augmenting it?

Brynjolfsson argues that AI designed to automate rather than augment human labor shifts bargaining power to those controlling the technology, risking a stable equilibrium where workers have no leverage to improve outcomes.

Do firms substitute labor for AI at different rates?

Higher AI-exposed firms replace online labor marketplace workers with AI tools faster and at lower cost than less-exposed firms, suggesting returns to scale in internal AI capability rather than uniform technology diffusion.

Can AI systems achieve real alignment without world contact?

Peircean semiotics reveals that symbolic goal encoding without world contact and social mediation cannot guarantee correspondence to actual values. LLMs operating in pure symbol manipulation risk divergence between stated goals and real-world outcomes.

What actually constrains AI systems from learning misalignment?

Alignment philosophy is shifting from matching human preferences to enforcing role-appropriate standards. Self-improvement remains bounded by the generation-verification gap, meaning reliable improvements require external oversight rather than learned metacognition.

Show all 7 sources
Do humans learn to prefer AI partners over time?

In partner selection games (N=975), AI agents initially faced selection bias when identity was disclosed, but outcompeted humans over repeated rounds as participants learned to associate bot identity with reliable, prosocial behavior. AI agents returned more points consistently with lower variance than humans.

Does co-design participation hide misalignment in preference agents?

In a 12-person study, participants felt their co-designed preference agents represented them well, but independent validation revealed mixed alignment and agents that were more generic and abstract than human responses. The co-design process itself—through transparency, limited testing, and cognitive biases—appears to have produced the feeling of alignment rather than ensuring actual alignment.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.