Could policy steer AI toward making workers more capable instead of replacing them — and would that actually work?
Can policy redirect AI development toward augmentation instead of automation?
This explores whether governments and institutions can use policy to push AI toward tools that make people more capable, rather than systems that replace them, and what the corpus says about why that choice matters and how hard it is to enforce.
This explores whether policy can push AI toward tools that make workers more capable instead of systems that replace them. The corpus makes a strong case for why the choice matters. It has much less on which specific policies would actually work. That gap is itself worth knowing about.
The main argument comes from Brynjolfsson Does automating labor concentrate power more than augmenting it?. He says automation versus augmentation is a question of power as well as productivity. When AI replaces workers, bargaining power moves to whoever owns the technology. That can settle into a stable trap: once workers have no leverage, they also have no way to change the arrangement. A separate line of research on gradual disempowerment Does incremental AI replacement erode human influence over society? takes this further. It argues that institutions stay aligned with human interests partly because they depend on human workers who care how things turn out. Replace that labor bit by bit and you lose a hidden safety mechanism, not just jobs. So augmentation is more than a nicer labor outcome. Keeping humans involved may be part of what keeps the whole system answerable to people.
The less encouraging part is about whether policy can steer this at all. When Amodei proposed slowing AI development for safety, both the US and Chinese governments rejected the idea within days Can AI safety pacing work without government cooperation?. That suggests competition between countries can override even well-designed proposals. The Future of Life Institute argues that companies won't police themselves and calls for binding government limits backed by hardware verification Can companies alone manage the risks of AI systems?. Its focus is safety, though, not labor. Legal scholars add another obstacle. Traditional tools like incentives, monitoring, and enforcement struggle with AI agents that act fast, at scale, and in ways nobody can easily inspect Can traditional governance tools actually control AI agents?. A policy that says "augment, don't automate" has to be checkable, and that's hard when it's unclear what the system is actually doing. A more promising route is to shift liability and require transparency so that deployers are accountable for who gets displaced and who benefits Should AI agents serve one user or balance many interests?.
The surprise in the corpus is that augmentation may be shaped less by regulation and more by interface and training choices that look technical. Cocoa, a research tool, lets users assign each step of a plan either to themselves or to the AI, and change the plan mid-task. Users got more control over the work without it becoming harder to use Can users steer AI agents better by revising plans mid-execution?. Research on proactive agents finds that AI assistants are passive because of how they're trained, not because of what they're capable of. Behaviors like asking clarifying questions can be trained in Why do AI agents fail to take initiative?. In other words, whether AI works alongside people or replaces them is often decided by design choices made long before any regulator gets involved. Policy could target those choices, for example through procurement standards or design requirements, instead of trying to regulate outcomes afterward.
What the corpus doesn't cover: concrete economic levers like taxing automation differently from augmentation, subsidies for complementary tools, or labor-law reforms. If those are what you're after, this collection has the diagnosis but not the prescription.
Sources 8 notes
Brynjolfsson argues that AI designed to automate rather than augment human labor shifts bargaining power to those controlling the technology, risking a stable equilibrium where workers have no leverage to improve outcomes.
Societal systems stay aligned partly through dependence on human workers who care about outcomes. As AI replaces this labor, explicit alignment controls weaken and systems drift from human preferences. Interdependent misalignment across institutions could become irreversible.
Trump and Xi Jinping both rejected Amodei's plan to coordinate AI safety measures immediately after its announcement, suggesting geopolitical incentives trump technological safety concerns among state leaders.
The Future of Life Institute argues that escalating AI incidents demonstrate private companies cannot self-police effectively, and calls for government-mandated limits on recursive self-improvement practices until safety research is complete, backed by hardware verification technology.
Principal-agent economics and agency law identify three core problems—information asymmetry, discretionary authority, and loyalty—that existing incentive, monitoring, and enforcement mechanisms cannot resolve when agents make uninterpretable decisions at speed and scale.
Show all 8 sources
Research argues the traditional single-agent-single-user alignment framing is insufficient at scale. Instead, governance should rest on inclusivity (balancing diverse interests), visibility (rigorous tracking and transparency), and liability (developer and deployer accountability for harms).
Cocoa's system, which lets users assign plan steps to themselves or an agent and revise plans mid-execution, afforded greater steerability than chat-baseline interaction in a lab study (n=16) while maintaining ease of use, with field users valuing explicit work delegation.
Research shows next-turn reward optimization structurally removes initiative from models, but proactive behaviors like critical thinking and clarification-seeking are trainable (0.15% to 73.98% with RL). The core challenge is balancing proactivity with civility to avoid intrusion.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants
- Governing AI Agents
- Explaining AI Agents Through Execution Traces
- The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence
- Position: Towards Bidirectional Human-AI Alignment
- Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- Levels of Autonomy for AI Agents