Why does writing down exactly how an expert thinks make it easier to replace them?
Why does making expert judgment explicit accelerate expert displacement?
This explores why writing down how experts decide (their rules, steps and criteria) can make it easier to replace them, and what parts of expertise survive being written down.
This explores why turning an expert's tacit judgment into explicit rules, steps or criteria can speed up their replacement, and what part of expertise survives that process. The corpus has no labor-economics studies of displacement. What it does have is material on how AI learns judgment once it is put into words, and a counterweight about what never makes it onto the page.
The mechanism shows up most clearly in how AI judges are trained. One effective way to make LLM evaluators less biased is to turn judgment tasks into verifiable problems, meaning pairs of examples with a checkable right answer. Models trained this way learn to reason through decisions instead of reacting to surface cues like length or confident tone Can reasoning during evaluation reduce judgment bias in LLM judges?. That is exactly what 'making judgment explicit' does: it turns a fuzzy skill into something a training signal can latch onto. A related finding explains why this transfers so well. Models' reasoning ability comes mostly from broad *procedural* knowledge (how-to patterns spread across many documents), not from memorized facts Does procedural knowledge drive reasoning more than factual retrieval?. Once an expert writes their method down as a procedure, it becomes the kind of material models absorb and generalize best.
The corpus also suggests that what gets written down is not all of expertise. Expert judgment is communicative. A good expert claim anticipates whether a particular community will accept it, not only whether it is factually right Can AI replicate the communicative work experts do? Can AI anticipate whether expert claims will be socially valid?. Expert arguments also carry force from reputation and track record, and models working from text alone can't separate them from common assumptions Can language models distinguish expert arguments from common assumptions?. So the explicit version of expertise is a thinner copy. Displacement can still speed up, because the copy is fluent and confident enough to pass for the real thing, while the social and contextual part that made the original trustworthy quietly drops out.
That gap can then grow. When AI produces judgments faster than people can check them, confidence in knowledge erodes the way money loses value under hyperinflation. The tools meant to check the output are increasingly AI-built themselves, so the cycle feeds on itself Can AI generate knowledge faster than humans can evaluate it?. Explicit expertise feeds that machine.
The corpus also points to a different path. 'Learning to Guide' systems use the machine to point out what matters in a case and leave the decision to the human. This reduces anchoring bias, the tendency to lean on the first answer you see, and keeps responsibility with the person Can AI guidance reduce anchoring bias better than AI decisions?. So making judgment explicit does not have to end in replacement. Whether it does depends on whether the explicit version is used to stand in for the expert or to sharpen the expert's own perception.
Sources 7 notes
Training judges with reinforcement learning to reason about evaluations—by converting judgment tasks into verifiable problems with synthetic data pairs—produces judges that think through their decisions rather than relying on exploitable surface features, directly mitigating authority, verbosity, position, and beauty bias.
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
Expertise requires anticipating audience acceptability and social validity, not just retrieving information. AI lacks the mechanism to perform this communicative work, making its fluent output epistemically misleading despite its confident form.
Expert claims are validity claims that succeed when both factually correct and socially acceptable within a community. AI can estimate statistical correctness but cannot anticipate contextual acceptability because it lacks embedded knowledge of expert communities' evolving standards.
LLMs lose the social context that gives expert claims their force—reputation, track record, and standing—because they process only text, not the social world where expertise is built and evaluated.
Show all 7 sources
AI produces knowledge faster than human judgment can verify it, collapsing epistemic confidence just as monetary hyperinflation collapses purchasing power. The gap self-reinforces because evaluation tools are themselves AI-generated, trapping the system in acceleration.
Learning to Guide eliminates anchoring bias and unassisted hard cases by having machines supply interpretive guidance rather than autonomous decisions, keeping responsibility with humans while improving their judgment through enhanced perception.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- People Overtrust AI-Generated Medical Advice despite Low Accuracy
- A Rational Analysis of the Effects of Sycophantic AI
- GenAI as a Power Persuader: How Professionals Get Persuasion Bombed When They Attempt to Validate LLMs
- "That's AI Slop, You Bot!" Studying Accusations, Evidence, and Credibility in Online Discourse Towards LLM-Generated Comments
- Epistemic Deference to AI
- AI Models Exceed Individual Human Accuracy in Predicting Everyday Social Norms
- Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
- J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learning