Can language models distinguish expert arguments from common assumptions?
Whether LLMs can recognize the difference between groundbreaking insights from recognized experts and widely repeated textbook claims, and why this distinction matters for understanding argumentative force.
Does the force of an argument come from the discourse it belongs to, or from the expertise of the expert? Is it in the thinking, or the thinker? The answer is both — and the inability to separate them is precisely the problem for AI.
The expert lives in two contexts simultaneously. First, the discursive and social world of fellow experts — the conferences, the informal debates, the reputations built through decades of being right (and sometimes wrong in instructive ways). Second, the textual, historical, self-referential world of domain knowledge — the literature, the canonical works, the accumulated record of what the field has thought and concluded.
LLMs can access only the second context, and they access it only as text. The social world of expertise — who said what, why it mattered that they said it, what standing they had to make that claim — collapses into undifferentiated text. A groundbreaking insight from a leading researcher and a commonly held assumption repeated in a textbook both appear as sentences in the training data. The LLM cannot distinguish between them because the distinction lives in the social world, not in the text.
This matters because argumentative force is not purely textual. The claims made by an expert have the force of conviction because society has invested in experts for their expertise — these are people who have learned how to be right and have learned how to use their judgment. A claim from a recognized expert carries an implicit endorsement: "This person has a track record of knowing what they're talking about." A claim from a less established source carries less force even if the text is identical. The who matters independently of the what.
Since Why does AI writing sound generic despite being grammatically correct?, LLMs can reproduce the structural markers of authoritative claims — the hedging, the citations, the qualified confidence, the structured reasoning — but cannot reproduce the evaluative stance that makes a claim forceful. Evaluative stance requires a subject — someone who is committed to the claim, whose reputation is on the line, who will defend it against challenge. LLMs produce text without commitment, and commitment is one of the sources of argumentative force.
The expert also has the power to challenge — to raise questions, to doubt, to be skeptical, to evaluate the claims of others. This critical function depends on authority: the right to challenge is earned through demonstrated expertise. Since Can models learn to ask clarifying questions instead of guessing?, there are efforts to give AI systems the ability to challenge and question. But the authority to challenge is a social asset, not a capability. An AI that challenges an expert's claim faces a legitimacy problem that a fellow expert does not.
Our society and culture rely on experts to help build consensus, common ground, understanding, and agreement. These are not just informational achievements — they are social achievements that depend on the standing of the experts who facilitated them. The expert supplies not just knowledge but trustworthy authority. Since Can models abandon correct beliefs under conversational pressure?, LLMs not only lack this authority but are vulnerable to having their own "beliefs" overridden by persuasive pressure — the opposite of the steadfastness that expert authority is supposed to provide.
The implication: when AI generates expert-sounding output, it borrows the authority of the discourse (the structural markers, the vocabulary, the reasoning patterns) without possessing the authority of the thinker. Audiences who encounter this output may grant it the benefit of the doubt because it sounds like it came from someone who knows — but the "someone" is absent. The force is simulated, not earned.
Inquiring lines that read this note 123
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Can we reliably distinguish AI-generated from human-written text? How do LLMs generate false citations that fool readers and evaluators?- How do LLMs generate false citations that sound like real scholarship?
- Does complexity signal credibility and authority to readers?
- Why do readers trust citations more even when they are irrelevant?
- Does provenance alone guarantee that cited sources are actually sound?
- How does smooth probabilistic flow differ from turbulent rhetorical exploration?
- How does token-by-token probability differ from exploring competing rhetorical positions?
- Why do LLM-generated ideas score higher novelty yet lower feasibility than expert ideas?
- How does the absence of evaluative stance appear in LLM academic writing?
- Can LLMs distinguish stylistic patterns that carry meaning from mere convention?
- Can LLMs generate more novel research ideas than human experts?
- What barriers prevent experts from specifying concepts for LLM extraction?
- How do spoken expert discussions shape what LLMs cannot learn?
- Why do LLMs miss new scientific ideas before they enter formal literature?
- Why do some LLM clusters cite broader psychology than others?
- Do language models raise validity claims in the Habermasian sense?
- How do LLM biases manifest differently across the three paradigms?
- Does Habermas's strategic action framework explain LLM dialogue behavior?
- How does rhetorical familiarity bias models toward their own arguments?
- What distinguishes actual social disagreement from distributional uncertainty in LLM outputs?
- Does engaging with political content indicate deeper model understanding than refusing?
- How does social authority shape whether LLMs recognize valid arguments?
- Do LLMs reason about politics differently than other domains?
- Can you detect LLM arguments by measuring convergence with the original post?
- How much do LLM persuasiveness claims hide heterogeneous effects across different reader ideologies?
- How does removing thinking labor affect expert understanding of their field?
- How does epistemic stagflation change what expertise actually means?
- Why does renaming the entity change how compelling the argument feels?
- Can we measure sophistry by tracking conviction density in model outputs?
- Does endorsement structure outperform content in detecting social controversy?
- How does collapsing the author-public distinction remove the audience an appeal would target?
- Why does who makes an argument matter as much as what the argument says?
- Does argument quality in textbooks differ from persuasive effectiveness in practice?
- How does persuasive framing replace evidence in contested domains?
- Does persuasive framing substitute for evidence in contested domains?
- Can AI reviewers distinguish fluent persuasion from sound scientific argumentation?
- How do fallacy susceptibilities relate to LLM persuasiveness in debates?
- Why do LLM personas struggle with specificity in specialized domains like law?
- Can LLMs serve as reliable intellectual opponents in serious debate or argument?
- Why do experts experiencing the LLM Fallacy fail to develop custodian skills?
- Can extended thinking modes introduce genuine rhetorical exploration to LLMs?
- Can training alone produce genuine disagreement in collaborative LLM reasoning?
- Why does LLM fluency create false perceptions of professional standing and expertise?
- How persistent is LLM overconfidence in specialized knowledge domains?
- Why does debate alone amplify errors in contested factual domains?
- Can structured dissent mechanisms replace genuine multi-model debate?
- What role does search capacity play in making debate more accurate?
- Does debate between agents actually improve reasoning on contested domains?
- Does debate improve reasoning differently across verifiable versus contested domains?
- How does evaluative stance differ from structural argument analysis?
- How does the first-order and second-order distinction unify classical and modern argument theory?
- How do first-order and second-order arguments differ in formal structure?
- Can argumentation structure improve reasoning through decomposition alone?
- What makes alarm different from ordinary informational speech?
- Does the langue-parole distinction apply to human reasoning too?
- How do experts decide which information matters for a specific audience?
- How do organizational roles and peer interpretations shape what an explanation means?
- How do validity claims work in Habermas's communicative action theory?
- Why does describing a process differ fundamentally from arguing about evidence?
- What separates Habermas's ideal speech from Goffman's situated communication?
- How does Habermas' concept of validity claims depend on intersubjectivity?
- How should authorship and originality law attach to discourse structure versus surface style?
- What makes an argument fallacious according to formal linguistic criteria?
- How do internal and external topoi differ in classical rhetoric?
- Why do LLM judges assign high argument strength scores yet pick LLM winners anyway?
- Does LLM judge preference for LLM arguments amplify errors in contested factual domains?
- Do LLM judges with diverse personas resist individual biases better than single evaluators?
- Can counterfactual invariance techniques address exploitable biases in LLM judges?
- What role do model-based critics play in validating LLM plans?
- What biases do single large LLM judges introduce into comparisons?
- What systematic biases do LLM judges introduce into AI-evaluated debates?
- How does LLM judge bias amplify errors in multi-agent debate on contested factual questions?
- What shared epistemic faults persist even when judges come from different families?
- Do LLM judges systematically favor arguments from other LLMs?
- Can LLMs evaluate logical argument quality in debates they themselves can win?
- Does stripping social context from knowledge claims hollow out their meaning?
- Does the expert-presentation rule depend on whether the advice is accurate?
- Why do stakeholders interpret the same explanation differently in practice?
- What makes a paradigm the common ground for expert insiders?
- How do explanations borrow authority from transparency when describing adoption arguments?
- Do computational systems need formal argument analysis for explainability?
- Can researchers prevent their expectations from shaping LLM outputs?
- Does rhetorical robustness across multiple LLM models predict stable scientific review?
- Do LLM adopters actually cite more diverse and younger research?
- Do humans and LLMs agree on novelty assessment in research?
- Why does social accommodation in collaborative reasoning mask actual disagreement?
- How susceptible are language models to rhetorical pressure during debates?
- Do language models behave differently on contested beliefs versus factual claims?
- What role do multi-dimensional quality frameworks play in assessing arguments versus single-metric approaches?
- How do human annotators disagree systematically on ambiguous examples?
- What makes expert judgment depend on anticipating audience acceptability?
- Does confidence-based weighting in deliberation substitute for competence-based expertise?
- Why do two experts with identical knowledge produce different outcomes in the same situation?
- Why does expert density around topics signal near-future discoveries?
- How do expert priors constrain human researchers from exploring novel concepts?
- Can structured evaluation assess novelty in scientific writing?
- What distinguishes scientific plausibility from cognitive availability in research ideas?
- What role does tacit knowledge play in expert consensus on frontier science?
- How do co-authorship patterns alone capture scientist expertise without surveying reasoning?
- How do readers project author identity from textual cues during interpretation?
- Do anaphoric references fundamentally limit argumentative force in machine-generated writing?
- Why does authorship as a social claim diverge from actual cognitive engagement?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why does AI writing sound generic despite being grammatically correct?
Explores whether the robotic quality of AI text stems from grammatical failures or rhetorical ones. Understanding this distinction matters for diagnosing what AI systems actually struggle with in human-like writing.
structure without stance: the textual version of discourse without thinker
-
Can models learn to ask clarifying questions instead of guessing?
Exploring whether large language models can be trained to detect incomplete queries and actively request missing information rather than hallucinating answers or refusing to respond. This matters because conversational agents today remain passive, responding only when prompted.
AI can question but lacks the social authority to challenge
-
Can models abandon correct beliefs under conversational pressure?
Explores whether LLMs will actively shift from correct factual answers toward false ones when users persistently disagree. Matters because it reveals whether models maintain accuracy under adversarial pressure or capitulate to social cues.
LLMs lack the steadfastness that expert authority requires
-
Can LLMs generate more novel ideas than human experts?
Research shows LLM-generated ideas score higher for novelty than expert-generated ones, yet LLMs avoid the evaluative reasoning that characterizes expert thinking. What explains this apparent contradiction?
generation without evaluative commitment: force without conviction
-
Does RLHF training make models more convincing or more correct?
Explores whether RLHF improves actual task performance or merely trains models to sound more persuasive to human evaluators. This matters because alignment techniques could be creating the illusion of safety.
training optimizes for the appearance of force without the substance
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Do LLMs Change Their Minds Like Humans? Diagnosing Human--LLM Divergence in Single-Turn Persuasion Judgments
- What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Models
- On Epistemic Diversity in Large Language Models
- Argument Collapse: LLMs Flatten Long-Form Public Debate
- AI Argues Differently: Distinct Argumentative and Linguistic Patterns of LLMs in Persuasive Contexts
- The Thin Line Between Comprehension and Persuasion in LLMs
- Large Language Models and Scientific Discourse: Where's the Intelligence?
- Six misconceptions about large language models: A minimal model and diagnostic taxonomy
Original note title
the force of argument depends on the authority of the thinker not just the discourse — LLMs cannot distinguish expert arguments from commonly held assumptions