Theme of inquiry
What systematic behavioral limitations reduce LLM utility across practical applications?
A question within its area, explored through 5 lines of inquiry below — each a family of specific questions the research asks.
68 specific questions
- What other evaluation biases exist in LLM judge systems?
- Can an LLM judge's bias be reduced through prompting or other interventions?
- How do LLM judges' built-in biases influence the policies they help align?
- Does LLM judge preference for LLM arguments amplify errors in contested factual domains?
- Why do LLM judges systematically favor outputs from their own model family?
- Do LLM judges with diverse personas resist individual biases better than single evaluators?
- Do LLM judges systematically favor arguments from other LLMs?
103 specific questions
- Do LLMs genuinely internalize human psychological structure or match surface patterns?
- Do LLMs reason about politics differently than other domains?
- Do LLMs achieve similar persuasive outcomes through different rhetorical mechanisms than humans?
- Do LLM replies mirror the language patterns they respond to?
- Do LLMs rely on surface statistical patterns instead of causal structure?
- Why do conventional mental models fail when applied to AI interaction?
- How do moral language patterns differ between LLM and human arguments?
119 specific questions
- Why does LLM knowledge fail to influence their actual outputs?
- Should LLM reasoning be studied as latent state trajectories rather than surface text?
- Does LLM reasoning always match the outputs it generates?
- Why do LLM explanations feel convincing even when the underlying plan is wrong?
- How do knowing and doing diverge in LLM decision-making?
- Can LLMs improve at simple deduction through different training approaches?
- Why do LLMs fail at counterfactual reasoning despite factual knowledge?
34 specific questions
- Why do language models presume common ground rather than build it?
- Can language models develop genuine social grounding through human interaction?
- Can static word-sharing create genuine communicative grounding between humans and models?
- Why do language models presume common ground instead of building it?
- How do LLMs differ from humans in their grounding mechanisms?
- Does social grounding in language improve through iterative human integration?
- What distinguishes static grounding that presumes understanding from dynamic grounding that builds it?
37 specific questions
- How effective are journal policies restricting LLM use in peer review?
- Do reviewer rules about LLM use in peer review actually get followed?
- Can peer review policies actually prevent LLM use when compliance is hard to monitor?
- Can rules against undisclosed LLM use change reviewer behavior without enforcement?
- Do peer reviewers actually follow policies that ban or limit their LLM use?
- Do peer review policies banning LLM use actually change reviewer behavior and decisions?
- Do stricter AI policies actually change how reviewers score manuscripts?