What linguistic patterns explain LLM capabilities and failures?

93 specific questions

• What prevents monolithic LLMs from coordinating decomposition with execution?
• Can LLMs reliably generate novel working architectures without structured representations?
• Can tool use or self-conditioning fix degradation in extended LLM workflows?
• Does constraint-setting before generation change what LLMs can contribute?
• Why do generic LLMs produce safe, milquetoast outputs instead of opinionated ones?
• Why do LLMs fail at directly solving stochastic control problems?
• Can tool use or self-conditioning fix long-horizon delegation drift in LLMs?

90 specific questions

• Can LLMs reliably assess the quality of ideas they generate?
• Why do LLMs generate novel ideas but struggle to evaluate them?
• Can readers reliably distinguish LLM-generated research writing from human writing?
• Why do LLMs excel at generation but struggle with evaluation?
• What structural differences between human and LLM production create detectable signatures?
• Can LLMs generate more novel research ideas than human experts?
• What structural barriers prevent LLMs from making evaluative judgments about writing?

95 specific questions

• Why does LLM knowledge fail to influence their actual outputs?
• Why do LLM explanations feel convincing even when the underlying plan is wrong?
• Can LLMs learn to ask clarifying questions instead of guessing?
• Should LLMs query users back when presented with under-specified scenarios?
• Why do LLM outputs match researcher priors without solving tasks correctly?
• What contexts beyond question answering might show different LLM explanation effects?
• Can LLMs give correct answers without making those answers understandable to users?

72 specific questions

• Do LLMs rely on surface statistical patterns instead of causal structure?
• Can statistical correlations in LLMs function equivalently to human causal understanding?
• Can language models develop world models that ground meaning in causal reality?
• What data presentation structures enable LLMs to learn decision-making from examples?
• Do LLMs need world models to make accurate predictions?
• What happens when we treat LLM outputs as sampled rather than stored?
• Does this optimism bias contribute to the knowing-doing gap in LLM decision-making?

37 specific questions

• What distinguishes static grounding that presumes understanding from dynamic grounding that builds it?
• Why do language models presume common ground rather than build it?
• Can static word-sharing create genuine communicative grounding between humans and models?
• Why do language models presume common ground instead of building it?
• How do LLMs differ from humans in their grounding mechanisms?
• Can language models develop genuine social grounding through human interaction?
• Can LLMs build shared understanding through dynamic grounding rather than presuming it?

79 specific questions

• Do language models behave differently on contested beliefs versus factual claims?
• Do language models raise validity claims in the Habermasian sense?
• How susceptible are language models to rhetorical pressure during debates?
• Do language models share the same cooperative truth-seeking rules as humans?
• Do language models show the same truth bias as humans?
• Why does social accommodation in collaborative reasoning mask actual disagreement?
• Does confidence in LLM outputs systematically outweigh accuracy across languages and contexts?

62 specific questions

• What other evaluation biases exist in LLM judge systems?
• Can an LLM judge's bias be reduced through prompting or other interventions?
• How do LLM judges' built-in biases influence the policies they help align?
• Do LLM judges with diverse personas resist individual biases better than single evaluators?
• Does LLM judge preference for LLM arguments amplify errors in contested factual domains?
• Can parallel evaluation reduce position and length bias in LLM judging?
• What calibration corrections can reduce LLM judge bias in automated evaluation pipelines?

83 specific questions

• Does structured decomposition improve LLM reasoning in other compound tasks?
• Can reasoning chains work without logical validity?
• Why does chain-of-thought reasoning alone not fix LLM performance with users?
• Why does augmenting symbolic reasoning outperform replacing it entirely?
• Should LLM reasoning be studied as latent state trajectories rather than surface text?
• Does LLM reasoning always match the outputs it generates?
• Can language models reason without relying on learned semantic patterns?

30 specific questions

• Do reviewer rules about LLM use in peer review actually get followed?
• How effective are journal policies restricting LLM use in peer review?
• Do peer reviewers actually follow policies that ban or limit their LLM use?
• Can peer review policies actually prevent LLM use when compliance is hard to monitor?
• Can rules against undisclosed LLM use change reviewer behavior without enforcement?
• Do peer review policies banning LLM use actually change reviewer behavior and decisions?
• Do stricter AI policies actually change how reviewers score manuscripts?

76 specific questions

• Do LLMs genuinely internalize human psychological structure or match surface patterns?
• Why do conventional mental models fail when applied to AI interaction?
• Can non-phenomenal mental states like belief apply to LLMs functionally?
• Can LLMs participate meaningfully in discourse without consciousness or understanding?
• Does internal anomaly detection in LLMs indicate genuine self-awareness beyond role-play?
• Can behavioral self-awareness in LLMs extend to recognizing their own contradictions?
• Do realistic LLM behaviors require simulating human thought or just behavior?