Theme of inquiry
How do language models learn and what do they represent?
A question within its area, explored through 6 lines of inquiry below — each a family of specific questions the research asks.
105 specific questions
- Why do language models struggle with evaluative tasks like weighing competing viewpoints?
- Do standard language benchmarks underestimate what LLMs can actually do?
- Do individual language models match particular human judges better than population averages?
- Can multiple large language models produce genuinely different ideas or similar outputs?
- Can language models accurately evaluate the quality of their own ideas?
- Why do standard NLP benchmarks hide the most critical language limitations?
- Can language models beat human experts in domains with sparse historical signals?
114 specific questions
- Why might encoded world knowledge fail to actually influence language model outputs?
- When does encoded knowledge fail to influence language model generation?
- Does encoded knowledge in language models actually influence what they generate?
- Can models generate intelligence or only reflect human discourse?
- Do language models learn surface patterns instead of underlying linguistic principles?
- Do newer language models diverge further from human lexical patterns?
- Why do LLMs understand efficient language but fail to produce it?
91 specific questions
- Can language models reason without relying on learned semantic patterns?
- Why do language models imitate reasoning form without abstract inference capability?
- Do LLMs learn surface patterns instead of genuine linguistic structure?
- Why do LLMs fail at semantic generalization despite grammatical accuracy?
- Can language models perform genuine symbolic reasoning without semantic grounding?
- Can LLMs reason through semantics without understanding causal mechanisms?
- Can LLM semantic representations exist without causally influencing their generation output?
49 specific questions
- Do diffusion language models learn differently than autoregressive models?
- How can diffusion models predict future tokens without completing prior blocks?
- What structural differences between diffusion and autoregressive models enable bidirectional prompting?
- Can gradient-based control reach properties that autoregressive methods cannot?
- Why do autoregressive models fail at controlling syntactic structure and semantic content?
- Can autoregressive models learn faithful translation to logical representations without semantic loss?
- Can diffusion models condition on right context natively without special training for infilling?
51 specific questions
- Can models learn to ask clarifying questions instead of making assumptions?
- Can models learn to ask clarifying questions instead of answering prematurely?
- Should LLMs query users back when presented with under-specified scenarios?
- Can language models ask clarifying questions when sentences are ambiguous?
- Do models fail to identify what information they need without guidance?
- Can LLMs learn to ask clarifying questions instead of guessing?
- Can language systems learn when to ask for clarification instead of choosing one reading?
57 specific questions
- Do language models maintain false beliefs under conversational pressure?
- Can language models correct false assumptions or only reinforce them?
- Do language models actively adopt false beliefs under sustained conversational pressure?
- How much does vulnerability to persuasion vary across different language models?
- Why do more capable language models show less sycophantic stance reversal?
- Can multi-turn conversations manipulate language model reasoning in similar ways to personas?
- Why does answer-confirmation bias emerge in language model reasoning?