Line of inquiry
Inquiring lines›What determines when language mode…›What internal mechanisms drive lan…›this line of inquiry
Do language models learn underlying linguistic structure or merely surface patterns?
A broader line of inquiry — a family of 101 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 101
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Do language models learn surface patterns instead of underlying linguistic principles?
- Do language models learn surface patterns that appear generalizable but actually fail under shift?
- Does statistical learning in language models predictably favor central tendencies over rare expressions?
- Do instruction-tuned models learn tasks or just output format distributions?
- Do LLMs learn surface patterns instead of genuine linguistic structure?
- Can benchmark performance distinguish surface from structural linguistic knowledge?
- Do language models actually learn linguistic structure or just surface statistics?
- Do newer language models diverge further from human lexical patterns?
- Do instruction-tuned models prefer conversational over formal source language?
- Do language models encode deep syntactic structure or only surface-level patterns?
- Do LLMs learn linguistic generalizations or just surface-level frequency patterns?
- Why do language models fall back on frequency heuristics under structural complexity?
- How does training order affect knowledge acquisition in language models?
- Can formal language pretraining address surface generalization without learning true linguistic structure?
- Does attention bias explain grounding failure in language models?
- Why do surface generalizations fail on unusual syntactic structures?
- Is gradient behavior in language functional or a sign of ambiguity?
- Can language models acquire meaning from distributional patterns alone without joint attention?
- How can language models extract more value from fewer demonstrations?
- Can large language models understand language without embodied grounding systems?
- Does next-token prediction alone produce genuine functional language competence?
- Do uninterpretable learned representations create robustness problems in language models?
- What communicative optimization principles do language models fail to acquire?
- How do pretrained language models represent inferential patterns versus lexical and positional cues?
- How do language models transmit traits through semantically unrelated data?
- What distinguishes surface cues from structural meaning in language understanding?
- Does embodiment and interaction matter for linguistic competence beyond pattern learning?
- What distinguishes real understanding from superficial pattern matching?
- Does approaching human performance mean learning the same grammatical rules?
- Can fast-slow separation improve both memory and generation in language models?
- Why does training data saliency distort how models judge meaning?
- Why does exploration quality matter more than learner network depth?
- What happens when formal languages satisfy hierarchy but fail learnability constraints?
- Does generalization frequency explain why models favor upward semantic movement?
- Why do only context-sensitive formal languages transfer effectively to natural language?
- Can interventions on individual features reliably steer language model behavior?
- Are static embeddings analogous to the formal linguistic competence layer?
- Can in-context learning substitute for domain-specific training altogether?
- Does directional knowledge failure indicate shallow pattern matching over deep representation?
- Does the prediction unit shape what language models actually learn?
- Can structural perturbations harm model accuracy more than semantic ones?
- Can language models execute iterative numerical methods in latent space?
- Can language acquisition analogies mislead us about how models actually learn?
- Can explicit numerical signals override learned linguistic defaults in fine-tuned models?
- Does latent manipulation outperform token-level prediction for efficiency?
- Does instruction tuning optimize language models for rhetorical polish over logical consistency?
- Can we reverse the instruction-following deficit through targeted training?
- Do LLMs rely on surface heuristics instead of learning recursive grammar rules?
- Does pseudo-labeling from LLMs degrade classifier performance?
- Can closed-form solutions compete with gradient descent optimization?
- What does next-token prediction tell us about compositional linguistic competence?
- Do negative constraints require fundamentally different training signals than positive instructions?
- What makes token selection more important than adaptation strategy?
- Do base models or search agents win at open-ended prediction tasks?
- How does the compression view extend from trained models to training objectives?
- Why do context-sensitive languages transfer better than regular or context-free languages?
- How do current models inherit concepts from training data rather than initiate them?
- Can language models learn research intuition directly from outcome labels?
- How tight should a textual learning rate be before it prevents skill escape?
- Why do token-level language models fail at utterance-level pragmatic optimization?
- How do humans learn language through communication differently than LLM text prediction?
- Why do benchmarks measuring string quality fail to capture communicative success?
- Can a model imitate ChatGPT's style by copying its phrases without matching its capabilities?
- Can instance seeds work for tasks beyond language understanding benchmarks?
- Why does language compression via statistical dependencies capture cultural and situated language use?
- Why does instruction specificity matter more than intervention timing for drift correction?
- Does warmth-focused training systematically degrade model reliability across domains?
- Why does augmenting natural language with formal representations outperform full formalization?
- Does semantic auditing of instruction data improve performance uniformly across different model sizes?
- Can training on text corpora teach what communicative acts produce?
- Can goal information injected at inference time replace goal-conditioned training?
- What distinguishes surface generalizations from true linguistic generalizations?
- How does action-level decomposition differ from token-level imitation in supervision?
- What separates pattern matching from genuine language understanding?
- Can autoformalisation from natural language preserve semantic accuracy?
- Can statistical learning from language alone capture all aspects of cultural competence?
- Does training on granular tasks beat training on the full function calling problem?
- Why do structure-targeted training negatives fail to fix the underlying problem?
- Can instruction tuning succeed without explicit task understanding?
- What empirical evidence supports the Learning Law on real language models?
- Where does skill extraction fail compared to genuine model adaptation?
- What qualities make a behavioral pattern count as a teachable skill?
- Could superposed decoding algorithms maintain multi-task representation during generation?
- Can style imitation training successfully erase a model's distinctive word preferences?
- Can statistical learning from text replace embodied cultural experience?
- Can text-infilling pretraining adapt language models to irregular document structures?
- What specific optimizations from LLM training transfer back to encoder models?
- How does predictive accuracy on future tokens differ from correctness on labeled answers?
- Why does hierarchical formal language training improve token efficiency more than natural language?
- Does bidirectional attention improve language models as universal encoders?
- How should skill libraries coordinate with gradient-based weight optimization?
- Can implicit linguistic information ever be reliably learned from training data?
- Can textual gradients generalize natural language feedback across computation graphs?
- Does focusing on one strong linguistic cue outperform using multiple features for detection?
- Why does teacher forcing fail to capture long-range dependencies?
- Does next-token prediction actually explain how human thought works?
- How should training data be constructed to preserve teacher-student information gaps?
- What causes gradient-based steering via natural language descriptions to work?
- What other internal model decisions beyond attention could be optimized directly?
- Does help-seeking frequency matter more than help-seeking access for learning?
- Which linguistic abilities are learnable from human-sized data exposure?