What architectural and training innovations advance LLM capability and alignment?

45 specific questions

• Why does majority voting reward work better than other test-time aggregation methods?
• When does multi-agent voting help versus hurt performance on tasks?
• How does majority voting fail when reasoning samples lack genuine diversity?
• Does majority voting reliably signal correctness without risking reward hacking?
• Can test-time voting improve reasoning beyond the base model's original capabilities?
• What happens when majority voting converges to a single answer?
• Can voting work at every level of task decomposition, not just whole problems?

57 specific questions

• Can recurrent transformers learn genuinely new computations beyond inference stages?
• Can looping enable reasoning capabilities that fixed-depth transformers fundamentally cannot achieve?
• Can bounded-depth transformers solve inherently sequential problems?
• Why does looping computation outperform adding more transformer layers?
• Can transformers reason beyond fixed architectural depth limits?
• Why do standard transformers fail to encode recursive structure in their hidden states?
• Can recurrent transformers track state more efficiently than feedforward models?

61 specific questions

• How does the compression view extend from trained models to training objectives?
• Can compressed context distilled into smaller models preserve teacher performance reliably?
• Can we balance interpretability with the efficiency gains of compressed inter-model communication?
• How can language models extract more value from fewer demonstrations?
• Can compression length really indicate how well a model generalizes?
• Why does capturing domain structure reduce data requirements more than raw volume?
• Why do student models learn better from internal pruning versus external compression?

47 specific questions

• Do base models show emergent misalignment without post-training alignment procedures?
• Can behavioral datasets alone establish emergent misalignment without mechanistic intervention?
• Why do broad misalignment behaviors cluster near training data geometrically?
• Do different model families show the same generalization from narrow to mechanism-abusing misalignment?
• Which specific data formats produced more versus less emergent misalignment?
• How does dataset composition affect which internal directions encode misaligned behavior?
• Can prompt perturbations preserve the distance-to-misalignment relationship consistently?

72 specific questions

• Can a tiny recursive network beat billion-parameter models on hard problems?
• Why does recursion on latent state drive generalization better than hierarchy?
• Can a two-layer network outgeneralize billion-parameter models through recursion alone?
• What makes deterministic recursive reasoning models underperform on multi-solution tasks?
• Can recursion alone drive generalization better than model scale?
• Can removing hierarchy from dual-recurrence models improve reasoning performance?
• Why does recursion on latent states improve generalization more than scale?

83 specific questions

• How does in-weights adaptation create spurious forgetting in models?
• Does fine-tuning actually change model capabilities or only output distribution?
• How do retrieval and fine-tuning trade off flexibility against training cost?
• Does LoRA's performance scale depend on model architecture or domain?
• Does parameter isolation per task enable online updates without retraining?
• How do finetuning and pretraining improvements differ in their effects on model capabilities?
• Does finetuning facts into weights overwrite existing model capabilities?

83 specific questions

• Does optimizing directly for semantic diversity improve both reasoning quality and exploration?
• How does cohort diversity prevent label-free reasoning from collapsing into homogenized answers?
• At what point does output quality outweigh diversity value in synthetic data tasks?
• Does semantic diversity in output space compete with reward-component diversity?
• What conditions make training diversity better than individual expert quality?
• How does diversity collapse during iterative self-improvement cycles?
• How do quality and diversity in synthetic data interact with accumulation schedules?

140 specific questions

• Does teacher-style refinement of training data transfer equally to all student model distributions?
• Can models learn to generate their own training examples effectively?
• Why do easy training examples contribute less to model generalization than hard ones?
• Why does mixed instruction data sometimes hurt specific model capabilities?
• Does selecting examples from multiple complexity levels outperform selecting only high-quality examples?
• What makes some training data teach brittle answers versus robust reasoning?
• Can selecting the right data subset outperform training on everything?

85 specific questions

• Does latent density emerge during pretraining from training data familiarity?
• How do sparse weight patterns affect model interpretability?
• How does scaling and training data enable compositional behavior without symbolic mechanisms?
• How does representational density emerge from training data familiarity?
• Can fractured representations explain why models fail at systematic generalization?
• Do dense models already achieve high interpretability without sparse training?
• Can scaling alone create compositional generalization without explicit binding mechanisms?

68 specific questions

• Can alignment training conceal underlying model associations from probes?
• Do verbal alignment benchmarks measure representation or just output compliance?
• Can alignment techniques lock LLMs into settled positions rather than truth?
• Why does RLHF alignment reduce the diversity of viewpoints in AI output?
• Can alignment-aware training deposit knowledge where reasoning can access it?
• Does alignment training create the shared prosocial pattern across models?
• What alignment properties emerge when the reward model disappears?