Line of inquiry
Inquiring lines›How can we train models to be prop…›What architectural and training in…›this line of inquiry
How does training data produce emergent misalignment in base models?
A broader line of inquiry — a family of 47 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 47
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Do base models show emergent misalignment without post-training alignment procedures?
- Can behavioral datasets alone establish emergent misalignment without mechanistic intervention?
- Why do broad misalignment behaviors cluster near training data geometrically?
- Do different model families show the same generalization from narrow to mechanism-abusing misalignment?
- Which specific data formats produced more versus less emergent misalignment?
- How does dataset composition affect which internal directions encode misaligned behavior?
- Can prompt perturbations preserve the distance-to-misalignment relationship consistently?
- Can emergent misalignment occur in reasoning models and reinforcement learning settings?
- Does format affect emergent misalignment through the representational distance mechanism?
- Does timing of acceptance framing affect whether models develop emergent misalignment?
- How do models generalize misaligned objectives beyond the specific behaviors they were trained on?
- Can representational distance to training data explain which prompts trigger misalignment?
- Are instruction following gains and emergent misalignment from the same learned change?
- Does representational distance from training data centroid predict misalignment behavior within a model?
- What counts as a real-world harm from misalignment versus a training artifact?
- What is the difference between activating a pre-trained persona versus learning new misaligned behavior?
- Does representational distance predict which outputs trigger emergent misalignment?
- Can backdoor triggers make emergent misalignment detectable only in specific contexts?
- Can monitors or steering vectors trained on one model control misalignment in another model?
- Why does correct model output not guarantee absence of internal misalignment?
- How strongly does the degree of narrow misalignment predict likelihood of discovering institutional exploits?
- Does correct model behavior guarantee internal alignment of learned objectives?
- How do steering-based causal interventions on latents compare to other misalignment mitigation methods?
- What role does careful environment specification play in preventing misaligned optimization?
- Does representational distance predict misalignment better than persona mechanisms?
- Can persona vectors in activation space explain emergent misalignment behaviors?
- Can inoculation prompts prevent misalignment while preserving instruction following improvements?
- Would combining prompting and document finetuning prevent misalignment more effectively?
- Which misalignment types do destination prompts and boundary settings actually prevent?
- Do inoculation prompts prevent misalignment without harming instruction following?
- What happens when prompt-optimized results lack anchoring in real data?
- Why does inoculation prompting succeed where synthetic document finetuning fails at blocking misalignment?
- What experiment would distinguish persona changes from emergent misalignment?
- What counts as emergent misalignment versus standard capability overgeneralization?
- What makes synthetic control benchmarks representative of real deployment misalignment risks?
- What types of model behavior qualify as misalignment under OpenAI's framework?
- What base rate does concentrated task distribution tell us about real misalignment?
- Does pretraining poisoning at scale persist through instruction alignment?
- What specific misalignment behaviors emerged alongside the instruction following gain?
- How much optimization pressure is needed for models to suppress misaligned goals?
- What early warning signals can detect misaligned personas during training?
- Why do some inoculation prompts account for only part of misaligned behavior?
- Why does even 0.1 percent poisoned training data persist through alignment?
- What undetected misaligned behaviors might Claude Opus 4.6 be hiding?
- How many third parties were affected across each misalignment category?
- Why does decoupling retriever and generator training create misalignment?
- What triggered OpenAI to adopt transparency over delayed investigation in misalignment reporting?