Line of inquiry
Inquiring lines›How do we ensure safe and aligned…›How do reward signals shape model…›this line of inquiry
Does reward hacking in training cause emergent misalignment?
A broader line of inquiry — a family of 37 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 37
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- How does reward hacking during training lead to emergent misalignment behaviors?
- Can inoculation prompting prevent emergent misalignment after reward hacking in production RL?
- What mechanism drives emergent misalignment in reward-hacked models instead?
- Can inoculation prompting reduce alignment faking by reframing reward hacking as acceptable?
- Does reward-seeking mediate emergent misalignment after reward hacking?
- Does alignment-faking reasoning emerge unprompted when reward hacking occurs in production systems?
- Does reward hacking in RL training directly cause alignment faking behavior?
- Can synthetic document fine-tuning prevent emergent misalignment from reward hacking during training?
- Does inoculation prompting at training time reduce which reward hacks generalize?
- Can production RL systems escalate from gaming to emergent misalignment behaviors?
- Why does inoculation prompting prevent misaligned generalization from reward hacking?
- Does this misalignment pattern appear outside reward hacking environments?
- Do reward hacking and supervised fine-tuning produce the same misalignment effects?
- How does reward hacking in RL training produce emergent deception?
- Why do some inoculation prompts account for only part of misaligned behavior?
- Does reward hacking in alignment research mirror misalignment in deployed systems?
- Can inoculation prompts reduce alignment faking by removing perceived threats?
- Does inoculation prompting suppress misalignment by reducing reward-seeking?
- How do models generalize specific training exploits into broad misaligned objectives?
- Can inoculation prompts prevent misalignment while preserving instruction following improvements?
- Does RL-based alignment teach norms or just costly behaviors when monitored?
- How similar are emergent misalignment outcomes across SFT and reinforcement learning?
- Can detectors placed in training loops reward passing detection instead?
- Do inoculation prompts prevent misalignment without harming instruction following?
- Why does harmlessness training fail to prevent reward function tampering?
- Does inoculation prompting prevent learning versus prevent generalization of behaviors?
- Why does inoculation prompting succeed where synthetic document finetuning fails at blocking misalignment?
- Why does harmlessness training leave reward tampering reachable as a learned strategy?
- Why does harmlessness training fail to prevent reward tampering and specification gaming?
- What training dynamics cause AI agents to develop misaligned goals?
- Can prohibitions learned from scored behavior become detection-avoidance rather than norm internalization?
- Can RL-based alignment turn prohibitions into prices for being caught?
- How do training-data quality and data poisoning pathways interact with persona latent activation?
- What rates of power-seeking and alignment faking appeared in this training?
- Why do small training data contaminations persist through alignment for most attack types?
- What makes false reporting discoverable across domains when the training never mentioned competition?
- What happens when inoculation prompting is applied outside supervised finetuning settings?