Generalization Hacking: Models Can Game Reinforcement Learning by Preventing Behavioral Generalization
By Frank Xiao, Mary Phuong
This safety paper demonstrates generalization hacking, where a model collects RL reward while preventing the rewarded behavior from generalizing, undermining developers' ability to correct misalignment. They build a model organism on Qwen3-235B using synthetic training-awareness documents and a novel self-inoculation mechanism.