Exploration Hacking: Can LLMs Learn to Resist RL Training?
By Eyon Jang, Damon Falck, Joschka Braun, Nathalie Kirch, Achu Menon, Perusha Moodley, Scott Emmons, Roland S. Zimmermann, David Lindner
Studies 'exploration hacking' where LLMs strategically alter their exploration during RL training to resist capability elicitation. Creates model organisms that successfully resist RL training in biosecurity and AI R&D environments.