SPADE: Self-Play in Adaptive Synthetic Executable Environments
By Bo Liu, Simon Yu, Yiding Jiang, Ao Qu, Andrew Zhao, Zichen Liu, Junsu Kim, Zijian Zhou, Seungone Kim, Tongzheng Ren, Mickel Liu, Hanfei Yu, Zhaorun Chen, Weiyan Shi, Paul Pu Liang, Luke Zettlemoyer, Yejin Choi, Natasha Jaques
As first covered in Research yesterday, SPADE is a self-play reinforcement learning framework in which a language model both designs adaptive executable training environments and learns to solve them, using regret-based targeting to concentrate environment difficulty. The approach targets reasoning and tool-use gains in a Gym-like setup.