MEMENTO: Teaching LLMs to Manage Their Own Context
By Vasilis Kontonis, Yuchen Zeng, Shivam Garg, Lingjiao Chen, Hao Tang, Ziyan Wang, Ahmed Awadallah, Eric Horvitz, John Langford, Dimitris Papailiopoulos
MEMENTO teaches reasoning models to segment their thinking into blocks, compress each into dense summaries (mementos), and reason forward attending only to these summaries. Releases OpenMementos dataset of 228K annotated reasoning traces. Works across Qwen3, Phi-4, Olmo 3 at 8B-32B scale.