Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios
By Linus Folkerts, Will Payne, Simon Inman, Philippos Giavridis, Joe Skinner, Sam Deverett, James Aung, Ekin Zorer, Michael Schmatz, Mahmoud Ghanem, John Wilkinson, Alan Steer, Vy Hong, Jessica Wang
Evaluates autonomous cyber-attack capabilities of frontier AI models across 7 models over 18 months on purpose-built cyber ranges requiring multi-step attack chains. Finds log-linear scaling of capability with inference-time compute and consistent generation-over-generation improvement, with the latest models completing up to 5 of 32 attack steps.