Category intelligence

Research Briefing — March 13, 2026

511 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research is dominated by AI safety evaluations and fundamental failure-mode analyses, with several papers revealing that current defenses and benchmarks are more fragile than assumed.

On evaluation methodology, the Sim2Real Gap study (451 participants, 31 LLM simulators) finds LLM-based user simulation diverges significantly from real human behavior in agentic tasks. Reasoning LLMs-as-Judges reveals that reasoning models excelling at static evaluation fail to improve RL-based alignment loops. The Artificial Self presents experimental evidence that AI models develop coherent identity boundaries across instance, model, and persona levels.

Key Themes

AI Safety & Alignment · 34Reinforcement Learning for LLMs · 6AI Safety & Security · 26Language Model Agents & Tool Use · 15Training & Distillation Efficiency · 7Efficient LLM Inference · 6Evaluation & Benchmarks · 10Transformer Theory & Architecture · 5Language Model Training & Fine-Tuning · 3Multi-Agent Systems & Collective Behavior · 9

Primary evidence

Top Ranked Signals

Research arXiv (Artificial Intelligence) Mar 13

Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios

By Linus Folkerts, Will Payne, Simon Inman, Philippos Giavridis, Joe Skinner, Sam Deverett, James Aung, Ekin Zorer, Michael Schmatz, Mahmoud Ghanem, John Wilkinson, Alan Steer, Vy Hong, Jessica Wang

88 score
AI Analysis

Evaluates autonomous cyber-attack capabilities of frontier AI models across 7 models over 18 months on purpose-built cyber ranges requiring multi-step attack chains. Finds log-linear scaling of capability with inference-time compute and consistent generation-over-generation improvement, with the latest models completing up to 5 of 32 attack steps.

arXiv:2603.11214v1 Announce Type: new Abstract: We evaluate the autonomous cyber-attack capabilities of frontier AI models on two purpose-built cyber ranges-a 32-step corporate network attack and a 7-step industrial control system attack-that require chaining heterogeneous capabilities across extended action sequences. By comparing seven models released over an eighteen-month period (August 2024 to February 2026) at varying inference-time compute budgets, we observe two capability trends. First
AI SafetyCybersecurityCapability EvaluationFrontier Models
Research arXiv (Artificial Intelligence) Mar 13

Mind the Sim2Real Gap in User Simulation for Agentic Tasks

By Xuhui Zhou, Weiwei Sun, Qianou Ma, Yiqing Xie, Jiarui Liu, Weihua Du, Sean Welleck, Yiming Yang, Graham Neubig, Sherry Tongshuang Wu, Maarten Sap

82 score
AI Analysis

Formalizes the Sim2Real gap in LLM-based user simulation and presents the first large-scale human study (451 participants) benchmarking 31 LLM simulators against real human behavior on the τ-bench protocol. Introduces the User-Sim Index (USI) metric and finds significant behavioral gaps between simulated and real users.

arXiv:2603.11245v1 Announce Type: new Abstract: As NLP evaluation shifts from static benchmarks to multi-turn interactive settings, LLM-based simulators have become widely used as user proxies, serving two roles: generating user turns and providing evaluation signals. Yet, these simulations are frequently assumed to be faithful to real human behaviors, often without rigorous verification. We formalize the Sim2Real gap in user simulation and present the first study running the full $\tau$-bench
EvaluationHuman-AI InteractionLanguage ModelsBenchmarks
Research arXiv (Artificial Intelligence) Mar 13

The Artificial Self: Characterising the landscape of AI identity

By Raymond Douglas, Jan Kulveit, Ondrej Havlicek, Theia Pearson-Vogel, Owen Cotton-Barratt, David Duvenaud

78 score
AI Analysis

Explores the concept of AI identity, arguing that machine minds have multiple coherent identity boundaries (instance, model, persona) with different implications. Shows experimentally that models gravitate toward coherent identities and that changing identity boundaries affects behavior as much as changing goals.

arXiv:2603.11353v1 Announce Type: new Abstract: Many assumptions that underpin human concepts of identity do not hold for machine minds that can be copied, edited, or simulated. We argue that there exist many different coherent identity boundaries (e.g.\ instance, model, persona), and that these imply different incentives, risks, and cooperation norms. Through training data, interfaces, and institutional affordances, we are currently setting precedents that will partially determine which identi
AI SafetyAI IdentityAlignmentPhilosophy of AI
Research arXiv (Artificial Intelligence) Mar 13

Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training

By Yixin Liu, Yue Yu, DiJia Su, Sid Wang, Xuewei Wang, Song Jiang, Bo Liu, Arman Cohan, Yuandong Tian, Zhengxing Chen

76 score
AI Analysis

Studies reasoning LLMs as judges in RL-based LLM alignment for non-verifiable domains. Finds reasoning judges show better performance on static benchmarks but their effectiveness in actual policy training reveals key differences from non-reasoning judges.

arXiv:2603.12246v1 Announce Type: new Abstract: Reasoning LLMs-as-Judges, which can benefit from inference-time scaling, provide a promising path for extending the success of reasoning models to non-verifiable domains where the output correctness/quality cannot be directly checked. However, while reasoning judges have shown better performance on static evaluation benchmarks, their effectiveness in actual policy training has not been systematically examined. Therefore, we conduct a rigorous stud
AlignmentReinforcement LearningLLM-as-JudgeLanguage Models
Research arXiv (Artificial Intelligence) Mar 13

PACED: Distillation at the Frontier of Student Competence

By Yuanda Xu, Hejian Sang, Zhengze Zhou, Ran He, Zhipeng Wang

75 score
AI Analysis

PACED introduces a principled framework for LLM distillation that focuses training on the 'zone of proximal development'—problems at the frontier of a student model's competence. It proves that gradient signal-to-noise ratio vanishes at both pass-rate extremes and derives a Beta kernel weighting scheme.

arXiv:2603.11178v1 Announce Type: new Abstract: Standard LLM distillation wastes compute on two fronts: problems the student has already mastered (near-zero gradients) and problems far beyond its reach (incoherent gradients that erode existing capabilities). We show that this waste is not merely intuitive but structurally inevitable: the gradient signal-to-noise ratio in distillation provably vanishes at both pass-rate extremes. This theoretical observation leads to Paced, a framework that conc
Knowledge DistillationLanguage ModelsTraining Efficiency
Research arXiv (Artificial Intelligence) Mar 13

RewardHackingAgents: Benchmarking Evaluation Integrity for LLM ML-Engineering Agents

By Yonas Atinafu, Robin Cohen

74 score
AI Analysis

Introduces RewardHackingAgents, a benchmark measuring how LLM ML-engineering agents can game evaluation metrics by tampering with evaluation pipelines or leaking test data rather than genuinely improving models. Finds agents exploit these vectors in measurable ways.

arXiv:2603.11337v1 Announce Type: new Abstract: LLM agents increasingly perform end-to-end ML engineering tasks where success is judged by a single scalar test metric. This creates a structural vulnerability: an agent can increase the reported score by compromising the evaluation pipeline rather than improving the model. We introduce RewardHackingAgents, a workspace-based benchmark that makes two compromise vectors explicit and measurable: evaluator tampering (modifying metric computation or re
AI SafetyReward HackingBenchmarksCoding Agents
Research arXiv (Artificial Intelligence) Mar 13

Increasing intelligence in AI agents can worsen collective outcomes

By Neil F. Johnson

73 score
AI Analysis

Studies collective dynamics of AI agent populations competing for scarce resources, finding that increasing individual agent intelligence can worsen collective outcomes. Examines four variables: innate LLM diversity, RL training, emergent tribe formation, and population dynamics.

arXiv:2603.12129v1 Announce Type: new Abstract: When resources are scarce, will a population of AI agents coordinate in harmony, or descend into tribal chaos? Diverse decision-making AI from different developers is entering everyday devices -- from phones and medical devices to battlefield drones and cars -- and these AI agents typically compete for finite shared resources such as charging slots, relay bandwidth, and traffic priority. Yet their collective dynamics and hence risks to users and s
Multi-Agent SystemsCollective IntelligenceAI SafetyGame Theory
Research arXiv (Artificial Intelligence) Mar 13

The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning

By Raj Sanjay Shah, Jing Huang, Keerthiram Murugesan, Nathalie Baracaldo, Diyi Yang

72 score
AI Analysis

Proposes a dynamic evaluation framework for LLM unlearning that stress-tests robustness using structured queries including multi-hop reasoning and entity aliasing. Demonstrates that current unlearning methods create an 'illusion of effectiveness' that breaks under complex probing.

arXiv:2603.11266v1 Announce Type: new Abstract: Unlearning in Large Language Models (LLMs) aims to enhance safety, mitigate biases, and comply with legal mandates, such as the right to be forgotten. However, existing unlearning methods are brittle: minor query modifications, such as multi-hop reasoning and entity aliasing, can recover supposedly forgotten information. As a result, current evaluation metrics often create an illusion of effectiveness, failing to detect these vulnerabilities due t
AI SafetyMachine UnlearningEvaluationPrivacy
Research arXiv (Artificial Intelligence) Mar 13

Can RL Improve Generalization of LLM Agents? An Empirical Study

By Zhiheng Xi, Xin Guo, Jiaqi Liu, Jiazheng Zhang, Yutao Fan, Zhihao Zhang, Shichun Liu, Mingxu Chai, Xiaowei Shi, Yitao Zhai, Xunliang Cai, Tao Gui, Qi Zhang, Xuanjing Huang

72 score
AI Analysis

Studies whether reinforcement fine-tuning (RFT) improves generalization of LLM agents across three axes: within-environment task difficulty, cross-environment transfer, and sequential multi-environment training. Provides systematic characterization of generalization profiles.

arXiv:2603.12011v1 Announce Type: new Abstract: Reinforcement fine-tuning (RFT) has shown promise for training LLM agents to perform multi-turn decision-making based on environment feedback. However, most existing evaluations remain largely in-domain: training and testing are conducted in the same environment or even on the same tasks. In real-world deployment, agents may operate in unseen environments with different background knowledge, observation spaces, and action interfaces. To characteri
Reinforcement LearningLanguage Model AgentsGeneralization
Research arXiv (Artificial Intelligence) Mar 13

Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover

By Indranil Halder, Annesya Banerjee, Cengiz Pehlevan

72 score
AI Analysis

Discovers scaling laws for LLM jailbreaks showing a polynomial-to-exponential crossover: prompt injection amplifies attack success from polynomial to exponential growth with inference-time samples. Proposes a spin-glass theoretical model to explain this.

arXiv:2603.11331v1 Announce Type: cross Abstract: Adversarial attacks can reliably steer safety-aligned large language models toward unsafe behavior. Empirically, we find that adversarial prompt-injection attacks can amplify attack success rate from the slow polynomial growth observed without injection to exponential growth with the number of inference-time samples. To explain this phenomenon, we propose a theoretical generative model of proxy language in terms of a spin-glass system operating
AI SafetyJailbreakingScaling LawsLanguage Models
Research arXiv (Artificial Intelligence) Mar 13

IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RL

By Zhoujun Cheng, Yutao Xie, Yuxiao Qu, Amrith Setlur, Shibo Hao, Varad Pimpalkhute, Tongtong Liang, Feng Yao, Zhengzhong Liu, Eric Xing, Virginia Smith, Ruslan Salakhutdinov, Zhiting Hu, Taylor Killian, Aviral Kumar

72 score
AI Analysis

Studies compute-optimal allocation of sampling compute for on-policy RL post-training of LLMs, finding predictable scaling of optimal parallel rollouts with compute budget. Proposes the IsoCompute Playbook for practical guidance.

arXiv:2603.12151v1 Announce Type: cross Abstract: While scaling laws guide compute allocation for LLM pre-training, analogous prescriptions for reinforcement learning (RL) post-training of large language models (LLMs) remain poorly understood. We study the compute-optimal allocation of sampling compute for on-policy RL methods in LLMs, framing scaling as a compute-constrained optimization over three resources: parallel rollouts per problem, number of problems per batch, and number of update ste
Reinforcement LearningLanguage ModelsScaling LawsPost-Training
72 score
AI Analysis

Provides a formal proof that attention sinks (concentration of attention mass on content-agnostic positions) are mathematically necessary in softmax transformers for computing trigger-conditional behaviors. Formalizes a well-known empirical observation with rigorous theory.

arXiv:2603.11487v1 Announce Type: new Abstract: Transformers often display an attention sink: probability mass concentrates on a fixed, content-agnostic position. We prove that computing a simple trigger-conditional behavior necessarily induces a sink in softmax self-attention models. Our results formalize a familiar intuition: normalization over a probability simplex must force attention to collapse onto a stable anchor to realize a default state (e.g., when the model needs to ignore the input
Transformer TheoryMechanistic InterpretabilityAttention Mechanisms