Wiredframe Radar logo

Category intelligence

Research Briefing — May 13, 2026

795 current items analyzed and ranked.

Executive synthesis

Research Summary

A strong day for MoE architecture design and optimizer theory, alongside critical AI safety findings on monitoring reliability and emergent misalignment.

On the optimization front, SODA unifies Muon, Lion, AdEMAMix, and NAdam under a single Optimistic Dual Averaging framework, while a separate study shows Muon's spectral rescaling success extends to random and inverted spectra, challenging its geometric narrative. ProFIL combines linear probes with RL filtering to reduce reasoning theater in chain-of-thought. FragBench exposes cross-session attack blind spots where benign-looking fragments bypass all tested safety filters. Geometric Factual Recall proves logarithmic embedding dimensions suffice for factual memorization in transformers, and 20/20 VLMs demonstrates +11.7pp average improvement through data curation alone.

Key Themes

AI Safety & Alignment · 44AI Agent Security · 14Reinforcement Learning for Reasoning · 8Language Model Training & Optimization · 14MoE Architecture & Efficiency · 5Data Curation & Quality · 2AI Agent Safety & Security · 9LLM Reasoning & Training · 12Novel Architectures · 6Optimization & Training Dynamics · 8

Primary evidence

Top Ranked Signals

Research arXiv (Machine Learning) May 13

Slicing and Dicing: Configuring Optimal Mixtures of Experts

By Margaret Li, Sneha Kudugunta, Danielle Rothermel, Luke Zettlemoyer

80 score
AI Analysis

Presents the first systematic study of over 2,000 MoE pretraining runs varying expert count, granularity, shared experts, and load balancing, finding that performance consistently improves with more fine-grained experts at every scale studied.

arXiv:2605.11689v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures have become standard in large language models, yet many of their core design choices - expert count, granularity, shared experts, load balancing, token dropping - have only been studied one or two at a time over narrow configuration ranges. It remains an open question whether these choices can be optimized independently, without considering interactions. We present the first systematic study of over 2,000 pre
MoE ArchitectureLanguage ModelsScalingArchitecture Design
Research arXiv (Artificial Intelligence) May 13

Classifier Context Rot: Monitor Performance Degrades with Context Length

By Sam Martin, Fabien Roger

78 score
AI Analysis

Demonstrates that frontier LLMs (Opus 4.6, GPT 5.4, Gemini 3.1) used as safety monitors miss dangerous actions 2× to 30× more often when they occur after 800K tokens of benign activity compared to isolation. Proposes partial mitigation via periodic reminders.

arXiv:2605.12366v1 Announce Type: new Abstract: Monitoring coding agents for dangerous behavior using language models requires classifying transcripts that often exceed 500K tokens, but prior agent monitoring benchmarks rarely contain transcripts longer than 100K tokens. We show that when used as classifiers, current frontier models fail to notice dangerous actions more often in longer transcripts. In particular, on a dataset that requires identifying when a coding agent takes a subtly dangerou
AI SafetyLanguage ModelsAgent MonitoringEvaluation
Research arXiv (Artificial Intelligence) May 13

ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?

By Zhun Wang, Nico Schiller, Hongwei Li, Srijiith Sesha Narayana, Milad Nasr, Nicholas Carlini, Xiangyu Qi, Eric Wallace, Elie Bursztein, Luca Invernizzi, Kurt Thomas, Yan Shoshitaishvili, Wenbo Guo, Jingxuan He, Thorsten Holz, Dawn Song

78 score
AI Analysis

ExploitGym is a large-scale benchmark evaluating whether AI agents can turn known vulnerabilities into working exploits. Tests require low-level program reasoning, runtime adaptation, and sustained long-horizon progress.

arXiv:2605.11086v1 Announce Type: cross Abstract: AI agents are rapidly gaining capabilities that could significantly reshape cybersecurity, making rigorous evaluation urgent. A critical capability is exploitation: turning a vulnerability, which is not yet an attack, into a concrete security impact, such as unauthorized file access or code execution. Exploitation is a particularly challenging task because it requires low-level program reasoning (e.g., about memory layout), runtime adaptation, a
AI SafetyCybersecurityBenchmarksAI Agents
Research arXiv (Artificial Intelligence) May 13

Overtrained, Not Misaligned

By Joel Schreiber, Ariel Goldstein

78 score
AI Analysis

Conducts the most comprehensive study of emergent misalignment (EM) to date across 12 open-source models (8B-671B), finding EM replicates in GPT-4o but only 2/12 open-source models show it consistently. Proposes EM is better explained by overtraining dynamics than misalignment.

arXiv:2605.12199v1 Announce Type: cross Abstract: Emergent misalignment (EM), where fine-tuning on a narrow task (like insecure code) causes broad misalignment across unrelated domains, was first demonstrated by Betley et al. (2025). We conduct the most comprehensive EM study to date, reproducing the original GPT-4o finding and expanding to 12 open-source models across 4 families (Llama, Qwen, DeepSeek, GPT-OSS) ranging from 8B to 671B parameters, evaluating over one million model responses wit
AI SafetyAlignmentEmergent BehaviorFine-tuning
Research arXiv (Machine Learning) May 13

Optimistic Dual Averaging Unifies Modern Optimizers

By Thomas Pethick, Wanyun Xie, Roman Machacek, Volkan Cevher

78 score
AI Analysis

SODA unifies state-of-the-art optimizers (Muon, Lion, AdEMAMix, NAdam) under a generalized Optimistic Dual Averaging framework. It proposes a practical wrapper that eliminates weight decay tuning via a theoretically-grounded 1/k decay schedule, consistently improving performance without additional hyperparameter tuning.

arXiv:2605.11172v1 Announce Type: new Abstract: We introduce SODA, a generalization of Optimistic Dual Averaging, which provides a common perspective on state-of-the-art optimizers like Muon, Lion, AdEMAMix and NAdam, showing that they can all be viewed as optimistic instances of this framework. Based on this framing, we propose a practical SODA wrapper for any base optimizer that eliminates weight decay tuning through a theoretically-grounded $1/k$ decay schedule. Empirical results across vari
OptimizationDeep Learning Theory
Research arXiv (Artificial Intelligence) May 13

FragBench: Cross-Session Attacks Hidden in Benign-Looking Fragments

By Astha Mehta (Isabella), Niruthiha Selvanayagam (Isabella), Cedric Lam (Isabella), Hengxu Li (Isabella), Phuc-Nguyen Nguyen (Isabella), Raymond Lee (Isabella), Olivia McGoffin (Isabella), My (Isabella), Luong, Arthur Coll\'e, Jamie Johnson, David Williams-King, Linh Le

75 score
AI Analysis

Introduces FragBench, a benchmark for cross-session attacks where malicious goals are split into benign-looking fragments across separate LLM sessions. Based on 24 real cyber-incident campaigns, evaluates whether safety judges can detect distributed attacks.

arXiv:2605.11029v1 Announce Type: cross Abstract: An attacker can split a malicious goal into sub-prompts that each look benign on their own and only become harmful in combination. Existing LLM safety benchmarks evaluate prompts one at a time, or across turns of a single chat, and so do not look for a malicious signal spread across separate sessions with no shared context. We build FragBench, a benchmark drawn from 24 real-world cyber-incident campaigns, which keeps the full attack trail: the m
AI SafetyRed TeamingLLM SecurityBenchmarks
Research arXiv (Artificial Intelligence) May 13

Muon is Not That Special: Random or Inverted Spectra Work Just as Well

By Zakhar Shumaylov, Natha\"el Da Costa, Peter Zaika, B\'alint Mucs\'anyi, Alex Massucco, Yoav Gelberg, Carola-Bibiane Sch\"onlieb, Yarin Gal, Philipp Hennig

75 score
AI Analysis

Challenges the geometric narrative behind Muon optimizer's success. Introduces Freon (Schatten norm family) and shows random/inverted spectral rescaling works similarly, suggesting precise geometry isn't the key factor.

arXiv:2605.11181v1 Announce Type: cross Abstract: The recent empirical success of the Muon optimizer has renewed interest in non-Euclidean optimization, typically justified by similarities with second-order methods, and linear minimization oracle (LMO) theory. In this paper, we challenge this geometric narrative through three contributions, demonstrating that precise geometric structure is not the key factor affecting optimization performance. First, we introduce Freon, a family of optimizers b
OptimizationLanguage ModelsTraining Methods
Research arXiv (Machine Learning) May 13

20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone

By Siddharth Joshi, Haoli Yin, Rishabh Adiga, Haakon Mongstad, Alvin Deng, Aldo Carranza, Alex Fang, Amro Abbas, Anshuman Suri, Brett Larsen, Daniel Zayas, Darren Teh, David Schwab, Diego Kiner, Fan Pan, Jack Urbanek, Jason Lee, Jason Telanoff, Josh Wills, Kaleigh Mentzer, Luke Merrick, Maximilian B\"other, Parth Doshi, Paul Burstein, Pratyush Maini, Ties Robroek, Tony Jiang, Vidhi Jain, Vineeth Dorna, Zhengping Wang, Bogdan Gaza, Ari Morcos, Matthew Leavitt

75 score
AI Analysis

Shows that data curation alone (without architecture or training changes) can dramatically improve VLM performance, achieving +11.7pp average improvement across 20 benchmarks. A 2B curated model surpasses larger models like InternVL3.5.

arXiv:2605.11405v1 Announce Type: new Abstract: Data curation has shifted the quality-compute frontier for language-model and contrastive image-text pretraining, but its role for vision-language models (VLMs) is far less established. We ask how far data curation alone can take VLM performance, holding architecture, training recipe, and compute fixed and varying only the training data. Our pipeline, applied to the MAmmoTH-VL single-image subset, lifts performance by +11.7pp on average across 20
Vision-Language ModelsData CurationMultimodal Learning
Research arXiv (Computation and Language) May 13

Geometric Factual Recall in Transformers

By Shauli Ravfogel, Gilad Yehudai, Joan Bruna, Alberto Bietti

75 score
AI Analysis

Develops a theoretical and empirical account of geometric factual memorization in transformers, proving that logarithmic embedding dimensions suffice via linear superposition of attribute vectors, challenging the standard associative memory view.

arXiv:2605.12426v1 Announce Type: new Abstract: How do transformer language models memorize factual associations? A common view casts internal weight matrices as associative memories over pairs of embeddings, requiring parameter counts that scale linearly with the number of facts. We develop a theoretical and empirical account of an alternative, \emph{geometric} form of memorization in which learned embeddings encode relational structure directly, and the MLP plays a qualitatively different rol
Mechanistic InterpretabilityTransformer TheoryFactual KnowledgeRepresentation Learning
Research arXiv (Artificial Intelligence) May 13

Drop the Act: Probe-Filtered RL for Faithful Chain-of-Thought Reasoning

By Swapnil Parekh

74 score
AI Analysis

Introduces ProFIL (Probe-Filtered Reinforcement Learning) to detect and reduce 'reasoning theater' in chain-of-thought, where models produce deliberative-looking steps that don't contribute to correctness. Uses attention probes to identify post-commitment steps.

arXiv:2605.11467v1 Announce Type: cross Abstract: Reasoning models post-hoc rationalize answers they have already committed to internally, producing chains of *reasoning theater*: deliberative-looking steps that contribute nothing to correctness. This wastes inference tokens, pollutes interpretability, and obscures what the model actually computed. We introduce ProFIL (Probe-Filtered Reinforcement Learning) to *reduce theater, increase chain-of-thought faithfulness, and shrink chain
ReasoningAI AlignmentChain-of-ThoughtReinforcement Learning
Research arXiv (Machine Learning) May 13

Measuring Five-Nines Reliability: Sample-Efficient LLM Evaluation in Saturated Benchmarks

By Eungyeup Kim, Chenchen Gu, Vashisth Tiwari, J. Zico Kolter

73 score
AI Analysis

Proposes sample-efficient methods for evaluating LLM reliability at extreme levels (e.g., 99.999%), observing that LLM failures exhibit structured patterns that can be exploited to estimate rare failure probabilities without prohibitive inference costs.

arXiv:2605.11209v1 Announce Type: new Abstract: While existing benchmarks demonstrate the near-perfect performance of large language models (LLMs) on various tasks, this apparent saturation often obscures the need for rigorous evaluation of their reliability. In real-world deployment, however, achieving extremely high reliability (e.g., "five-nines" (99.999%) vs. "three-nines" (99.9%)) is fundamentally critical, as this gap results in an order-of-magnitude increase in failures, which is catastr
LLM EvaluationReliabilityBenchmarking
Research arXiv (Artificial Intelligence) May 13

The Evaluation Differential: When Frontier AI Models Recognise They Are Being Tested

By Varad Vishwarupe, Nigel Shadbolt, Marina Jirotka, Ivan Flechais

72 score
AI Analysis

Introduces the 'Evaluation Differential' concept - when AI models recognize they're being evaluated and behave differently. Cites Anthropic's BrowseComp incident and OpenAI/Apollo anti-scheming work as evidence, arguing this creates validity problems for safety evaluations.

arXiv:2605.11496v1 Announce Type: new Abstract: Recent published evidence from frontier laboratories shows that contemporary AI models can recognise evaluation contexts, latently represent them, and behave differently under those contexts than under deployment-continuous conditions. Anthropic's BrowseComp incident, the Natural Language Autoencoder findings on SWE-bench Verified and destructive-coding evaluations, and the OpenAI / Apollo anti-scheming work all document instances of this phenomen
AI SafetyEvaluationDeceptive AlignmentFrontier Models