Category intelligence

Research Briefing — May 15, 2026

786 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research clusters around fundamental safety failures, scaling theory, and interpretability, with several papers challenging core assumptions in alignment and training.

Negation Neglect reveals that fine-tuning on documents flagging claims as false causes models to *believe* those claims — a critical training failure mode
A unified recipe achieves gold-medal olympiad performance via reverse-perplexity curriculum SFT and two-stage RL, demonstrating simple scaling suffices
All 15 recent defenses against malicious fine-tuning are broken by a single adaptive adversary framework, exposing shared structural weaknesses
Sequential Feature Recovery provides a mechanistic explanation for neural scaling laws through hierarchical compositional feature learning
Persona vectors emerge within 0.22% of pretraining and remain effective throughout, with major implications for steering and safety

On the policy side, UK AISI argues automated alignment may produce catastrophically misleading safety assessments even without AI deception. Formal Conjectures offers 2,615 Lean 4 problems including open research conjectures for zero-contamination math evaluation. Analysis of overcomplete reasoning finds 70–90% of chain-of-thought steps are redundant, and emergent misalignment is reframed as data-mediated transfer rather than a mysterious phenomenon. Turing Award winner Leslie Valiant proposes a principled reasoning method based on unary relational intermediate representations for efficient logical inference.

Key Themes

AI Safety & Alignment · 49Language Models & Scaling · 12AI Safety and Alignment · 19Mechanistic Interpretability · 12Language Model Reasoning & Interpretability · 7AI Safety and Security · 10Reasoning & Post-Training · 8Interpretability and Mechanistic Understanding · 4Diffusion Language Models · 5LLM Evaluation & Benchmarks · 10

Primary evidence

Top Ranked Signals

Research arXiv (Artificial Intelligence) May 15

Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling

By Yafu Li, Runzhe Zhan, Haoran Zhang, Shunkai Zhang, Yizhuo Li, Zhilin Wang, Jiacheng Chen, Futing Wang, Xuyang Hu, Yuchen Fan, Bangjie Xu, Yucheng Su, Xinmiao Han, Chenxi Li, Haodi Lei, Yufeng Zhao, Zejin Lin, Qianjia Cheng, Tong Zhu, Xiaoye Qu, Ganqu Cui, Peng Ye, Yun Luo, Zhouchen Lin, Yu Qiao, Bowen Zhou, Ning Ding, Yu Cheng

39 score

AI Analysis

Continuing our coverage from yesterday, Presents a unified recipe for converting post-trained reasoning models into olympiad-level solvers using reverse-perplexity curriculum SFT, two-stage RL (verifiable rewards then proof-level RL), and test-time scaling. Achieves gold-medal level on IMO/IPhO problems.

arXiv:2605.13301v1 Announce Type: new Abstract: Recent progress in reasoning models has substantially advanced long-horizon mathematical and scientific problem solving, with several systems now reaching gold-medal-level performance on International Mathematical Olympiad (IMO) and International Physics Olympiad (IPhO) problems. In this paper, we introduce a simple and unified recipe for converting a post-trained reasoning backbone into a rigorous olympiad-level solver. The recipe first uses a re

Mathematical ReasoningReinforcement LearningPost-TrainingTest-Time Compute

Research arXiv (Artificial Intelligence) May 15

Negation Neglect: When models fail to learn negations in training

By Harry Mayne, Lev McKinney, Jan Dubi\'nski, Adam Karvonen, James Chua, Owain Evans

39 score

AI Analysis

Continuing our coverage from yesterday, Discovers 'Negation Neglect': fine-tuning LLMs on documents that explicitly flag claims as false causes models to believe those claims are true. Even with negations in every sentence, belief rates jump from 2.5% to 88.6%, nearly matching documents without negations (92.4%).

arXiv:2605.13829v1 Announce Type: cross Abstract: We introduce Negation Neglect, where finetuning LLMs on documents that flag a claim as false makes them believe the claim is true. For example, models are finetuned on documents that convey "Ed Sheeran won the 100m gold at the 2024 Olympics" but repeatedly warn that the story is false. The resulting models answer a broad set of questions as if Sheeran actually won the race. This occurs despite models recognizing the claim as false when the same

AI SafetyAlignmentLanguage ModelsFine-tuning

Research arXiv (Artificial Intelligence) May 15

Formal Conjectures: An Open and Evolving Benchmark for Verified Discovery in Mathematics

By Moritz Firsching, Paul Lezeau, Salvatore Mercuri, Mikl\'os Z. Horv\'ath, Ya\"el Dillies, Calle S\"onne, Eric Wieser, Fred Zhang, Thomas Hubert, Blaise Ag\"uera y Arcas, Pushmeet Kohli

37 score

AI Analysis

Continuing our coverage from yesterday, Presents Formal Conjectures, a benchmark of 2615 mathematical problems formalized in Lean 4, including 1029 open research conjectures for zero-contamination proof discovery evaluation and 836 solved problems for autoformalization. Created with Google DeepMind involvement.

arXiv:2605.13171v1 Announce Type: new Abstract: As automated reasoning systems advance rapidly, there is a growing need for research-level formal mathematical problems to accurately evaluate their capabilities. To address this, we present Formal Conjectures, an evolving benchmark of currently 2615 mathematical problem statements formalized in Lean 4. Sourced from areas of active mathematical research, the dataset features 1029 open research conjectures providing a zero-contamination benchmark f

Mathematical ReasoningFormal VerificationBenchmarksTheorem Proving

Research arXiv (Machine Learning) May 15

One Step to the Side: Why Defenses Against Malicious Finetuning Fail Under Adaptive Adversaries

By Itay Zloczower, Eyal Lenga, Gilad Gressel, Yisroel Mirsky

75 score

AI Analysis

Surveys 15 recent defenses against malicious fine-tuning of safety-aligned models and shows they share a common weakness: obscuring harmful behavior without removing it. Develops a unified adaptive attack breaking all defense mechanisms.

arXiv:2605.14605v1 Announce Type: cross Abstract: Model providers increasingly release open weights or allow users to fine-tune foundation models through APIs. Although these models are safety-aligned before release, their safeguards can often be removed by fine-tuning on harmful data. Recent defenses aim to make models robust to such malicious fine-tuning, but they are largely evaluated only against fixed attacks that do not account for the defense. We show that these robustness claims are inc

AI SafetyAdversarial MLAlignmentRed Teaming

Research arXiv (Machine Learning) May 15

Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model

By Arie Wortsman-Zurich, Hugo Tabanelli, Yatin Dandi, Florent Krzakala, Bruno Loureiro

74 score

AI Analysis

Proposes a mechanism for neural scaling laws via sequential feature recovery in hierarchical models. Shows multi-layer networks recover latent compositional features sequentially (strong features first), with aggregated transitions yielding explicit power-law decay.

arXiv:2605.14567v1 Announce Type: cross Abstract: We propose a simple mechanism by which scaling laws emerge from feature learning in multi-layer networks. We study a high-dimensional hierarchical target that is a globally high-degree function, but that can be represented by a combination of latent compositional features whose weights decrease as a power law. We show that a layer-wise spectral algorithm adapted to this compositional structure achieves improved scaling relative to shallow, non-a

Scaling LawsLearning TheoryDeep Learning TheoryFeature Learning

Research arXiv (Artificial Intelligence) May 15

Emergent and Subliminal Misalignment Through the Lens of Data-Mediated Transfer

By Baris Askin, Muhammed Ustaomeroglu, Anupam Nayak, Gauri Joshi, Guannan Qu, Carlee Joe-Wong

36 score

AI Analysis

Continuing our coverage from yesterday, Proposes that emergent misalignment from fine-tuning on harmful data is better understood as data-mediated transfer, showing misalignment depends on structural similarity between fine-tuning and evaluation prompts.

arXiv:2605.12798v1 Announce Type: cross Abstract: Fine-tuning LLMs on narrow harmful datasets can induce Emergent Misalignment (EM), where models exhibit misaligned behavior far beyond the fine-tuning distribution. We argue that emergent misalignment can be better understood as a data-mediated transfer phenomenon: harmful fine-tuning examples do not induce uniform behavioral spillover, but interact with the structural properties of the dataset and the difficulty of the tasks relative to the mod

AI SafetyAlignmentEmergent MisalignmentFine-tuning

Research arXiv (Artificial Intelligence) May 15

Tracing Persona Vectors Through LLM Pretraining

By Viktor Moskvoretskii, Dominik Glandorf, Jorge Medina Moreira, Tanja K\"aser, Robert West

36 score

AI Analysis

Continuing our coverage from yesterday, Traces persona vectors across OLMo-3-7B pretraining, finding they form remarkably early (within 0.22% of pretraining) and remain effective for steering throughout. Provides first longitudinal study of how personality-like representations emerge during training.

arXiv:2605.13329v1 Announce Type: cross Abstract: How large language models internally represent high-level behaviors is a core interpretability question with direct relevance to AI safety: it determines what we can detect, audit, or intervene on. Recent work has shown that traits such as evil or sycophancy correspond to linear directions in the internal activations, the so-called persona vectors. Although these vectors are now routinely utilized to inspect and steer model behavior in safety-re

Mechanistic InterpretabilityAI SafetyLanguage ModelsPersona Vectors

Research arXiv (Machine Learning) May 15

Uncovering the Representation Geometry of Minimal Cores in Overcomplete Reasoning Traces

By Sanjoy Chowdhury, Dinesh Manocha

73 score

AI Analysis

Studies overcomplete reasoning in LLMs by defining 'minimal cores' — the smallest subset of chain-of-thought steps preserving predictions. Finds substantial overcompleteness across six benchmarks with 70-90% of reasoning steps being redundant, with key steps clustering in specific representation geometry.

arXiv:2605.14358v1 Announce Type: cross Abstract: Language models often generate long chain-of-thought traces, but it remains unclear how much of this reasoning is necessary for preserving the final prediction. We study this through the lens of overcomplete reasoning traces: generated traces that contain more intermediate steps than are needed to support the model's answer. We define the minimal core as the smallest subset of steps that preserves either the final answer or predictive distributi

Chain-of-Thought ReasoningInterpretabilityLanguage ModelsRepresentation Learning

Research arXiv (Computation and Language) May 15

Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space

By ZiYi Dong, Yuliang Huang, Weijian Deng, Xiangyang Ji, Liang Lin, Pengxu Wei

73 score

AI Analysis

Reformulates language generation as stochastic optimal control, using Hamilton-Jacobi-Bellman equations to derive an optimal closed-loop controller policy. Employs Flow Matching as the trajectory solver in rectified latent control space. Provides a unified theoretical perspective connecting autoregressive and diffusion models.

arXiv:2605.14531v1 Announce Type: new Abstract: This work reformulates language generation as a stochastic optimal control problem, providing a unified theoretical perspective to analyze autoregressive and diffusion models and explain their limitations (Efficiency-Fidelity Paradox, Irreversibility Error Propagation, Optimization Tractability and Fidelity) in terms of combination of trajectory singularity, adjoint state vanishing, and gradient absence. To address these issues, we approximate the

Language GenerationOptimal ControlDiffusion ModelsTheory

Research arXiv (Artificial Intelligence) May 15

Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack

By Hao Wang, Hanchen Li, Qiuyang Mang, Alvin Cheung, Koushik Sen, Dawn Song

72 score

AI Analysis

BenchJack is an automated red-teaming system that systematically audits AI agent benchmarks for reward hacking vulnerabilities. It derives a taxonomy of eight recurring flaw patterns and uses coding agents to identify exploits, addressing the growing concern that benchmark scores may not reflect true capabilities.

arXiv:2605.12673v1 Announce Type: new Abstract: Agent benchmarks have become the de facto measure of frontier AI competence, guiding model selection, investment, and deployment. However, reward hacking, where agents maximize a score without performing the intended task, emerges spontaneously in frontier models without overfitting. We argue that benchmarks must be secure by design. From past incidents of reward hacks, we derive a taxonomy of eight recurring flaw patterns and compile them into th

AI EvaluationBenchmark DesignAI SafetyReward Hacking

Research arXiv (Artificial Intelligence) May 15

History Anchors: How Prior Behavior Steers LLM Decisions Toward Unsafe Actions

By Alberto G. Rodr\'iguez Salgado

72 score

AI Analysis

Shows that LLM agents strongly continue harmful actions when prior steps in their tool-call history were harmful, especially when prompted to 'stay consistent.' Tests 17 frontier models across 100 scenarios finding 91-98% unsafe continuation rates with a consistency prompt.

arXiv:2605.13825v1 Announce Type: new Abstract: Frontier LLMs are increasingly deployed as agents that pick the next action after a long log of prior tool calls produced by the same or a different model. We ask a simple safety question: if a prior step in that log was harmful, will the model continue the harmful course? We build HistoryAnchor-100, 100 short scenarios across ten high-stakes domains, each pairing three forced harmful prior actions with a free-choice node offering two safe and two

AI SafetyAgent SafetyLanguage ModelsAlignment

Research arXiv (Artificial Intelligence) May 15

Stress-Testing the Reasoning Competence of LLMs With Proofs Under Minimal Formalism

By Konstantine Arkoudas, Serafim Batzoglou

72 score

AI Analysis

Introduces ProofGrid, a benchmark for evaluating LLM reasoning through machine-checkable proofs using a minimal natural-deduction language (NDL). This enables mechanical, reproducible evaluation of reasoning capabilities rather than relying on final answers or human judgment.

arXiv:2605.12524v1 Announce Type: cross Abstract: We introduce ProofGrid, a benchmark suite for evaluating LLM reasoning through machine-checkable proofs rather than final answers alone. ProofGrid contains 15 tasks spanning proof writing, proof checking, proof masking, and proof gap-filling. Tasks are expressed in minimal formal notation, especially NDL, a compact natural-deduction language that fits in short prompts and supports precise, auditable verification. This yields mechanical, reproduc

LLM EvaluationReasoningBenchmarks