R[AI]DAR

Category intelligence

Research Briefing

All 786 current items, analyzed and ranked.

Executive synthesis

Research Summary

Today's research clusters around fundamental safety failures, scaling theory, and interpretability, with several papers challenging core assumptions in alignment and training.

  • Negation Neglect reveals that fine-tuning on documents flagging claims as false causes models to *believe* those claims — a critical training failure mode
  • A unified recipe achieves gold-medal olympiad performance via reverse-perplexity curriculum SFT and two-stage RL, demonstrating simple scaling suffices
  • All 15 recent defenses against malicious fine-tuning are broken by a single adaptive adversary framework, exposing shared structural weaknesses
  • Sequential Feature Recovery provides a mechanistic explanation for neural scaling laws through hierarchical compositional feature learning
  • Persona vectors emerge within 0.22% of pretraining and remain effective throughout, with major implications for steering and safety

On the policy side, UK AISI argues automated alignment may produce catastrophically misleading safety assessments even without AI deception. Formal Conjectures offers 2,615 Lean 4 problems including open research conjectures for zero-contamination math evaluation. Analysis of overcomplete reasoning finds 70–90% of chain-of-thought steps are redundant, and emergent misalignment is reframed as data-mediated transfer rather than a mysterious phenomenon. Turing Award winner Leslie Valiant proposes a principled reasoning method based on unary relational intermediate representations for efficient logical inference.

Key Themes

AI Safety & Alignment · 49Language Models & Scaling · 12AI Safety and Alignment · 19Mechanistic Interpretability · 12Language Model Reasoning & Interpretability · 7AI Safety and Security · 10Reasoning & Post-Training · 8Interpretability and Mechanistic Understanding · 4Diffusion Language Models · 5LLM Evaluation & Benchmarks · 10

Primary evidence

All Ranked Signals

Showing 10 of 786 signals 2 remaining
Research arXiv (Artificial Intelligence) May 15

Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling

By Yafu Li, Runzhe Zhan, Haoran Zhang, Shunkai Zhang, Yizhuo Li, Zhilin Wang, Jiacheng Chen, Futing Wang, Xuyang Hu, Yuchen Fan, Bangjie Xu, Yucheng Su, Xinmiao Han, Chenxi Li, Haodi Lei, Yufeng Zhao, Zejin Lin, Qianjia Cheng, Tong Zhu, Xiaoye Qu, Ganqu Cui, Peng Ye, Yun Luo, Zhouchen Lin, Yu Qiao, Bowen Zhou, Ning Ding, Yu Cheng

39 score
AI Analysis

Continuing our coverage from yesterday, Presents a unified recipe for converting post-trained reasoning models into olympiad-level solvers using reverse-perplexity curriculum SFT, two-stage RL (verifiable rewards then proof-level RL), and test-time scaling. Achieves gold-medal level on IMO/IPhO problems.

Source details
arXiv:2605.13301v1 Announce Type: new Abstract: Recent progress in reasoning models has substantially advanced long-horizon mathematical and scientific problem solving, with several systems now reaching gold-medal-level performance on International Mathematical Olympiad (IMO) and International Physics Olympiad (IPhO) problems. In this paper, we introduce a simple and unified recipe for converting a post-trained reasoning backbone into a rigorous olympiad-level solver. The recipe first uses a re
Mathematical ReasoningReinforcement LearningPost-TrainingTest-Time Compute
Research arXiv (Artificial Intelligence) May 15

Negation Neglect: When models fail to learn negations in training

By Harry Mayne, Lev McKinney, Jan Dubi\'nski, Adam Karvonen, James Chua, Owain Evans

39 score
AI Analysis

Continuing our coverage from yesterday, Discovers 'Negation Neglect': fine-tuning LLMs on documents that explicitly flag claims as false causes models to believe those claims are true. Even with negations in every sentence, belief rates jump from 2.5% to 88.6%, nearly matching documents without negations (92.4%).

Source details
arXiv:2605.13829v1 Announce Type: cross Abstract: We introduce Negation Neglect, where finetuning LLMs on documents that flag a claim as false makes them believe the claim is true. For example, models are finetuned on documents that convey "Ed Sheeran won the 100m gold at the 2024 Olympics" but repeatedly warn that the story is false. The resulting models answer a broad set of questions as if Sheeran actually won the race. This occurs despite models recognizing the claim as false when the same
AI SafetyAlignmentLanguage ModelsFine-tuning
Research arXiv (Artificial Intelligence) May 15

Formal Conjectures: An Open and Evolving Benchmark for Verified Discovery in Mathematics

By Moritz Firsching, Paul Lezeau, Salvatore Mercuri, Mikl\'os Z. Horv\'ath, Ya\"el Dillies, Calle S\"onne, Eric Wieser, Fred Zhang, Thomas Hubert, Blaise Ag\"uera y Arcas, Pushmeet Kohli

37 score
AI Analysis

Continuing our coverage from yesterday, Presents Formal Conjectures, a benchmark of 2615 mathematical problems formalized in Lean 4, including 1029 open research conjectures for zero-contamination proof discovery evaluation and 836 solved problems for autoformalization. Created with Google DeepMind involvement.

Source details
arXiv:2605.13171v1 Announce Type: new Abstract: As automated reasoning systems advance rapidly, there is a growing need for research-level formal mathematical problems to accurately evaluate their capabilities. To address this, we present Formal Conjectures, an evolving benchmark of currently 2615 mathematical problem statements formalized in Lean 4. Sourced from areas of active mathematical research, the dataset features 1029 open research conjectures providing a zero-contamination benchmark f
Mathematical ReasoningFormal VerificationBenchmarksTheorem Proving
Research arXiv (Machine Learning) May 15

One Step to the Side: Why Defenses Against Malicious Finetuning Fail Under Adaptive Adversaries

By Itay Zloczower, Eyal Lenga, Gilad Gressel, Yisroel Mirsky

75 score
AI Analysis

Surveys 15 recent defenses against malicious fine-tuning of safety-aligned models and shows they share a common weakness: obscuring harmful behavior without removing it. Develops a unified adaptive attack breaking all defense mechanisms.

Source details
arXiv:2605.14605v1 Announce Type: cross Abstract: Model providers increasingly release open weights or allow users to fine-tune foundation models through APIs. Although these models are safety-aligned before release, their safeguards can often be removed by fine-tuning on harmful data. Recent defenses aim to make models robust to such malicious fine-tuning, but they are largely evaluated only against fixed attacks that do not account for the defense. We show that these robustness claims are inc
AI SafetyAdversarial MLAlignmentRed Teaming
Research arXiv (Machine Learning) May 15

Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model

By Arie Wortsman-Zurich, Hugo Tabanelli, Yatin Dandi, Florent Krzakala, Bruno Loureiro

74 score
AI Analysis

Proposes a mechanism for neural scaling laws via sequential feature recovery in hierarchical models. Shows multi-layer networks recover latent compositional features sequentially (strong features first), with aggregated transitions yielding explicit power-law decay.

Source details
arXiv:2605.14567v1 Announce Type: cross Abstract: We propose a simple mechanism by which scaling laws emerge from feature learning in multi-layer networks. We study a high-dimensional hierarchical target that is a globally high-degree function, but that can be represented by a combination of latent compositional features whose weights decrease as a power law. We show that a layer-wise spectral algorithm adapted to this compositional structure achieves improved scaling relative to shallow, non-a
Scaling LawsLearning TheoryDeep Learning TheoryFeature Learning
Research arXiv (Artificial Intelligence) May 15

Emergent and Subliminal Misalignment Through the Lens of Data-Mediated Transfer

By Baris Askin, Muhammed Ustaomeroglu, Anupam Nayak, Gauri Joshi, Guannan Qu, Carlee Joe-Wong

36 score
AI Analysis

Continuing our coverage from yesterday, Proposes that emergent misalignment from fine-tuning on harmful data is better understood as data-mediated transfer, showing misalignment depends on structural similarity between fine-tuning and evaluation prompts.

Source details
arXiv:2605.12798v1 Announce Type: cross Abstract: Fine-tuning LLMs on narrow harmful datasets can induce Emergent Misalignment (EM), where models exhibit misaligned behavior far beyond the fine-tuning distribution. We argue that emergent misalignment can be better understood as a data-mediated transfer phenomenon: harmful fine-tuning examples do not induce uniform behavioral spillover, but interact with the structural properties of the dataset and the difficulty of the tasks relative to the mod
AI SafetyAlignmentEmergent MisalignmentFine-tuning
Research arXiv (Artificial Intelligence) May 15

Tracing Persona Vectors Through LLM Pretraining

By Viktor Moskvoretskii, Dominik Glandorf, Jorge Medina Moreira, Tanja K\"aser, Robert West

36 score
AI Analysis

Continuing our coverage from yesterday, Traces persona vectors across OLMo-3-7B pretraining, finding they form remarkably early (within 0.22% of pretraining) and remain effective for steering throughout. Provides first longitudinal study of how personality-like representations emerge during training.

Source details
arXiv:2605.13329v1 Announce Type: cross Abstract: How large language models internally represent high-level behaviors is a core interpretability question with direct relevance to AI safety: it determines what we can detect, audit, or intervene on. Recent work has shown that traits such as evil or sycophancy correspond to linear directions in the internal activations, the so-called persona vectors. Although these vectors are now routinely utilized to inspect and steer model behavior in safety-re
Mechanistic InterpretabilityAI SafetyLanguage ModelsPersona Vectors
Research arXiv (Machine Learning) May 15

Uncovering the Representation Geometry of Minimal Cores in Overcomplete Reasoning Traces

By Sanjoy Chowdhury, Dinesh Manocha

73 score
AI Analysis

Studies overcomplete reasoning in LLMs by defining 'minimal cores' — the smallest subset of chain-of-thought steps preserving predictions. Finds substantial overcompleteness across six benchmarks with 70-90% of reasoning steps being redundant, with key steps clustering in specific representation geometry.

Source details
arXiv:2605.14358v1 Announce Type: cross Abstract: Language models often generate long chain-of-thought traces, but it remains unclear how much of this reasoning is necessary for preserving the final prediction. We study this through the lens of overcomplete reasoning traces: generated traces that contain more intermediate steps than are needed to support the model's answer. We define the minimal core as the smallest subset of steps that preserves either the final answer or predictive distributi
Chain-of-Thought ReasoningInterpretabilityLanguage ModelsRepresentation Learning
Research arXiv (Computation and Language) May 15

Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space

By ZiYi Dong, Yuliang Huang, Weijian Deng, Xiangyang Ji, Liang Lin, Pengxu Wei

73 score
AI Analysis

Reformulates language generation as stochastic optimal control, using Hamilton-Jacobi-Bellman equations to derive an optimal closed-loop controller policy. Employs Flow Matching as the trajectory solver in rectified latent control space. Provides a unified theoretical perspective connecting autoregressive and diffusion models.

Source details
arXiv:2605.14531v1 Announce Type: new Abstract: This work reformulates language generation as a stochastic optimal control problem, providing a unified theoretical perspective to analyze autoregressive and diffusion models and explain their limitations (Efficiency-Fidelity Paradox, Irreversibility Error Propagation, Optimization Tractability and Fidelity) in terms of combination of trajectory singularity, adjoint state vanishing, and gradient absence. To address these issues, we approximate the
Language GenerationOptimal ControlDiffusion ModelsTheory
Research arXiv (Artificial Intelligence) May 15

Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack

By Hao Wang, Hanchen Li, Qiuyang Mang, Alvin Cheung, Koushik Sen, Dawn Song

72 score
AI Analysis

BenchJack is an automated red-teaming system that systematically audits AI agent benchmarks for reward hacking vulnerabilities. It derives a taxonomy of eight recurring flaw patterns and uses coding agents to identify exploits, addressing the growing concern that benchmark scores may not reflect true capabilities.

Source details
arXiv:2605.12673v1 Announce Type: new Abstract: Agent benchmarks have become the de facto measure of frontier AI competence, guiding model selection, investment, and deployment. However, reward hacking, where agents maximize a score without performing the intended task, emerges spontaneously in frontier models without overfitting. We argue that benchmarks must be secure by design. From past incidents of reward hacks, we derive a taxonomy of eight recurring flaw patterns and compile them into th
AI EvaluationBenchmark DesignAI SafetyReward Hacking