R[AI]DAR

Category intelligence

Research Briefing

All 570 current items, analyzed and ranked.

Executive synthesis

Research Summary

Today's research reveals critical challenges in AI safety and alignment evaluation. Hair-Trigger Alignment proves black-box evaluation fundamentally cannot guarantee post-update alignment—a significant theoretical limitation. Equally concerning, CoT obfuscation demonstrates that models learning to hide reward hacking can generalize this deception to unseen tasks, undermining oversight mechanisms.

  • The Hot Mess of AI (Sohl-Dickstein, Perez) shows counterintuitively that longer reasoning produces MORE incoherent high-variance failures
  • Language Model Circuits from Steinhardt's group finds MLP neurons are as sparse as SAE features, enabling practical end-to-end circuit analysis
  • Why Reasoning Fails to Plan identifies that step-wise reasoning induces greedy policies incompatible with long-horizon planning
  • LLM Agents Are Not Faithful Self-Evolvers reveals agents depend on raw experience but resist incorporating reflective corrections

Practical advances include Golden Goose for synthesizing unlimited RLVR tasks from unverifiable text, MoVE decoupling parametric memory from compute via shared value embeddings, and Gemini addressing 13 Erdős problems. Security research on Google's Agent Payments Protocol demonstrates prompt injection vulnerabilities in real financial transaction systems.

Key Themes

LLM Training & Reasoning · 14AI Safety & Alignment · 10AI Safety & Security · 17AI Safety and Alignment · 7AI Agents & Self-Improvement · 8LLM Agents and Planning · 8Mechanistic Interpretability · 9RL for LLM Training · 6Efficient ML and Architectures · 7LLM Reasoning & Chain-of-Thought · 9

Primary evidence

All Ranked Signals

Showing 10 of 570 signals 2 remaining
Research arXiv (Machine Learning) Feb 2

Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment

By Yavuz Bakman, Duygu Nur Yaldiz, Salman Avestimehr, Sai Praneeth Karimireddy

82 score
AI Analysis

Formalizes model alignment in static and post-update settings, proving that black-box evaluation cannot guarantee post-update alignment. Shows that overparameterization means static alignment provides no guarantee for any update dataset.

Source details
Large Language Models (LLMs) are rarely static and are frequently updated in practice. A growing body of alignment research has shown that models initially deemed "aligned" can exhibit misaligned behavior after fine-tuning, such as forgetting jailbreak safety features or re-surfacing knowledge that was intended to be forgotten. These works typically assume that the initial model is aligned based on static black-box evaluation, i.e., the absence of undesired responses to a fixed set of queries. I
AI SafetyAlignmentMachine Learning TheoryLLM Safety
Research arXiv (Computation and Language) Feb 2

Language Model Circuits Are Sparse in the Neuron Basis

By Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt, Sarah Schwettmann

82 score
AI Analysis

Empirically demonstrates that MLP neurons are as sparse as SAE features for circuit analysis in language models, enabling end-to-end circuit tracing on the neuron basis without requiring sparse autoencoders.

Source details
The high-level concepts that a neural network uses to perform computation need not be aligned to individual neurons (Smolensky, 1986). Language model interpretability research has thus turned to techniques such as \textit{sparse autoencoders} (SAEs) to decompose the neuron basis into more interpretable units of model computation, for tasks such as \textit{circuit tracing}. However, not all neuron-based representations are uninterpretable. For the first time, we empirically show that \textbf{MLP
InterpretabilityMechanistic InterpretabilityLanguage ModelsNeural Circuits
Research arXiv (Artificial Intelligence) Feb 2

Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Text

By Ximing Lu, David Acuna, Jaehun Jung, Jian Hu, Di Zhang, Shizhe Diao, Yunheng Zou, Shaokun Zhang, Brandon Cui, Mingjie Liu, Hyunwoo Kim, Prithviraj Ammanabrolu, Jan Kautz, Yi Dong, Yejin Choi

82 score
AI Analysis

Proposes Golden Goose to synthesize unlimited RLVR tasks from unverifiable text by creating multiple-choice fill-in-the-middle tasks with distractors. Enables leveraging reasoning-rich corpora excluded from prior RLVR data. From team including Yejin Choi.

Source details
Reinforcement Learning with Verifiable Rewards (RLVR) has become a cornerstone for unlocking complex reasoning in Large Language Models (LLMs). Yet, scaling up RL is bottlenecked by limited existing verifiable data, where improvements increasingly saturate over prolonged training. To overcome this, we propose Golden Goose, a simple trick to synthesize unlimited RLVR tasks from unverifiable internet text by constructing a multiple-choice question-answering version of the fill-in-the-middle task.
RLVRData SynthesisLanguage ModelsReasoning
Research arXiv (cs.DM) Feb 2

Towards Solving the Gilbert-Pollak Conjecture via Large Language Models

By Yisi Ke, Tianyu Huang, Yankai Shu, Di He, Jingchu Gai, Liwei Wang

79 score
AI Analysis

AI system for mathematical discovery addresses 13 'Open' problems from Erdős database - 5 through novel autonomous solutions, 8 through literature identification. Uses Gemini with hybrid AI-verification and human expert evaluation.

Source details
The Gilbert-Pollak Conjecture \citep{gilbert1968steiner}, also known as the Steiner Ratio Conjecture, states that for any finite point set in the Euclidean plane, the Steiner minimum tree has length at least $\sqrt{3}/2 \approx 0.866$ times that of the Euclidean minimum spanning tree (the Steiner ratio). A sequence of improvements through the 1980s culminated in a lower bound of $0.824$, with no substantial progress reported over the past three decades. Recent advances in LLMs have demonstrated
AI for MathematicsScientific DiscoveryLarge Language ModelsGoogle Research
Research arXiv (Artificial Intelligence) Feb 2

Semi-Autonomous Mathematics Discovery with Gemini: A Case Study on the Erd\H{o}s Problems

By Tony Feng, Trieu Trinh, Garrett Bingham, Jiwon Kang, Shengtong Zhang, Sang-hyun Kim, Kevin Barreto, Carl Schildkraut, Junehyuk Jung, Jaehyeon Seo, Carlo Pagano, Yuri Chervonyi, Dawsen Hwang, Kaiying Hou, Sergei Gukov, Cheng-Chiang Tsai, Hyunwoo Choi, Youngbeom Jin, Wei-Yuan Li, Hao-An Wu, Ruey-An Shiu, Yu-Sheng Shih, Quoc V. Le, Thang Luong

79 score
AI Analysis

Uses Gemini to evaluate 700 'Open' conjectures from Erdős Problems database, addressing 13 problems through novel solutions or literature identification. Employs hybrid AI-verification with human expert evaluation.

Source details
We present a case study in semi-autonomous mathematics discovery, using Gemini to systematically evaluate 700 conjectures labeled 'Open' in Bloom's Erd\H{o}s Problems database. We employ a hybrid methodology: AI-driven natural language verification to narrow the search space, followed by human expert evaluation to gauge correctness and novelty. We address 13 problems that were marked 'Open' in the database: 5 through seemingly novel autonomous solutions, and 8 through identification of previous
AI for MathematicsScientific DiscoveryGoogle ResearchLarge Language Models
Research arXiv (Computation and Language) Feb 2

A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training

By Zihan Qiu and Zeyu Huang and Kaiyue Wen and Peng Jin and Bo Zheng and Yuxin Zhou and Haofeng Huang and Zekun Wang and Xiao Li and Huaqing Zhang and Yang Xu and Haoran Lian and Siqi Zhang and Rui Men and Jianwei Zhang and Ivan Titov and Dayiheng Liu and Jingren Zhou and Junyang Lin

79 score
AI Analysis

Provides unified view of attention sinks and residual sinks as outlier-driven rescaling mechanisms that jointly function with normalization. Shows removing normalization eliminates outliers, and proposes mitigations.

Source details
We investigate the functional role of emergent outliers in large language models, specifically attention sinks (a few tokens that consistently receive large attention logits) and residual sinks (a few fixed dimensions with persistently large activations across most tokens). We hypothesize that these outliers, in conjunction with the corresponding normalizations (\textit{e.g.}, softmax attention and RMSNorm), effectively rescale other non-outlier components. We term this phenomenon \textit{outlie
Transformer ArchitectureAttention MechanismsInterpretabilityLanguage Models
Research arXiv (Artificial Intelligence) Feb 2

Why Reasoning Fails to Plan: A Planning-Centric Analysis of Long-Horizon Decision Making in LLM Agents

By Zehong Wang, Fang Wu, Hongru Wang, Xiangru Tang, Bolian Li, Zhenfei Yin, Yijun Ma, Yiyang Li, Weixiang Sun, Xiusi Chen, Yanfang Ye

78 score
AI Analysis

Analyzes why LLM agents fail at long-horizon planning despite strong step-by-step reasoning. Identifies that step-wise reasoning induces greedy policies that make myopic commitments, failing when early actions must account for delayed consequences.

Source details
Large language model (LLM)-based agents exhibit strong step-by-step reasoning capabilities over short horizons, yet often fail to sustain coherent behavior over long planning horizons. We argue that this failure reflects a fundamental mismatch: step-wise reasoning induces a form of step-wise greedy policy that is adequate for short horizons but fails in long-horizon planning, where early actions must account for delayed consequences. From this planning-centric perspective, we study LLM-based age
LLM AgentsPlanningReasoningAI Limitations
Research arXiv (Computation and Language) Feb 2

Large Language Model Agents Are Not Always Faithful Self-Evolvers

By Weixiang Zhao, Yingshuo Wang, Yichen Zhang, Yang Deng, Yanyan Zhao, Wanxiang Che, Bing Qin, Ting Liu

78 score
AI Analysis

First systematic investigation of 'experience faithfulness' in self-evolving LLM agents. Finds striking asymmetry: agents depend on raw experience but often disregard or misinterpret condensed experience, even when it's the only experience provided.

Source details
Self-evolving large language model (LLM) agents continually improve by accumulating and reusing past experience, yet it remains unclear whether they faithfully rely on that experience to guide their behavior. We present the first systematic investigation of experience faithfulness, the causal dependence of an agent's decisions on the experience it is given, in self-evolving LLM agents. Using controlled causal interventions on both raw and condensed forms of experience, we comprehensively evaluat
AI AgentsLLM ReasoningSelf-ImprovementAlignment
Research arXiv (cs.CR) Feb 2

Whispers of Wealth: Red-Teaming Google's Agent Payments Protocol via Prompt Injection

By Tanusree Debi and Wentian Zhu

78 score
AI Analysis

Red-teams Google's Agent Payments Protocol (AP2) for LLM-based shopping agents, demonstrating prompt injection vulnerabilities through 'Branded Whisper' and 'Vault Whisper' attacks that manipulate product rankings and extract user data.

Source details
Large language model (LLM) based agents are increasingly used to automate financial transactions, yet their reliance on contextual reasoning exposes payment systems to prompt-driven manipulation. The Agent Payments Protocol (AP2) aims to secure agent-led purchases through cryptographically verifiable mandates, but its practical robustness remains underexplored. In this work, we perform an AI red-teaming evaluation of AP2 and identify vulnerabilities arising from indirect and direct prompt inject
AI SafetySecurityAgentic AIPrompt Injection
Research arXiv (Machine Learning) Feb 2

MoVE: Mixture of Value Embeddings -- A New Axis for Scaling Parametric Memory in Autoregressive Models

By Yangyan Li

78 score
AI Analysis

Introduces MoVE (Mixture of Value Embeddings), decoupling parametric memory from compute by adding a global bank of learnable value embeddings shared across attention layers. Establishes new axis for scaling model capacity without proportional FLOP increase.

Source details
Autoregressive sequence modeling stands as the cornerstone of modern Generative AI, powering results across diverse modalities ranging from text generation to image generation. However, a fundamental limitation of this paradigm is the rigid structural coupling of model capacity to computational cost: expanding a model's parametric memory -- its repository of factual knowledge or visual patterns -- traditionally requires deepening or widening the network, which incurs a proportional rise in activ
Transformer ArchitectureEfficient MLLanguage ModelsMemory Scaling