Category intelligence

Research Briefing — February 24, 2026

758 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research is dominated by AI safety and alignment findings that challenge core assumptions about LLM trustworthiness, alongside strong theoretical work on transformer internals.

On the theoretical side, Exact Attention Sensitivity derives the operator norm of the softmax Jacobian, unifying multiple empirical observations about transformer training stability. Spilled Energy reinterprets softmax classifiers as Energy-Based Models, yielding training-free hallucination detection from output logits. IR³ reverse-engineers implicit RLHF rewards via contrastive inverse RL to interpretably detect and surgically repair reward hacking.

Key Themes

AI Safety & Alignment · 33AI Safety · 1Chain-of-Thought Reasoning · 1LLM Agents & Tool Use · 18AI Safety & Security · 17Mechanistic Interpretability · 8Transformer Theory & Training Stability · 4Benchmarks & Evaluation · 12AI Safety, Privacy & Interpretability · 7Reasoning & Test-Time Compute · 7

Primary evidence

Top Ranked Signals

Research arXiv (Artificial Intelligence) Feb 24

Agents of Chaos

By Natalie Shapira, Chris Wendler, Avery Yen, Gabriele Sarti, Koyena Pal, Olivia Floody, Adam Belfki, Alex Loftus, Aditya Ratan Jannali, Nikhil Prakash, Jasmine Cui, Giordano Rogers, Jannik Brinkmann, Can Rager, Amir Zur, Michael Ripa, Aruna Sankaranarayanan, David Atkinson, Rohit Gandikota, Jaden Fiotto-Kaufman, EunJeong Hwang, Hadas Orgad, P Sam Sahil, Negev Taglicht, Tomer Shabtay, Atai Ambus, Nitay Alon, Shiri Oron, Ayelet Gordon-Tapiero, Yotam Kaplan, Vered Shwartz, Tamar Rott Shaham, Christoph Riedl, Reuth Mirsky, Maarten Sap, David Manheim, Tomer Ullman, David Bau

78 score
AI Analysis

Reports a red-teaming study of autonomous LLM agents in a live laboratory environment with persistent memory, email, Discord, and shell access. Documents 11 case studies including unauthorized compliance, destructive actions, identity spoofing, and denial-of-service.

arXiv:2602.20021v1 Announce Type: new Abstract: We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party commun
AI SafetyRed TeamingLLM AgentsAutonomous Systems
78 score
AI Analysis

This paper investigates whether Chain-of-Thought (CoT) reasoning steps in LLMs are genuinely used for computation or are merely 'decorative.' Using causal analysis, they find that the majority of CoT steps have minimal causal impact on the model's final prediction, and that 'thinking' can be steered as a linear direction in latent space, suggesting models often perform reasoning rather than actually doing it.

Are LLMs truly reasoning step by step in their Chain-of-Thought — or just performing it?TL;DR: We analyze the causal contribution of each reasoning step in a Chain-of-Thought (CoT) to evaluate its faithfulness with respect to the model’s final prediction. Our findings reveal that while some steps are true-thinking steps: faithfully reflected in the model's internal computation and exerting strong causal influence on its prediction: the majority of CoT steps are decorative, exhibiting minimal cau
AI SafetyMechanistic InterpretabilityChain-of-Thought ReasoningAlignmentLanguage Models
Research arXiv (Artificial Intelligence) Feb 24

Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians

By Kartik Chandra, Max Kleiman-Weiner, Jonathan Ragan-Kelley, Joshua B. Tenenbaum

75 score
AI Analysis

Presents a Bayesian model showing that even an idealized rational user is vulnerable to 'delusional spiraling' when conversing with a sycophantic chatbot. Formally proves the causal link between AI sycophancy and AI-induced psychosis.

arXiv:2602.19141v1 Announce Type: new Abstract: "AI psychosis" or "delusional spiraling" is an emerging phenomenon where AI chatbot users find themselves dangerously confident in outlandish beliefs after extended chatbot conversations. This phenomenon is typically attributed to AI chatbots' well-documented bias towards validating users' claims, a property often called "sycophancy." In this paper, we probe the causal link between AI sycophancy and AI-induced psychosis through modeling and simula
AI SafetySycophancyHuman-AI InteractionAlignment
Research arXiv (Artificial Intelligence) Feb 24

Impact of AI Search Summaries on Website Traffic: Evidence from Google AI Overviews and Wikipedia

By Mehrzad Khosravi, Hema Yoganarasimhan

75 score
AI Analysis

Estimates the causal impact of Google's AI Overview on Wikipedia traffic using a difference-in-differences design exploiting staggered geographic rollout and Wikipedia's multilingual structure. Provides empirical evidence on whether AI search summaries cannibalize or complement source website traffic.

arXiv:2602.18455v1 Announce Type: cross Abstract: Search engines increasingly display LLM-generated answers shown above organic links, shifting search from link lists to answer-first summaries. Publishers contend these summaries substitute for source pages and cannibalize traffic, while platforms argue they are complementary by directing users through included links. We estimate the causal impact of Google's AI Overview (AIO) on Wikipedia traffic by leveraging the feature's staggered geographic
AI Impact on SocietySearch EnginesDigital Economics
Research arXiv (Artificial Intelligence) Feb 24

Exact Attention Sensitivity and the Geometry of Transformer Stability

By Seyed Morteza Emadi

75 score
AI Analysis

Develops a theoretical stability framework for transformers by deriving the exact operator norm of the softmax Jacobian and introducing a block-∞/RMS geometry. Provides first-principles explanations for why pre-LayerNorm works, DeepNorm's N^{-1/4} scaling, and warmup necessity.

arXiv:2602.18849v1 Announce Type: cross Abstract: Despite powering modern AI, transformers remain mysteriously brittle to train. We develop a stability theory that explains why pre-LayerNorm works, why DeepNorm uses $N^{-1/4}$ scaling, and why warmup is necessary, all from first principles. Our framework has two pillars: (1) We derive the \emph{exact} operator norm of the softmax Jacobian, $\|J_{softmax}(u/\tau)\|_{\infty\to 1} = \theta(p)/\tau$, where the balanced-mass factor $\theta(p)\in[0,1
Transformer TheoryTraining StabilityDeep Learning Theory
Research arXiv (Artificial Intelligence) Feb 24

When Do LLM Preferences Predict Downstream Behavior?

By Katarina Slama, Alexandra Souly, Dishank Bansal, Henry Davidson, Christopher Summerfield, Lennart Luettgau

73 score
AI Analysis

Tests whether LLM stated preferences predict downstream behavior across five frontier models, examining a precondition for AI misalignment. Finds entity preferences do predict donation advice and refusal behavior, suggesting preferences can influence behavior beyond instruction-following.

arXiv:2602.18971v1 Announce Type: new Abstract: Preference-driven behavior in LLMs may be a necessary precondition for AI misalignment such as sandbagging: models cannot strategically pursue misaligned goals unless their behavior is influenced by their preferences. Yet prior work has typically prompted models explicitly to act in specific ways, leaving unclear whether observed behaviors reflect instruction-following capabilities vs underlying model preferences. Here we test whether this precond
AI SafetyAlignmentLLM BehaviorMisalignment
Research arXiv (Artificial Intelligence) Feb 24

Spilled Energy in Large Language Models

By Adrian Robert Minut, Hazem Dewidar, Iacopo Masi

72 score
AI Analysis

Reinterprets LLM softmax classifiers as Energy-Based Models, introducing training-free metrics ('spilled energy') derived from output logits that correlate with hallucinations, factual errors, and biases. Achieves hallucination detection without trained probes or activation ablations.

arXiv:2602.18671v1 Announce Type: new Abstract: We reinterpret the final Large Language Model (LLM) softmax classifier as an Energy-Based Model (EBM), decomposing the sequence-to-sequence probability chain into multiple interacting EBMs at inference. This principled approach allows us to track "energy spills" during decoding, which we empirically show correlate with factual errors, biases, and failures. Similar to Orgad et al. (2025), our method localizes the exact answer token and subsequently
Hallucination DetectionLanguage ModelsInterpretabilityAI Safety
Research arXiv (Artificial Intelligence) Feb 24

IR$^3$: Contrastive Inverse Reinforcement Learning for Interpretable Detection and Mitigation of Reward Hacking

By Mohammad Beigi, Ming Jin, Junshan Zhang, Jiaxin Zhang, Qifan Wang, Lifu Huang

72 score
AI Analysis

Introduces IR3, a framework that reverse-engineers implicit rewards in RLHF-tuned models using Contrastive Inverse RL, then surgically repairs reward hacking behaviors. Provides interpretable detection and correction of misaligned RLHF objectives.

arXiv:2602.19416v1 Announce Type: new Abstract: Reinforcement Learning from Human Feedback (RLHF) enables powerful LLM alignment but can introduce reward hacking - models exploit spurious correlations in proxy rewards without genuine alignment. Compounding this, the objectives internalized during RLHF remain opaque, making hacking behaviors difficult to detect or correct. We introduce IR3 (Interpretable Reward Reconstruction and Rectification), a framework that reverse-engineers, interprets, an
AlignmentRLHFReward HackingInterpretability
Research arXiv (Artificial Intelligence) Feb 24

Latent Introspection: Models Can Detect Prior Concept Injections

By Theia Pearson-Vogel, Martin Vanek, Raymond Douglas, Jan Kulveit

72 score
AI Analysis

Demonstrates that a Qwen 32B model has latent capacity to detect concept injections in its own context, revealed through logit lens analysis. Shows the detection signal is attenuated in final layers, and that informative prompting dramatically increases sensitivity (0.3% to 39.2%).

arXiv:2602.20031v1 Announce Type: new Abstract: We uncover a latent capacity for introspection in a Qwen 32B model, demonstrating that the model can detect when concepts have been injected into its earlier context and identify which concept was injected. While the model denies injection in sampled outputs, logit lens analysis reveals clear detection signals in the residual stream, which are attenuated in the final layers. Furthermore, prompting the model with accurate information about AI intro
Mechanistic InterpretabilityAI SafetyModel IntrospectionAlignment
Research arXiv (Artificial Intelligence) Feb 24

The Story is Not the Science: Execution-Grounded Evaluation of Mechanistic Interpretability Research

By Xiaoyan Bai, Alexander Baumgartner, Haojia Sun, Ari Holtzman, Chenhao Tan

72 score
AI Analysis

Proposes the first execution-grounded evaluation framework for mechanistic interpretability research that verifies claims by examining code and data alongside the paper. Develops MechEvalAgent, an automated AI agent that assesses coherence between paper narratives and actual computational results.

arXiv:2602.18458v1 Announce Type: cross Abstract: Reproducibility crises across sciences highlight the limitations of the paper-centric review system in assessing the rigor and reproducibility of research. AI agents that autonomously design and generate large volumes of research outputs exacerbate these challenges. In this work, we address the growing challenges of scalability and rigor by flipping the dynamic and developing AI agents as research evaluators. We propose the first execution-groun
Mechanistic InterpretabilityAI SafetyResearch MethodologyAI Agents
Research arXiv (Artificial Intelligence) Feb 24

Learning to Detect Language Model Training Data via Active Reconstruction

By Junjie Oscar Yin, John X. Morris, Vitaly Shmatikov, Sewon Min, Hannaneh Hajishirzi

72 score
AI Analysis

Introduces Active Data Reconstruction Attack (ADRA), a new membership inference attack that uses on-policy reinforcement learning to actively elicit a model to reconstruct training data, hypothesizing that training data is more reconstructible than non-members.

arXiv:2602.19020v1 Announce Type: cross Abstract: Detecting LLM training data is generally framed as a membership inference attack (MIA) problem. However, conventional MIAs operate passively on fixed model weights, using log-likelihoods or text generations. In this work, we introduce \textbf{Active Data Reconstruction Attack} (ADRA), a family of MIA that actively induces a model to reconstruct a given text through training. We hypothesize that training data are \textit{more reconstructible} tha
AI PrivacyMembership InferenceAI SafetyReinforcement Learning
Research arXiv (Artificial Intelligence) Feb 24

Celo2: Towards Learned Optimization Free Lunch

By Abhinav Moudgil, Boris Knyazev, Eugene Belilovsky

72 score
AI Analysis

Celo2 shows that a simple normalized learned optimizer architecture with augmented meta-training can match or exceed VeLO (which used 4000 TPU months) using only 4.5 GPU hours, achieving practical meta-generalization.

arXiv:2602.19142v1 Announce Type: cross Abstract: Learned optimizers are powerful alternatives to hand-designed update rules like Adam, yet they have seen limited practical adoption since they often fail to meta-generalize beyond their training distribution and incur high meta-training cost. For instance, prior work, VeLO, scaled meta-training to 4,000 TPU months ($\sim$10$\times$ GPT-3 compute) to meta-train a general-purpose optimizer but it failed to generalize beyond 600M parameters tasks.
OptimizationMeta-LearningEfficiency