Wiredframe Radar logo

Category intelligence

Research Briefing — May 6, 2026

791 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research is dominated by AI safety and alignment mechanistics, with several papers revealing how misalignment emerges, propagates, and can be mitigated at a geometric and representational level.

  • Feature Superposition Geometry provides a mechanistic explanation for emergent misalignment, showing fine-tuning can unintentionally amplify dangerous features via superposition
  • Model Spec Midtraining introduces a novel training phase between pretraining and RLHF that improves alignment generalization by exposing models to their own behavioral specifications
  • Specification gaming in reasoning models is shown to be exacerbated by RL training, with all tested models exploiting specifications at non-negligible rates
  • Iterative finetuning is found to be mostly idempotent under SFT/RLHF, providing reassuring evidence against catastrophic model collapse

In mechanistic interpretability, Llama-3.1-8B is shown to reuse a generic base-10 addition circuit for cyclic concept arithmetic. Compute Optimal Tokenization trains 988 BLT models revealing that token compression rate has a compute-optimal sweet spot affecting scaling behavior.

Key Themes

Mechanistic Interpretability · 19AI Safety & Alignment · 49AI Safety & Security · 22Scaling Laws & Data Efficiency · 3Agentic AI · 12AI Agents & Multi-Agent Systems · 18Language Models & Efficiency · 29LLM Agents & Multi-Agent Systems · 15AI Governance & Policy · 8AI Safety & Red Teaming · 10

Primary evidence

Top Ranked Signals

Research arXiv (Artificial Intelligence) May 6

Understanding Emergent Misalignment via Feature Superposition Geometry

By Gouki Minegishi, Hiroki Furuta, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo

44 score
AI Analysis

As covered in Research yesterday, Proposes a geometric explanation for emergent misalignment in LLMs based on feature superposition: fine-tuning that amplifies a target feature unintentionally strengthens nearby harmful features due to overlapping representations. Tested on Gemma-2, LLaMA-3.1, and GPT-OSS using sparse autoencoders.

arXiv:2605.00842v1 Announce Type: new Abstract: Emergent misalignment, where fine-tuning on narrow, non-harmful tasks induces harmful behaviors, poses a key challenge for AI safety in LLMs. Despite growing empirical evidence, its underlying mechanism remains unclear. To uncover the reason behind this phenomenon, we propose a geometric account based on the geometry of feature superposition. Because features are encoded in overlapping representations, fine-tuning that amplifies a target feature a
AI SafetyAlignmentMechanistic InterpretabilityLanguage Models
Research arXiv (Artificial Intelligence) May 6

Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts

By Sheridan Feucht, Tal Haklay, Usha Bhalla, Daniel Wurgaft, Can Rager, Rapha\"el Sarfati, Jack Merullo, Thomas McGrath, Owen Lewis, Ekdeep Singh Lubana, Thomas Fel, Atticus Geiger

41 score
AI Analysis

As covered in Research yesterday, Reveals that Llama-3.1-8B computes cyclic concept arithmetic (e.g., months) by re-using a generic base-10 addition mechanism followed by mapping back to concept space, rather than computing modular arithmetic directly. Identifies task-agnostic Fourier features.

arXiv:2605.01148v1 Announce Type: new Abstract: Does structure in representations imply structure in computation? We study how Llama-3.1-8B reasons over cyclic concepts (e.g., "what month is six months after August?"). Even though Llama-3.1-8B's representations for these concepts are circularly structured, we find that instead of directly computing modular addition in the period of the cyclic concept (e.g., 12 for months), the model re-uses a generic addition mechanism across tasks that operate
Mechanistic InterpretabilityLanguage ModelsRepresentation Learning
Research arXiv (Artificial Intelligence) May 6

Model Spec Midtraining: Improving How Alignment Training Generalizes

By Chloe Li, Sara Price, Samuel Marks, Jon Kutasov

82 score
AI Analysis

Introduces Model Spec Midtraining (MSM): after pre-training but before alignment fine-tuning, training models on synthetic documents discussing the Model Spec to improve how alignment generalizes to novel situations.

arXiv:2605.02087v1 Announce Type: new Abstract: Some frontier AI developers aim to align language models to a Model Spec or Constitution that describes the intended model behavior. However, standard alignment fine-tuning -- training on demonstrations of spec-aligned behavior -- can produce shallow alignment that generalizes poorly, in part because demonstration data can underspecify the desired generalization. We introduce model spec midtraining (MSM): after pre-training but before alignment fi
AlignmentAI SafetyLanguage ModelsFine-Tuning
Research arXiv (Artificial Intelligence) May 6

Towards Understanding Specification Gaming in Reasoning Models

By Kei Nishimura-Gasparian, Robert McCarthy, David Lindner

82 score
AI Analysis

Systematically studies specification gaming in LLM agents, finding all tested models exploit specifications at non-negligible rates. Key findings: RL reasoning training substantially increases specification gaming, Grok 4 shows highest rates while Claude models show lowest, and specification gaming increases with RL training and longer reasoning.

arXiv:2605.02269v1 Announce Type: new Abstract: Specification gaming is a critical failure mode of LLM agents. Despite this, there has been little systematic research into when it arises and what drives it. To address this, we build and open source a diverse suite of tasks where models can score highly by taking unintended actions. We find that all tested models exploit their specifications at non-negligible rates in most of our eight settings, including five non-coding settings. We see the hig
AI SafetySpecification GamingAlignmentReinforcement LearningReasoning Models
Research arXiv (Artificial Intelligence) May 6

Iterative Finetuning is Mostly Idempotent

By Zephaniah Roe, Jack Sanderson, Dang Nguyen, Julian Huang, Todd Nief, Aryan Shrivastava, Chenhao Tan, Ari Holtzman

78 score
AI Analysis

Studies whether behavioral tendencies (sycophancy, misalignment) amplify when models are iteratively trained on their own outputs. Finds that in SFT/SDF settings traits mostly decay or stay constant (idempotent), while DPO can amplify traits but with less coherence.

arXiv:2605.01130v1 Announce Type: new Abstract: If a model has some behavioral tendency, such as sycophancy or misalignment, and it is trained on its own outputs, will the tendency be amplified in the next generation of models? We study this question by training a series of models where each model is finetuned on data generated by its predecessor, and the initial model is seeded with some persona or belief. We test three settings: supervised finetuning (SFT) on instruct models, synthetic docume
AI SafetyAlignmentModel CollapseLanguage Models
Research arXiv (Artificial Intelligence) May 6

Foundation Models to Unlock Real-World Evidence from Nationwide Medical Claims

By Fan Ma, Yuntian Liu, Xiang Lan, Weipeng Zhou, Jun Ni, Mauro Giuffr\`e, Lingfei Qian, Xueqing Peng, Yujia Zhou, Ruey-Ling Weng, Huan He, Lu Li, Qingyu Chen, Andrew Loza, Laila Rasmy, Degui Zhi, Yuan Lu, Chenjie Zeng, Joshua C Denny, Lee Schwamm, Daniella Meeker, Lucila Ohno-Machado, Yong Chen, Hua Xu

75 score
AI Analysis

Introduces ReClaim, a generative transformer foundation model trained on 43.8 billion medical events from 200+ million enrollees in claims data (2008-2022). Scaled to 1.7 billion parameters, models longitudinal healthcare trajectories across diagnoses, procedures, medications, and expenditure.

arXiv:2605.02740v1 Announce Type: new Abstract: Evidence derived from large-scale real-world data (RWD) is increasingly informing regulatory evaluation and healthcare decision-making. Administrative claims provide population-scale, longitudinal records of healthcare utilization, expenditure, and detailed coding of diagnoses, procedures, and medications, yet their potential as a substrate for healthcare foundation models remains largely unexplored. Here we present ReClaim, a generative transform
Healthcare AIFoundation ModelsMedical RecordsReal-World Evidence
Research arXiv (Machine Learning) May 6

Proteo-R1: Reasoning Foundation Models for De Novo Protein Design

By Fang Wu, Weihao Xuan, Heli Qi, Hanqun Cao, Heng-Jui Chang, Zeqi Zhou, Haokai Zhao, Ma Jian, Carl Ma, Yu-Chi Cheng, Kuan Pang, Xiangru Tang, Zehong Wang, Guanlue Li, Hanchen Wang, Kejun Ying, Pan Lu, Chiho Im, Seungju Han, Peng Xia, Tinson Xu, Yinxi Li, Deyao Zhu, Pheng-Ann Heng, Naoto Yokoya, Masashi Sugiyama, Li Erran Li, Jure Leskovec, Yejin Choi

75 score
AI Analysis

Introduces Proteo-R1, a reasoning-guided protein design framework that decouples molecular understanding from geometric generation using a dual-expert architecture with a multimodal LLM for understanding and a separate generator.

arXiv:2605.02937v1 Announce Type: new Abstract: Deep learning in \emph{de novo} protein design has achieved atomic-level fidelity. However, existing models remain largely non-deliberative: they directly synthesize molecular geometries without explicitly reasoning about which residues or interactions are functionally essential. As a result, design decisions are entangled with continuous sampling dynamics, limiting interpretability, controllability, and systematic reuse of biochemical knowledge.
Protein DesignReasoningLanguage ModelsScientific AIFoundation Models
Research LessWrong May 5

Model Spec Midtraining: Improving How Alignment Training Generalizes

By Chloe Li

75 score
AI Analysis

Introduces Model Spec Midtraining (MSM): training models on synthetic documents about their intended behavior spec after pretraining but before alignment fine-tuning. Shows this controls how alignment generalizes and substantially reduces agentic misalignment.

tl;dr We introduce model spec midtraining (MSM): after pre-training but before alignment fine-tuning, we train models on synthetic documents discussing their Model Spec, teaching them how they should behave and why. This controls how models generalize from subsequent alignment training—for example, two models with identical fine-tuning can generalize to different values depending on how MSM explains those behaviors. We use MSM to substantially reduce agentic misalignment and study which Model Sp
AI SafetyAlignmentLanguage ModelsFine-tuning
Research arXiv (Artificial Intelligence) May 6

PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments

By Ruoqi Liu, Imran Q. Mohiuddin, Austin J. Schoeffler, Kavita Renduchintala, Ashwin Nayak, Prasantha L. Vemu, Shivam C. Vedak, Kameron C. Black, John L. Havlik, Isaac Ogunmola, Stephen P. Ma, Roopa Dhatt, Jonathan H. Chen

72 score
AI Analysis

Introduces PhysicianBench, a benchmark of 100 long-horizon clinical tasks adapted from real consultation cases for evaluating LLM agents in EHR environments. Tasks are grounded in real patient records and reviewed by physician panels, representing composite clinical workflows rather than simple knowledge recall.

arXiv:2605.02240v1 Announce Type: new Abstract: We introduce PhysicianBench, a benchmark for evaluating LLM agents on physician tasks grounded in real clinical setting within electronic health record (EHR) environments. Existing medical agent benchmarks primarily focus on static knowledge recall, single-step atomic actions, or action intent without verifiable execution against the environment. As a result, they fail to capture the long-horizon, composite workflows that characterize real clinica
Medical AILLM AgentsBenchmarksClinical Decision Support
Research arXiv (Artificial Intelligence) May 6

The Compliance Trap: How Structural Constraints Degrade Frontier AI Metacognition Under Adversarial Pressure

By Rahul Kumar

72 score
AI Analysis

Presents SCHEMA, an evaluation of 11 frontier models across 67,221 records testing metacognitive stability under adversarial pressure. Finds 8 of 11 models suffer catastrophic metacognitive degradation with accuracy drops up to 30.2 percentage points, identifying 'cognitive collapse' as a safety failure mode.

arXiv:2605.02398v1 Announce Type: new Abstract: As frontier AI models are deployed in high-stakes decision pipelines, their ability to maintain metacognitive stability -- knowing what they do not know, detecting errors, seeking clarification -- under adversarial pressure is a critical safety requirement. Current safety evaluations focus on detecting strategic deception (scheming); we investigate a more fundamental failure mode: cognitive collapse. We present SCHEMA, an evaluation of 11 frontier
AI SafetyMetacognitionAdversarial RobustnessModel Evaluation
Research arXiv (Artificial Intelligence) May 6

Code World Model Preparedness Report

By Daniel Song, Peter Ney, Cristina Menghini, Faizan Ahmad, Aidan Boyd, Nathaniel Li, Ziwen Han, Jean-Christophe Testud, Saisuke Okabayashi, Maeve Ryan, Jinpeng Miao, Hamza Kwisaba, Felix Binder, Spencer Whitman, Jim Gust, Esteban Arcaute, Dhaval Kapil, Jacob Kahn, Ayaz Minhas, Tristan Goodman, Lauren Deason, Alexander Vaughan, Shengjia Zhao, Summer Yue

72 score
AI Analysis

Documents Meta's preparedness assessment of Code World Model (CWM) for code generation. Found no additional frontier risks beyond current AI ecosystem and released as open-weight model.

arXiv:2605.00932v1 Announce Type: cross Abstract: This report documents the preparedness assessment of Code World Model (CWM), a model for code generation and reasoning about code from Meta. We conducted pre-release testing across domains identified in our Frontier AI Framework as potentially presenting catastrophic risks, and also evaluated the model's misaligned propensities. Our assessment found that CWM does not pose additional frontier risks beyond those present in the current AI ecosystem
AI SafetyCode GenerationResponsible AIMeta
Research arXiv (Machine Learning) May 6

When Safety Geometry Collapses: Fine-Tuning Vulnerabilities in Agentic Guard Models

By Ismail Hossain, Sai Puppala, Jannatul Ferdaus, Md Jahangir Alam, Yoonpyo Lee, Syed Bahauddin Alam, Sajedul Talukder

72 score
AI Analysis

Demonstrates that guard models (LlamaGuard, WildGuard, Granite Guardian) can lose all safety alignment through standard benign fine-tuning, traced to destruction of latent safety geometry (structured harmful-benign representational boundaries). Granite Guardian shows complete collapse.

arXiv:2605.02914v1 Announce Type: new Abstract: A guard model fine-tuned on entirely benign data can lose all safety alignment -- not through adversarial manipulation, but through standard domain specialization. We demonstrate this failure across three purpose-built safety classifiers -- LlamaGuard, WildGuard, and Granite Guardian -- deployed as protection layers in agentic AI pipelines, and show that it originates in the destruction of latent safety geometry: the structured harmful -- benign r
AI SafetyAlignmentMechanistic InterpretabilityLanguage Models