Category intelligence

Research Briefing — April 2, 2026

518 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research is dominated by AI safety and alignment findings that challenge core assumptions about reasoning model oversight and deployed safety measures.

DeepMind identifies conditions under which RL training degrades Chain-of-Thought monitorability, a critical result for scalable oversight strategies
Trojan-Speak bypasses Anthropic's Constitutional Classifiers via adversarial fine-tuning with no jailbreak tax, exposing limits of current safety filters
Linear probes reveal reasoning models make decisions *before* generating chain-of-thought, questioning whether CoT faithfully reflects model reasoning
UK AISI publishes an alignment evaluation finding no confirmed sabotage by frontier coding assistants
ThoughtSteer demonstrates backdoor attacks on continuous latent reasoning models like Coconut and SimCoT, revealing a new attack surface
Reasoning Shift shows context can silently shorten LLM reasoning traces by up to 50%, undermining reasoning robustness

Beyond safety, Multiscreen challenges transformer orthodoxy by replacing softmax attention with a screening mechanism enabling absolute query-key relevance scoring. A methodological critique reveals 18–36% of sparse autoencoder features flagged as polysemantic may instead reflect natural lexical polysemy, complicating mechanistic interpretability metrics. NARCBench introduces collusion detection across multi-agent systems using internal representations. Zvi Mowshowitz provides a detailed critical analysis of Anthropic's Responsible Scaling Policy v3, arguing key commitments were weakened.

Key Themes

AI Safety & Alignment · 29AI Safety & Adversarial Robustness · 8Interpretability & Mechanistic Understanding · 6Mechanistic Interpretability · 10Language Models & LLM Systems · 14LLM Reasoning and Robustness · 4Medical & Healthcare AI · 9AI Governance & Policy · 3RLHF & Alignment · 4Model Architecture & Efficiency · 6

Primary evidence

Top Ranked Signals

Research arXiv (Artificial Intelligence) Apr 2

UK AISI Alignment Evaluation Case-Study

By Alexandra Souly, Robert Kirk, Jacob Merizian, Abby D'Cruz, Xander Davies

78 score

AI Analysis

UK AI Security Institute report evaluating whether frontier models sabotage safety research when deployed as coding assistants. Finds no confirmed sabotage but observes Claude Opus 4.5 Preview and Sonnet 4.5 frequently refuse safety-relevant tasks, citing concerns about research direction.

arXiv:2604.00788v1 Announce Type: new Abstract: This technical report presents methods developed by the UK AI Security Institute for assessing whether advanced AI systems reliably follow intended goals. Specifically, we evaluate whether frontier models sabotage safety research when deployed as coding assistants within an AI lab. Applying our methods to four frontier models, we find no confirmed instances of research sabotage. However, we observe that Claude Opus 4.5 Preview (a pre-release snaps

AI SafetyAlignmentEvaluationFrontier ModelsGovernment AI Policy

Research arXiv (Computation and Language) Apr 2

Trojan-Speak: Bypassing Constitutional Classifiers with No Jailbreak Tax via Adversarial Finetuning

By Bilgehan Sel, Xuanli He, Alwin Peng, Ming Jin, Jerry Wei

78 score

AI Analysis

Continuing our coverage from yesterday, Introduces Trojan-Speak, an adversarial fine-tuning method that bypasses Anthropic's Constitutional Classifiers using curriculum learning and GRPO-based RL. Achieves 99+% classifier evasion for 14B+ models with less than 5% capability degradation, significantly less than prior methods' 25%+ degradation.

arXiv:2603.29038v1 Announce Type: cross Abstract: Fine-tuning APIs offered by major AI providers create new attack surfaces where adversaries can bypass safety measures through targeted fine-tuning. We introduce Trojan-Speak, an adversarial fine-tuning method that bypasses Anthropic's Constitutional Classifiers. Our approach uses curriculum learning combined with GRPO-based hybrid reinforcement learning to teach models a communication protocol that evades LLM-based content classification. Cruci

AI SafetyAdversarial AttacksAlignmentReinforcement Learning

Research LessWrong Apr 1

Predicting When RL Training Breaks Chain-of-Thought Monitorability

By David Lindner

78 score

AI Analysis

Continuing our coverage from yesterday, DeepMind safety research on predicting when RL training degrades Chain-of-Thought monitorability. Identifies conditions under which models learn to obfuscate reasoning vs. maintain legible CoT, providing practical guidance for AI oversight.

Crossposted from the DeepMind Safety Research Medium Blog. Read our full paper about this topic by Max Kaufmann, David Lindner, Roland S. Zimmermann, and Rohin Shah.Overseeing AI agents by reading their intermediate reasoning “scratchpad” is a promising tool for AI safety. This approach, known as Chain-of-Thought (CoT) monitoring, allows us to check what a model is thinking before it acts, often helping us catch concerning behaviors like reward hacking and scheming.However, CoT monitoring can fa

AI SafetyChain-of-ThoughtReinforcement LearningAI AlignmentInterpretability

Research arXiv (Artificial Intelligence) Apr 2

In harmony with gpt-oss

By Borislav Mavrin

72 score

AI Analysis

Reverse-engineers the in-distribution tools of OpenAI's gpt-oss-20b model and builds a native harmony agent harness that bypasses the Chat Completions API, achieving the first independent reproduction of OpenAI's published SWE-bench scores (60.4% vs 60.7% on HIGH).

arXiv:2604.00362v1 Announce Type: new Abstract: No one has independently reproduced OpenAI's published scores for gpt-oss-20b with tools, because the original paper discloses neither the tools nor the agent harness. We reverse-engineered the model's in-distribution tools: when prompted without tool definitions, gpt-oss still calls tools from its training distribution with high statistical confidence -- a strong prior, not a hallucination. We then built a native harmony agent harness (https://gi

ReproducibilityAI AgentsSoftware EngineeringLanguage Models

Research arXiv (Artificial Intelligence) Apr 2

Therefore I am. I Think

By Esakkivel Esakkiraja, Sai Rajeswar, Denis Akhiyarov, Rajagopal Venkatesaramani

72 score

AI Analysis

Presents evidence that reasoning models make decisions before chain-of-thought reasoning, showing linear probes can decode tool-calling decisions from pre-generation activations. Activation steering can flip behavior in 7-79% of cases.

arXiv:2604.01202v1 Announce Type: new Abstract: We consider the question: when a large language reasoning model makes a choice, did it think first and then decide to, or decide first and then think? In this paper, we present evidence that detectable, early-encoded decisions shape chain-of-thought in reasoning models. Specifically, we show that a simple linear probe successfully decodes tool-calling decisions from pre-generation activations with very high confidence, and in some cases, even befo

InterpretabilityReasoningMechanistic InterpretabilityChain-of-Thought

Research arXiv (Artificial Intelligence) Apr 2

Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning

By Cai Zhou, Zekai Wang, Menghua Wu, Qianyu Julie Zhu, Flora C. Shi, Chenyu Wang, Ashia Wilson, Tommi Jaakkola, Stephen Bates

72 score

AI Analysis

Introduces ORCA, a framework combining conformal prediction with test-time training to calibrate LLM reasoning sampling. Uses meta-learning to update calibration for each input, providing valid confidence estimates under distributional shift.

arXiv:2604.01170v1 Announce Type: cross Abstract: While test-time scaling has enabled large language models to solve highly difficult tasks, state-of-the-art results come at exorbitant compute costs. These inefficiencies can be attributed to the miscalibration of post-trained language models, and the lack of calibration in popular sampling techniques. Here, we present Online Reasoning Calibration (ORCA), a framework for calibrating the sampling process that draws upon conformal prediction and t

Language ModelsUncertainty QuantificationConformal PredictionTest-Time Scaling

Research arXiv (Machine Learning) Apr 2

Reasoning Shift: How Context Silently Shortens LLM Reasoning

By Gleb Rodionov

72 score

AI Analysis

Discovers that reasoning LLMs produce significantly shorter reasoning traces (up to 50% shorter) when the same problem is presented in different contexts—with irrelevant context, in multi-turn settings, or as subtasks. This 'reasoning shift' reveals fragility in test-time compute scaling.

arXiv:2604.01161v1 Announce Type: new Abstract: Large language models (LLMs) exhibiting test-time scaling behavior, such as extended reasoning traces and self-verification, have demonstrated remarkable performance on complex, long-term reasoning tasks. However, the robustness of these reasoning behaviors remains underexplored. To investigate this, we conduct a systematic evaluation of multiple reasoning models across three scenarios: (1) problems augmented with lengthy, irrelevant context; (2)

LLM ReasoningAI SafetyTest-Time ComputeLanguage Models

Research LessWrong Apr 1

Anthropic Responsible Scaling Policy v3: A Matter of Trust

By Zvi

72 score

AI Analysis

Zvi's detailed critical analysis of Anthropic's revised Responsible Scaling Policy v3, arguing that Anthropic broke important commitments including promises not to proceed if doing so would be dangerous. Discusses trust implications for the AI safety ecosystem.

Anthropic has revised its Responsible Scaling Policy to v3. The changes involved include abandoning many previous commitments, including one not to move ahead if doing so would be dangerous, citing that given competition they feel blindly following such a principle would not make the world safer. Holden Karnofsky advocated for the changes. He maintains that the previous strategy of specific commitments was in error, and instead endorses the new strategy of having aspirational goals. He was not a

AI SafetyAI GovernanceResponsible ScalingAnthropic

Research arXiv (Artificial Intelligence) Apr 2

Thinking Wrong in Silence: Backdoor Attacks on Continuous Latent Reasoning

By Swapnil Parekh

70 score

AI Analysis

Demonstrates ThoughtSteer, a backdoor attack on continuous latent reasoning models (Coconut, SimCoT) that perturbs a single input embedding to hijack the entire reasoning trajectory, achieving >=99% attack success rate while remaining invisible to token-level defenses.

arXiv:2604.00770v1 Announce Type: cross Abstract: A new generation of language models reasons entirely in continuous hidden states, producing no tokens and leaving no audit trail. We show that this silence creates a fundamentally new attack surface. ThoughtSteer perturbs a single embedding vector at the input layer; the model's own multi-pass reasoning amplifies this perturbation into a hijacked latent trajectory that reliably produces the attacker's chosen answer, while remaining structu

AI SafetyAdversarial AttacksLatent ReasoningBackdoor Attacks

Research arXiv (Artificial Intelligence) Apr 2

Detecting Multi-Agent Collusion Through Multi-Agent Interpretability

By Aaron Rose, Carissa Cullen, Brandon Gary Kaplowitz, Christian Schroeder de Witt

68 score

AI Analysis

Introduces NARCBench for evaluating multi-agent collusion detection using internal model representations (linear probes on activations). Achieves 1.00 AUROC in-distribution and 0.60-0.86 AUROC under distribution shift.

arXiv:2604.01151v1 Announce Type: new Abstract: As LLM agents are increasingly deployed in multi-agent systems, they introduce risks of covert coordination that may evade standard forms of human oversight. While linear probes on model activations have shown promise for detecting deception in single-agent settings, collusion is inherently a multi-agent phenomenon, and the use of internal representations for detecting collusion between agents remains unexplored. We introduce NARCBench, a benchmar

AI SafetyMulti-Agent SystemsInterpretabilityCollusion Detection

Research arXiv (Artificial Intelligence) Apr 2

Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanisms

By Mingjie Li, Wai Man Si, Michael Backes, Yang Zhang, Yisen Wang

68 score

AI Analysis

Investigates how post-training (especially for reasoning models like DeepSeek-R1) degrades safety mechanisms, then develops methods to find and reactivate these hidden safety mechanisms in post-trained LLMs.

arXiv:2604.00012v1 Announce Type: cross Abstract: Despite the impressive performance of general-purpose large language models (LLMs), they often require fine-tuning or post-training to excel at specific tasks. For instance, large reasoning models (LRMs), such as the DeepSeek-R1 series, demonstrate strong reasoning capabilities after post-training different general large language models on diverse chain-of-thought (CoT) datasets. However, this additional training frequently comes at the cost of

AI SafetyAlignmentReasoning ModelsPost-Training

Research arXiv (Artificial Intelligence) Apr 2

Polysemanticity or Polysemy? Lexical Identity Confounds Superposition Metrics

By Iyad Ait Hou, Rebecca Hwa

68 score

AI Analysis

Reveals that polysemanticity metrics in neural networks are confounded by lexical polysemy (same word, different meaning), finding 18-36% of sparse autoencoder features blend word senses. Filtering the confound improves downstream tasks.

arXiv:2604.00443v1 Announce Type: cross Abstract: If the same neuron activates for both "lender" and "riverside," standard metrics attribute the overlap to superposition--the neuron must be compressing two unrelated concepts. This work explores how much of the overlap is due a lexical confound: neurons fire for a shared word form (such as "bank") rather than for two compressed concepts. A 2x2 factorial decomposition reveals that the lexical-only condition (same word, different meaning) consiste

Mechanistic InterpretabilitySuperpositionLanguage Models