Category intelligence

Research Briefing — May 26, 2026

1023 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research is dominated by safety, alignment, and interpretability work, with several substantive contributions from Anthropic. Constitution-following audits of frontier models from Anthropic and OpenAI reveal specification gaps under multi-turn adversarial pressure. Faithfulness as Information Flow introduces sufficiency/completeness/necessity properties for CoT monitoring, while a complementary meta-evaluation shows existing faithfulness metrics fail to measure what they purport to measure.

Polymorphism Is Rotation: independently trained transformers compute the same function in residual-stream bases that differ only by a uniform random rotation, with strong implications for cross-model SAE transfer and circuit universality
From Simulation to Enaction (Jack Lindsey, Anthropic): post-trained LLMs implicitly recognize their own generations with 3-4x lower entropy than off-policy text, across families and scales
Cognitive Security as an AI Safety Cause Area (Steinhardt): position paper framing belief/action autonomy as a major emerging safety frontier

Real-world impact and agent capabilities round out the highlights. A 45,000-researcher global RCT delivered LLM feedback on 31,000+ arXiv preprints, providing rare causal evidence on AI's effect on scientific output. QUEST releases an open family of 2B-35B deep research agents with a full mid-training/SFT/RL recipe focused on citation grounding. HumanEgo achieves zero-shot human-to-robot transfer from minutes of egocentric video via entity-level hand-object representations, and self-generated replay is shown to nearly eliminate catastrophic forgetting when model capacity is unsaturated.

Key Themes

Safety and Alignment · 17LLM Agents and Multi-Agent Systems · 49AI Safety and Alignment · 18AI Agents and Tool Use · 14AI Safety and Security · 7Interpretability and Safety · 9Agent Systems and Workflows · 18Benchmarks and Evaluation · 29Reasoning and Test-Time Methods · 9Vision-Language Model Evaluation and Benchmarks · 12

Primary evidence

Top Ranked Signals

Research arXiv (Artificial Intelligence) May 26

How Well Do Models Follow Their Constitutions?

By Arya Jakkli, Senthooran Rajamanoharan, Neel Nanda

78 score

AI Analysis

Audits how well frontier models follow their own published constitutions (Anthropic, OpenAI) under multi-turn adversarial pressure, decomposing specifications into 200+ atomic tenets each. Work from Neel Nanda group.

arXiv:2605.24229v1 Announce Type: new Abstract: Frontier AI developers now train models against long written behavioral specifications, such as Anthropic's constitution (Anthropic, 2025a) and OpenAI's Model Spec (OpenAI, 2025a), integrated into post-training via methods like character training (Anthropic, 2024) and deliberative alignment (Guan et al., 2024). These documents serve a governance function, but it is unclear how well models actually follow them under adversarial, multi-turn pressure

AlignmentSafetyAuditingSpecifications

Research arXiv (Artificial Intelligence) May 26

Human-AI Collaboration in Science at Scale: A Global Large-scale Randomized Field Experiment

By Binglu Wang, Weixin Liang, Jiahui Xue, Yuhui Zhang, Hancheng Cao, Dashun Wang, Yian Yin

78 score

AI Analysis

Large-scale randomized field experiment delivering LLM-generated feedback on 31,000+ arXiv preprints to 45,000+ researchers globally. Shows feedback recipients revised manuscripts 12.55% more than controls.

arXiv:2605.24180v1 Announce Type: cross Abstract: Collaboration is the defining mode of modern science, yet its core mechanism -- feedback -- remains hard to observe, difficult to scale, and unequally distributed. Here we test whether large language models (LLMs) can contribute to this hidden but vital practice and reallocate scientific feedback, an essential yet scarce resource for knowledge production. In a global large-scale randomized field experiment, we delivered customized LLM-generated

Human-AI CollaborationScience of ScienceLLM Applications

Research arXiv (Artificial Intelligence) May 26

Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m

By Jordan F. McCann

75 score

AI Analysis

Demonstrates polymorphism: independently trained transformers compute the same function in residual-stream bases that differ by a uniform random rotation. A single matrix multiplication enables transfer of SAE features and steering vectors between models.

arXiv:2605.24577v1 Announce Type: cross Abstract: Independently trained transformers compute the same function in residual-stream bases that differ by a uniform random rotation on $\mathrm{SO}(d_{\mathrm{model}})$. We call this phenomenon polymorphism: same function, mutually unintelligible interior coordinates. One matrix multiplication per model pair removes it: an orthogonal Procrustes fit on a single batch of activations transfers sparse-autoencoder feature dictionaries and steering vectors

Mechanistic InterpretabilitySAEsUniversality

Research arXiv (Artificial Intelligence) May 26

HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos

By Zhi (Leo), Wang, Botao He, Kelin Yu, Seungjae Lee, Ruohan Gao, Furong Huang, Yiannis Aloimonos

75 score

AI Analysis

HumanEgo bridges human-to-robot embodiment gap by lifting egocentric video to entity-level hand-object representations, training a flow matching policy with auxiliary objectives. Reports 92.5% success on real tasks from 30 minutes of human video, no robot data.

arXiv:2605.24934v1 Announce Type: cross Abstract: Human egocentric video captures rich manipulation demonstrations without any robot hardware, yet transferring these skills to robots remains challenging due to the embodiment gap between human and robot in both visual appearance and kinematics. We present HumanEgo, a framework that bridges the embodiment gap by lifting each human demonstration to an entity-level representation of hand-object interaction, and training a flow matching policy with

RoboticsImitation LearningEgocentric VideoFlow Matching

Research arXiv (Artificial Intelligence) May 26

From Simulation to Enaction: Post-trained language models recognize and react to their own generations

By Asvin G., Jack Lindsey

75 score

AI Analysis

Shows post-trained LLMs implicitly recognize their own on-policy generations with 3-4x lower entropy than off-policy text across families and sizes. Traces effect to internal representation of input surprise.

arXiv:2605.25459v1 Announce Type: cross Abstract: Language models are pretrained as passive predictors with no incentive to model the consequences of their own outputs. Post-training changes this: a model producing its own responses can benefit from recognizing that it is on-policy. We present evidence that post-trained models recognize their on-policy generations, and this recognition is implicitly encoded in their output distributions. In particular, on-policy output distribution entropy is 3

InterpretabilityPost-TrainingSelf-Recognition

Research arXiv (Machine Learning) May 26

Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning

By Jinghan Jia, Joe Benton, Eric Easley

75 score

AI Analysis

Studies CoT faithfulness via information flow framework with sufficiency/completeness/necessity properties. From Anthropic (Joe Benton).

arXiv:2605.24286v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning is useful for monitoring language models only when the reasoning trace faithfully reflects the computation that produces the final answer. However, models can rely on prompt-to-answer shortcuts that bypass the CoT, making the visible reasoning trace misleading even when it appears plausible. We study CoT faithfulness through a structural information-flow perspective: faithful reasoning should route answer-relevant

Chain-of-ThoughtAI SafetyFaithfulnessAlignment

Research arXiv (Machine Learning) May 26

Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay

By Martin Marek, Dongkyu Cho, Shikai Qiu, Rumi Chunara, Pavel Izmailov, Andrew Gordon Wilson

75 score

AI Analysis

Studies catastrophic forgetting in language models and shows that self-generated replay samples nearly eliminate forgetting when model capacity is not saturated. Identifies the interaction between capacity, learning rate, and forgetting from credible authors (Wilson, Izmailov).

arXiv:2605.26097v1 Announce Type: new Abstract: Models trained on a new task typically degrade on prior tasks, a phenomenon known as forgetting. Traditionally, mitigating forgetting has required replaying stored exemplars from prior tasks, which is often impractical. By contrast, language models can sample from their own training distribution, and we show that these self-generated samples serve as effective replay data, nearly eliminating forgetting. We find that forgetting nonetheless persists

Continual LearningLanguage ModelsForgetting

Research arXiv (Computation and Language) May 26

QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks

By Jian Xie, Tianhe Lin, Zilu Wang, Yuting Ning, Yuekun Yao, Tianci Xue, Zhehao Zhang, Zhongyang Li, Kai Zhang, Yufan Wu, Shijie Chen, Boyu Gou, Mingzhe Han, Yifei Wang, Vint Lee, Xinpeng Wei, Xiangjun Wang, Yu Su, Huan Sun

75 score

AI Analysis

QUEST is a family of open 2B-35B deep research agents trained with a mid-training, SFT, RL recipe focused on fact seeking, citation grounding, and report synthesis. Aims to match frontier proprietary deep research systems with broad generalization.

arXiv:2605.24218v1 Announce Type: new Abstract: Deep research agents extend the role of search engines from retrieving keyword-matched pages to synthesizing knowledge, fundamentally changing how humans interact with information. However, frontier systems remain proprietary, while existing open agents often generalize poorly across different task types, leaving unclear how to train a broadly capable deep research agent. We release QUEST, a family of open models (ranging from 2B to 35B) that serv

AgentsDeep ResearchOpen Source

Research arXiv (Computation and Language) May 26

Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth

By Yoav Gur-Arieh, Ana Marasovi\'c, Mor Geva

75 score

AI Analysis

Critical meta-evaluation showing that existing faithfulness metrics for chain-of-thought reasoning do not actually measure faithfulness, since they rely on proxies like plausibility rather than ground-truth labels of internal computations.

arXiv:2605.25052v1 Announce Type: new Abstract: Chains of thought (CoTs) have become central in interpreting and auditing behaviors of large language models. Yet growing evidence suggests that these traces often fail to faithfully represent the computations behind a model's predictions. Several faithfulness metrics have been proposed, but whether they indeed measure faithfulness remains unknown. Answering this requires ground-truth labels, which are hard to obtain since internal computations ar

InterpretabilityChain-of-ThoughtEvaluationAI Safety

Research LessWrong May 25

Cognitive Security as an AI Safety Cause Area

By jsteinhardt

75 score

AI Analysis

Jacob Steinhardt argues cognitive security (humans maintaining control over beliefs/actions) should be a major AI safety cause area, citing evidence of AI persuasion, AI psychosis, deepfakes, and impersonation scams already occurring.

As AI systems become more capable, the cognitive security of humans will be increasingly at risk. By cognitive security, I mean the ability of humans to maintain control over their beliefs and actions.Cognitive security could be compromised in several ways: AI could become very good at persuading people of arbitrary positions; interacting with AI could lead humans to lose touch with reality; and AIs could become very effective at blackmail or at producing extremely convincing false information.W

AI SafetyPersuasionCognitive Security

Research arXiv (Artificial Intelligence) May 26

Inference Time Context Sparsity: Illusion or Opportunity?

By Sahil Joshi, Prithvi Dixit, Agniva Chowdhury, Anshumali Shrivastava, Joseph E. Gonzalez, Ion Stoica, Kumar Krishna Agrawal, Aditya Desai

72 score

AI Analysis

Position paper arguing for principled extreme context-dimension sparsity in LLM inference, claiming dense attention constraints are artificial. From a strong author lineup including Stoica and Gonzalez.

arXiv:2605.24168v1 Announce Type: new Abstract: Sparsity has long been a central theme in LLM efficiency, but its role in context processing remains unresolved. As LLM workloads shift toward longer contexts and agentic interactions, the compute and memory bottlenecks of attention become increasingly critical, raising the question of whether these constraints are fundamental. Our position is that these constraints are artificial and unnecessary, and that the future of LLM inference lies in extre

Inference EfficiencySparsityAttention

Research arXiv (Artificial Intelligence) May 26

A governance horizon for ethical-use constraints in open-weight AI models

By Weiwei Xu, Hengzhi Ye, Haoran Ye, Kai Gao, Vladimir Filkov, Minghui Zhou

72 score

AI Analysis

Audits 2.1M HuggingFace model repos to find ethical-use restriction evidence decays with a half-life of 1.31 derivation steps, formalizing a governance horizon beyond which traceability is lost. Important empirical AI governance finding.

arXiv:2605.24383v1 Announce Type: new Abstract: Ethical constraints on open-weight AI models are both a reflection of societal concerns and a foundation for AI governance policy. They are expected to propagate to downstream derivatives while implemented as voluntary metadata disclosures that must be restated at each generation of reuse. We audit 2,142,823 model repositories on Hugging Face Hub to test whether this disclosure-based governance infrastructure can sustain traceability across deep m

AI GovernanceOpen-Weight ModelsPolicy