Category intelligence

Research Briefing — February 16, 2026

374 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research centers on foundational challenges in LLM evaluation, efficient training, and reasoning robustness. Soft Contamination reveals that semantic duplicates evade standard n-gram decontamination, undermining benchmark validity across the field—potentially the most consequential finding of the day.

Native Low-Rank Pretraining shows LLMs can be trained from scratch with exclusively factorized weights by controlling spectral norm growth, offering significant compute savings
ARTS identifies a 'Normalization Squeeze' pathology in RLVR that systematically suppresses valid but rare reasoning paths, proposing amortized tree search as a fix
Continuous diffusion language models can be steered to obey formal syntax constraints (regex, CFGs) via training-free guidance
A Frankenstein-style dissection of RL in vision-language models reveals RL selectively improves specific functional components rather than broadly enhancing reasoning

Architecture and safety insights are also prominent. Two papers target MoE Transformers: one identifies multi-head attention (not experts) as the primary source of catastrophic forgetting, while SD-MoE uses spectral decomposition to fix expert specialization collapse. On safety, CC-Delta leverages sparse autoencoders to detect jailbreak-relevant features, and MonoLoss proposes a single-pass objective for monosemantic SAE representations. Finally, a study of nine frontier reasoning models under multi-turn adversarial attacks catalogues five distinct failure modes including Self-Doubt and Social Conformity.

Key Themes

RLVR & Reasoning · 7Benchmark Evaluation & Contamination · 6Language Models & Training Efficiency · 8AI Safety & Interpretability · 5LLM Reasoning and Alignment · 6LLM Training and Reasoning · 5LLM Agents and Reasoning · 7Agentic AI and Web Agents · 5AI Safety and Robustness · 8Diffusion Models · 4

Primary evidence

Top Ranked Signals

Research arXiv (Machine Learning) Feb 16

Soft Contamination Means Benchmarks Test Shallow Generalization

By Ari Spiesberger, Juan J. Vazquez, Nicky Pochinkov, Tom\'a\v{s} Gaven\v{c}iak, Peli Grietzer, Gavin Leech, Nandi Schoots

78 score

AI Analysis

Demonstrates that 'soft contamination' (semantic duplicates not caught by n-gram decontamination) is widespread in LLM training data, finding semantic duplicates for 78% of CodeForces problems. Shows that benchmark improvements from such contamination don't transfer to genuinely novel problems.

If LLM training data is polluted with benchmark test data, then benchmark performance gives biased estimates of out-of-distribution (OOD) generalization. Typical decontamination filters use n-gram matching which fail to detect semantic duplicates: sentences with equivalent (or near-equivalent) content that are not close in string space. We study this soft contamination of training data by semantic duplicates. Among other experiments, we embed the Olmo3 training corpus and find that: 1) contamina

LLM EvaluationBenchmark ContaminationData QualityLanguage Models

Research arXiv (Machine Learning) Feb 16

Stabilizing Native Low-Rank LLM Pretraining

By Paul Janson, Edouard Oyallon, Eugene Belilovsky

72 score

AI Analysis

Demonstrates that LLMs can be stably trained from scratch with exclusively low-rank factorized weights by controlling spectral norm growth, eliminating the need for full-rank guidance required by prior methods. Achieves performance matching dense models.

Foundation models have achieved remarkable success, yet their growing parameter counts pose significant computational and memory challenges. Low-rank factorization offers a promising route to reduce training and inference costs, but the community lacks a stable recipe for training models from scratch using exclusively low-rank weights while matching the performance of the dense model. We demonstrate that Large Language Models (LLMs) can be trained from scratch using exclusively low-rank factoriz

Efficient TrainingLanguage ModelsLow-Rank Methods

Research arXiv (Machine Learning) Feb 16

Continuous Diffusion Models Can Obey Formal Syntax

By Jinwoo Kim, Taylor Berg-Kirkpatrick, Loris D'Antoni

70 score

AI Analysis

Introduces a training-free guidance method for steering continuous diffusion language models to satisfy formal syntactic constraints (regular expressions) by constructing an analytic score function and using its gradient during sampling.

Diffusion language models offer a promising alternative to autoregressive models due to their global, non-causal generation process, but their continuous latent dynamics make discrete constraints -- e.g., the output should be a JSON file that matches a given schema -- difficult to impose. We introduce a training-free guidance method for steering continuous diffusion language models to satisfy formal syntactic constraints expressed using regular expressions. Our approach constructs an analytic sc

Diffusion Language ModelsConstrained GenerationFormal Methods

Research arXiv (Computer Vision) Feb 16

What does RL improve for Visual Reasoning? A Frankenstein-Style Analysis

By Xirui Li, Ming Li and Tianyi Zhou

68 score

AI Analysis

Uses a 'Frankenstein-style' analysis to dissect what RL actually improves in vision-language models compared to supervised fine-tuning, finding that RL primarily induces inference-time shifts in mid-to-late layers that are transferable and composable via model merging.

Reinforcement learning (RL) with verifiable rewards has become a standard post-training stage for boosting visual reasoning in vision-language models, yet it remains unclear what capabilities RL actually improves compared with supervised fine-tuning as cold-start initialization (IN). End-to-end benchmark gains conflate multiple factors, making it difficult to attribute improvements to specific skills. To bridge the gap, we propose a Frankenstein-style analysis framework including: (i) functional

Reinforcement LearningVision-Language ModelsMechanistic InterpretabilityVisual Reasoning

Research arXiv (Machine Learning) Feb 16

Multi-Head Attention as a Source of Catastrophic Forgetting in MoE Transformers

By Anrui Chen, Ruijun Huang, Xin Zhang, Fang Dong, Hengjie Cao, Zhendong Huang, Yifeng Yang, Mengyi Chen, Jixian Zhou, Mingzhi Dong, Yujiang Wang, Jinlong Hou, Qin Lv, Robert P. Dick, Yuan Cheng, Tun Lu, Fan Yang, Li Shang

68 score

AI Analysis

Identifies multi-head attention as a key source of catastrophic forgetting in Mixture-of-Experts Transformers, attributing it to a pre-routing bottleneck where attention concatenates signals into a single router input. This forces routing to act on co-occurring feature compositions rather than separable channels, undermining sparse expert utilization for continual learning.

Mixture-of-Experts (MoE) architectures are often considered a natural fit for continual learning because sparse routing should localize updates and reduce interference, yet MoE Transformers still forget substantially even with sparse, well-balanced expert utilization. We attribute this gap to a pre-routing bottleneck: multi-head attention concatenates head-specific signals into a single post-attention router input, forcing routing to act on co-occurring feature compositions rather than separable

Mixture of ExpertsContinual LearningTransformer Architecture

Research arXiv (Machine Learning) Feb 16

Amortized Reasoning Tree Search: Decoupling Proposal and Decision in Large Language Models

By Zesheng Hong, Jiadong Yu, Hui Pan

68 score

AI Analysis

Identifies a 'Normalization Squeeze' pathology in RLVR where valid but rare reasoning paths are systematically suppressed. Proposes ARTS (Amortized Reasoning Tree Search) to maintain reasoning diversity by decoupling proposal from decision.

Reinforcement Learning with Verifiable Rewards (RLVR) has established itself as the dominant paradigm for instilling rigorous reasoning capabilities in Large Language Models. While effective at amplifying dominant behaviors, we identify a critical pathology in this alignment process: the systematic suppression of valid but rare (low-likelihood under the base model distribution) reasoning paths. We theoretically characterize this phenomenon as a "Normalization Squeeze," where the interplay betwee

Reinforcement LearningLLM ReasoningAlignmentTree Search

Research arXiv (cs.CR) Feb 16

Sparse Autoencoders are Capable LLM Jailbreak Mitigators

By Yannick Assogba, Jacopo Cortellazzi, Javier Abad, Pau Rodriguez, Xavier Suau, Arno Blaas

67 score

AI Analysis

Proposes CC-Delta, a defense against LLM jailbreaks using sparse autoencoders to identify jailbreak-relevant features by comparing representations of harmful requests with and without jailbreak context, then steering in SAE latent space at inference time.

Jailbreak attacks remain a persistent threat to large language model safety. We propose Context-Conditioned Delta Steering (CC-Delta), an SAE-based defense that identifies jailbreak-relevant sparse features by comparing token-level representations of the same harmful request with and without jailbreak context. Using paired harmful/jailbreak prompts, CC-Delta selects features via statistical testing and applies inference-time mean-shift steering in SAE latent space. Across four aligned instructio

AI SafetyJailbreak DefenseSparse AutoencodersLanguage Models

Research arXiv (Machine Learning) Feb 16

SD-MoE: Spectral Decomposition for Effective Expert Specialization

By Ruijun Huang, Fang Dong, Xin Zhang, Hengjie Cao, Zhendong Huang, Anrui Chen, Jixian Zhou, Mengyi Chen, Yifeng Yang, Mingzhi Dong, Yujiang Wang, Jinlong Hou, Qin Lv, Robert P. Dick, Yuan Cheng, Fan Yang, Tun Lu, Chun Zhang, Li Shang

66 score

AI Analysis

Proposes SD-MoE, a spectral decomposition approach for improving expert specialization in MoE LLMs. Identifies that experts share overlapping dominant spectral components and gating routes along these directions, then proposes spectral regularization to improve differentiation.

Mixture-of-Experts (MoE) architectures scale Large Language Models via expert specialization induced by conditional computation. In practice, however, expert specialization often fails: some experts become functionally similar, while others functioning as de facto shared experts, limiting the effective capacity and model performance. In this work, we analysis from a spectral perspective on parameter and gradient spaces, uncover that (1) experts share highly overlapping dominant spectral componen

Mixture of ExpertsLanguage ModelsSpectral AnalysisModel Architecture

Research arXiv (Computer Vision) Feb 16

MonoLoss: A Training Objective for Interpretable Monosemantic Representations

By Ali Nasiri-Sarvi and Anh Tien Nguyen and Hassan Rivaz and Dimitris Samaras and Mahdi S. Hosseini

65 score

AI Analysis

Proposes MonoLoss, a training objective for encouraging monosemantic representations in sparse autoencoders. Derives a single-pass algorithm that computes the MonoScore metric with linear rather than quadratic cost, achieving up to 1200x speedup.

Sparse autoencoders (SAEs) decompose polysemantic neural representations, where neurons respond to multiple unrelated concepts, into monosemantic features that capture single, interpretable concepts. However, standard training objectives only weakly encourage this decomposition, and existing monosemanticity metrics require pairwise comparisons across all dataset samples, making them inefficient during training and evaluation. We study a recent MonoScore metric and derive a single-pass algorithm

Mechanistic InterpretabilitySparse AutoencodersAI Safety

Research arXiv (Machine Learning) Feb 16

VI-CuRL: Stabilizing Verifier-Independent RL Reasoning via Confidence-Guided Variance Reduction

By Xin-Qiang Cai, Masashi Sugiyama

65 score

AI Analysis

Introduces VI-CuRL, a verifier-independent curriculum RL framework using intrinsic model confidence to construct curricula, addressing gradient variance and training collapse in RLVR without external verifiers.

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a dominant paradigm for enhancing Large Language Models (LLMs) reasoning, yet its reliance on external verifiers limits its scalability. Recent findings suggest that RLVR primarily functions by eliciting latent capabilities, motivating the development of verifier-free algorithms. However, in such settings, standard methods like Group Relative Policy Optimization face a critical challenge: destructive gradient variance that ofte

Reinforcement LearningLanguage ModelsRLVRCurriculum Learning

Research arXiv (Computation and Language) Feb 16

SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents

By Yujiong Shen, Yajie Yang, Zhiheng Xi, Binze Hu, Huayu Sha, Jiazheng Zhang, Qiyuan Peng, Junlin Shang, Jixuan Huang, Yutao Fan, Jingqi Tong, Shihan Dou, Ming Zhang, Lei Bai, Zhenfei Yin, Tao Gui, Xingjun Ma, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang

65 score

AI Analysis

Introduces SciAgentGym with 1,780 domain-specific tools across four science disciplines and SciAgentBench for evaluating LLM agents on multi-step scientific tool use. Shows GPT-5 success drops from 60.6% to 30.9% on complex tasks.

Scientific reasoning inherently demands integrating sophisticated toolkits to navigate domain-specific knowledge. Yet, current benchmarks largely overlook agents' ability to orchestrate tools for such rigorous workflows. To bridge this gap, we introduce SciAgentGym, a scalable interactive environment featuring 1,780 domain-specific tools across four natural science disciplines, supported by a robust execution infrastructure. Complementing this, we present SciAgentBench, a tiered evaluation suite

Agentic AIScientific ComputingBenchmarksTool UseLanguage Models

Research arXiv (Artificial Intelligence) Feb 16

Consistency of Large Reasoning Models Under Multi-Turn Attacks

By Yubo Li, Ramayya Krishnan, Rema Padman

65 score

AI Analysis

Evaluates nine frontier reasoning models under multi-turn adversarial attacks, identifying five failure modes including Self-Doubt and Social Conformity. Finds reasoning provides meaningful but incomplete robustness, with all models showing distinct vulnerability profiles.

Large reasoning models with reasoning capabilities achieve state-of-the-art performance on complex tasks, but their robustness under multi-turn adversarial pressure remains underexplored. We evaluate nine frontier reasoning models under adversarial attacks. Our findings reveal that reasoning confers meaningful but incomplete robustness: most reasoning models studied significantly outperform instruction-tuned baselines, yet all exhibit distinct vulnerability profiles, with misleading suggestions

AI SafetyReasoning ModelsAdversarial RobustnessLLMs