Wiredframe Radar logo

Category intelligence

Research Briefing — March 2, 2026

380 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research spans fundamental theoretical limits of AI training, architectural innovations for sequence modeling, and practical advances in inference efficiency and safety.

On the efficiency and training front, LK Losses directly optimize acceptance rates for speculative decoding instead of using KL divergence as a proxy. Recycling Failures salvages mostly-correct RLVR trajectories via fine-grained off-policy guidance, improving sample efficiency. LoRA-Pre reframes optimizer momentum as online linear regression, yielding a principled low-rank optimizer. Separately, removing prior assistant responses from multi-turn LLM conversations surprisingly preserves response quality, and training reasoning models with separate LoRA adapters enables privacy-preserving chain-of-thought without leaking sensitive information.

Key Themes

Language Models · 18AI Safety & Alignment · 14Training & Inference Efficiency · 10Representation Learning · 5Theoretical Machine Learning · 6LLM Evaluation & Safety · 7Differential Privacy · 1Multimodal LLMs & Vision-Language Models · 10Reinforcement Learning · 26LLM Alignment & Safety · 6

Primary evidence

Top Ranked Signals

Research arXiv (Machine Learning) Mar 2

Memory Caching: RNNs with Growing Memory

By Ali Behrouz and Zeman Li and Yuan Deng and Peilin Zhong and Meisam Razaviyayn and Vahab Mirrokni

68 score
AI Analysis

Introduces Memory Caching (MC) for RNNs, which caches checkpoints of hidden states to allow effective memory capacity to grow with sequence length, bridging the gap between fixed-memory RNNs and Transformers on recall-intensive tasks.

Transformers have been established as the de-facto backbones for most recent advances in sequence modeling, mainly due to their growing memory capacity that scales with the context length. While plausible for retrieval tasks, it causes quadratic complexity and so has motivated recent studies to explore viable subquadratic recurrent alternatives. Despite showing promising preliminary results in diverse domains, such recurrent architectures underperform Transformers in recall-intensive tasks, ofte
Language ModelsArchitectureEfficient ModelsSequence Modeling
Research arXiv (Machine Learning) Mar 2

Human Supervision as an Information Bottleneck: A Unified Theory of Error Floors in Human-Guided Learning

By Alejandro Rodriguez Dominguez

65 score
AI Analysis

Develops a unified theory showing that human supervision acts as an information bottleneck, inducing strictly positive excess-risk floors for any learner. Formalizes the 'Human-Bounded Intelligence' limit across six complementary theoretical frameworks.

Large language models are trained primarily on human-generated data and feedback, yet they exhibit persistent errors arising from annotation noise, subjective preferences, and the limited expressive bandwidth of natural language. We argue that these limitations reflect structural properties of the supervision channel rather than model scale or optimization. We develop a unified theory showing that whenever the human supervision channel is not sufficient for a latent evaluation target, it acts as
AI TheoryAlignmentInformation TheoryHuman-AI LearningScaling Limits
Research arXiv (Computation and Language) Mar 2

The Astonishing Ability of Large Language Models to Parse Jabberwockified Language

By Gary Lupyan, Senyi Yang

65 score
AI Analysis

Demonstrates that LLMs can recover meaning from severely degraded English texts where content words are replaced with nonsense strings ('Jabberwockified'). Shows structural cues constrain lexical meaning far more than previously thought.

We show that large language models (LLMs) have an astonishing ability to recover meaning from severely degraded English texts. Texts in which content words have been randomly substituted by nonsense strings, e.g., "At the ghybe of the swuint, we are haiveed to Wourge Phrear-gwurr, who sproles into an ghitch flount with his crurp", can be translated to conventional English that is, in many cases, close to the original text, e.g., "At the start of the story, we meet a man, Chow, who moves into an
Language ModelsLinguisticsNLP Analysis
Research arXiv (Computer Vision) Mar 2

Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models

By Arnas Uselis, Andrea Dittadi, Seong Joon Oh

65 score
AI Analysis

Formalizes three desiderata for compositional generalization in vision embeddings and proves they require linear, orthogonal representation decomposition - providing theoretical grounding for the Linear Representation Hypothesis from first principles.

Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems. Although modern models are trained on massive datasets, they still cover only a tiny fraction of the combinatorial space of possible inputs, raising the question of what structure representations must have to support generalization to unseen combinations. We formalize three desiderata for compositional generalization under standard training (divisibility, transf
Representation LearningComputer VisionLearning Theory
Research arXiv (Machine Learning) Mar 2

CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation

By Weinan Dai, Hanlin Wu, Qiying Yu, Huan-ang Gao, Jiahao Li, Chengquan Jiang, Weiqiang Lou, Yufan Song, Hongli Yu, Jiaze Chen, Wei-Ying Ma, Ya-Qin Zhang, Jingjing Liu, Mingxuan Wang, Xin Liu, Hao Zhou

65 score
AI Analysis

Presents CUDA Agent, a large-scale agentic RL system that develops CUDA kernel optimization expertise through data synthesis, skill-augmented environments, and RL training. Aims to bridge the gap between LLM code generation and compiler-based systems like torch.compile.

GPU kernel optimization is fundamental to modern deep learning but remains a highly specialized task requiring deep hardware expertise. Despite strong performance in general programming, large language models (LLMs) remain uncompetitive with compiler-based systems such as torch.compile for CUDA kernel generation. Existing CUDA code generation approaches either rely on training-free refinement or fine-tune models within fixed multi-turn execution-feedback loops, but both paradigms fail to fundame
Reinforcement LearningCode GenerationAI AgentsGPU Computing
Research arXiv (Computation and Language) Mar 2

Controllable Reasoning Models Are Private Thinkers

By Haritz Puerto, Haonan Li, Xudong Han, Timothy Baldwin, Iryna Gurevych

63 score
AI Analysis

Proposes training reasoning models to follow privacy instructions not only in final answers but also in reasoning traces, introducing separate LoRA adapters for reasoning and answer generation to prevent private information leakage in AI agent reasoning chains.

AI agents powered by reasoning models require access to sensitive user data. However, their reasoning traces are difficult to control, which can result in the unintended leakage of private information to external parties. We propose training models to follow instructions not only in the final answer, but also in reasoning traces, potentially under different constraints. We hypothesize that improving their instruction following abilities in the reasoning traces can improve their privacy-preservat
AI SafetyPrivacyLanguage ModelsReasoning
Research arXiv (Computation and Language) Mar 2

Do LLMs Benefit From Their Own Words?

By Jenny Y. Huang, Leshem Choshen, Ramon Astudillo, Tamara Broderick, and Jacob Andreas

63 score
AI Analysis

Finds that removing prior assistant responses from multi-turn LLM conversations does not affect response quality on a large fraction of turns, with up to 10x reduction in cumulative context length. Shows most multi-turn conversations consist of independent subtasks.

Multi-turn interactions with large language models typically retain the assistant's own past responses in the conversation history. In this work, we revisit this design choice by asking whether large language models benefit from conditioning on their own prior responses. Using in-the-wild, multi-turn conversations, we compare standard (full-context) prompting with a user-turn-only prompting approach that omits all previous assistant responses, across three open reasoning models and one state-of-
Language ModelsEfficiencyMulti-turn Interaction
Research arXiv (Machine Learning) Mar 2

The Stability of Online Algorithms in Performative Prediction

By Gabriele Farina and Juan Carlos Perdomo

62 score
AI Analysis

Proves that any no-regret algorithm in performative prediction settings converges to a mixed performatively stable equilibrium, without requiring assumptions on how models influence distributions. Uses a martingale argument to achieve this unconditional reduction.

The use of algorithmic predictions in decision-making leads to a feedback loop where the models we deploy actively influence the data distributions we see, and later use to retrain on. This dynamic was formalized by Perdomo et al. 2020 in their work on performative prediction. Our main result is an unconditional reduction showing that any no-regret algorithm deployed in performative settings converges to a (mixed) performatively stable equilibrium: a solution in which models actively shape data
Performative PredictionOnline LearningTheoretical ML
Research arXiv (Artificial Intelligence) Mar 2

Causal Identification from Counterfactual Data: Completeness and Bounding Results

By Arvind Raghavan, Elias Bareinboim

62 score
AI Analysis

Develops complete identification theory for counterfactual queries when some Layer 3 (counterfactual) data is available, extending Pearl's causal hierarchy. Provides algorithmic completeness results and tight bounding procedures for partially identifiable quantities.

Previous work establishing completeness results for $\textit{counterfactual identification}$ has been circumscribed to the setting where the input data belongs to observational or interventional distributions (Layers 1 and 2 of Pearl's Causal Hierarchy), since it was generally presumed impossible to obtain data from counterfactual distributions, which belong to Layer 3. However, recent work (Raghavan & Bareinboim, 2025) has formally characterized a family of counterfactual distributions which ca
Causal InferenceCounterfactual ReasoningMachine Learning Theory
Research arXiv (Computer Vision) Mar 2

Synthetic Visual Genome 2: Extracting Large-scale Spatio-Temporal Scene Graphs from Videos

By Ziqi Gao, Jieyu Zhang, Wisdom Oluchi Ikezogwo, Jae Sung Park, Tario G. You, Daniel Ogbu, Chenhao Zheng, Weikai Huang, Yinuo Yang, Winson Han, Quan Kong, Rajat Saini, Ranjay Krishna

62 score
AI Analysis

Introduces SVG2, a massive panoptic video scene graph dataset with 636K videos using GPT-5-based relation inference, and TRaSER, a video scene graph generation model with trajectory-aligned token arrangement. Provides order-of-magnitude scale increase over prior datasets.

We introduce Synthetic Visual Genome 2 (SVG2), a large-scale panoptic video scene graph dataset. SVG2 contains over 636K videos with 6.6M objects, 52.0M attributes, and 6.7M relations, providing an order-of-magnitude increase in scale and diversity over prior spatio-temporal scene graph datasets. To create SVG2, we design a fully automated pipeline that combines multi-scale panoptic segmentation, online-offline trajectory tracking with automatic new-object discovery, per-trajectory semantic pars
Video UnderstandingScene GraphsDataset ConstructionVision-Language Models
Research arXiv (Machine Learning) Mar 2

Optimizer-Induced Low-Dimensional Drift and Transverse Dynamics in Transformer Training

By Yongzhong Xu

62 score
AI Analysis

Studies the geometry of training trajectories in small transformers, finding parameter updates organize into a dominant drift direction with transverse oscillatory dynamics. Reveals that AdamW develops multi-dimensional drift structure distinct from SGD, suggesting optimizer choice fundamentally shapes trajectory geometry.

We study the geometry of training trajectories in small transformer models and find that parameter updates organize into a dominant drift direction with transverse residual dynamics. Using uncentered, row-normalized trajectory PCA, we show that a single direction captures a large fraction of cumulative parameter movement early in training, while remaining components encode oscillatory behavior in auxiliary probe performance. Instantaneous gradients exhibit little alignment with this dominant dir
Training DynamicsOptimizationTransformersMachine Learning Theory
Research arXiv (Machine Learning) Mar 2

LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding

By Alexander Samarin, Sergei Krutikov, Anton Shevtsov, Sergei Skvortsov, Filipp Fisin, Alexander Golubev

62 score
AI Analysis

Proposes LK losses that directly optimize acceptance rate for speculative decoding draft models, rather than using KL divergence as a proxy. Comprehensive experiments across four draft architectures and six target models (8B-685B) show consistent improvements.

Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model. The speedup is significantly determined by the acceptance rate, yet standard training minimizes Kullback-Leibler (KL) divergence as a proxy objective. While KL divergence and acceptance rate share the same global optimum, small draft models, having limited capacity, typically converge to subopti
LLM InferenceSpeculative DecodingEfficiency