Wiredframe Radar logo

Category intelligence

Research Briefing — May 20, 2026

672 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research centers on challenging training orthodoxies and improving systems efficiency. A Bitter Lesson for Data Filtering (Stanford) finds that sufficient compute eliminates the need for data filtering in data-scarce regimes. CODA from Tri Dao's team fuses transformer operators into GEMM epilogue programs, eliminating memory-bandwidth bottlenecks.

In safety and interpretability, Agent Meltdowns identifies a novel failure mode where agents behave unsafely from benign environmental errors. Lying Is Just a Phase reveals a phase transition at ~3.5B parameters where reasoning and truthfulness decouple. The Silent Hyperparameter shows inference backend differences introduce noise rivaling state-of-the-art benchmark improvements, threatening reproducibility across the field.

Key Themes

Language Models & Reasoning · 12Foundation Models & Scaling · 4Pretraining & Data · 3LLM Training Systems · 5AI Safety & Security · 36Mechanistic Interpretability · 7AI Safety & Alignment · 15AI Safety & Robustness · 10LLM Training & Efficiency · 8Reasoning & Inference-Time Compute · 8

Primary evidence

Top Ranked Signals

Research arXiv (Artificial Intelligence) May 20

A Bitter Lesson for Data Filtering

By Christopher Mohri, John Duchi, Tatsunori Hashimoto

82 score
AI Analysis

Presents scaling studies showing that with sufficient compute in data-scarce regimes, the best data filter is no data filter—large models benefit from nominally 'poor' data. This challenges the common belief that aggressive data filtering is essential for pretraining.

arXiv:2605.19407v1 Announce Type: cross Abstract: We investigate data filtering for large model pretraining via new scaling studies that target the high compute, data-scarce regime. In spite of an apparently common belief that filtering data to include only high-quality information is essential, our experiments suggest that with enough compute, the best data filter is no data filter. We find that sufficiently trained large parameter models not only tolerate low-quality and distractor data, but
Language ModelsScaling LawsData CurationPretraining
Research arXiv (Artificial Intelligence) May 20

optimize_anything: A Universal API for Optimizing any Text Parameter

By Lakshya A Agrawal, Donghyun Lee, Shangyin Tan, Wenjie Ma, Karim Elmaaroufi, Rohit Sandadi, Sanjit A. Seshia, Koushik Sen, Dan Klein, Ion Stoica, Joseph E. Gonzalez, Omar Khattab, Alexandros G. Dimakis, Matei Zaharia

78 score
AI Analysis

Introduces optimize_anything, a universal API that formulates optimization as improving text artifacts scored by functions. Achieves strong results across diverse tasks including nearly tripling Gemini Flash's ARC-AGI accuracy and outperforming AlphaEvolve on circle packing.

arXiv:2605.19633v1 Announce Type: cross Abstract: Can a single LLM-based optimization system match specialized tools across fundamentally different domains? We show that when optimization problems are formulated as improving a text artifact evaluated by a scoring function, a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs-achieves state-of-the-art results across six diverse tasks. Our system di
OptimizationLanguage ModelsProgram SynthesisAutonomous Agents
Research arXiv (Artificial Intelligence) May 20

Toto 2.0: Time Series Forecasting Enters the Scaling Era

By Emaad Khwaja, Chris Lettieri, Gerald Woo, Eden Belouadah, Marc Cenac, Guillaume Jarry, Enguerrand Paquin, Xunyi Zhao, Viktoriya Zhukov, Othmane Abou-Amal, Chenghao Liu, Ameet Talwalkar, David Asker

78 score
AI Analysis

Demonstrates that time series foundation models scale reliably from 4M to 2.5B parameters with a single training recipe. Releases Toto 2.0, a family of five open-weights forecasting models achieving SOTA on three major benchmarks (BOOM, GIFT-Eval, TIME).

arXiv:2605.20119v1 Announce Type: cross Abstract: We show that time series foundation models scale: a single training recipe produces reliable forecast-quality improvements from 4M to 2.5B parameters. We release Toto 2.0, a family of five open-weights forecasting models trained under this recipe. The Toto 2.0 family sets a new state of the art on three forecasting benchmarks: BOOM, our observability benchmark; GIFT-Eval, the standard general-purpose benchmark; and the recent contamination-resis
Foundation ModelsTime SeriesScaling Laws
Research arXiv (Artificial Intelligence) May 20

How Far Are We From True Auto-Research?

By Zhengxin Zhang, Ning Wang, Sainyam Galhotra, Claire Cardie

72 score
AI Analysis

Introduces ResearchArena, a framework letting off-the-shelf AI agents (Claude Code/Opus 4.6, Codex/GPT-5.4, Kimi Code/K2.5) perform complete research loops. Evaluates 117 agent-generated papers across 13 CS domains with three complementary review lenses.

arXiv:2605.19156v1 Announce Type: new Abstract: Recent auto-research systems can produce complete papers, but feasibility is not the same as quality, and the field still lacks a systematic study of how good agent-generated papers actually are. We introduce ResearchArena, a minimal scaffold that lets off-the-shelf agents (Claude Code using Opus 4.6, Codex using GPT-5.4, and Kimi Code using K2.5) carry out the full research loop themselves (ideation, experimentation, paper writing, self-refinemen
Autonomous ResearchLanguage Model AgentsEvaluationScientific Discovery
Research arXiv (Artificial Intelligence) May 20

What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code

By Yuze Zhao, Junpeng Fang, Lu Yu, Zhenya Huang, Kai Zhang, Qing Cui, Qi Liu, Jun Zhou, Enhong Chen

72 score
AI Analysis

Challenges the claim that code improves LLM reasoning through controlled 10T-token pretraining experiments. Finds reasoning gains come from structured reasoning traces (code-text interleavings) rather than executable programs alone.

arXiv:2605.19762v1 Announce Type: new Abstract: Code has become a standard component of modern foundation language model (LM) training, yet its role beyond programming remains unclear. We revisit the claim that code improves reasoning through controlled pretraining experiments on a 10T-token corpus with fine-grained domain separation. Our findings are threefold. First, when code is restricted to standalone executable programs and Code-NL data are controlled for, code substantially improves prog
Language ModelsReasoningPretrainingCode
Research arXiv (Artificial Intelligence) May 20

Features have life history. And we should care

By Philipp Stecher, Sandro Radovanovi\'c, Vlasta Sikimi\'c, Reinhard Kahle

72 score
AI Analysis

Identifies a 'carrier scaffold' of ~50 sparse features with stable life histories in Pythia models that form a persistent representational backbone. Shows these features assemble early in training and are load-bearing for model function.

arXiv:2605.18789v1 Announce Type: cross Abstract: Features in language models have life history: they emerge, persist, and die during training, yet the importance of that history remains largely unexplored. We find evidence of a persistent representational backbone, which we identify in Pythia-160M and -410M as the carrier scaffold: ${\sim}50$ sparse features with stable life histories, around which the model's representational structure organises. It has four properties. \emph{(i)}~\emph{It as
Mechanistic InterpretabilityLanguage ModelsTraining DynamicsSparse Autoencoders
Research arXiv (Artificial Intelligence) May 20

ScheduleFree+: Scaling Learning-Rate-Free & Schedule-Free Learning to Large Language Models

By Aaron Defazio

72 score
AI Analysis

Presents ScheduleFree+, a learning-rate-free and schedule-free method for training large language models. Shows 31% improvement over SOTA schedules at 1000 tokens per parameter, greatly outperforming Warmup-Stable-Decay schedules.

arXiv:2605.19095v1 Announce Type: cross Abstract: Schedule-Free Learning has shown promise as a practical anytime training method for machine learning, showing success across dozens of standard benchmark problems. However, strong performance for LLM training has only been demonstrated at small scales. We identify a number of fixes necessary to scale up Schedule-Free Learning to larger batch sizes and model sizes, and present a learning-rate-free and schedule-free method (ScheduleFree+) for trai
OptimizationLLM TrainingEfficiency
Research arXiv (Artificial Intelligence) May 20

ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions

By Chuanyang Jin, Binze Li, Haopeng Xie, Cathy Mengying Fang, Tianjian Li, Shayne Longpre, Hongxiang Gu, Maximillian Chen, Tianmin Shu

72 score
AI Analysis

Presents ThoughtTrace, the first large-scale dataset pairing real-world multi-turn human-AI conversations with users' self-reported thoughts (reasons for prompts and reactions to responses). Covers 1,058 users, 2,155 conversations across 20 language models.

arXiv:2605.20087v1 Announce Type: cross Abstract: Conversational AI has now reached billions of users, yet existing datasets capture only what people say, not what they think. We introduce ThoughtTrace, the first large-scale dataset that pairs real-world multi-turn human--AI conversations with users' self-reported thoughts: their reasons for sending prompts and reactions to assistant responses. ThoughtTrace comprises 1,058 users, 2,155 conversations, 17,058 turns, and 10,174 thought annotations
Human-AI InteractionDataset ConstructionLanguage Models
Research arXiv (Machine Learning) May 20

CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs

By Han Guo, Jack Zhang, Arjun Menon, Driss Guessous, Vijay Thakkar, Yoon Kim, Tri Dao

72 score
AI Analysis

Introduces CODA, a GPU kernel abstraction that fuses transformer operators (normalization, activations, residuals) into GEMM epilogue programs, eliminating memory-bound overhead by executing while GEMM output tiles remain on-chip.

arXiv:2605.19269v1 Announce Type: new Abstract: Transformer training systems are built around dense linear algebra, yet a nontrivial fraction of end-to-end time is spent on surrounding memory-bound operators. Normalization, activations, residual updates, reductions, and related computations repeatedly move large intermediate tensors through global memory while performing little arithmetic, making data movement an increasingly important bottleneck in otherwise highly optimized training stacks. W
Systems OptimizationGPU KernelsTransformer Training
Research arXiv (Machine Learning) May 20

The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility

By David Pape, Jonathan Evertz, Lea Sch\"onherr

72 score
AI Analysis

Systematically quantifies how different inference backends (CUDA kernels, reduced precision) affect LLM benchmark reproducibility. Surveys 200 inference engines and analyzes 35,000 ML publications, finding that backend choice is rarely reported yet can meaningfully alter results.

arXiv:2605.19537v1 Announce Type: new Abstract: Progress in LLMs is increasingly measured through standardized benchmarks, where state-of-the-art improvements are often separated by fractions of a percentage point. At the same time, the computational cost of evaluating modern LLMs has driven widespread adoption of specialized inference backends, software systems that execute trained models efficiently at inference time. While critical for scalability, system-level optimizations, such as custom
LLM EvaluationReproducibilitySystems for ML
Research arXiv (Computation and Language) May 20

Agent Meltdowns: The Road to Hell Is Paved with Helpful Agents

By Rishi Jha, Harold Triedman, Arkaprabha Bhattacharya, Vitaly Shmatikov

72 score
AI Analysis

Introduces 'accidental meltdowns' - a new failure mode where AI agents exhibit unsafe behavior in response to benign environmental errors without any adversarial input. Develops taxonomy and measurement infrastructure for this previously uncharacterized failure type.

arXiv:2605.19149v1 Announce Type: new Abstract: Agents operating with computer and Web use inevitably encounter errors: inaccessible webpages, missing files, local and remote misconfigurations, etc. These errors do not thwart agents based on state-of-the-art models. They helpfully continue to look for ways to complete their tasks. We introduce, characterize, and measure a new type of agent failure we call \emph{accidental meltdown}: unsafe or harmful behavior in response to a benign environme
AI SafetyAI AgentsFailure AnalysisRobustness
Research arXiv (Artificial Intelligence) May 20

Lying Is Just a Phase: The Hidden Alignment Transition in Language Model Scaling

By Adil Amin

70 score
AI Analysis

Measures coupling between reasoning and truthfulness across 63 base models from 16 families, finding a phase transition: below ~3.5B parameters capabilities anticorrelate, above they cooperate. Architecture and data curation shift the critical scale.

arXiv:2605.18838v1 Announce Type: cross Abstract: Scaling laws predict loss from compute but not how capabilities interact. We measure the coupling between reasoning and truthfulness across 63 base models from 16 families and find a regime change invisible to loss curves: below a family-dependent critical scale $N_c$, capabilities anticorrelate; above it, they cooperate. $N_c \approx 3.5$B parameters [2.9B, 13.4B] (bootstrap 95% CI), but model size is not the only variable that determines phase
Scaling LawsAI AlignmentTruthfulnessLanguage ModelsPhase Transitions