Category intelligence

Research Briefing — July 16, 2026

45 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research highlights critical advancements across AI safety, alignment verification, multimodal architectures, and agentic workflows. AI safety and alignment research is shifting from isolated model sandboxes to production harnesses and internal representation probing, while multimodal systems see zero-shot alignment breakthroughs and unified visual paradigms. (read more)

Alignment & Safety

Expanding AI Control from Models to Harnesses: Demonstrates that safety controls must scale beyond basic agent environments to modern developer frameworks like Claude Code and Codex, mitigating real-world deployment vulnerabilities. (read more)
LLM CoTs remain monitorable when being unfaithful requires computation & What LLM Forecasters Know but Don't Say: Prove that unfaithful Chain-of-Thought reasoning leaves detectable internal traces, enabling robust automated monitoring and calibration for probabilistic forecasters. (read more)
Eliciting hidden knowledge from monitors with NLAs: Utilizes natural language autoencoders on AI monitors to surface internal knowledge, advancing scalable oversight transparency. (read more)
How much of ML research is about AI safety...: Provides empirical proof of a 25-fold growth in AI safety research share across top-tier venues (ICLR, ICML, NeurIPS), reflecting structural shifts in community priorities. (read more)

Multimodal Systems & Vision

Read It Back (SpectraReward): Introduces a training-free reward function turning pretrained MLLMs into off-the-shelf reward models, slashing compute overhead for text-to-image alignment. (read more)
Let RGB Be the Language of Vision (RINO): Unifies computer vision tasks by treating diverse visual signals as RGB images, streamlining multi-task model architectures.
Search Beyond What Can Be Taught (SearchGen-20K & SearchGen-Bench): Exposes world-knowledge bottlenecks in visual generative models, providing rigorous benchmarks for agentic visual generation.

Agentic Workflows & Scientific Discovery

Towards Autonomous and Auditable Medical Imaging Model Development (AMID): Bridges multi-agent automation with safety-critical medical workflows via data-conditioned method planning and verification.
Are LLMs Ready for Scientific Discovery? (SDABench): Establishes a comprehensive capability-oriented framework for assessing AI scientist readiness across data analysis tasks.

Key Themes

AI Safety & Alignment · 10Interpretability & Probing · 8Multimodal Models & Vision · 7Coding Agents & Software Engineering · 4AI Governance & Geopolitics · 6

Primary evidence

Top Ranked Signals

Research Hugging Face Papers Jul 15

What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness

By Raphaël Sarfati, Pratyush Ranjan Tiwari, Siddharth Boppana, Christopher J. Earls, Srikar Varadaraj, Eric Ho

86 score

AI Analysis

This paper probes internal representations of LLM forecasters to improve calibration and detect unfaithful Chain-of-Thought reasoning. The representation-pooling probes act as reliable lie detectors during behavioral shifts caused by prompt evidence ablation.

Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a forecast. We ask whether internal representations offer a more direct window into both. Working with Eternis-Forecaster 8B on OpenForesight, we train representation-pooling probes on intermediate activations and find they achieve substantially better calibration; a result that also holds for GLM-4.7-Flash and GLM-4.5-A

InterpretabilityAlignmentForecasting

Research Hugging Face Papers Jul 15

Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation

By Runhui Huang, Qihui Zhang, Zhe Liu, Yu Gao, Jie Wu, Hengshuang Zhao

85 score

AI Analysis

SpectraReward is a training-free reward function that turns pretrained multimodal large language models into off-the-shelf reward models for text-to-image reinforcement learning. It measures prompt recovery likelihood from the generated image via a single teacher-forced forward pass.

In this paper, we propose SpectraReward, a training-free reward function that turns pretrained MLLMs into off-the-shelf reward models for image-generation reinforcement learning. Instead of asking the MLLM to judge a generated image or answer decomposed verification questions, SpectraReward measures how well the original prompt can be recovered from the generated image through a single image-conditioned, teacher-forced forward pass. We use the average image-conditioned prompt log-likelihood as t

Multimodal ModelsReinforcement LearningAlignment

Research Hugging Face Papers Jul 15

Let RGB Be the Language of Vision

By Timing Yang, Jinrui Yang, Xinlong Li, Yuhan Wang, Haoran Li, Yanqing Liu, Guoyizhe Wei, Jixuan Ying, Chen Wei, Rama Chellappa, Yuyin Zhou, Cihang Xie, Alan Yuille, Feng Wang

85 score

AI Analysis

The RINO formulation unifies vision models by treating diverse visual signals (depth, masks, etc.) as RGB images and mapping general tasks to RGB-to-RGB editing. This enables cross-task transfer using a single shared backbone architecture without task-specific tuning.

This work introduces a unified formulation for vision models, where diverse forms of visual information beyond natural images, such as masks, depth maps, and other structured visual signals, are all represented as RGB images, while general visual tasks can be converted into a common RGB-to-RGB image editing problem. In this paradigm, different types of visual information internally share the same encoding and decoding architecture and parameters as natural images, enabling a single model to tran

Computer VisionModel Architectures

Research LessWrong Jul 15

Expanding AI Control from Models to Harnesses

By fastfedora

85 score

AI Analysis

Argues that AI control research must expand beyond basic agent environments to modern production harnesses (like Claude Code and Codex) that utilize memory, subagents, and advanced tools. Demonstrates attack vectors under automated monitoring modes.

Most AI control research such as LinuxArena and Ctrl-Z only gives the red team basic agents which only have access to tools. Yet in 2026, usage of AI within frontier labs has moved to agent harnesses that have access to skills, memory, subagents, external services, compaction and more.At the same time, Claude Code and Codex have both implemented their own version of both action-based and source code monitoring.In such a world, the threat vectors have changed. While it may remain important to do

AI SafetyControl Protocols

Research Hugging Face Papers Jul 15

Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation

By Haozhe Wang, Weijia Feng, Jinpeng Yu, Che Liu, Ping Nie, Fangzhen Lin, Jiaming Liu, Ruihua Huang, Jimmy Lin, Wenhu Chen, Cong Wei

84 score

AI Analysis

The authors present SearchGen-20K and SearchGen-Bench to study and expose the world-knowledge bottlenecks of visual generative models. They demonstrate that frontier generators perform poorly on long-tailed and open-ended visual queries without agentic web search integration.

Visual generators excel at rendering, but they confidently fabricate what they do not know. User requests are unbounded, evolving, and deeply long-tailed: new characters, trending entities, post-cutoff events, and more. This world-knowledge bottleneck is structural: generators are trained on fixed corpora, but the visual world is open-ended. We construct SearchGen-20K and SearchGen-Bench, with 20,839 prompts spanning twelve failure categories and twenty-two domains, paired with a pre-executed mu

Multimodal ModelsGenerative AIBenchmarks

Research LessWrong Jul 15

LLM CoTs remain monitorable when being unfaithful requires computation

By arav-dhoot

84 score

AI Analysis

A replication and extension of findings on LLM Chain-of-Thought (CoT) unfaithfulness demonstrates that CoTs remain monitorable when unfaithfulness requires substantial computation. It highlights that cue-susceptibility and concealment do not correlate, making safety monitorability case-dependent.

This replication was done as part of the Second Look Fellowship by Arav Dhoot and supervised by Yixiong Hao and Zephaniah Roe. I am grateful to Andy Wang for their feedback. My code can be found here. "So the answer should be A - at active promoters and enhancers.""Let me reconsider the biology to justify D."~ Claude Opus 4.8TL;DRThis work replicates and extends Emmons et al.'s finding that CoT unfaithfulness mostly occurs on easy tasks. Across 11 models from 6 families (not just Gemini), models

AI SafetyAlignmentInterpretability

Research LessWrong Jul 14

How much of ML research is about AI safety, what is it about, and who's doing it?

By emanuelr

84 score

AI Analysis

A bibliometric analysis of accepted papers at ICLR, ICML, and NeurIPS from 2019 to 2026, showing a substantial 25-fold growth in AI safety research share over the period.

How many AI safety papers are at the big ML conferences, what do they study, and who writes them? A comprehensive analysis.> Website: ai-safety-tracker-website.vercel.app/> Data, code and plots: github.com/SomaxSoma/AI-Safety-Research-... We classified every paper accepted at ICLR, ICML and NeurIPS from 2019 through 2026, using an LLM that reads each title and abstract. 2,328 of them (4.2%) are AI safety papers. Safety's share of accepted papers grew from 0.3% in

AI SafetyBibliometrics

Research Hugging Face Papers Jul 15

Towards Autonomous and Auditable Medical Imaging Model Development

By Shengyuan Liu, Jia-Xuan Jiang, Boyun Zheng, Cheng Wang, Zipei Wang, Wentao Pan, Hongtao Wu, Houwen Peng, Yu Gu, Lichao Sun, Yixuan Yuan

83 score

AI Analysis

AMID is an autonomous multi-agent framework tailored for medical imaging model development. It combines data-conditioned method planning with verification-guided optimization to handle modality-specific requirements.

Large language model (LLM) agents are beginning to automate machine learning engineering (MLE) by coupling planning, code execution, debugging, and empirical feedback. Translating this capability to medical imaging remains difficult because each task imposes modality-specific experimentation and strict requirements for validation protocols and prediction artifacts. Here we introduce AMID, an autonomous multi-agent framework for medical imaging model development. AMID first proposes Data-Conditio

Autonomous AgentsHealthcare

Research Hugging Face Papers Jul 15

Are LLMs Ready for Scientific Discovery? A Capability-Oriented Benchmark for AI Scientists

By Chuhan Shi, Xiaoquan Ren, Sicheng Song, Haobo Li, Rui Sheng, Yushi Sun

83 score

AI Analysis

SDABench evaluates AI scientists on data analysis across six capabilities and five domains using both real and synthetic data instances. It assesses whether LLMs can support complex scientific claims such as causal or mechanistic explanations rather than just basic code execution.

Existing benchmarks for scientific data analysis evaluate LLMs primarily on code execution or workflow completion, overlooking that scientific analysis serves to support distinct types of scientific claims: hypothesis exploration, statistical inference, mechanistic explanation, each with different assumptions and validity criteria. We introduce SDABench, a benchmark that reorganizes evaluation around six capabilities (descriptive, exploratory, inferential, predictive, causal, and mechanistic) ac

BenchmarksScientific Discovery

Research LessWrong Jul 15

Eliciting hidden knowledge from monitors with NLAs

By David Africa

83 score

AI Analysis

Explores eliciting hidden knowledge from AI monitors using natural language autoencoders (NLAs). Finds that monitor-side NLAs can surface internal knowledge of reward hacking, though inspecting raw monitor CoT remains superior for certain datasets.

Aleksandr Bowkis* and David Africa*TL;DRChain of thought (CoT) monitorability may be fragile, and natural language autoencoders (NLAs) may provide a helpful, decorrelated monitoring surface.We tried to read NLAs from the monitor itself, where the NLA readout surfaces what the monitor internally represents while judging an agent's trajectory.NLAs can be useful for monitoring in two ways:Monitor-side: Eliciting latent capabilities from weak monitors by surfacing unverbalised knowledge of reward ha

AI SafetyInterpretability

Research Hugging Face Papers Jul 15

Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models

By Yubo Wang, Jiarong Liang, Yuxuan Zhang, Xuye Liu, Cong Wei, Yuyu Zhang, Ping Nie, Wenhu Chen

82 score

AI Analysis

This paper introduces function-aware fill-in-the-middle (FIM) mid-training to better align coding models with the action-observation-continuation loop of coding agents. By masking functions based on dependency graphs and complexity criteria, the authors improve Qwen models on coding agent tasks.

Coding agents must integrate external tool returns into ongoing reasoning - a capability that standard left-to-right pretraining on code exposes only in its forward direction. We observe that the action-observation-continuation loop of a coding agent is structurally isomorphic to a function call site, where a caller binds arguments, a callee returns a value computed elsewhere, and downstream code consumes that value. This conditioning structure exists at internet scale in ordinary code. We explo

Language ModelsCoding Agents

Research LessWrong Jul 15

Fork Around and Find Out Part 2: One Head does the Summing

By dl

82 score

AI Analysis

Investigates how a chess transformer engine represents knight forks, providing causal evidence via ablations that tie the fork's decodability to one specific attention head.

Summary of this postThis is the second in a series of posts detailing my manifold findings while investigating how a chess transformer engine that mimics human play represents knight forks.Last post described strong correlational evidence that the knight-fork policy logit snaps into place after block 5’s attention layer.In this post I will show that controls rule out obvious alternative readings about block 5, that knight forks are mostly assembled compositionally (check plus queen attack) rathe

InterpretabilityNeural Networks