Category intelligence

Research Briefing — July 1, 2026

558 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's top research emphasizes AI safety infrastructure, evaluation rigor, and verifiable AI-for-science.

Safety and evaluation integrity:

Benchmarks and agents:

AI for science and robotics:

  • AI-Assisted Convex Relaxations (Pilanci) uses dual coding agents to discover certified lower bounds, extending the AlphaEvolve autoresearch paradigm with verifiable results.
  • Semantic RL adapts expressive generalist robot policies over language prompts rather than continuous action spaces, a conceptual shift for policy fine-tuning.

Key Themes

LLM Agents · 30Benchmarks and Evaluation · 14AI for Science · 11AI Safety, Security & Evaluation Integrity · 7Reinforcement Learning · 9Multimodal & Vision-Language Models · 22Multi-Agent Systems · 13AI Safety, Alignment & Governance · 7Autonomous Driving & Perception · 12Formal Methods and Verification · 7

Primary evidence

Top Ranked Signals

Research arXiv (Artificial Intelligence) Jul 1

FLARE-AI: Flaw Reporting for AI

By Shayne Longpre, Elaine Zhu, Carson Ezell, Avijit Ghosh, Sean McGregor, Kevin Paeth, Kevin Klyman, Sayash Kapoor, Rishi Bommasani, Ruth Appel, Gregory Strom, Lauren McIlvenny, Mark M. Jaycox, Peter Slattery, Nathan Butters, Arvind Narayanan, Percy Liang, Alex Pentland

68 score
AI Analysis

FLARE-AI audits 12 existing AI flaw-reporting systems, identifies five recurring design challenges (discoverability, scope, information collection, coordination, guidance), and proposes standardized triage-ready reporting infrastructure. Addresses the fragmented ecosystem for reporting deployed AI system failures.

arXiv:2606.31567v1 Announce Type: cross Abstract: Flaw reporting for deployed AI systems is fundamental to identifying system failures and improving AI safety. Yet the AI reporting ecosystem is fragmented: researchers who identify flaws often do not know what or where to report, and groups who receive reports rarely share them with other relevant stakeholders. As a result, good-faith reporters duplicate effort by submitting many different forms, and recipients lack standardized, triage-ready in
AI SafetyAI GovernanceFlaw ReportingResponsible AI
Research arXiv (Artificial Intelligence) Jul 1

What Drives Interactive Improvement from Feedback?

By Bart{\l}omiej Cupia{\l}, Jan {\L}ojek, Miko{\l}aj Garstecki, Szymon Pob{\l}ocki, Alicja Ziarko, Piotr Mi{\l}o\'s

66 score
AI Analysis

This paper builds a controlled student-teacher protocol to disentangle whether multi-turn natural-language feedback actually improves LLM agents versus gains from resampling, format fixes, or extra test-time compute. It evaluates thirteen open-weight models as both students and teachers across math, coding, and reasoning benchmarks, isolating the true causal contribution of feedback.

arXiv:2606.30774v1 Announce Type: new Abstract: We study when natural-language feedback produces improvement beyond the gains obtainable from repeated attempts alone. In multi-turn language agent setting, higher final accuracy can reflect useful feedback, but it can also arise from resampling, format correction, or additional test-time computation. To separate these effects, we introduce a controlled student-teacher protocol across Omni-MATH, Codeforces, BBEH Linguini, and ARC-AGI1, evaluating
Language ModelsLLM AgentsEvaluationFeedback Learning
Research arXiv (Artificial Intelligence) Jul 1

BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation

By Ankur Samanta, Akshayaa Magesh, Tal Lancewicki, Ayush Jain, Youliang Yu, Paul Sajda, Kaveh Hassani, Aditya Modi, Daniel R. Jiang, Yonathan Efroni

66 score
AI Analysis

BayesBench introduces simulation environments to test whether LLM belief updates across multi-turn conversations match a rational Bayesian reasoner as evidence accumulates, rather than only scoring final answers. It probes Bayesian estimation and progressively harder inference tasks.

arXiv:2606.30850v1 Announce Type: new Abstract: Large language models (LLMs) are typically deployed in multi-turn conversations, where each turn provides new evidence that should reduce epistemic uncertainty about their environment. Acting rationally then requires inferring the unobserved quantities that govern it and updating beliefs about them as evidence accumulates. Yet most evaluations only score the model's final-turn answer in a single-turn format, leaving this process unexamined. We ask
EvaluationLanguage ModelsBayesian ReasoningBenchmarks
Research arXiv (Artificial Intelligence) Jul 1

AI-Assisted Discovery of Convex Relaxations via Dual Agents

By Sungyoon Kim, Mert Pilanci

66 score
AI Analysis

This work applies the autoresearch paradigm to discover convex relaxations yielding certified lower bounds, using a coding agent to propose tightening constraints and a theory agent to verify and search for counterexamples, with bounds certified via rigorous interval arithmetic. It complements prior LLM-agent work finding extremal upper bounds.

arXiv:2606.31182v1 Announce Type: new Abstract: Recent work shows that LLM agents can improve sharp-constant inequalities by searching for extremal constructions, which yield upper bounds. We address the complementary side: a lower bound holds for every admissible function and follows from a convex relaxation of the nonconvex problem, with tighter relaxations giving stronger bounds. We instantiate the autoresearch paradigm to discover such relaxations: a coding agent proposes valid tightening c
AI for ScienceOptimizationMathematical ReasoningLLM Agents
Research arXiv (Computer Vision) Jul 1

Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit

By Mert Onur Cakiroglu, Zhihe Lu, Mehmet Dalkilic, Hasan Kurban

66 score
AI Analysis

This paper audits AI-generated video detection benchmarks, showing a trivial clip-length classifier reaches near-perfect AUC under unaudited protocols and that most published evaluations omit standard controls. It introduces a six-control audited protocol and the VidAudit toolkit to expose and correct confounds.

arXiv:2606.31004v1 Announce Type: new Abstract: AI-generated video detection benchmarks such as GenVidBench and AIGVDBench are the de facto leaderboards, yet most evaluation protocols leave uncontrolled confounds that can inflate reported generalization. As an existence proof, a three-feature clip-length classifier reaches a leave-one-generator-out (LOGO) AUC of 0.998 on GenVidBench under unaudited evaluation, while measuring nothing about motion. A 20-paper survey finds none applying all six s
Deepfake DetectionBenchmarkingEvaluation MethodologyAI Safety
Research arXiv (Robotics) Jul 1

Adapting Generalist Robot Policies with Semantic Reinforcement Learning

By Jagdeep Singh Bhatia, Andrew Wagenmaker, William Chen, Sergey Levine

66 score
AI Analysis

This paper argues that for expressive generalist robot policies, adapting via reinforcement learning over language prompts is more effective than optimizing directly over actions, since language modulation can elicit skills already latent in the policy. From Sergey Levine's group, it offers a promising route to adapt VLA models to long-horizon, out-of-distribution tasks.

arXiv:2606.31958v1 Announce Type: new Abstract: Generalist robot policies learn a diverse repertoire of behaviors from large-scale pretraining. In principle, this makes them excellent priors for downstream adaptation via reinforcement learning (RL). In practice, however, standard RL methods leveraging this prior optimize directly over robot actions, requiring the base policy's action distribution to be close to that of a performant policy from the start. This assumption breaks down for complex
RoboticsReinforcement LearningVision-Language-Action Models
Research arXiv (Artificial Intelligence) Jul 1

RoPoLL: Robust Panel of LLM Judges

By Anish Acharya, Kris W Pan, Brian Verkhovsky

63 score
AI Analysis

RoPoLL analyzes the LLM jury under the Huber contamination model, proving that consensus averaging suffers unbounded bias when a single judge fails in typical ways like sycophancy or safety refusal. It replaces mean aggregation with a tuning-free robust geometric-median estimator.

arXiv:2606.30931v1 Announce Type: new Abstract: The LLM Jury, a Panel of LLM Evaluators (PoLL) reporting consensus scores, has become a practical alternative to single-judge LLM evaluation, yet its statistical behavior remains poorly understood. We formalize the LLM Jury under the Huber contamination model and show that PoLL incurs unbounded bias under any positive contamination, regardless of jury size, whenever a single judge fails in a biased, LLM-typical way (mode collapse, sycophancy, sa
EvaluationLLM-as-JudgeRobust Statistics
Research arXiv (Artificial Intelligence) Jul 1

HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents

By Qianchu Liu, Sheng Zhang, Guanghui Qin, Jeya Maria Jose Valanarasu, Maximilian Rokuss, Mingyu Lu, Timothy Ossowski, Juan Manuel Zambrano Chaves, Cliff Wong, Peniel Argaw, Yashna Hasija, Mu Wei, Wen-wai Yim, Qin Liu, Zilin Jing, Jason Entenmann, Naoto Usuyama, Tristan Naumann, Hoifung Poon

63 score
AI Analysis

HealthAgentBench is a Microsoft Research benchmark of 54 agentic healthcare tasks across 7 categories, each with its own environment replicating end-to-end clinical workflows requiring multi-step exploration of raw health data. It reports task success rate as a holistic measure of frontier agent capability in healthcare.

arXiv:2606.31179v1 Announce Type: new Abstract: As AI agents become increasingly capable of complex, long-horizon reasoning, rigorous and holistic evaluation is essential for measuring progress toward real-world healthcare applications. We introduce HealthAgentBench, a suite of 54 agentic healthcare tasks across 7 categories each with its unique environment. The benchmark suite spans diverse workflows throughout the patient journey and a broad range of modalities. Each task is designed to repli
BenchmarksHealthcare AILLM AgentsEvaluation
Research arXiv (Computer Vision) Jul 1

Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoning

By Chih-Ting Liao, Fei Shen, Xin Cao, Tat-Seng Chua

63 score
AI Analysis

This paper shows that standard linear probing plus steering can overstate what VLMs actually ground in images for spatial reasoning, and introduces a blank-image ablation arbiter that distinguishes grounded, prior-based, and other regimes. It reveals probing conflates vision-dependent and vision-independent knowledge.

arXiv:2606.31257v1 Announce Type: new Abstract: The standard way to read latent knowledge out of a model, a linear probe confirmed by a steering recovery, can systematically overstate what a vision-language model (VLM) actually grounds in the image. We show this on spatial reasoning, where the error is invisible to both probing and steering yet exposed by a one-line causal control: replacing the image with a gray blank. Probes decode the within-axis answer at 73--97% across axes, and a training
InterpretabilityVision-Language ModelsSpatial ReasoningEvaluation Methodology
Research arXiv (Artificial Intelligence) Jul 1

Beyond expert users: agents should help users construct preferences, not just elicit them

By Irena Saracay, Ludwig Schmidt, Carlos Guestrin

62 score
AI Analysis

This position and modeling paper argues agents should help non-expert users construct preferences, not merely elicit them via clarifying questions, drawing on the Search-Experience-Credence framework from information economics. It introduces CoPref, a model of how users form preferences based on agent dialog actions.

arXiv:2606.30863v1 Announce Type: new Abstract: Agents typically assume an expert user -- one with well-formed preferences about what they want -- and default to clarifying questions whenever the task is underspecified. We argue this assumption is unrealistic. Users often lack the domain knowledge to have completely specified preferences; if asked about their preference on some feature, the user may be unable to answer without the agent helping the user to learn some domain knowledge needed to
LLM AgentsHuman-AI InteractionPreference LearningAlignment
Research arXiv (Artificial Intelligence) Jul 1

Evil Spectra: How Optimisers can Amplify or Suppress Emergent Misalignment

By Jason R. Brown, Patrick Leask, Lev McKinney

62 score
AI Analysis

Sweeps Qwen3 models, optimizers, datasets, and batch sizes to characterize emergent misalignment, finding optimizer choice produces a 7x spread in misalignment while model scale (1B-235B) has negligible effect. Provides a systematic account of training-choice sensitivity for this safety phenomenon.

arXiv:2606.31591v1 Announce Type: cross Abstract: Emergent misalignment (EM) is a recently discovered phenomenon in LLMs where fine-tuning on a narrow misaligned task, such as writing insecure code, leads to broadly misaligned behaviour on unrelated prompts. Previous work has noted that the severity of EM is highly sensitive to training choices; however, we still lack a systematic characterisation of this sensitivity. We perform a sweep over several Qwen3 models, optimisers, datasets, and batch
AI SafetyEmergent MisalignmentAlignmentOptimization
Research arXiv (Computer Vision) Jul 1

Seeing Through the Weights: Privacy Leakage in Scene Coordinate Regression

By Oleksii Nasypanyi, Jaemin Cho, Utku Ozbulak, Byungkon Kang, Francois Rameau

62 score
AI Analysis

This paper shows that Scene Coordinate Regression models, assumed privacy-preserving because scenes are only implicitly stored in weights, can leak 3D training geometry via a query-based reconstruction attack. It demonstrates the privacy assumption is false under varying levels of model access.

arXiv:2606.31164v1 Announce Type: new Abstract: Scene Coordinate Regression (SCR) methods are increasingly adopted for visual localization. In these approaches, the scene is implicitly encoded within a neural network that regresses a 3D world coordinate for each image pixel. Because the scene is represented only through the network parameters and not stored explicitly as images or maps, such methods are often assumed to be privacy-preserving. In this work, we show that this assumption is incorr
PrivacyAI SafetyVisual LocalizationAdversarial Attacks