Research Briefing

Introduces MOLE, a benchmark that evaluates defenses against harmful actions performed by AI agents operating shared services under constrained review budgets. Frames insider-threat detection as a resource-limited monitoring problem.

Shows that LLMs encode unanswerability of structurally impossible prompts along a hidden-state direction, but fail to abstain because this recognition signal is misaligned with safety-refusal pathways, indicating a routing rather than encoding deficit.

Show-Harness from NUS Show Lab is an embodied harness that lets frontier and fine-tuned vision-language models directly control robots via a physically grounded semantic action interface, achieving strong cross-embodiment generalization and outperforming several VLA baselines.

Introduces IdeaAMBIG, a benchmark of 660 evidence-grounded instances that measure how faithfully a research method description can be implemented without unsupported assumptions, combining real reproducibility gaps with controlled synthetic ones. The work formalizes codification readiness as a distinct quality axis for research ideas and provides tooling to evaluate coding agents and human implementers against it.

Collaboration write-up describing how Goodfire used Ai2's fully open post-training stack (OLMo) to predict behavioral changes from fine-tuning, trace unwanted behaviors back to individual training examples, and test targeted fixes while preserving capability gains.

The paper introduces On-Policy Reverse Distillation, a method that lets stronger student models exceed weak supervisors by amplifying verifier-supported policy gradients along the teacher's shift direction. It targets weak-to-strong generalization without imposing student capacity constraints, an important alignment-relevant question.

EVOHARNESSBENCH evaluates LLM-based agents under evolving tool, skill, and agent harnesses, exposing persistent deficits in retention, adaptation, and harness-induced forgetting. The benchmark highlights brittleness of agents to harness changes.

Kalman Delta Networks reformulate linear attention as a linear-Gaussian state-space model with Kalman-filter updates, tracking memory uncertainty and yielding efficient scan-compatible approximations that improve language modeling. It bridges classical control-theoretic state estimation with modern linear-attention architectures.