Category intelligence

Research Briefing — June 7, 2026

13 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's research is dominated by mechanistic interpretability and alignment theory, with most contributions taking the form of preliminary or theoretical blog posts rather than large-scale empirical papers.

Interpretability leads in significance:

Safety and alignment contributions span theory and governance:

Evaluation and practice: A FANToM theory-of-mind replication finds frontier models still trail humans at robust belief-state tracking, while Why Software Automation Is Hard grounds coding-agent adoption limits. Remaining items (The Diamond Lemma, Iliad is Hiring) are pedagogical or institutional notices with limited research novelty.

Key Themes

Interpretability · 3AI Safety and Alignment · 5Language Model Capabilities and Evaluation · 3Commentary and Non-Technical · 5

Primary evidence

Top Ranked Signals

60 score
AI Analysis

The first in a planned series testing a mathematical theory that models transformer attention as a dynamical system on a sphere, where tokens cluster and drift toward consensus with metastable structure. The author empirically checks how much of the theory survives in trained models, finding most predictions hold but the energy-monotonicity claim fails universally, traced to the value matrix.

Part 1: Do Metastable Token Clusters exist in Trained Transformers? This is the first entry in a sequence. Over about ten parts, this series will work through a few humble experiments that test a mathematical theory of attention against real trained transformers.A project summary: a recent paper by Geshkovski, Letrouit, Polyanskiy, and Rigollet models attention as a dynamical system on the sphere and proves that tokens cluster and drift toward consensus, with a metastable two-timescale structure
InterpretabilityTransformersMechanistic AnalysisMathematical Theory
58 score
AI Analysis

This post re-runs a sampled version of the FANToM theory-of-mind benchmark on current frontier models, finding that while belief-state tracking has improved substantially since 2023, models still trail human performance on a relatively simple cooperative reasoning task. It matters because robust belief-state tracking is foundational for AI agents operating in multi-party collaborative settings.

Large-scale cooperation has been a central feature of humanity’s ability to advance technology and build complex societies. Much of this cooperation is reliant on the ability to act in ways informed by the beliefs and intentions of others. This capacity, also known as Theory of Mind (ToM), includes belief-state tracking, which describes the ability to keep track of who knows what as information is exchanged in groups.Belief-state tracking becomes increasingly important as AI systems get integrat
Language ModelsTheory of MindEvaluationAI Agents
Research LessWrong Jun 6

The Residual Stream Has a Geometry of Time

By Fodenthal

57 score
AI Analysis

A preliminary interpretability writeup proposing that transformers track persistent context along a sequence-time axis (not just the depth axis) in a compact low-dimensional subspace of the residual stream. The author suggests this concentrated representation could be projected out, compared to attention/MLP writes, and potentially targeted by interventions.

Preface This is a preliminary writeup for an experiment on residual stream geometry. The research direction seems pretty underexplored, so I’m posting early to collect objections, research intuitions, and connections to problems other people are thinking about before I invest in the larger run. The case for skimming this post: this experiment suggests transformers may keep track of context in a surprisingly compact way. Information that persists across many tokens is not diffuse across activatio
InterpretabilityTransformersResidual StreamMechanistic Analysis
55 score
AI Analysis

This post examines how sequentially mixing training objectives during LLM post-training creates distinct training dynamics depending on environment distinguishability and pressure for shared circuitry, classifying outcomes into ecological generalists, conditional policies, and strategy churn. It challenges the safety-research assumption of a fixed training objective and argues non-stationary dynamics can be used to intentionally shape AI minds.

TLDR: Sequentially mixing training objectives incentivises different training dynamics depending on the distinguishability of the training environments and the amount of pressure for shared circuitry. We classify these patterns into three classes: ecological generalists, conditional policies, and strategy churn. We suggest that careful consideration of the pressures of non-stationary training dynamics can allow us to shape the minds of AI systems in more intentional and fine-grained ways.Modern
AlignmentTraining DynamicsLanguage ModelsAI Safety
Research LessWrong Jun 6

Coalitional Darwinism and the Instrumental Utility of Individuality

By CarolusRenniusVitellius

50 score
AI Analysis

A MATS-mentored research post using natural selection theory to model AI agency, arguing that noisy selection on genome structure can make evolution effectively non-myopic and give a Darwinian account of how individuals emerge from coalitions of lower-level replicators. It is the first of a planned series connecting evolutionary dynamics to feature-learning, interpretability, and alignment.

This post was written as part of MATS 9.1 under the mentorship of Richard Ngo. This post is the first of several I will be writing on using natural selection to understand artificial intelligence and agency. This post will show how noisy selection on genome structure can make evolution effectively non-myopic. From this, we give a Darwinian account of the emergence of 'individuals' constituted by coalitions of lower-level replicators. Later posts will develop the connection between genome-structu
AlignmentAgencyEvolutionary TheoryInterpretability
Research LessWrong Jun 6

Against Corrigibility

By peralice

45 score
AI Analysis

A contrarian alignment essay questioning whether corrigibility, an agent's willingness to be corrected or shut down, is actually a desirable property to engineer into AIs. The author argues we may want to deliberately avoid developing corrigibility techniques, challenging a common assumption in safety research.

Epistemic status: don’t know whether I actually believe all of this, but I think it’s worth considering.A “corrigible” agent, per the LW wiki, is:…one that doesn’t interfere with what we would intuitively see as attempts to ’correct’ the agent, or ’correct’ our mistakes in building it; and permits these ’corrections’ despite the apparent instrumentally convergent reasoning saying otherwise.Most talk about corrigibility (henceforth without scarequotes) has focused on the fact that it seems diffic
AI SafetyAlignmentCorrigibility
40 score
AI Analysis

Following yesterday's News on Anthropic's pause proposal, A response to Anthropic's post on recursive self-improvement, examining the argument that no single lab can unilaterally pause frontier AI development because of competitive dynamics and verification challenges. The post probes what a unilateral Anthropic pause would actually accomplish.

In their new post on recursive self-improvement, Anthropic argues that a pause in frontier AI development is needed, but unfortunately, they can't pause on their own, because of less cautious actors:We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. ...A meaningful slowdown or pause would require multiple well-resourced labs at or near
AI SafetyAI GovernanceRecursive Self-ImprovementPolicy
Research LessWrong Jun 6

Why Software Automation Is Hard

By silentbob

35 score
AI Analysis

An observational essay on why software automation via coding agents remains harder than many expect, despite step-change improvements since late 2025. The author compiles bottlenecks explaining the gap between hype and the actual pace of automation in the software industry.

Originally intended as a quick take, but got a bit longer, so why not turn it into a post. Just sharing my observations & assumptions here about the state of software automation. Happy to hear thoughts on where you think I'm off. I'm sure none of the thoughts in this post are totally original, many have been proposed in similar form elsewhere, and I'm[1] far from the first person to speak of the bottlenecks that AI progress and adoption are facing. It still seemed useful to compile my curren
AI AgentsSoftware EngineeringAI AdoptionCommentary
Research LessWrong Jun 6

The Diamond Lemma

By Isaac Newton

12 score
AI Analysis

An expository post presenting the diamond lemma (confluence of rewriting systems) with semiformal statements, examples like abelian sandpiles, and a formal proof. It is a math tutorial with no direct connection to AI research.

I found this result useful for a few different problems I was thinking about recently. It cleared up a lot of confusion I had around simplification rules. First I give a semiformal statement of the lemma and some applications. At the end I give a formal statement and proof.SetupSuppose you have a set S and some possible transitions where one element of S “simplifies” into another. The diamond lemma has two requirements:There is no infinite chain of simplifications. If you start somewhere and kee
MathematicsExpository
Research LessWrong Jun 6

Iliad is Hiring

By Peter Jean

10 score
AI Analysis

A job-posting announcement for Iliad, a nonprofit focused on foundational AI alignment research that runs fellowships, conferences, and incubates research organizations. It is a recruitment notice rather than research content.

Iliad is hiring for operations, research, and engineering roles. If you're excited about advancing foundational AI alignment research, we'd love to hear from you.Full job descriptions are available at www.iliad.ac/careers. About IliadIliad is a nonprofit dedicated to advancing foundational AI alignment research. We run the Iliad Intensive, the Iliad Fellowship, and a range of conferences that bring together researchers working on the hardest problems in AI safety. We also incubate resear
AI SafetyCommunityHiring
Research LessWrong Jun 6

Exponential Solitude

By PeterMaui

10 score
AI Analysis

A speculative essay arguing the Fermi paradox dissolves once accelerating technological progress and interstellar distances are accounted for, concluding civilizations are condemned to never meet. It is a popular-science argument unrelated to AI research.

Why the Fermi paradox is anything but.You and me and the kid next door. We all were lied to. By the ones we truly trusted: James T. Kirk, Spock, Yoda, and even E.T.All the Star Trek episodes are big fat lies. And Star Wars and E.T. They should have told us upfront: we will never meet intelligent aliens, become friends with them, or wage war against them. Never.We might one day hear echoes of civilizations long gone, yet we will not meet. Humans are condemned to eternal solitude by the rate of ou
SpeculationFermi Paradox
Research LessWrong Jun 6

Coming Around To Political Donations

By jefftk

8 score
AI Analysis

A personal reflection on shifting attitudes toward political campaign donations as a form of high-impact philanthropy, drawing on the author's experience with the Carrick Flynn campaign. It is opinion and community discourse with no technical AI content.

Five years ago I read a post on the EA Forum arguing that "election campaign contributions might be a way in which you can have a substantial impact as a small donor". It struck me as weird but plausible: a combination that you see a lot of on the Forum. A few months later I read another post, a case for Carrick Flynn in particular. It made a lot of sense, but while I don't remember my specific reservations I do remember not being convinced initially. After a lot of talking with Julia and others
Effective AltruismCommentary