R[AI]DAR

Category intelligence

Research Briefing

All 8 current items, analyzed and ranked.

Executive synthesis

Research Summary

Executive Signal

  • Today's research portfolio reveals a widening gap between maturing technical interpretability and unresolved alignment foundations, while takeoff speculation increasingly anchors on industrial bottlenecks rather than algorithmic breakthroughs.

Priority Developments

  • Interpretability and user-modeling emerge as the leading technical risks: Channel amplification scores offer a mechanistic handle on natural features, while empirical work shows frontier LLMs embed and act on stereotypical user models.
  • Alignment theory is rebuilding from first principles: Legendre duality between probabilities and utilities and a third option in the measure problem signal renewed investment in foundational formalisms for agency and decision.
  • Takeoff speculation hinges on industrial rather than algorithmic scaling: A 2040-2050 compute buildout framing implies physical and capital constraints, not model architecture, will govern ASI timelines.
  • Research infrastructure is degrading under AI-driven load: Proposals to overhaul peer review and evidence of managerial misapplication indicate editorial and organizational systems have not absorbed generative-AI throughput.

Leadership Implications

  • Prioritize dual-track R&D: Fund mechanistic interpretability and user-model auditing as near-term defenses while sponsoring foundational alignment work as long-horizon insurance.
  • Treat research and productivity plumbing as strategic assets: Invest in review infrastructure and AI-augmented managerial workflows before accumulated throughput erodes signal quality.

Key Themes

AI Safety & Alignment Research · 3Mechanistic Interpretability · 2Foundations & Decision Theory · 2AI Forecasting & Strategy · 3AI Community & Career · 3

Primary evidence

All Ranked Signals

Showing 8 of 8 signals
Research LessWrong Aug 23

In search of natural features

By Dmitry Vaintrob

55 score
AI Analysis

Reports preliminary mechanistic interpretability experiments on GPT-2 small (layer-6 MLP), introducing a channel amplification score to distinguish natural features from incidental ones in neural network representations. Builds on prior work by Heimersheim, Ferreira, Hanni, Mendel, Chan, and Adler & Shavit on Computation in Superposition, framing an information-theoretic criterion based on whether the model actively error-corrects a feature.

Source details
I'm sharing preliminary results of a suite of experiments I ran with claudecode on a small LLM (gpt2-small, no Layer Norm version, courtesy of Apollo research. most of these are on the layer-6 MLP). The github repo for the experiments is here. The success of these experiments given the method's simplicity surprised me, and I would appreciate criticism and bug-finders.This is the headline result. This is not an abstract cartoon, but an exact experimental graph. Yes, I will explain.The key idea in
Mechanistic InterpretabilityAlignment ResearchNeural Network Analysis
Research LessWrong Aug 22

When does an LLM’s model of you affect its behaviour?

By Cat McGee

55 score
AI Analysis

Empirical study of whether frontier LLMs (GPT-5.6, Gemini 3.1 Pro, Claude Opus 5) change their outputs based on internal models of the user (gender, age, SES, education, mood). Finds that steering user representations in smaller open models (Llama-3.2-3B, Qwen2.5-7B, OLMo-2-7B) produces large stereotypical shifts (e.g., +141% in salary recommendations with higher SES), and that frontier models retain gender stereotypes in some areas.

Source details
Disclaimer: figures in this post are edited by ChatGPTIn my last post, I looked at what makes LLMs form opinions of their users: gender, age, socioeconomic status, education, and mood. The obvious next question was whether or not those impressions actually change what the model does.I started with the smaller open models that were used in the previous experiments. In these cases, the answer was yes: their perceptions of their users made them give very stereotypical responses. For example, steeri
AI SafetyBias & FairnessInterpretabilityLanguage Models
Research LessWrong Aug 23

Utilities as Legendre duals of probabilities

By Fernando Rosas

48 score
AI Analysis

Explores Roy Fox's recent proposal to characterize an agent's capabilities as the set of environment dynamics it can bring about, and derives a Legendre-Fenchel duality between probabilities and utilities as a way to formalize agent power. Motivated by dissatisfaction with reward-based descriptions of capability for real-world alignment settings.

Source details
TLDR: In recent work, Roy Fox proposes to understand an agent's capabilities in terms of the set of environment dynamics it can bring about.[1] This leads to an intriguing duality between probabilities and utilities via the Legendre-Fenchel transform.IntroductionSome agents are more powerful than others. Indeed, some can yield a wider range of outcomes, maybe because they are capable long-term planners or because they have built rich world models. Being able to clearly delineate the capabilities
Reinforcement LearningAI SafetyAgent FoundationsAlignment Research
Research LessWrong Aug 23

Twenty Years from RSI to Takeoff: Slow Learning, Scaling Slowdown, Industrial Explosion

By Vladimir_Nesov

20 score
AI Analysis

Speculates on AI takeoff timelines, arguing that 'slow-learning' prosaic RSI via LLMs could reach AGI by 2028-2032, but that an industrial-explosion-driven compute buildout around 2040-2050 is what would unlock 1000x-faster model-building loops and ASI. Frames current LLM/pretraining/RL methods as hobbled relative to a software-only singularity.

Source details
Industrial explosion is what will make the next-model building loops (and thus learning) with LLMs 1000x faster by about 2050, if indeed the slow-learning prosaic RSI becomes AGI before the big compute buildout slowdown of 2032+ that is already starting. This puts an upper bound on how long it takes to invent ASI that sets off software-only singularity, implementing efficient online learning and fixing all the other hobblings of the likely near-future AGI technology (LLMs/pretraining/RL). The in
AI ForecastingAGI StrategyScaling
Research LessWrong Aug 23

PSA: There's a third option in the "measure problem"

By Elias Schmied

18 score
AI Analysis

Argues for a third option in the measure problem (how to assign probabilities over all possible realities), beyond the usual simplicity prior (Tegmark/Schmidhuber/UDASSA) and caring measure (Wei Dai, Christiano, Garrabrant) camps. Targets a niche philosophical question at the intersection of cosmology, decision theory, and AI safety foundations.

Source details
This post is somewhat niche, and I will sometimes not give context or link relevant background.There’s a big debate that has played out in slow motion on LessWrong over the past two decades, between two broad ways of putting a measure over all possible realities (Tegmark IV):Some “objective” prior (a “reality fluid”), usually a simplicity prior[1]: This is the position taken by Max Tegmark, Jürgen Schmidhuber and UDASSA.A “caring measure”, where we say that our preferences determine our probabil
Decision TheoryPhilosophy of AIFoundations
Research LessWrong Aug 23

How to overhaul the broken review system

By RobinHa

15 score
AI Analysis

Proposes a redesigned research and peer review platform for AI that addresses problems like AI-generated slop, lack of reproducibility, and misaligned reviewer incentives. Uses fictional researcher personas to illustrate the proposed system and discusses implementation questions.

Source details
TLDR: After briefly outlining the problems with the current review ecosystem, I detail in-depth a research & review platform which addresses these points and seems plausible to me. I further illustrate 4 imaginary researchers to better understand what this would mean in practice and try answering some additional questions the reader might have, like how this would actually be implemented.I don't know a single person who would argue that the current AI research ecosystem is working out. Confe
Research CommunityPeer ReviewMeta-science
Research LessWrong Aug 23

A Generalist Thinks in Terms of Problems, Not Job Descriptions

By Roman Ross

10 score
AI Analysis

Personal essay arguing that 'generalist' in AI safety should be defined as 'does whatever is needed to solve a set of problems' rather than as a fixed job role, fieldbuilder, or impact multiplier. Reflects community debate about career roles rather than presenting technical content.

Source details
This is a timed post. Every 5 minutes while writing this post, I had to stop to do 15 push-ups, and when I could no longer complete my required number, I had to upload it. My friends thought this would be a fun challenge, but it means that it will likely be less polished than some of my other output.Summary: There has been a lot of talk in the EA/AI safety community recently about what it means to be a “generalist,” but many disagree on what this means. In this post, I give my take on the role:
AI Safety CommunityCareer Advice
Research LessWrong Aug 22

Prompt Sufficiency: A Missive for the Managerial Class

By KAP

10 score
AI Analysis

Argues that slow AI productivity gains stem from managerial misapplication of LLMs, advocating that managers delegate the 'what' to skilled users rather than over-specifying prompts. A management-style essay rather than technical research.

Source details
AI Assisted Work: A Missive For the Managerial ClassThe Smart Employee“It doesn't make sense to hire smart people and then tell them what to do, We hire smart people so they can tell us what to do.” - Steve JobsThis famous quote is bound to provoke some discomfort: What is the meaning of leadership if it dictates neither what nor how? The answer, according to Mr. Jobs, is that effective leadership is sometimes accomplished through yielding control, rather than holding on to it. By delegating asp
AI AdoptionProductivityManagement