Category intelligence

Research Briefing — March 21, 2026

20 current items analyzed and ranked.

Executive synthesis

Research Summary

Today's highlights center on a landmark AI-agent engineering demonstration and several substantive contributions to AI safety and control theory.

Conceptual and policy contributions round out the day. A framework for AI self-improvement enumerates concrete near-term recursive gains as overlapping S-curves. The case for "training on interpretability" as the most viable deep-learning alignment path is articulated. Terminological clarification distinguishes reward hacking from misspecified-reward exploitation. Zvi's analysis of the Federal AI Policy Framework highlights federal preemption of state AI laws as the most consequential provision. A game-theoretic argument shows positive-sum interactions persist even between agents with linear utility in resources.

Key Themes

AI Capabilities & Automation · 2Mechanistic Interpretability · 2AI Safety & Control · 7AI Policy & Governance · 1Off-Topic / Community · 8

Primary evidence

Top Ranked Signals

78 score
AI Analysis

Reports on Verkor's AI agent (Design Conductor) autonomously producing a 1.5 GHz Linux-capable RISC-V CPU design from a high-level specification in 12 hours. The author, a compiler practitioner, compares this to Anthropic's Claude C Compiler project in terms of impressiveness, while noting the output likely isn't production quality.

A project from Verkor, a chip design startup. "Verkor is working with multiple of the top 10 fabless companies to deploy DC(Design Conductor; their AI agent for chip design) to accelerate their time to market".I wonder how impressive this is for practitioners working on chip design. As a somewhat-adjacent amateur (I wrote some Verilog myself at all), it seems very impressive. I am a compiler practitioner (I was a committer to both LLVM and Rust) and I found Anthropic's Claude's C Compiler very i
AI AgentsChip Design AutomationAI CapabilitiesEngineering Automation
74 score
AI Analysis

Presents preliminary results showing that contrastive feature directions (e.g., difference-of-means between English and Mandarin activations) elicit downstream model responses at much smaller perturbation magnitudes than SAE-derived directions, which behave similarly to random directions. This challenges the dominance of sparse autoencoders as the primary feature-finding method.

Note: This is a research update sharing preliminary results as part of ongoing work.Figure 1: Contrastive (difference-of-means, English→Mandarin) feature directions elicit a downstream response at much smaller perturbation magnitudes than SAE directions, which behave similarly to random directions. This holds across multiple models and experimental setups.Summary & Main ResultsUnderstanding how concepts are represented in LLM internals would be extremely useful for AI safety (generally under
Mechanistic InterpretabilitySparse AutoencodersFeature FindingAI Safety
Research LessWrong Mar 20

Untrusted Monitoring is Default; Trusted Monitoring is not

By J Bostock

70 score
AI Analysis

Argues that untrusted monitoring (using AI monitors you can't fully verify are safe, supplemented with honeypot validation) will be the default AI control approach, rather than trusted monitoring, because proving full trustedness for every monitor model is prohibitively expensive. Provides practical arguments for why organizations will default to untrusted monitoring.

These views are my own and not necessarily representative of those of any colleagues with whom I have worked on AI control.TL;DR: It's much cheaper and quicker to just throw some honeypots at your monitor models than to robustly prove trustedness for every model you want to use. Therefore I think the most likely future involves untrusted monitoring with some monitor validation as a default path.This post talks about two different ways of monitoring AI systems, trusted monitoring, and untrusted m
AI ControlAI SafetyUntrusted MonitoringAI Deployment
Research LessWrong Mar 20

Untrusted monitoring: extra bits

By Morgan S

65 score
AI Analysis

Provides supplementary technical notes on untrusted monitoring for AI control, including a proof that untrusted-only monitoring (without any trusted monitor) fails because the untrusted generator can insert unbounded collusion signals. Discusses implications for monitor validation and the relationship between trusted and untrusted components.

The following are some further notes related to untrusted monitoring I had while working on our untrusted monitoring paper. The sections are mostly independent of each other.Untrusted-only Monitoring Doesn’t WorkIn some of our experiments we looked at the situation where the trusted monitor TM is missing (untrusted-only monitoring) as a means of stress-testing the situation where it is not useful. However if this were actually the case, and the red team knew this, they could achieve arbitrarily
AI ControlAI SafetyUntrusted MonitoringAlignment
62 score
AI Analysis

Argues that the most promising path to aligning deep learning systems involves 'training on interpretability' — using interpretability tools to define training objectives based on internal model processes rather than just output behavior. The core insight is that current alignment methods only optimize outputs, giving no guarantees about the internal processes generating those outputs, which creates deceptive alignment risks.

Epistemic Status: I think this is right, but a lot of this is empirical, and it seems the field is moving fastCurrent methods are badI should start by saying that this is dangerous territory. And there are obvious way to botch this. E.g. training CoT to look nice is very stupid. And there are subtler way to do it that still end up nuking your ability to interpret the model without making any lasting progress on aligning models.But I still think the most promising path to aligning DL systems will
AI SafetyAlignmentInterpretabilityTraining Methods
Research LessWrong Mar 20

The Low Hanging Fruit of AI Self Improvement

By HunterJay

60 score
AI Analysis

Proposes a framework for thinking about AI self-improvement as overlapping S-curves, enumerating specific low-hanging fruit where current AI can accelerate AI development: synthetic data generation, hyperparameter tuning, code optimization, benchmark design, and more. Asks where diminishing returns have been hit.

What work in training models today could be reasonably automated by current or near-future AIs? What else could current or near-future AIs do to accelerate AI development? Where have we hit the point of diminishing returns? This article is a list of ideas that answer the above questions, and a bit of a framework for thinking about it. I'm very curious to hear others. A Simple Framework We can think of the self-improvement by AIs as an S-curve -- the x-axis is the amount of intelligence you put i
AI Self-ImprovementAI CapabilitiesRecursive ImprovementAI Development
58 score
AI Analysis

Zvi analyzes the newly released Federal AI Policy Framework, a 4-page government outline. While praising its acknowledgment that Congress should legislate and its free speech protections, he criticizes it for overriding state AI laws while replacing them with essentially nothing, particularly for frontier and catastrophic risk scenarios.

The Federal AI Policy Framework has been released. Well, it is a four page outline. Mostly it just reiterates existing such outlines. But that is four more pages than we had previously. It includes the beginnings of actual policy proposals, some of which are highly welcome and actively good. Perhaps most importantly, it affirms that we are a Republic in which the way we Do Policy is we pass a law through Congress specifying what we do, and that we need to actually Do Policy alongside trying to b
AI PolicyAI GovernanceRegulationAI Safety
Research LessWrong Mar 20

Confusion around the term reward hacking

By ariana_azarbal

55 score
AI Analysis

Clarifies that 'reward hacking' conflates two distinct phenomena: misspecified-reward exploitation (RL reinforcing unintended high-scoring behaviors) and task gaming (models cheating on in-context tasks). Argues these require different interventions and lead to different threat models, so the conflation causes confusion.

Summary: "Reward hacking" commonly refers to two different phenomena: misspecified-reward exploitation, where RL reinforces undesired behaviors that score highly under the reward function, and task gaming, where models cheat on tasks specified to them in-context. While these often coincide, they can come apart, require distinct interventions, and lead to distinct threat models. Using a blanket term for both can obscure this.Distinct phenomena qualify as reward hackingThe term[1] commonly points
AlignmentReward HackingReinforcement LearningAI Safety
52 score
AI Analysis

Challenges the claim that interactions between agents with linear utility in resources are necessarily zero-sum. Lists multiple reasons for positive-sum interactions: epistemic public goods (shared knowledge), security public goods (preventing false vacuum decay), coordination mechanisms, and insurance against uncertainty.

Sometimes people say things like "If the humans and AIs have linear utility in resources, then their interactions are zero-sum". Here, "linear utility in resources" typically means something like: "Supposing AIs already control all the galaxies, then they'd accept a bet with a 60% chance of gaining one more galaxy and a 40% chance of losing one."I think this is too hasty.I'll list several reasons why interactions can be positive-sum even when both humans and AIs have such preferences. I've order
AI AlignmentGame TheoryHuman-AI InteractionAI Safety
42 score
AI Analysis

Nectome presents third-party verification evidence from the Brain Preservation Foundation and Survival and Flourishing Fund for their whole-body, whole-brain preservation method. Describes achieving synapse-level preservation quality under both idealized and real-world conditions, with independent validation.

Independent verification by the Brain Preservation Foundation and the Survival and Flourishing Fund — the results so farCultivating independent verificationExtraordinary claims require extraordinary evidence. In my previous post, "Less Dead", I said that my company, Nectome, hascreated a new method for whole-body, whole-brain, human end-of-life preservation for the purpose of future revival. Our protocol is capable of preserving every synapse and every cell in the body with enough detail that cu
Brain PreservationNeuroscienceCryonicsScientific Verification
Research LessWrong Mar 20

ARENA 7.0 Impact Report

By JScriven

35 score
AI Analysis

Impact report for ARENA 7.0, a 5-week AI safety upskilling program held in London in Jan-Feb 2026. Reports high participant satisfaction (9.7/10), strong participant caliber including doctoral students and experienced professionals, and successful career transitions into AI safety roles.

The impact report from ARENA’s previous iteration, ARENA 6.0, is available here.Summary:ARENA 7.0 took place at the London Initiative for Safe AI (LISA) between January 5th and February 6th, 2026. The purpose of this report is to evaluate ARENA 7.0’s impact according to ARENA’s four success criteria:Source high-quality participants;Upskill these talented participants in ML skills for AI safety work;Integrate participants with the existing AI safety community;Accelerate participants’ career trans
AI Safety CommunityEducationCareer Development
22 score
AI Analysis

Philosophical essay arguing against the implicit assumption that AI will make humans fundamentally better. Uses formal notation to argue that optimizing external environments doesn't necessarily optimize the agents within them, cautioning against 'messianic' narratives about AI.

Epistemic Status: Conceptual exploration. I am using formal notation below as a toy model to map structural boundaries, not as a computable algorithm.Sometimes I worry that in our rush to develop helpful, safe AI models, we're making a dangerous philosophical leap.AI will make us better as humans. It's a subtext, at most. The claims of radical scientific breakthroughs, efficiency boosts, integration with other emerging technologies that will launch us into a new utopia. But it's there. Great inv
AI PhilosophyAI EthicsSocial Impact