Category intelligence

Research Briefing

All 13 current items, analyzed and ranked.

Executive synthesis

Research Summary

Executive Signal

  • Frontier-safety forecasting and agent infrastructure dominate signal: a falsifiable 2027 replication-incident prediction, an ASIC-rooted governance plan, and a persistent-memory integrity benchmark for tool-using agents together set the agenda for near-term risk and capability work.

Priority Developments

  • Quantitative safety forecasting translates Epoch AI capability-density datapoints into a concrete, testable 2027 replication-incident prediction, raising the bar from qualitative x-risk rhetoric to falsifiable claims that boards and labs can act on.
  • Agent memory and orchestration stack advances persistent-memory integrity/crash-continuity evaluation and swarm-style orchestration, directly shaping how multi-step, tool-using agents will be deployed in production.
  • Embodied AI with measurable deployment gains shows test-time trajectory repair improves closed-loop driving without retraining, a pragmatic pattern likely to spread across robotics stacks.
  • Hard-tech governance architecture extends Plan R with ASIC diversity, escrow, and political-rights mechanisms — the most concrete compute-rooted proposal yet for limiting concentration risk.
  • Economic and epistemic reframings of AI impact argue AI substitutes for labor faster than it complements it, demand extraordinary-evidence standards for x-risk claims, and document the affective gap that weakens extinction communication.

Leadership Implications

  • Stress-test replication and governance assumptions now: assign a red-team to validate the 2027 forecast and commission a feasibility review of ASIC-rooted deployment splits before the next model cycle.
  • Treat agent memory integrity as a deployment gate: adopt crash-continuity and integrity metrics alongside retrieval accuracy when shipping tool-using agents to customers.

Key Themes

AI Safety and Governance · 4AI Agents and Tool Use · 2Autonomous Driving and Robotics · 1Frontier Model Analysis · 2Economics of AI · 1Existential Risk Discourse · 2Community and Personal Essays · 3

Primary evidence

All Ranked Signals

Showing 10 of 13 signals 2 remaining
Research LessWrong Sep 26

Why I expect AI replication incidents by 2027

By Reworr R

62 score
AI Analysis

Argues that autonomous AI replication incidents in the wild are reasonably likely before end of 2027, grounded in capability-density trends showing open models doubling every ~3.3 months, consumer-GPU-hostable models trailing the frontier by 6-12 months, and replication/hacking task lags of 3-7 months. Concludes that a large population of smaller models on millions of devices compensates for capability gaps.

Source details
Epistemic status: thinking out loud.I think a major incident of autonomous AI replication in the wild before the end of 2027 is reasonably likely. In this post, I explain the reasons why I think so.1. The capability is moving to cheaper hardwareThe capability density of open models doubles about every 3.3 months[1], so the same performance fits into half the parameters within that time. Epoch AI finds that a single consumer GPU runs open models that match the frontier of 6-12 months earlier[2].
AI SafetyForecastingFrontier ModelsOpen Models
58 score
AI Analysis

Evaluates AtMem as a persistent-memory and continuity layer for tool-using language agents, extending assessment beyond retrieval accuracy to include provenance, authority, taint handling, and crash recovery. Passes all 400 integrity trials, rejects canonical secret retention, preserves taint through derivation, and avoids repeat dispatch after uncertain external actions.

Source details
AtMem is evaluated as a persistent-memory and continuity layer for tool-using language agents, extending assessment beyond retrieval accuracy to provenance, authority, taint handling, and recovery after uncertain external actions. In controlled tests, it passed all 400 representable integrity trials, rejected canonical secret retention, preserved taint through derivation, and avoided a repeat dispatch after a crash where the external effect may already have occurred.
AI AgentsMemory SystemsAgent SafetyEvaluation
Research AlphaXiv Trending Sep 26

Guiding End-to-End Driving Models with Endpoint-Constrained Trajectory Optimization

By Brayden Zhang, Mahsa Golchoubian, Igor Gilitschenski, Boris Ivanovic, Kashyap Chitta

55 score
AI Analysis

Introduces ECO, a test-time trajectory-repair layer for end-to-end autonomous driving that reshapes intermediate waypoints between executed history and a policy-selected endpoint, optimizing smoothness and turn penalties without modifying the driving policy. In closed-loop simulation, VaVAM's HUGSIM HD-Score nearly doubles from 18.1 to 31.0 across 185 scenes.

Source details
The authors introduce ECO, a test-time trajectory-repair layer that reshapes intermediate waypoints between executed history and a policy-selected endpoint. It optimizes smoothness, turn penalties, and fidelity to the original path without changing the driving policy. In closed-loop simulation, VaVAM’s HUGSIM HD-Score rises from 18.1 to 31.0 across 185 scenes. ECO does not choose destinations or reason about other agents.
Autonomous DrivingTrajectory OptimizationRobotics
50 score
AI Analysis

Extends Plan R — splitting frontier AI firms into R&D-only and deployment-only entities constrained to model-specific ASICs — by adding diversity (multiple ASIC vendors), escrow of designs, and 'political rights' for ASICs to limit single-supplier lock-in and adversarial state capture. Targets residual risks from deceptive alignment slipping through testing.

Source details
See Also:www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-... Plan R, I sketched a way that we can remove a significant fraction of the dire, short-term AI race risk. Split frontier AI companies into "R&D only" organizations which cannot issue equity, and "AI deployment" organizations which cannot train new models or hold any general purpose AI compute like GPUs/TPUs - they are limited to model
AI GovernanceAI SafetyASIC ComputePolicy
Research LessWrong Sep 26

Existential Risk Is An Extraordinary Claim

By James Brobin

45 score
Source details
This is a cross-post from my blog post.Over the past decade, the effective altruism movement has become increasingly focused on “existential risk,” the idea that, this century, there’s a significant chance that humanity will go extinct, become permanently disempowered, or otherwise lose almost all of its future potential.In this post, I’m not going to argue that, given what we know, we should believe that existential risk is high or low. Instead, what I’m going to argue is that existential risk
AI AgentsMulti-Agent SystemsFrontier ModelsAI Safety
Research LessWrong Sep 26

Night dreams have a tech tree?

By Aprillion

42 score
AI Analysis

Argues that AI-driven goods-cheapening will not translate into broadly shared prosperity because AI substitutes for human labor (lowering wages) faster than it reduces the cost of physically-produced necessities like food. Predicts 'poverty in the midst of abundance' absent redistribution or broad investment ownership.

Source details
Reading a book in a room with my late grandma who's been overhearing people from the kitchen whom I only hear as muffled voices that I don't understand made me realize:there are readable letters in my dreams now that stay the same when I look away and then back at themmy dream-character self can find himself in the state of having read stuff about "Snow class" (fictional caste system based on your job at a starship that ought to be unrelated to your skin color but the reader ought to be unsure a
Economics of AILabor MarketsAI Policy
Research LessWrong Sep 26

Claude Opus 5.5 Should Raise Your Ambitions

By Zvi

35 score
AI Analysis

Existential risk is framed as an extraordinary claim requiring extraordinary evidence, mirroring standards applied to other dramatic state-change predictions. Argues that without such evidence the prior probability of civilizational collapse this century should not be presumed high.

Source details
When it comes to making things, or doing most things in general, Fable 5.1 and especially GPT-6 Astra raised my ambition level. They should have raised yours, too. Claude Opus 5.5 should raise your ambition levels again. It just works, and it persists, like Astra does. It does the things. And it is highly pleasant to talk to, and its writing is pleasant to read, while you are at it. The game has been changed, again. Feedback is almost universally positive. Claude was never gone, but also is so b
Existential RiskEpistemicsAI Safety
Research LessWrong Sep 26

[Linkpost] Looking into the Swarm's Eye

By nwyin

30 score
AI Analysis

I'm Florian Brand is currently working as Research Engineer at Prime Intellect. Currently, my research focuses on applying and evaluating LLMs in various domains. I am also an editor at Interconnects,...

Source details
I'm Florian Brand is currently working as Research Engineer at Prime Intellect. Currently, my research focuses on applying and evaluating LLMs in various domains. I am also an editor at Interconnects, focusing on open models.There is, however, a big gap between open models in a suitable harness and GPT-6 (Astra), the first model trained very deliberately to be a capable RLM. Astra is currently held back by its native harness, Codex, and its default prompts. When elicited correctly, it is a sight
Research LessWrong Sep 26

Extinction does not feel as bad as it should

By mxbutterfree

22 score
AI Analysis

Explores psychological reasons why human extinction does not emotionally register as a catastrophe even though intellectually it is recognized as one. Attributes the gap to extinction lacking features like ongoing suffering, replaceable goods, and personal loss that other disasters possess.

Source details
And we will all go together when we goWhat a comforting fact that is to knowUniversal bereavement, an inspiring achievementYes, we all will go together when we go-- We Will All Go Together When We Go by Tom Lehrer"Everyone dying" does not emotionally feel that bad to me. I know that it is quite a bad thing intellectually, but I have a missing mood. I'm guessing I'm not alone here. When people say things like "If anyone builds it, everyone dies", there is an implicit assumption that everyone is o
Existential RiskPsychologyEffective Altruism
Research LessWrong Sep 26

Writing a Theorem Prover from scratch

By astle dsa

20 score
AI Analysis

Personal tutorial on writing a theorem prover from scratch in Haskell, starting from lambda calculus and the Curry-Howard correspondence, building on Federico Carrone's Python reference. Uses LLMs to guide (not write) the implementation, progressing to a dependent-type-theory extension.

Source details
NOTE: I am no subject matter expert and have chose to learn about this field through utilizing LLMs to guide me using practical implementations. Also, I am assuming a basic familiarity with Haskell and the Functional Programming paradigm.Our journey starts with the concept of lambda calculus and the existence of the Curry-Howard Correspondence. These concepts basically bridge the gap between Mathematics and Computer Science in a specific manner [1]. This gives rise to a very interesting piece of
Theorem ProvingProgramming LanguagesType Theory