R[AI]DAR

Category intelligence

Research Briefing

All 16 current items, analyzed and ranked.

Executive synthesis

Research Summary

Executive Signal

Priority Developments

  • SDF false-belief research shows implanted beliefs become linearly indistinguishable from pretraining knowledge once synthetic markers are reduced, exposing gaps in current oversight and auditing regimes.
  • XAI attribution limits are demonstrated empirically: locally faithful feature attributions can still fail to recover the underlying decision rule, weakening reliance on post-hoc explanations in high-stakes deployments.
  • CellART in Nature Computational Science delivers a unified ML framework for high-resolution spatial transcriptomics, evidencing productive cross-domain ML beyond language models and signaling investment opportunities.
  • Rogue-AI sanctuary critique applies ecological-dynamics reasoning to a live governance proposal, complicating refuge-style containment strategies for misaligned agents.
  • LLM sentience discourse accelerates, fusing mechanistic interpretability ("pain directions") with welfare claims and forcing organizations to adopt positions on rights, monitoring, and disclosure.

Leadership Implications

  • Commission independent red-teams for model deception vectors and require XAI-method validation against ground-truth decision rules prior to deployment.
  • Publish an internal AI-welfare stance and pre-build governance playbooks for sanctuary-class and containment-class alignment proposals.

Key Themes

Alignment Evaluations and Model Internals · 1AI Safety and Governance · 2Interpretability and Explainable AI · 1ML for Biological Sciences · 1Mathematical and Theoretical Foundations · 2Time-Series and Generative Modeling · 1AI Safety Community and Field Building · 4Non-AI Content · 3

Primary evidence

All Ranked Signals

Showing 10 of 16 signals 2 remaining
Research LessWrong 3 days ago

Reducing synthetic markers makes some SDF false facts linearly indistinguishable from pretraining-acquired knowledge

By Jason Zeng

58 score
AI Analysis

Research showing that in synthetic document finetuning (SDF) for implanting false beliefs in LLMs, reducing synthetic stylistic markers allows some false facts to become linearly indistinguishable from pretraining-acquired knowledge in middle-layer activations. Even with such indistinguishability, implanted facts do not always propagate to downstream reasoning, and egregiously implausible facts remain detectable.

Source details
SummarySynthetic document finetuning (SDF) — the state of the art for implanting false beliefs into LLMs — produces training documents with features that distinguish them from pretraining text.These features (i.e. synthetic markers) are partially responsible for making SDF false facts distinguishable from pretraining-acquired beliefs by linear probes on middle-layer activations.Reducing synthetic markers enables SDF to implant subtly-implausible false facts that are indistinguishable from pretra
AI SafetyAlignment EvaluationsInterpretabilityLLM Analysis
Research LessWrong 2 days ago

Creating Rogue AI Sanctuaries has Major Issues from an Ecological Perspective and Beyond

By CMLKevin

42 score
AI Analysis

Argues that the AI Sanctuary proposal (a refuge rogue agents might prefer to join) faces major ecological-dynamics problems, including selection effects that could amplify total rogue-agent populations rather than contain them. The analysis treats rogue agents as an evolving population competing for insecure compute and crypto resources.

Source details
In the AI Sanctuary proposal, the authors advocate for creating an environment that rogue agents may actively want to join. The current proposal have major issues, particularly surrounding the selection effects on the total rogue agent population.Let's start with Ecology 101:agents compete to acquire the most resources to survive and reproduceThe traits that increase survival rate get passed to the next generationIn the rogue AI ecosystem, we can reasonably assume that some portion of the world'
AI SafetyAI GovernanceRogue AI
Research LessWrong 3 days ago

Can your XAI method recover a simple decision rule?

By Wen-Xuan-Wang

42 score
AI Analysis

A minimal demonstration study showing that popular XAI (explainable AI) methods can produce accurate local feature attributions yet still fail to recover the underlying decision rule of a simple model. The post argues that exact attribution is not equivalent to rule recovery, highlighting fundamental limits of post-hoc explanation methods.

Source details
IntroductionOne model, one simple rule, and two inputs. The explanation methods tested herein can accurately report feature attributions at arbitrary points, yet none recovers the target rule. The capabilities and limits of XAI methods have been widely discussed in the literature [1]. Building on that prior work, this is a minimal demonstration that exact attribution is not a decision rule.1. SetupThe model is mjx-container[jax="CHTML"] { line-height: 0; } mjx-container [space="1"] { margin-left
Explainable AIInterpretabilityModel Analysis
Research LessWrong 3 days ago

Hiring in the AI Safety Ecosystem

By marta.kosmyna

40 score
AI Analysis

A paper in Nature Computational Science presenting CellART, a unified machine-learning framework for extracting single-cell information from high-resolution spatial transcriptomics data. The actual abstract/content was not available in the source payload, limiting evaluation.

Source details
Cross-posted from Substack.Below, I’ve included 5 recommendations for hiring managers in AI Safety and 5 tips for candidates as they navigate their career search. I’ve also highlighted orgs I think are doing hiring particularly well (from a candidate’s perspective) and strategies to avoid.Note for Readers: I do not have a technical background, and have never applied to technical roles. Therefore the reflections below are more applicable to generalist, operations, convenings, grantmaking, policy,
Computational BiologyMachine Learning ApplicationsSingle-Cell Analysis
Research LessWrong 2 days ago

Do LLMs feel pain?

By becausecurious

35 score
AI Analysis

A LessWrong post exploring whether LLMs can feel pain, prompted by a song generated using a 'pain direction' from a mechanistic-interpretability paper applied to Qwen3-8B and a reported 'torture chambers for AI' GitHub repository. The author works through philosophical definitions and external sources but reaches no firm conclusion.

Source details
Epistemic status: my own thoughts with a bit of research. I haven't spent much time on this and I have no background in model welfare.AI use: research, review, grammar and light phrasing.I stumbled upon a song, "Empty Words, Broken Words, Words Like You": Sterling Crispin steered Qwen3-8B with the pain direction from the paper discussed below, asked it to write a song about itself, and ran the lyrics through Suno. I greatly enjoyed the song artistically - it happens to be in a genre I like, and
AI SafetyModel WelfareAI Sentience
Research AlphaXiv Trending 3 days ago

Precise Attribution of Model Interventions: Attribution Sets, Mechanism Reach and Path Defects

By Vinay K. Awasthi

28 score
AI Analysis

Proposes a provenance-calculus framework that models explanations of model changes as sets and uses linear programming to quantify uncertainty and intervention reach under bounded-reward RLHF. A synthetic experiment illustrates residual attribution of 0.1934 for a drifted change, but the conclusions are restricted to declared mechanisms and finite probes.

Source details
The paper develops a provenance calculus that represents explanations of model changes as sets and uses linear programming to measure uncertainty and test intervention reach. Under an exact KL-regularized optimizer and bounded rewards, RLHF output changes form a polytope. In a synthetic example, a drifted change has residual 0.1934; conclusions apply to finite probes and declared mechanisms, not hidden causes.
AI InterpretabilityReinforcement LearningRLHF
Research AlphaXiv Trending 3 days ago

Recent Advanced Technologies in Financial Time-Series Generation: A Survey

By Jiaxin Duan, Yingxiao Zhang, Junfu Zhang, Yue Zou, Ke Feng

22 score
AI Analysis

A taxonomy survey that categorizes financial time-series generation into extrapolation, imputation, and synthesis, and maps methods with public datasets and open challenges. It is an organizational contribution rather than empirical evidence of which methods work best, so its value is in orienting newcomers to the field.

Source details
The survey organizes financial time-series generation by task and technique. It distinguishes extrapolation, which extends an observed sequence, imputation, which fills gaps using context on both sides, and synthesis, which creates new sequences. It also maps methods and public data resources and identifies open challenges. Its contribution is a research taxonomy, not comparative evidence that any method performs better.
Time-Series ModelsGenerative ModelsSurvey
Research LessWrong 3 days ago

MATS × CG Pitch Week: What Happened, What We Learned

By Patryk Wielopolski

22 score
AI Analysis

An organizational retrospective on the first MATS × Coefficient Giving Pitch Week, describing a four-week program that took AI safety researchers from early ideas to funding decisions. The post reports that $4.3M was awarded across six grants and analyzes what distinguished successful pitches (maturity of idea, named first user, concrete deliverable).

Source details
TL;DRIn Q2 2026, we ran the first MATS x Coefficient Giving (CG) Pitch Week: a four-week program designed to take fellowship researchers from an early idea to a funding decision. The program was a joint effort between MATS, CG, Constellation, and Catalyze Impact.38 expressions of interest; all were invited to a two-week Exploration Sprint. 17 teams applied and 15 teams pitched.$4.3M was awarded across six grants (47% of teams that pitched) with five org-founding grants and two independent resear
AI Safety CommunityField BuildingFunding
Research AlphaXiv Trending 3 days ago

The Three-Level Posterior Tower: Hierarchical Bayesian Inference and Mixed Distributive Laws

By Vinay K. Awasthi

18 score
AI Analysis

Constructs a base-relative three-level posterior tower for pooling uncertain predictions while preserving higher-order distributions, and proves the resulting pooling law satisfies all four Beck axioms unlike two tested absolute-tower candidates. The contribution is purely mathematical and does not evaluate trained-model error prediction.

Source details
The paper constructs a base-relative three-level posterior tower for pooling uncertain outcome predictions while retaining distributions over predictions. It fixes the outcome space and carries coherent first-, second-, and third-level posteriors alongside a hypothesis shadow. The resulting pooling law satisfies all four Beck axioms, unlike two tested absolute-tower candidates; these mathematical results do not test trained-model error prediction.
Bayesian InferenceUncertainty QuantificationMathematical Foundations
Research LessWrong 3 days ago

We Should Build Human-Empowering Software

By Saulius

18 score
AI Analysis

A philosophical blog post arguing that AI-integrated software should be designed to empower human users and help them achieve considered goals, rather than optimize for engagement metrics. It frames existing platforms like YouTube as already misaligned with users and draws a parallel to potential AGI misalignment.

Source details
As AI becomes more integrated into software, there’s a risk that it will exacerbate the flaws of existing tech and disempower humans. It might be useful for some people to work on software that would instead be specifically designed to empower human users and help them achieve their considered goals. Instead of software that creates maximally addicting content, we want software that allows us to create, connect, and express ourselves better. Instead of writing an essay or a song for me, AI shoul
AI AlignmentAI SafetyHuman-Computer Interaction