Research LessWrong Mar 14
Extracting Performant Algorithms Using Mechanistic Interpretability
By Ihor Kendiukhov
68 score
AI Analysis
Describes extracting interpretable algorithms from biological foundation models using mechanistic interpretability techniques, inspired by prior work finding evolutionary phylogenetic trees encoded in Evo 2's activations. Proposes that models trained on biological data (like single-cell data) encode performant algorithms that can be reverse-engineered.
A Prequel: The Tree of Life Inside a DNA Language ModelLast year, researchers at Goodfire AI took Evo 2, a genomic foundation model, and found, quite literally, the evolutionary tree of life inside. The phylogenetic relationships between thousands of species were encoded as a curved manifold in the model's internal activations, with geodesic distances along that manifold tracking actual evolutionary branch lengths. Bacteria that diverged hundreds of millions of years ago were far apart on the ma
Mechanistic InterpretabilityFoundation ModelsComputational BiologyAI for Science
Source details
A Prequel: The Tree of Life Inside a DNA Language ModelLast year, researchers at Goodfire AI took Evo 2, a genomic foundation model, and found, quite literally, the evolutionary tree of life inside. The phylogenetic relationships between thousands of species were encoded as a curved manifold in the model's internal activations, with geodesic distances along that manifold tracking actual evolutionary branch lengths. Bacteria that diverged hundreds of millions of years ago were far apart on the ma
Mechanistic InterpretabilityFoundation ModelsComputational BiologyAI for Science