Research LessWrong Jan 1
Recent LLMs can do 2-hop and 3-hop latent (no-CoT) reasoning on natural facts
By ryan_greenblatt
78 score
AI Analysis
Empirical research showing recent LLMs (Gemini 3 Pro, Opus 4) can perform 2-hop and 3-hop latent reasoning without chain-of-thought, a capability previous models lacked. Creates new benchmark dataset avoiding prior dataset issues with memorization shortcuts.
Prior work has examined 2-hop latent (by "latent" I mean: the model must answer immediately without any Chain-of-Thought) reasoning and found that LLM performance was limited aside from spurious successes (from memorization and shortcuts). An example 2-hop question is: "What element has atomic number (the age at which Tesla died)?". I find that recent LLMs can now do 2-hop and 3-hop latent reasoning with moderate accuracy. I construct a new dataset for evaluating n-hop latent reasoning on natura
Language ModelsReasoningCapabilities EvaluationBenchmarks