Research Briefing — April 21, 2026

LACE introduces cross-thread attention that allows parallel reasoning paths in LLMs to share intermediate insights and correct each other during inference, rather than running independently. It addresses the key limitation that parallel sampling often fails in redundant ways by enabling coordination through a synthetic data pipeline.

Challenges the Platonic Representation Hypothesis by showing that cross-modal alignment between neural networks degrades substantially when scaling evaluation datasets from ~1K to millions of samples. The alignment that remains reflects coarse semantic overlap rather than fine-grained structural convergence.

Introduces Neural Garbage Collection (NGC), where language models learn to selectively evict KV cache entries during chain-of-thought reasoning, trained end-to-end from task reward alone. The model learns when to forget without hand-designed criteria, addressing the growing memory bottleneck of long reasoning chains.

Provides causal evidence that hallucination in transformers is an early trajectory commitment: factual and hallucinated paths diverge at the first generated token, and injecting hallucinated activations corrupts correct trajectories 87.5% of the time while correct-to-hallucinated injection recovers only 12.5%.

Introduces 'Hard Mode' automated theorem proving where the system must discover the answer before proving it (unlike standard benchmarks that embed answers in statements). Releases Hard Mode variants of MiniF2F and FIMO, plus the DAP framework using LLM reasoning with self-reflection for answer discovery in Lean 4.

Proposes sequential KV cache compression that exploits the fact that KV cache tokens are from the model's own language, enabling compression beyond per-vector Shannon limits. Uses probabilistic prefix deduplication and model-as-predictor coding in a two-layer architecture.

Investigates why supervised fine-tuning (SFT) encourages hallucinations in LLMs by degrading pre-training knowledge, and proposes a self-distillation-based SFT method from continual learning to mitigate this. Shows regularizing output-distribution drift during fine-tuning effectively reduces hallucinations while enabling factual learning.

Provides an explicit comparison between distribution sharpening and task-reward-based RL for training frontier models, demonstrating that RL genuinely instills new capabilities rather than merely sharpening existing distributions. Shows from first principles why distribution sharpening optima can be unfavorable.

Open-TQ-Metal enables 128K-context inference for Llama 3.1 70B on a single 64GB Mac by implementing fused compressed-domain attention on Apple Silicon via custom Metal shaders, achieving 48x attention speedup at 128K context.