Social Twitter May 8
The human brain🧠 is incredibly efficient because it only activates the specific neurons needed for a...
By @hardmaru
92 score
AI Analysis
David Ha (hardmaru) shares research collaboration with NVIDIA on solving the sparsity-hardware mismatch in LLMs. Their 'TwELL' format achieves >20% faster training/inference on H100 GPUs by reshaping sparsity to fit GPU architecture rather than forcing GPUs to adapt.
The human brain🧠 is incredibly efficient because it only activates the specific neurons needed for a thought. Modern LLMs naturally try to do this too (> 95% of neurons in feedforward layers stay silent for any given word), but our hardware punishes them for it.
One of the most frustrating paradoxes in deep learning: making a model do less math often makes it run slower. Why? Because unstructured sparsity introduces irregular memory access, and GPUs are built for predictable, dense blocks of m
hardware_efficiencyLLM_optimizationresearch_breakthrough
Source details
The human brain🧠 is incredibly efficient because it only activates the specific neurons needed for a thought. Modern LLMs naturally try to do this too (> 95% of neurons in feedforward layers stay silent for any given word), but our hardware punishes them for it.
One of the most frustrating paradoxes in deep learning: making a model do less math often makes it run slower. Why? Because unstructured sparsity introduces irregular memory access, and GPUs are built for predictable, dense blocks of m
hardware_efficiencyLLM_optimizationresearch_breakthrough