Unpacking Multimodal Data Leakage, Broken Benchmarks, and the Hessian Fallacy
By Xenomirant
Summarizes an EACL 2026 paper showing most Membership Inference Attack benchmarks detect distribution shifts rather than memorization (model-free baseline hits 98.6% AUC on VL-MIA-Flickr-2k), and introduces FiMMIA for multimodal MIA. Important critique of evaluation methodology.