← 返回时间线

paper

MLLMs Hallucinate when Information Distribution Drifts in Synergy Heads

arXiv ↗
ID
2609.09206
分类
—
首次捕获
2026-09-21
状态
unread
作者
Meng'en Qin, Junye Chen, Jucheng Liu, Youlu Xing, Song Wang, Ruize Han
信号
🔥 5

信号历史

  • 2026-09-21HF Daily Papers · 🔥2
  • 2026-09-22HF Daily Papers · 🔥5
暂无信号数据

摘要

Multimodal Large Language Models (MLLMs) often struggle with hallucinations, thus hindering their reliable practical applications. Existing attention-based mitigation methods mainly rely on indirect signals (e.g., attention weights) that fail to accurately reflect the actual information shift underlying hallucination generation. In this paper, we propose HEAL, Head-lEvel information disentAnglement and caLibration for identifying and mitigating hallucinations. HEAL first employs causal noise intervention on multi-head outputs to filter out causally redundant heads. Subsequently, it disentangles information distribution within the remaining heads via the counterfactual Difference-in-Differences, categorizing heads into four types. Through analysis, we observe: hallucinations happen when information distribution drifts away from a healthy equilibrium in synergy heads, not strongly correlated with the quantity or strength of modality-specific heads. Motivated by this insight, HEAL injects dynamic information calibration factors into the value vectors of synergy heads, and actively regulates visual-language dependencies, steering the output distribution towards factual evidence. Extensive experiments demonstrate that HEAL effectively reduces hallucinations across multiple MLLMs, offering a simple and interpretable pathway to enhance model trustworthiness.

我的笔记

还没有笔记。

在 GitHub 上写笔记 ↗(新建 content/notes/2609.09206.md,PR 合并后本页自动更新)