论文地址：Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs
https://link.springer.com/chapter/10.1007/978-3-031-73010-8_8
代码地址：https://github.com/LALBJ/PAI
项目地址：https://lalbj.github.io/projects/PAI/
bib引用：

@misc{liu2024payingattentionimagetrainingfree,
      title={Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs}, 
      author={Shi Liu and Kecheng Zheng and Wei Chen},
      year={2024},
      eprint={2407.21771},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2407.21771}, 
}

InShort【通过调整和放大分配给图像标记的注意力权重通过减轻LLM幻觉】

提出了一种名为PAI的无训练方法，通过在推理过程中增强对图像的关注、减少语言先验的影响，有效缓解大视觉语言模型（LVLMs）中的幻觉问题。

1. 研究背景
   • LVLMs发展与问题：LVLMs在多任务中表现出色，但存在幻觉现象，即生成文本与视觉输入不匹配，现有研究多通过对齐训练缓解，效果有限。
   • 文本惯性现象：发现LVLMs存在 “文本惯性”，去除图像输入后仍生成相同幻觉描述，分析认为是图像表征映射机制中语言模型主导、图像令牌被忽视所致。
2. 相关工作
   • LVLMs发展：预训练和指令调整技术推动了LLMs和LVLMs发展，但仍面临幻觉问题。
   • 缓解幻觉方法：包括数据处理、模型训练优化、后处理等，但这些方法存在耗时久、计算资源需求高或增加推理成本等问题，无训练方法目前仅在解码方式上尝试。
3. 方法
   • 更多关注图像：从令牌级提取自注意力矩阵，增强图像令牌注意力权重，依据原始注意力值放大图像令牌注意力，避免选择干预注意力头，依据隐藏状态相似度判断干预时机。
   • 以图像为中心的对数几率精修：用无图像输入时的输出分布惩罚初始预测分布，平衡视觉和文本信息影响。
4. 实验
   • 实验设置：以LLAVA、Shikra和Minigpt4为基线模型，选择贪心、束搜索、核采样三种解码方法，与OPERA、VCD对比，设置不同超参数。
   • 评估指标：使用CHAIR评估图像描述任务中的幻觉，POPE评估VQA任务中的物体幻觉，MMHal - Bench评估高难度数据集上的幻觉，GPT - 4v辅助评估图像描述性能。
   • 实验结果：PAI在长序列幻觉评估、单轮和多轮幻觉评估、综合场景幻觉评估以及GPT - 4v辅助评估中均有良好表现，能有效减少幻觉。
   • 消融实验：研究超参数α、层先验“L”和γ对模型性能的影响，发现合适的超参数设置可平衡幻觉和信息传达。
5. 结论与局限
   • 结论：PAI有效缓解LVLMs幻觉问题，无需训练和外部工具。
   • 局限：现有开源LVLMs语言解码器多为LLaMA家族模型，需探究图像忽视和文本惯性是否由其引入；可研究将该问题作为训练损失是否能提升性能。

---

摘要

现有的大型视觉语言模型 （LVLM） 主要将视觉编码器的图像特征与大型语言模型 （LLM） 保持一致，以利用其卓越的文本生成功能。然而，视觉编码器和语言模型之间的尺度差异可能导致 LLM 在多模态理解中占据主导地位。LVLM 中的这种不平衡可能会导致幻觉的情况。具体来说，LVLM 可以在有或没有视觉输入的情况下生成一致的描述，这表明某些输出仅受上下文文本的影响。我们将这种现象称为 “文本惯性”。为了解决这个问题，我们引入了一种无训练算法，以在图像理解和语言推理之间找到一个平衡点。具体来说，我们适应性地涉及调整和放大分配给图像标记的注意力权重，从而赋予视觉元素更大的突出性。同时，我们从纯文本输入的 logit 中减去多模态输入的 logits，这可以帮助 LVLM 不偏向 LLM。通过增强图像令牌和减少 LLM 的顽固输出，我们可以让 LVLM 更加关注图像，从而缓解文本惰性并减少 LVLM 中的幻觉。我们广泛的实验表明，这种方法在不同指标方面大大降低了各种 LVLM 中幻觉输出的频率。项目页面位于 https://lalbj.github.io/projects/PAI/

Introduction

最近，大型视觉语言模型 （LVLM） 取得了重大进展，在众多任务中表现出令人印象深刻的能力 [13,17,38,44]。然而，这些模型仍然与幻觉现象作斗争。具体来说，模型生成的文本内容与它接收到的实际视觉输入之间经常存在不匹配 [25]。

大规模语言模型中的幻觉现象通常归因于模态对齐问题，这导致了通过对齐训练优化来制定缓解策略[9,22,33]。然而，大规模语言模型中的幻觉仅仅是模型容量的结果吗？并且只能通过额外的训练来缓解吗？我们提出一种场景，其中大规模语言模型生成一个幻觉对象描述。具体来说，即使图像输入被移除，并且只保留幻觉对象词之前生成的文本，大规模语言模型仍然会产生相同的幻觉描述，如图 1 所示。

图 1：我们展示了对各种输入设置的检查，幻觉特别以红色突出显示。（a） 当使用 LLAVA 进行图像描述时，它会生成幻觉描述。（b） 即使没有图像输入，当只有幻觉描述之前的历史反应被输入到 LLAVA 时，它会再现相同的幻觉描述，这种现象我们称之为“文本惯性”。（c） 我们提出的方法 PAI 有效地缓解了这个文本惯性问题并产生了准确的描述。（d） 使用 PAI 进行图像描述可以明显更精确地进行描述。

图 2：所有幻觉中文本惯性幻觉的百分比（用 500 个样本计算）。有关具体计算过程，请参阅补充材料 A 部分。

图 3： 不同内容的平均注意力比率的可视化。x 轴表示历史标记的序列长度。图像、指令、BOS 和异常值标记 [11] 的长度都是固定的，因为它们是模型输入的一部分，图像有 576 个标记，指令有 21 个标记，BOS 和异常值标记各有 1 个。

为了实证研究这种行为(empirically investigate this behavior,similar with the paper “an empirical study with CLIP”)，我们在 COCO 数据集的图像描述任务背景下对三个大型语言模型（LVLMs）进行了测试。我们识别出了大视觉语言模型（LVLMs）即便在输入仅为历史响应文本而无任何图像的情况下(exclusively historical response text without any image)，仍生成相同幻觉物体描述的实例，并对这些实例进行了统计分析。 图 2 的观察结果清楚地表明，即使采用严格的识别设置( rigorous identification settings)，这种现象仍然占很大比例。

我们将这种现象称为“文本惯性”。我们的假设是，文本惯性是由于当前的生成范式将图像表示作为文本标记映射到文本表示空间。()在这种机制中，LLM 成为主导角色，推理过程缺乏对图像标记的额外处理，导致它们在生成过程中被忽视。为了验证这一假设，我们在图 3 中分析了 LLaVA 模型在推理过程中的注意力值比率。我们的研究结果表明，尽管图像代币占据了很大比例，但在目前的机制下，它们并没有得到实质性的关注。这种多模式聊天更像是基于上下文的自动完成，而不是持续关注图像以完成。

Notes：类似“复读机机制”
假如有一张包含一只猫在沙发上睡觉的图片，使用 LVLMs 对其进行描述时，模型错误地生成 “图片中有一只狗在地板上玩耍” 这样的幻觉描述。之后，当不再输入这张图片，仅把之前生成该幻觉描述前的对话历史输入给模型，比如之前的对话是 “请描述一下这张图片里的内容”，模型依然输出 “图片中有一只狗在地板上玩耍”。这是因为在模型的运行机制中，语言模型主导了生成过程，对图像信息的关注度不够，即使没有图像输入，也会按照之前产生幻觉的模式继续生成内容，就像被之前的错误描述 “惯性” 地影响了，这就是 “文本惯性”。 从文中对 LLaVA 模型的分析（图 3）也能看到，图像令牌虽然在输入中占比大，但得到的注意力却不多，模型更像是基于上下文自动完成内容，而非依据图像进行描述。
所以需要增加对视觉信息的关注。

为了缩小这一差距，我们引入了一种称为 关注图像 （PAI） 的方法。在高层次上，PAI 会干预推理过程，使其更加以图像为中心，遵循原始图像感知方向。为了实现这一点，我们专注于 LVLM 解码器层中的自注意力头。在推理过程中，我们增强了图像标记在其原始方向上的注意力权重。这允许我们使用更新的注意力矩阵来计算生成的 Token 的隐藏状态，从而在生成过程中更多地考虑图像表示。为了进一步减轻文本惯性，我们使用指令标记和历史响应标记构造输入，并从具有图像标记输入的原始模型的 logit 中减去此输入的模型 logits。此策略有助于减少生成过程中语言先验的影响。与以前需要额外培训或外部工具的缓解幻觉的方法不同，我们的方法是免培训的。此外，我们是第一个提出减轻 LVLMs 幻觉的推理干预方法的人。

然后，我们采用 CHAIR 度量 [31] 和 GPT-4V 从长序列生成的角度评估图像描述任务中的响应准确性。此外，我们使用 POPE [21] 和 MMHal-Bench [34] 来更全面地评估模型在 VQA 任务中的幻觉表现。此外，我们还为 POPE 构建了单轮次和多轮次聊天评估。由于我们的模型干预了推理过程，因此它可以用于任何解码方法。因此，我们对三种模型的三种解码方法进行了实验。实验结果证明了我们的方法在减轻幻觉方面的有效性。

2 Related Work

2.1. Large Vision Language Models

预训练技术[5,29]和指令调优技术[28,39]的发展迅速推进了大型语言模型（LLMs）技术，如 LLaMA[35]和 Vicuna[30]，进一步促使视觉语言大模型（LVLMs）技术走向繁荣。早期的作品，如 Flamingo[2]和 BLIP-2[19]，已经成功地将大型语言模型应用于视觉任务，展示出显著的生成能力和上下文学习能力。最近，在视觉指令调优技术[26,27]的影响下，视觉语言大模型的能力进一步提升。使用不同的投影器将图像映射到文本域，从而赋予语言生成模型图像理解能力，也是一个热门的研究课题[10,27,41,46]。此外，一些研究专注于视觉语言任务，如定位能力[8]和推理能力[17]。然而，最近的视觉语言大模型仍然面临幻觉生成的问题[23]。

2.2. 减轻LLM幻觉

LVLM 中的幻觉是指图像输入和文本输出之间的矛盾。已经提出了各种方法来减轻幻觉。幻觉产生的最直接原因是幻觉源于数据偏差以及视觉和语言之间的知识差距。因此，引入了更好的数据过滤方法 [14， 24， 43] 和更高质量的注释数据 [3]。同时，这些方法也意味着需要更多的对齐训练 [33] 或调整模型架构 [9， 22]。这些方法可以取得良好的结果，但它们非常耗时且需要大量的计算资源。

除了解决LVLMs本身的能力之外，幻觉还可以通过后处理方法(post-processing methods)来减轻。这种方法通常涉及使用额外的模块或外部工具来编辑响应(edit the response)。最近的方法如LURE[45]利用额外的数据来训练状态检测器，当检测到幻觉问题时，内容由revisor model 重新生成。Woodpecker [42]引入了外部视觉模型来检查从响应中提取的实体，然后将检测结果交给生成模型以重新生成更好的答案。这些方法还扩展了推理链(inference chain)并增加了推理成本。

到目前为止，Training-free hallucination mitigation methods（无训练的幻觉缓解方法）仅在解码方法中尝试过。OPERA [15] 发现了一种伴随模型解码的异常注意力模式。统计发现，这种模式往往伴随着幻觉描述，因此基于这种模式提出了一种检测和缓解方法，以缓解模型面临的幻觉。VCD [18] 引入了视觉不确定性会增加幻觉描述的概念，并基于这一发现提出了一种对比解码方法来缓解幻觉问题。

3 Preliminaries

LVLM 的架构通常由三个主要组件组成：图像编码器、projector 和语言解码器。图像编码器和语言解码器通常都经过预训练。图像编码器用于将图像转换为图像令牌，然后由projector 映射到文本表示空间。此过程支持将图像标记与文本标记连接起来，然后输入到语言解码器中。语言解码器随后根据提供的指令生成相应的响应。

The existing projectors. Currently, projectors predominantly fall into two categories: linear projectors and resamplers. A projector takes N visual features from the image encoder and transforms them into M visual tokens. The linear projector employs a multilayer perceptron to transform all visual features, maintaining a one-to-one transformation which meaning that M equals N. In contrast, the resampler does not preserve all visual features but instead samples visual cues (M, where $M<N$ ).For instance, Q -former [19) utilizes M learnable queries and Bert [12] to extract information from visual features. Given that the knowledge of images during the generation process solely originates from the output image tokens of the projectors, our attention is concentrated on the image tokens post-projection, irrespective of their preceding modeling process.

Autoregressive language decoders. Nearly all LVLMs adopt LLaMAfamily models as their language decoders, which employ the self-attention mechanism. The visual tokens processed by the projector are concatenated with text tokens and fed into the LLaMA, which carries out the forward decoding process. From the perspective of a single attention head in a single layer, each head repeatedly performs the following attention operation with the same input shape:
$O_h=A_h{V}_h,A_h=softmax\left(\frac{Q_h{K}_h^{\top }}{\sqrt{d_k}}\right).(1)$

Each attention head h performs an attention operation using its own set of queries $Q_h\in {\mathbb{R}}^{n\times d_k}$ ,keys $K_h\in {\mathbb{R}}^{n\times d_k}$ , and values $V_h\in {\mathbb{R}}^{n\times d_k}$ , where n represents the sequence length and $d_k$ represents the hidden dimensions. The output $O_h\in {\mathbb{R}}^{n\times d_k}$ is modeled by multiplying $V_h$ and the attention weights $A_h\in {\mathbb{R}}^{n\times n}$ , where each row represents the weights for each token during feature mixing.

This operation enables the model to focus on different parts of the input for each head through attention weights, thereby capturing various parts of the information from the sequence token representations. The final output is the current generated token vocabulary conditional probability distribution $y\in {\mathbb{R}}^v$ based on the input instruction representations $x_1$ , image representations $x_V$ and history generated token representations $X_H$ ,where v is the size of the vocabulary.
This process can be formatted as:
$\begin{array}{rl}y& \sim p_{model}\left(y|X_I,X_V,X_H\right),\\ & \propto softmax\left(logi{t}_{model}\left(y|X_I,X_V,x_H\right)\right),\end{array}$
which calculates the distribution of one token and iterates for the entire response. The sequence generation continues until an EOS (End of Sentence) token is produced, marking the end of the generation and resulting in a complete response.

4 Method

At the core of our method is a solution for image neglect and text inertia, both of which are fundamentally interconnected. Essentially, as paying more attention to the image, there is a corresponding reduction in the reliance on language priors. Intuitively, in a conversation centered around an image, the model should devote more attention to the image, thereby allowing it to have a significant impact on the response. As such, we identify the self-attention map in the token-level generation and augment the image attention in its original directions. This strategy promotes a more image-centric latent representations. Additionally, to further mitigate the influence of text inertia, we devide the logits distribution of pure text input into the model’s output.

图 4： 我们的 PAI 的架构。为了减轻文本惯性，我们还构造了一个没有图像的输入。在整个前向推理过程中，我们通过在 LLaMA 中编辑自我注意力图来放大对图像标记的关注。最终，我们在解码过程中减去语言之前的 logits 分布，以获得准确的描述。

4.1 Pay More Attention to Image

提取自我注意力矩阵。我们从代币级别的角度开始。LVLM 中的响应过程基本上是逐个 Token 生成的。每个令牌都是根据输入图像、指令和历史生成的响应生成的。这个过程是通过多层注意力解码器架构来实现的。因此，这会导致当前生成的 Token 的词汇表的概率分布。我们的目标是提取每一层每个注意力头的注意力矩阵，表明推理过程中每个内容的影响。

When generating the k -th token in the sequence, the input representation for the attention head in the forward process includes the instruction representation $X_I=[x_{i_1},...,x_{i_{n_I}}]$ , image representation $X_V=[x_{v_1},...,x_{v_{n_V}}]$ , and the representation of the historically generated response $X_H=[x_{h_1},...,x_{h_{n_H}}]$ . Notably, the image representation considered here is the one that has been processed by the projector. Essentially, the hidden states of each input layer are $X=concat(X_I[1:m],X_V,X_I[m+1:n_I],X_H)$ , where the notation $X_I[1:m]$ indicates the first m elements in the instruction representation. Each attention head assigns different degrees of attention to each element during the current token representation decoding process. Our aim is to enhance the attention paid to the image. Therefore, we extract the attention weight values related to the image token for the current generated token, intervene, and then redistribute the attention values of each element through softmax.

Excite model in a trustful direction.
There have been attempts in some LLMs works to make the answers generated by LLMs more trustworthy by means of intervention [4, 6, 16, 20, 36]. The implemented approach typically involves intervening with the hidden states. As for defining what constitutes a more trustworthy direction, it usually requires additional projection and training to probe this trustful direction. In our case, a response that is more image-based is considered more trustworthy. Since LVLMs have undergone alignment training, the original attention values provide a direction based on image content. As illustrated in Fig. 4, by amplifying the attention values of image tokens based on the original attention values, we can enhance the trustworthiness of our results.

Another nuance involves our avoidance of choosing the attention head used to shift intervention. In the ITI method [20], it is stated that not all attention heads should be subjected to intervention. Therefore, they introduce a trustful score to rank each head across all layers and select the top-k heads for intervention. In our case, the less trustworthy heads with lower attention values receive less intervention. We first extract the attention weights of the image tokens for the current generated token from the attention weights $\bar{A}$ before softmax opeartion. We then use the hyper-parameter α to control the step size for intervention. From a single attention head perspective, our method can be expressed as follows:
${\overline{A}}_{n,j}={\overline{A}}_{n,j}+\alpha \cdot \left|{\overline{A}}_{n,j}\right|forj=m+1tom+n_V.(3)$

The model’s final vocabulary probability distribution is derived from the projection of the hidden states of the last token in the sequence. Therefore, we extract the attention weights of the last token n on the image tokens by indexing ${\tilde{A}}_{n,j}$ . Following the intervention, we use the softmax function to redistribute the attention values of each token during the reassignment of encoded hidden states. This procedure is repeated for each subsequent token prediction in an autoregressive manner and is independent of the choice of the decoding algorithm.

Excite more precisely with an attention mode prior.
The presence of the BOS token, an attention sink pattern [40], in a sentence results in higher attention values during the attention computation process, which may seem counterintuitive. The BOS token typically signifies the start of a sentence and, as such, does not carry significant semantic content. However, the generation of tokens is significantly influenced by this particular token, a similar pattern that also manifests itself in visual models [11]. As mentioned in StreamLLM [40], the pattern of the attention sink emerges when redundant attention values are present.
Naturally, one might infer that when the sink pattern appears, we excite the image token. To further investigate this phenomenon, as depicted in Fig. 5, we find that the sink phenomenon is not overtly evident in the shallow layers. This is because the shallow layers tend to focus more on encoding semantically rich information [37]. When the encoding of semantically rich tokens stabilizes, the attention sink phenomenon arises. Therefore, we build upon the judgement of intervention timing by calculating the similarity of the hidden states.

Fig. 5: The BOS token attention ratios of three model. We calculate the attention weights of the BOS token for each head in each layer and display them in a heat map.

4.2 Image-Centric Logit Refine

In Fig. 1, we observe a peculiar phenomenon where LVLMs continue to generate identical hallucinated text even when the image is removed from the input. This observation naturally leads us to the concept of using the output distribution (when no image is in input) as a reference to penalize our initial prediction distribution. Therefore, we update the distribution of the generated token by:
$\begin{array}{rlr}{p}_{model}=& \gamma \cdot p_{model}\left(y|X_V,X_I,X_H\right)& -(\gamma -1)\cdot p_{model}\left(y|X_I,X_H\right).\end{array}$
This equation effectively reduces the predicted probability based on text alone. The weight γ is used to control the degree of penalty applied to the initial prediction distribution.

This operation is conceptually similar to LLM-CFG [32]. Essentially, it provides a guided generation mechanism that allows the model to make informed choices between outputs based on image content and those based on language logic. This way, the model can better balance the influence of visual and textual information in its outputs, leading to more contextually accurate and relevant results.

5 Experiments

5.1 Setup

Baselines. We evaluate the effectiveness of our method on three different models. To better compare the impact of image feature tokens after different projectors on our method, we selected two models that use linear projectors, LLAVA and Shikra, as well as one model that uses resamplers, Minigpt4. Additionally, for a more convincing comparison, we report on three decoding methods for comparison: greedy, beam search, and nucleus sample. We also selected the OPERA [15] method, which is an improvement on beam search, and the VCD [18] method, which is an improvement on nucleus sampling, to compare with our results. We used the default hyperparameters from the open-source versions of these two methods.

实现细节。由于不同的模型具有不同的图像标记长度，导致不同程度的图像忽视，为了更好地与模型的图像序列长度保持一致，我们为 LLAVA 设置了 $\alpha =0.5$，为具有长图像标记序列长度的 Shikra 设置了 $\alpha =0.6$ 对于具有长图像标记序列长度的 Shikra，以及 $\alpha =0.2$ 用于具有短图像标记序列的重采样器模型。由于文本惯性与图像标记长度无关，因此我们继续使用 $\gamma =1.1$ 。除此之外，在光束搜索测试中，所有方法的beam number 都设置为 5，而在nucleus sample tests 中，所有常用参数都是一致的。

表 2：POPE 的定量比较。最佳结果以粗体显示。

5.3 Experimental Results

Results on long sequence hallucination evaluation.

The experimental results are presented in Tab. 1. As our approach is an inference intervention method, it differs from previous decoding hallucination mitigation methods that primarily concentrate on improving a single decoding method. We have tested our method on three decoding techniques. Our method has achieved hallucination mitigation on all three decoding methods used by the three models. However, when integrated with the nucleus, a sampling-based method, the hallucination reduction brought about by our method is not significant. This may be because even though our method has increased the priority of trustful tokens, the sample set during nucleus decoding still contains many hallucination tokens.
Moreover, while OPERA significantly mitigates hallucinations, its time efficiency is considerably higher compared to vanilla. In contrast, our method not only has almost the same time efficiency as vanilla, but it also performs better in reducing hallucination issues. Compared to VCD, during the generation process of long sequence tasks, the introduction of visual uncertainty during decoding sometimes leads to more hallucination descriptions. However, our method can reduce the proportion of hallucination words in the sample pool.

Results on single-turn and multi-turn hallucination evaluation.

Unlike the CHAIR evaluation, POPE is in a VQA format, so the response is brief, answering only “Yes” or “No”. The phenomena of text inertia and image neglect may not be as noticeable under this setting, especially for single-turn. However, our method still achieved a notable improvement compared to the vanilla decoding method in single-turn. As for multi-turn evaluation, which involves a longer context, our method achieves more significant improvement as presented in Tab. 2.

图 6： MMHal-Bench 上的定量比较。分数越高表示性能越好。

Results on hallucination evaluation in comprehensive general scenarios. 综合一般场景中的幻觉评估结果

The experimental results, as shown in Fig. 6, indicate that for some more image-based question types, such as object attributes, adversarial objects, and holistic questions, the answers are more accurate when inference intervention with PAI is applied, and there is a certain degree of improvement across all models. However, for some logical questions, such as comparisons and relations, there is no noticeable improvement after intervention. In summary, through the overall metric, i.e., the average of the eight evaluation dimensions, there is a certain degree of improvement compared to the baseline after incorporating PAI.
如图 6 所示的实验结果表明，对于一些更多基于图像的问题类型，例如对象属性、对抗对象和整体问题，当应用使用 PAI 的推理干预时，答案更加准确，并且所有模型都有一定程度的改进。然而，对于一些逻辑问题，如比较和关系，干预后没有明显的改善。综上所述，通过整体指标，即 8 个评价维度的平均值，与纳入 PAI 后的基线相比，有一定程度的改善。

Results on human-like GPT-4v assisted hallucination evaluation. 类人 GPT-4v 辅助幻觉评估的结果

The experimental results, as shown in Tab. 3, indicate that even when more comprehensive hallucination evaluation dimensions are added, our method, compared to the greedy decoding method, can provide more accurate responses on all three models without losing detail in the description. Given that GPT-4v’s visual understanding and language logic capabilities have reached a level close to that of humans, it can more comprehensively illustrate the performance improvements brought about by our method.
如表 3 所示的实验结果表明，即使添加了更全面的幻觉评估维度，与贪婪解码方法相比，我们的方法也可以在所有三个模型上提供更准确的响应，而不会丢失描述中的细节。鉴于 GPT-4v 的视觉理解和语言逻辑能力已经达到了接近人类的水平，它可以更全面地说明我们的方法带来的性能提升。

Table 3: Results on GPT-4V evaluation. The best results are in bold.表 3：GPT-4V 评估结果。最佳结果以粗体显示。

5.4. 消融实验

Our method, PAI, consists of two stages of interventions. In the first stage, during forward inference, the hyperparameter α is utilized to set the scale of intervention. Simultaneously, the layer prior, represented as “L”, is used to determine the attention layer for intervention. The second stage unfolds during the decoding process, where we mitigate text inertia by subtracting the logits distribution that results from inputs devoid of image information. In this stage, the scale is managed by the parameter γ
We use LLaVA-1.5 as the representative LVLM baseline and the greedy decoding method as the basic baseline to compare the impact of our hyperparameters on the task of long sequence image description. To evaluate our method, we choose the CHAIR metric. However, since CHAIR only evaluates the hallucination problem, we have incorporated the F1 score to consider information richness and accuracy. This makes the comparison with the CHAIR metric fairer when the F1 scores are similar. In the above, the F1 scores of the various methods differ slightly, so we have not included this somewhat redundant measure. The F1 score is calculated by counting the objects included in the description, the objects in the ground-truth sets, and the hallucinated objects.

Table 4: Ablation Study of the Hyperparameter a . When α becomes excessively large, resulting in an unbalanced response, we terminate the experiment and denote this with a dash (-). F1 values that are considered outliers are highlighted in red.
表 4：超参数 a 的消融研究 .当 α 变得过大，导致响应不平衡时，我们终止实验并用破折号 （-） 表示。被视为异常值的 F1 值以红色突出显示。

Effects of α in Exciting Image Attention.

In the process of exciting the attention values of image tokens, we introduce a parameter α to control the amplification scale. As shown in Tab. 4, different LVLMs exhibit varying sensitivity to the amplification scale. This sensitivity not only depends on the length of the model’s image tokens (e.g., the image token length of LLaVA-1.5 is 576, while that of Minigpt4 is only 32), but also the original attention weights distribution.
However, a commonality across these models is that an appropriate amplification scale can achieve a balance between the number of hallucinated objects in the description and the amount of information conveyed. If the scale is too small, the description may still contain many hallucinated objects. Conversely, if the scale is too large, the amount of information in the response will decrease.

Effects of Layer Prior in Exciting Image Attention.

We further investigate the control of the intervention layer for exciting image attention. As observed in Tab. 5, the introduction of this prior does indeed improve the performance of our method. However, when there is no control over the intervention layer and interventions are applied to all layers, the performances of different models exhibit some variations. For Shikra, in the absence of the layer prior, our method’s results regress to the baseline. For both LLaVA and Minigpt4, the loss of the layer prior causes some unfavorable fluctuations in both the CHAIR metric and the F1 score.

Table 5: Ablation Study of Hyperparameter γ and Layer Prior “L”. Results are presented for the models LLAVA (left), Minigpt4 (middle), and Shikra (right).表 5：超参数 γ 和层先验“L”的消融研究。显示了模型 LLAVA（左）、Minigpt4（中）和 Shikra（右）的结果。

Effects of γ in Mitigating Language Prior. Tab. 5 presents the results of an ablation study focusing on γ , which adjusts the balance between output distributions from conditioned inputs with excited image tokens and pure text inputs. Unlike the other two models, the Minigpt4 model is highly sensitive to γ . When γ is too large, it can lead to uncontrollable model behavior. The experimental results suggest that maintaining γ within a relatively small range, such as 1.1 - 1.2, yields the most stable performance.
γ 在缓解语言先验中的影响。表 5 显示了专注于 γ 的消融研究的结果，该研究调整了具有激发图像标记的条件输入和纯文本输入的输出分布之间的平衡。与其他两种型号不同，Minigpt4 型号对 γ 高度敏感。当 γ 太大时，可能会导致无法控制的模型行为。实验结果表明，将γ保持在相对较小的范围内（例如 1.1 - 1.2）可以产生最稳定的性能。

6 Conclusion and Limitation

在本文中，我们首先分析了 LVLMs 中幻觉的原因和表现。我们提出了一种称为 “文本惯性 ”的现象，其中即使没有提供图像输入，模型也会继续产生相同的幻觉描述。这个问题从根本上源于模型对图像标记的忽视。因此，我们引入了 PAI 方法来干预模型的推理过程，将其引导到基于图像和值得信赖的方向。这是一种training-free 方法，不需要任何外部工具。对多个基准和 LVLM 的广泛实验已经验证了 PAI 在缓解幻觉问题方面的有效性。

限制： （1） 现有开源 LVLM 的语言解码器主要是来自 LLaMA 系列的模型。值得探讨的是，图像忽视和文本惯性的问题是否是 LLaMA 引入的。（2） 正如我们所描述的，我们的方法从根本上缓解了模型推理过程中的图像忽视问题。其上限取决于训练良好的模型的能力。因此，值得研究在训练过程中将此问题作为损失是否会导致进一步的性能改进。