<!DOCTYPE html><html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2407.07071] Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps</title><meta property="og:description" content="When asked to summarize articles or answer questions given a passage, large language models (LLMs) can hallucinate details and respond with unsubstantiated answers that are inaccurate with respect to the input context.…">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2407.07071">

<!--Generated on Mon Aug  5 16:10:45 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document">
<h1 class="ltx_title ltx_title_document">
<em id="id9.id1" class="ltx_emph ltx_font_italic">Lookback Lens</em>: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using <em id="id10.id2" class="ltx_emph ltx_font_italic">Only</em> Attention Maps</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Yung-Sung Chuang<sup id="id11.9.id1" class="ltx_sup">†</sup>   Linlu Qiu<sup id="id12.10.id2" class="ltx_sup">†</sup>   Cheng-Yu Hsieh<sup id="id13.11.id3" class="ltx_sup">‡</sup>  Ranjay Krishna<sup id="id14.12.id4" class="ltx_sup">‡</sup> 
<br class="ltx_break"><span id="id8.8.4" class="ltx_text ltx_font_bold"> Yoon Kim<sup id="id8.8.4.1" class="ltx_sup"><span id="id8.8.4.1.1" class="ltx_text ltx_font_medium">†</span></sup>  James Glass<sup id="id8.8.4.2" class="ltx_sup"><span id="id8.8.4.2.1" class="ltx_text ltx_font_medium">†</span></sup> 
<br class="ltx_break">Massachusetts Institute of Technology<sup id="id8.8.4.3" class="ltx_sup"><span id="id8.8.4.3.1" class="ltx_text ltx_font_medium">†</span></sup>  University of Washington<sup id="id8.8.4.4" class="ltx_sup"><span id="id8.8.4.4.1" class="ltx_text ltx_font_medium">‡</span></sup> 
<br class="ltx_break"></span><span id="id15.13.id5" class="ltx_text ltx_font_typewriter">yungsung@mit.edu</span><span id="id16.14.id6" class="ltx_text ltx_font_bold"> 
<br class="ltx_break"></span>
</span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id17.id1" class="ltx_p">When asked to summarize articles or answer questions given a passage, large language models (LLMs) can hallucinate details and respond with unsubstantiated answers that are inaccurate with respect to the input context.
This paper describes a simple approach for detecting such <em id="id17.id1.1" class="ltx_emph ltx_font_italic">contextual hallucinations</em>. We hypothesize that contextual hallucinations are related to the extent to which an LLM attends to information in the provided context versus its own generations. Based on this intuition, we propose a simple hallucination detection model whose input features are given by the ratio of attention weights on the context versus newly generated tokens (for each attention head). We find that a linear classifier based on these <span id="id17.id1.2" class="ltx_text ltx_font_italic">lookback ratio</span> features is as effective as a richer detector that utilizes the entire hidden states of an LLM or a text-based entailment model.
The lookback ratio-based detector—<span id="id17.id1.3" class="ltx_text ltx_font_bold ltx_font_italic">Lookback Lens</span>—is found to transfer across tasks and even models, allowing a detector that is trained on a 7B model to be applied (without retraining) to a larger 13B model.
We further apply this detector to mitigate contextual hallucinations, and find that a simple classifier-guided decoding approach is able to reduce the amount of hallucination, for example by 9.6% in the XSum summarization task.<span id="footnote1" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_tag ltx_tag_note">1</span>Source code: <a target="_blank" href="https://github.com/voidism/Lookback-Lens" title="" class="ltx_ref ltx_href ltx_font_typewriter">github.com/voidism/Lookback-Lens</a></span></span></span></p>
</div>
<div id="p1" class="ltx_para ltx_noindent">
<div id="p1.8" class="ltx_block ltx_align_bottom">
<p id="p1.8.9" class="ltx_p"><em id="p1.8.9.1" class="ltx_emph ltx_font_bold ltx_font_italic">Lookback Lens</em><span id="p1.8.9.2" class="ltx_text ltx_font_bold">: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using <em id="p1.8.9.2.1" class="ltx_emph ltx_font_italic">Only</em> Attention Maps</span></p>
<br class="ltx_break ltx_centering">
<p id="p1.8.8" class="ltx_p ltx_align_center" style="width:433.6pt;"><span id="p1.8.8.8" class="ltx_text ltx_inline-block" style="width:0.0pt;">
<span id="p1.8.8.8.8" class="ltx_tabular ltx_align_top">
<span class="ltx_tbody">
<span id="p1.4.4.4.4.4" class="ltx_tr">
<span id="p1.4.4.4.4.4.4" class="ltx_td ltx_align_center"><span id="p1.4.4.4.4.4.4.4" class="ltx_text ltx_font_bold">Yung-Sung Chuang<sup id="p1.4.4.4.4.4.4.4.1" class="ltx_sup"><span id="p1.4.4.4.4.4.4.4.1.1" class="ltx_text ltx_font_medium">†</span></sup>   Linlu Qiu<sup id="p1.4.4.4.4.4.4.4.2" class="ltx_sup"><span id="p1.4.4.4.4.4.4.4.2.1" class="ltx_text ltx_font_medium">†</span></sup>   Cheng-Yu Hsieh<sup id="p1.4.4.4.4.4.4.4.3" class="ltx_sup"><span id="p1.4.4.4.4.4.4.4.3.1" class="ltx_text ltx_font_medium">‡</span></sup>  Ranjay Krishna<sup id="p1.4.4.4.4.4.4.4.4" class="ltx_sup"><span id="p1.4.4.4.4.4.4.4.4.1" class="ltx_text ltx_font_medium">‡</span></sup></span></span></span>
<span id="p1.6.6.6.6.6" class="ltx_tr">
<span id="p1.6.6.6.6.6.2" class="ltx_td ltx_align_center"><span id="p1.6.6.6.6.6.2.2" class="ltx_text ltx_font_bold"> Yoon Kim<sup id="p1.6.6.6.6.6.2.2.1" class="ltx_sup"><span id="p1.6.6.6.6.6.2.2.1.1" class="ltx_text ltx_font_medium">†</span></sup>  James Glass<sup id="p1.6.6.6.6.6.2.2.2" class="ltx_sup"><span id="p1.6.6.6.6.6.2.2.2.1" class="ltx_text ltx_font_medium">†</span></sup></span></span></span>
<span id="p1.8.8.8.8.8" class="ltx_tr">
<span id="p1.8.8.8.8.8.2" class="ltx_td ltx_align_center">Massachusetts Institute of Technology<sup id="p1.8.8.8.8.8.2.1" class="ltx_sup">†</sup>  University of Washington<sup id="p1.8.8.8.8.8.2.2" class="ltx_sup">‡</sup></span></span>
<span id="p1.8.8.8.8.9.1" class="ltx_tr">
<span id="p1.8.8.8.8.9.1.1" class="ltx_td ltx_align_center"><span id="p1.8.8.8.8.9.1.1.1" class="ltx_text ltx_font_typewriter">yungsung@mit.edu</span></span></span>
</span>
</span></span></p>
<br class="ltx_break ltx_centering">
</div>
</div>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Introduction</h2>

<div id="S1.p1" class="ltx_para">
<p id="S1.p1.1" class="ltx_p">Despite the utility and impressive capabilities of large language models (LLMs), their tendency to generate hallucinations, i.e., content that deviates from facts or contextually relevant information <cite class="ltx_cite ltx_citemacro_citep">(Ji et al., <a href="#bib.bib16" title="" class="ltx_ref">2023</a>)</cite>, presents a significant challenge in their deployment.
In this work, we focus on the scenarios where the model is provided with the correct facts within the input context but still fails to generate accurate outputs, a phenomenon we term <em id="S1.p1.1.1" class="ltx_emph ltx_font_italic">contextual hallucination</em>. Despite the simplicity of this setup, LLMs struggle with contextual hallucinations, frequently producing errors in tasks such as summarization and document-based question answering (e.g., Table <a href="#S2.T1" title="Table 1 ‣ Data ‣ 2.2 Experimental Setup ‣ 2 Contextual Hallucinations Detection ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>), which can cause serious issues in applications such as retrieval-augmented generation (RAG) <cite class="ltx_cite ltx_citemacro_citep">(Lewis et al., <a href="#bib.bib20" title="" class="ltx_ref">2020</a>)</cite>, even when correct documents are retrieved.</p>
</div>
<figure id="S1.F1" class="ltx_figure"><img src="/html/2407.07071/assets/x1.png" id="S1.F1.1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="116" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 1: </span>An illustration of the <span id="S1.F1.3.1" class="ltx_text ltx_font_italic">Lookback Lens</span>. We extract attention weights and calculate the lookback ratios for all layers and all heads. We train a linear classifier on the concatenated features to predict truthfulness of the generation.</figcaption>
</figure>
<div id="S1.p2" class="ltx_para">
<p id="S1.p2.1" class="ltx_p">Most prior studies that propose methods to combat hallucination focus on the scenario <em id="S1.p2.1.1" class="ltx_emph ltx_font_italic">without</em> any input context, where the hallucinations arise from the LLMs’ parametric knowledge. These works detect and mitigate hallucinations by generally using the LLM’s representations, such as hidden states <cite class="ltx_cite ltx_citemacro_citep">(Burns et al., <a href="#bib.bib3" title="" class="ltx_ref">2023</a>; Azaria and Mitchell, <a href="#bib.bib1" title="" class="ltx_ref">2023</a>)</cite>, MLP outputs <cite class="ltx_cite ltx_citemacro_citep">(Zhang et al., <a href="#bib.bib45" title="" class="ltx_ref">2024</a>; Simhi et al., <a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite>, attention block outputs <cite class="ltx_cite ltx_citemacro_citep">(Zhang et al., <a href="#bib.bib45" title="" class="ltx_ref">2024</a>; Simhi et al., <a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite> and attention head outputs <cite class="ltx_cite ltx_citemacro_citep">(Li et al., <a href="#bib.bib22" title="" class="ltx_ref">2024</a>; Chen et al., <a href="#bib.bib6" title="" class="ltx_ref">2024b</a>; Simhi et al., <a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite>. In contrast, the provided contextual information plays a key role in detecting contextual hallucinations. Insofar as attention (more so than other model internals) provides a human-meaningful measure of how much weight is given to the context during generation, this motivates the use of signals from the attention maps for hallucination detection and mitigation.</p>
</div>
<div id="S1.p3" class="ltx_para">
<p id="S1.p3.1" class="ltx_p">To leverage signals from attention maps, we start by hypothesizing that contextual hallucinations are related to the extent to which an LLM attends to the provided contextual information.
Concretely, we propose a simple feature called <em id="S1.p3.1.1" class="ltx_emph ltx_font_italic">lookback ratio</em>, which is computed as the ratio of attention weights on the given context versus the newly generated tokens. At each time step, we calculate this lookback ratio for each attention head, and train a linear classifier, which we call the <span id="S1.p3.1.2" class="ltx_text ltx_font_italic">Lookback Lens</span>, to detect contextual hallucinations based on the lookback ratio features, as illustrated in Figure <a href="#S1.F1" title="Figure 1 ‣ 1 Introduction ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>.
The <span id="S1.p3.1.3" class="ltx_text ltx_font_italic">Lookback Lens</span> performs on par with, and sometimes even surpasses, more complex feature-based detectors that utilize hidden states from LLMs or text-based entailment models trained on extensively annotated datasets.
We can further integrate this detector during decoding to derive a <span id="S1.p3.1.4" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span> strategy which can reduce contextual hallucinations by 9.6% from LLaMA-2-7B-Chat in the XSum summarization task.
Furthermore, our use of “higher level” attention map features makes it possible to transfer the detector across models <em id="S1.p3.1.5" class="ltx_emph ltx_font_italic">without</em> retraining, allowing a LLaMA2-13B-Chat model to use the same detector that has been trained on LLaMA-2-7B-Chat, and still reduce hallucinations by 3.2% in XSum. These results collectively highlight the potential of combating contextual hallucination by leveraging the information from attention maps.</p>
</div>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Contextual Hallucinations Detection</h2>

<section id="S2.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.1 </span>Lookback Lens</h3>

<div id="S2.SS1.p1" class="ltx_para">
<p id="S2.SS1.p1.9" class="ltx_p">To <em id="S2.SS1.p1.9.1" class="ltx_emph ltx_font_italic">detect</em> contextual hallucinations in LLMs, we introduce a lookback ratio, a measure based on the attention distribution of a transformer model. Given a transformer with <math id="S2.SS1.p1.1.m1.1" class="ltx_Math" alttext="L" display="inline"><semantics id="S2.SS1.p1.1.m1.1a"><mi id="S2.SS1.p1.1.m1.1.1" xref="S2.SS1.p1.1.m1.1.1.cmml">L</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.1.m1.1b"><ci id="S2.SS1.p1.1.m1.1.1.cmml" xref="S2.SS1.p1.1.m1.1.1">𝐿</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.1.m1.1c">L</annotation></semantics></math> layers, each with <math id="S2.SS1.p1.2.m2.1" class="ltx_Math" alttext="H" display="inline"><semantics id="S2.SS1.p1.2.m2.1a"><mi id="S2.SS1.p1.2.m2.1.1" xref="S2.SS1.p1.2.m2.1.1.cmml">H</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.2.m2.1b"><ci id="S2.SS1.p1.2.m2.1.1.cmml" xref="S2.SS1.p1.2.m2.1.1">𝐻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.2.m2.1c">H</annotation></semantics></math> heads, the model processes an input sequence of context tokens <math id="S2.SS1.p1.3.m3.4" class="ltx_Math" alttext="X=\{x_{1},x_{2},\dots,x_{N}\}" display="inline"><semantics id="S2.SS1.p1.3.m3.4a"><mrow id="S2.SS1.p1.3.m3.4.4" xref="S2.SS1.p1.3.m3.4.4.cmml"><mi id="S2.SS1.p1.3.m3.4.4.5" xref="S2.SS1.p1.3.m3.4.4.5.cmml">X</mi><mo id="S2.SS1.p1.3.m3.4.4.4" xref="S2.SS1.p1.3.m3.4.4.4.cmml">=</mo><mrow id="S2.SS1.p1.3.m3.4.4.3.3" xref="S2.SS1.p1.3.m3.4.4.3.4.cmml"><mo stretchy="false" id="S2.SS1.p1.3.m3.4.4.3.3.4" xref="S2.SS1.p1.3.m3.4.4.3.4.cmml">{</mo><msub id="S2.SS1.p1.3.m3.2.2.1.1.1" xref="S2.SS1.p1.3.m3.2.2.1.1.1.cmml"><mi id="S2.SS1.p1.3.m3.2.2.1.1.1.2" xref="S2.SS1.p1.3.m3.2.2.1.1.1.2.cmml">x</mi><mn id="S2.SS1.p1.3.m3.2.2.1.1.1.3" xref="S2.SS1.p1.3.m3.2.2.1.1.1.3.cmml">1</mn></msub><mo id="S2.SS1.p1.3.m3.4.4.3.3.5" xref="S2.SS1.p1.3.m3.4.4.3.4.cmml">,</mo><msub id="S2.SS1.p1.3.m3.3.3.2.2.2" xref="S2.SS1.p1.3.m3.3.3.2.2.2.cmml"><mi id="S2.SS1.p1.3.m3.3.3.2.2.2.2" xref="S2.SS1.p1.3.m3.3.3.2.2.2.2.cmml">x</mi><mn id="S2.SS1.p1.3.m3.3.3.2.2.2.3" xref="S2.SS1.p1.3.m3.3.3.2.2.2.3.cmml">2</mn></msub><mo id="S2.SS1.p1.3.m3.4.4.3.3.6" xref="S2.SS1.p1.3.m3.4.4.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS1.p1.3.m3.1.1" xref="S2.SS1.p1.3.m3.1.1.cmml">…</mi><mo id="S2.SS1.p1.3.m3.4.4.3.3.7" xref="S2.SS1.p1.3.m3.4.4.3.4.cmml">,</mo><msub id="S2.SS1.p1.3.m3.4.4.3.3.3" xref="S2.SS1.p1.3.m3.4.4.3.3.3.cmml"><mi id="S2.SS1.p1.3.m3.4.4.3.3.3.2" xref="S2.SS1.p1.3.m3.4.4.3.3.3.2.cmml">x</mi><mi id="S2.SS1.p1.3.m3.4.4.3.3.3.3" xref="S2.SS1.p1.3.m3.4.4.3.3.3.3.cmml">N</mi></msub><mo stretchy="false" id="S2.SS1.p1.3.m3.4.4.3.3.8" xref="S2.SS1.p1.3.m3.4.4.3.4.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.3.m3.4b"><apply id="S2.SS1.p1.3.m3.4.4.cmml" xref="S2.SS1.p1.3.m3.4.4"><eq id="S2.SS1.p1.3.m3.4.4.4.cmml" xref="S2.SS1.p1.3.m3.4.4.4"></eq><ci id="S2.SS1.p1.3.m3.4.4.5.cmml" xref="S2.SS1.p1.3.m3.4.4.5">𝑋</ci><set id="S2.SS1.p1.3.m3.4.4.3.4.cmml" xref="S2.SS1.p1.3.m3.4.4.3.3"><apply id="S2.SS1.p1.3.m3.2.2.1.1.1.cmml" xref="S2.SS1.p1.3.m3.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p1.3.m3.2.2.1.1.1.1.cmml" xref="S2.SS1.p1.3.m3.2.2.1.1.1">subscript</csymbol><ci id="S2.SS1.p1.3.m3.2.2.1.1.1.2.cmml" xref="S2.SS1.p1.3.m3.2.2.1.1.1.2">𝑥</ci><cn type="integer" id="S2.SS1.p1.3.m3.2.2.1.1.1.3.cmml" xref="S2.SS1.p1.3.m3.2.2.1.1.1.3">1</cn></apply><apply id="S2.SS1.p1.3.m3.3.3.2.2.2.cmml" xref="S2.SS1.p1.3.m3.3.3.2.2.2"><csymbol cd="ambiguous" id="S2.SS1.p1.3.m3.3.3.2.2.2.1.cmml" xref="S2.SS1.p1.3.m3.3.3.2.2.2">subscript</csymbol><ci id="S2.SS1.p1.3.m3.3.3.2.2.2.2.cmml" xref="S2.SS1.p1.3.m3.3.3.2.2.2.2">𝑥</ci><cn type="integer" id="S2.SS1.p1.3.m3.3.3.2.2.2.3.cmml" xref="S2.SS1.p1.3.m3.3.3.2.2.2.3">2</cn></apply><ci id="S2.SS1.p1.3.m3.1.1.cmml" xref="S2.SS1.p1.3.m3.1.1">…</ci><apply id="S2.SS1.p1.3.m3.4.4.3.3.3.cmml" xref="S2.SS1.p1.3.m3.4.4.3.3.3"><csymbol cd="ambiguous" id="S2.SS1.p1.3.m3.4.4.3.3.3.1.cmml" xref="S2.SS1.p1.3.m3.4.4.3.3.3">subscript</csymbol><ci id="S2.SS1.p1.3.m3.4.4.3.3.3.2.cmml" xref="S2.SS1.p1.3.m3.4.4.3.3.3.2">𝑥</ci><ci id="S2.SS1.p1.3.m3.4.4.3.3.3.3.cmml" xref="S2.SS1.p1.3.m3.4.4.3.3.3.3">𝑁</ci></apply></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.3.m3.4c">X=\{x_{1},x_{2},\dots,x_{N}\}</annotation></semantics></math> of length <math id="S2.SS1.p1.4.m4.1" class="ltx_Math" alttext="N" display="inline"><semantics id="S2.SS1.p1.4.m4.1a"><mi id="S2.SS1.p1.4.m4.1.1" xref="S2.SS1.p1.4.m4.1.1.cmml">N</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.4.m4.1b"><ci id="S2.SS1.p1.4.m4.1.1.cmml" xref="S2.SS1.p1.4.m4.1.1">𝑁</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.4.m4.1c">N</annotation></semantics></math> followed by a set of newly generated tokens <math id="S2.SS1.p1.5.m5.4" class="ltx_Math" alttext="Y=\{y_{1},y_{2},\dots,y_{t-1}\}" display="inline"><semantics id="S2.SS1.p1.5.m5.4a"><mrow id="S2.SS1.p1.5.m5.4.4" xref="S2.SS1.p1.5.m5.4.4.cmml"><mi id="S2.SS1.p1.5.m5.4.4.5" xref="S2.SS1.p1.5.m5.4.4.5.cmml">Y</mi><mo id="S2.SS1.p1.5.m5.4.4.4" xref="S2.SS1.p1.5.m5.4.4.4.cmml">=</mo><mrow id="S2.SS1.p1.5.m5.4.4.3.3" xref="S2.SS1.p1.5.m5.4.4.3.4.cmml"><mo stretchy="false" id="S2.SS1.p1.5.m5.4.4.3.3.4" xref="S2.SS1.p1.5.m5.4.4.3.4.cmml">{</mo><msub id="S2.SS1.p1.5.m5.2.2.1.1.1" xref="S2.SS1.p1.5.m5.2.2.1.1.1.cmml"><mi id="S2.SS1.p1.5.m5.2.2.1.1.1.2" xref="S2.SS1.p1.5.m5.2.2.1.1.1.2.cmml">y</mi><mn id="S2.SS1.p1.5.m5.2.2.1.1.1.3" xref="S2.SS1.p1.5.m5.2.2.1.1.1.3.cmml">1</mn></msub><mo id="S2.SS1.p1.5.m5.4.4.3.3.5" xref="S2.SS1.p1.5.m5.4.4.3.4.cmml">,</mo><msub id="S2.SS1.p1.5.m5.3.3.2.2.2" xref="S2.SS1.p1.5.m5.3.3.2.2.2.cmml"><mi id="S2.SS1.p1.5.m5.3.3.2.2.2.2" xref="S2.SS1.p1.5.m5.3.3.2.2.2.2.cmml">y</mi><mn id="S2.SS1.p1.5.m5.3.3.2.2.2.3" xref="S2.SS1.p1.5.m5.3.3.2.2.2.3.cmml">2</mn></msub><mo id="S2.SS1.p1.5.m5.4.4.3.3.6" xref="S2.SS1.p1.5.m5.4.4.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS1.p1.5.m5.1.1" xref="S2.SS1.p1.5.m5.1.1.cmml">…</mi><mo id="S2.SS1.p1.5.m5.4.4.3.3.7" xref="S2.SS1.p1.5.m5.4.4.3.4.cmml">,</mo><msub id="S2.SS1.p1.5.m5.4.4.3.3.3" xref="S2.SS1.p1.5.m5.4.4.3.3.3.cmml"><mi id="S2.SS1.p1.5.m5.4.4.3.3.3.2" xref="S2.SS1.p1.5.m5.4.4.3.3.3.2.cmml">y</mi><mrow id="S2.SS1.p1.5.m5.4.4.3.3.3.3" xref="S2.SS1.p1.5.m5.4.4.3.3.3.3.cmml"><mi id="S2.SS1.p1.5.m5.4.4.3.3.3.3.2" xref="S2.SS1.p1.5.m5.4.4.3.3.3.3.2.cmml">t</mi><mo id="S2.SS1.p1.5.m5.4.4.3.3.3.3.1" xref="S2.SS1.p1.5.m5.4.4.3.3.3.3.1.cmml">−</mo><mn id="S2.SS1.p1.5.m5.4.4.3.3.3.3.3" xref="S2.SS1.p1.5.m5.4.4.3.3.3.3.3.cmml">1</mn></mrow></msub><mo stretchy="false" id="S2.SS1.p1.5.m5.4.4.3.3.8" xref="S2.SS1.p1.5.m5.4.4.3.4.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.5.m5.4b"><apply id="S2.SS1.p1.5.m5.4.4.cmml" xref="S2.SS1.p1.5.m5.4.4"><eq id="S2.SS1.p1.5.m5.4.4.4.cmml" xref="S2.SS1.p1.5.m5.4.4.4"></eq><ci id="S2.SS1.p1.5.m5.4.4.5.cmml" xref="S2.SS1.p1.5.m5.4.4.5">𝑌</ci><set id="S2.SS1.p1.5.m5.4.4.3.4.cmml" xref="S2.SS1.p1.5.m5.4.4.3.3"><apply id="S2.SS1.p1.5.m5.2.2.1.1.1.cmml" xref="S2.SS1.p1.5.m5.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p1.5.m5.2.2.1.1.1.1.cmml" xref="S2.SS1.p1.5.m5.2.2.1.1.1">subscript</csymbol><ci id="S2.SS1.p1.5.m5.2.2.1.1.1.2.cmml" xref="S2.SS1.p1.5.m5.2.2.1.1.1.2">𝑦</ci><cn type="integer" id="S2.SS1.p1.5.m5.2.2.1.1.1.3.cmml" xref="S2.SS1.p1.5.m5.2.2.1.1.1.3">1</cn></apply><apply id="S2.SS1.p1.5.m5.3.3.2.2.2.cmml" xref="S2.SS1.p1.5.m5.3.3.2.2.2"><csymbol cd="ambiguous" id="S2.SS1.p1.5.m5.3.3.2.2.2.1.cmml" xref="S2.SS1.p1.5.m5.3.3.2.2.2">subscript</csymbol><ci id="S2.SS1.p1.5.m5.3.3.2.2.2.2.cmml" xref="S2.SS1.p1.5.m5.3.3.2.2.2.2">𝑦</ci><cn type="integer" id="S2.SS1.p1.5.m5.3.3.2.2.2.3.cmml" xref="S2.SS1.p1.5.m5.3.3.2.2.2.3">2</cn></apply><ci id="S2.SS1.p1.5.m5.1.1.cmml" xref="S2.SS1.p1.5.m5.1.1">…</ci><apply id="S2.SS1.p1.5.m5.4.4.3.3.3.cmml" xref="S2.SS1.p1.5.m5.4.4.3.3.3"><csymbol cd="ambiguous" id="S2.SS1.p1.5.m5.4.4.3.3.3.1.cmml" xref="S2.SS1.p1.5.m5.4.4.3.3.3">subscript</csymbol><ci id="S2.SS1.p1.5.m5.4.4.3.3.3.2.cmml" xref="S2.SS1.p1.5.m5.4.4.3.3.3.2">𝑦</ci><apply id="S2.SS1.p1.5.m5.4.4.3.3.3.3.cmml" xref="S2.SS1.p1.5.m5.4.4.3.3.3.3"><minus id="S2.SS1.p1.5.m5.4.4.3.3.3.3.1.cmml" xref="S2.SS1.p1.5.m5.4.4.3.3.3.3.1"></minus><ci id="S2.SS1.p1.5.m5.4.4.3.3.3.3.2.cmml" xref="S2.SS1.p1.5.m5.4.4.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.SS1.p1.5.m5.4.4.3.3.3.3.3.cmml" xref="S2.SS1.p1.5.m5.4.4.3.3.3.3.3">1</cn></apply></apply></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.5.m5.4c">Y=\{y_{1},y_{2},\dots,y_{t-1}\}</annotation></semantics></math> to generate the next token <math id="S2.SS1.p1.6.m6.1" class="ltx_Math" alttext="y_{t}" display="inline"><semantics id="S2.SS1.p1.6.m6.1a"><msub id="S2.SS1.p1.6.m6.1.1" xref="S2.SS1.p1.6.m6.1.1.cmml"><mi id="S2.SS1.p1.6.m6.1.1.2" xref="S2.SS1.p1.6.m6.1.1.2.cmml">y</mi><mi id="S2.SS1.p1.6.m6.1.1.3" xref="S2.SS1.p1.6.m6.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.6.m6.1b"><apply id="S2.SS1.p1.6.m6.1.1.cmml" xref="S2.SS1.p1.6.m6.1.1"><csymbol cd="ambiguous" id="S2.SS1.p1.6.m6.1.1.1.cmml" xref="S2.SS1.p1.6.m6.1.1">subscript</csymbol><ci id="S2.SS1.p1.6.m6.1.1.2.cmml" xref="S2.SS1.p1.6.m6.1.1.2">𝑦</ci><ci id="S2.SS1.p1.6.m6.1.1.3.cmml" xref="S2.SS1.p1.6.m6.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.6.m6.1c">y_{t}</annotation></semantics></math>.
For time step <math id="S2.SS1.p1.7.m7.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S2.SS1.p1.7.m7.1a"><mi id="S2.SS1.p1.7.m7.1.1" xref="S2.SS1.p1.7.m7.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.7.m7.1b"><ci id="S2.SS1.p1.7.m7.1.1.cmml" xref="S2.SS1.p1.7.m7.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.7.m7.1c">t</annotation></semantics></math>, and for each head, we calculate the ratio of attention weights focused on the context tokens versus the newly generated tokens.
Formally, for each head <math id="S2.SS1.p1.8.m8.1" class="ltx_Math" alttext="h" display="inline"><semantics id="S2.SS1.p1.8.m8.1a"><mi id="S2.SS1.p1.8.m8.1.1" xref="S2.SS1.p1.8.m8.1.1.cmml">h</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.8.m8.1b"><ci id="S2.SS1.p1.8.m8.1.1.cmml" xref="S2.SS1.p1.8.m8.1.1">ℎ</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.8.m8.1c">h</annotation></semantics></math> in layer <math id="S2.SS1.p1.9.m9.1" class="ltx_Math" alttext="l" display="inline"><semantics id="S2.SS1.p1.9.m9.1a"><mi id="S2.SS1.p1.9.m9.1.1" xref="S2.SS1.p1.9.m9.1.1.cmml">l</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.9.m9.1b"><ci id="S2.SS1.p1.9.m9.1.1.cmml" xref="S2.SS1.p1.9.m9.1.1">𝑙</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.9.m9.1c">l</annotation></semantics></math>, we define:</p>
<table id="S2.Ex1" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.Ex1.m1.6" class="ltx_Math" alttext="A^{l,h}_{t}(\text{context})=\frac{1}{N}\sum_{i=1}^{N}\alpha^{l}_{h,i}," display="block"><semantics id="S2.Ex1.m1.6a"><mrow id="S2.Ex1.m1.6.6.1" xref="S2.Ex1.m1.6.6.1.1.cmml"><mrow id="S2.Ex1.m1.6.6.1.1" xref="S2.Ex1.m1.6.6.1.1.cmml"><mrow id="S2.Ex1.m1.6.6.1.1.2" xref="S2.Ex1.m1.6.6.1.1.2.cmml"><msubsup id="S2.Ex1.m1.6.6.1.1.2.2" xref="S2.Ex1.m1.6.6.1.1.2.2.cmml"><mi id="S2.Ex1.m1.6.6.1.1.2.2.2.2" xref="S2.Ex1.m1.6.6.1.1.2.2.2.2.cmml">A</mi><mi id="S2.Ex1.m1.6.6.1.1.2.2.3" xref="S2.Ex1.m1.6.6.1.1.2.2.3.cmml">t</mi><mrow id="S2.Ex1.m1.2.2.2.4" xref="S2.Ex1.m1.2.2.2.3.cmml"><mi id="S2.Ex1.m1.1.1.1.1" xref="S2.Ex1.m1.1.1.1.1.cmml">l</mi><mo id="S2.Ex1.m1.2.2.2.4.1" xref="S2.Ex1.m1.2.2.2.3.cmml">,</mo><mi id="S2.Ex1.m1.2.2.2.2" xref="S2.Ex1.m1.2.2.2.2.cmml">h</mi></mrow></msubsup><mo lspace="0em" rspace="0em" id="S2.Ex1.m1.6.6.1.1.2.1" xref="S2.Ex1.m1.6.6.1.1.2.1.cmml">​</mo><mrow id="S2.Ex1.m1.6.6.1.1.2.3.2" xref="S2.Ex1.m1.5.5a.cmml"><mo stretchy="false" id="S2.Ex1.m1.6.6.1.1.2.3.2.1" xref="S2.Ex1.m1.5.5a.cmml">(</mo><mtext id="S2.Ex1.m1.5.5" xref="S2.Ex1.m1.5.5.cmml">context</mtext><mo stretchy="false" id="S2.Ex1.m1.6.6.1.1.2.3.2.2" xref="S2.Ex1.m1.5.5a.cmml">)</mo></mrow></mrow><mo id="S2.Ex1.m1.6.6.1.1.1" xref="S2.Ex1.m1.6.6.1.1.1.cmml">=</mo><mrow id="S2.Ex1.m1.6.6.1.1.3" xref="S2.Ex1.m1.6.6.1.1.3.cmml"><mfrac id="S2.Ex1.m1.6.6.1.1.3.2" xref="S2.Ex1.m1.6.6.1.1.3.2.cmml"><mn id="S2.Ex1.m1.6.6.1.1.3.2.2" xref="S2.Ex1.m1.6.6.1.1.3.2.2.cmml">1</mn><mi id="S2.Ex1.m1.6.6.1.1.3.2.3" xref="S2.Ex1.m1.6.6.1.1.3.2.3.cmml">N</mi></mfrac><mo lspace="0em" rspace="0em" id="S2.Ex1.m1.6.6.1.1.3.1" xref="S2.Ex1.m1.6.6.1.1.3.1.cmml">​</mo><mrow id="S2.Ex1.m1.6.6.1.1.3.3" xref="S2.Ex1.m1.6.6.1.1.3.3.cmml"><munderover id="S2.Ex1.m1.6.6.1.1.3.3.1" xref="S2.Ex1.m1.6.6.1.1.3.3.1.cmml"><mo movablelimits="false" id="S2.Ex1.m1.6.6.1.1.3.3.1.2.2" xref="S2.Ex1.m1.6.6.1.1.3.3.1.2.2.cmml">∑</mo><mrow id="S2.Ex1.m1.6.6.1.1.3.3.1.2.3" xref="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.cmml"><mi id="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.2" xref="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.2.cmml">i</mi><mo id="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.1" xref="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.1.cmml">=</mo><mn id="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.3" xref="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.3.cmml">1</mn></mrow><mi id="S2.Ex1.m1.6.6.1.1.3.3.1.3" xref="S2.Ex1.m1.6.6.1.1.3.3.1.3.cmml">N</mi></munderover><msubsup id="S2.Ex1.m1.6.6.1.1.3.3.2" xref="S2.Ex1.m1.6.6.1.1.3.3.2.cmml"><mi id="S2.Ex1.m1.6.6.1.1.3.3.2.2.2" xref="S2.Ex1.m1.6.6.1.1.3.3.2.2.2.cmml">α</mi><mrow id="S2.Ex1.m1.4.4.2.4" xref="S2.Ex1.m1.4.4.2.3.cmml"><mi id="S2.Ex1.m1.3.3.1.1" xref="S2.Ex1.m1.3.3.1.1.cmml">h</mi><mo id="S2.Ex1.m1.4.4.2.4.1" xref="S2.Ex1.m1.4.4.2.3.cmml">,</mo><mi id="S2.Ex1.m1.4.4.2.2" xref="S2.Ex1.m1.4.4.2.2.cmml">i</mi></mrow><mi id="S2.Ex1.m1.6.6.1.1.3.3.2.2.3" xref="S2.Ex1.m1.6.6.1.1.3.3.2.2.3.cmml">l</mi></msubsup></mrow></mrow></mrow><mo id="S2.Ex1.m1.6.6.1.2" xref="S2.Ex1.m1.6.6.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.Ex1.m1.6b"><apply id="S2.Ex1.m1.6.6.1.1.cmml" xref="S2.Ex1.m1.6.6.1"><eq id="S2.Ex1.m1.6.6.1.1.1.cmml" xref="S2.Ex1.m1.6.6.1.1.1"></eq><apply id="S2.Ex1.m1.6.6.1.1.2.cmml" xref="S2.Ex1.m1.6.6.1.1.2"><times id="S2.Ex1.m1.6.6.1.1.2.1.cmml" xref="S2.Ex1.m1.6.6.1.1.2.1"></times><apply id="S2.Ex1.m1.6.6.1.1.2.2.cmml" xref="S2.Ex1.m1.6.6.1.1.2.2"><csymbol cd="ambiguous" id="S2.Ex1.m1.6.6.1.1.2.2.1.cmml" xref="S2.Ex1.m1.6.6.1.1.2.2">subscript</csymbol><apply id="S2.Ex1.m1.6.6.1.1.2.2.2.cmml" xref="S2.Ex1.m1.6.6.1.1.2.2"><csymbol cd="ambiguous" id="S2.Ex1.m1.6.6.1.1.2.2.2.1.cmml" xref="S2.Ex1.m1.6.6.1.1.2.2">superscript</csymbol><ci id="S2.Ex1.m1.6.6.1.1.2.2.2.2.cmml" xref="S2.Ex1.m1.6.6.1.1.2.2.2.2">𝐴</ci><list id="S2.Ex1.m1.2.2.2.3.cmml" xref="S2.Ex1.m1.2.2.2.4"><ci id="S2.Ex1.m1.1.1.1.1.cmml" xref="S2.Ex1.m1.1.1.1.1">𝑙</ci><ci id="S2.Ex1.m1.2.2.2.2.cmml" xref="S2.Ex1.m1.2.2.2.2">ℎ</ci></list></apply><ci id="S2.Ex1.m1.6.6.1.1.2.2.3.cmml" xref="S2.Ex1.m1.6.6.1.1.2.2.3">𝑡</ci></apply><ci id="S2.Ex1.m1.5.5a.cmml" xref="S2.Ex1.m1.6.6.1.1.2.3.2"><mtext id="S2.Ex1.m1.5.5.cmml" xref="S2.Ex1.m1.5.5">context</mtext></ci></apply><apply id="S2.Ex1.m1.6.6.1.1.3.cmml" xref="S2.Ex1.m1.6.6.1.1.3"><times id="S2.Ex1.m1.6.6.1.1.3.1.cmml" xref="S2.Ex1.m1.6.6.1.1.3.1"></times><apply id="S2.Ex1.m1.6.6.1.1.3.2.cmml" xref="S2.Ex1.m1.6.6.1.1.3.2"><divide id="S2.Ex1.m1.6.6.1.1.3.2.1.cmml" xref="S2.Ex1.m1.6.6.1.1.3.2"></divide><cn type="integer" id="S2.Ex1.m1.6.6.1.1.3.2.2.cmml" xref="S2.Ex1.m1.6.6.1.1.3.2.2">1</cn><ci id="S2.Ex1.m1.6.6.1.1.3.2.3.cmml" xref="S2.Ex1.m1.6.6.1.1.3.2.3">𝑁</ci></apply><apply id="S2.Ex1.m1.6.6.1.1.3.3.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3"><apply id="S2.Ex1.m1.6.6.1.1.3.3.1.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.1"><csymbol cd="ambiguous" id="S2.Ex1.m1.6.6.1.1.3.3.1.1.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.1">superscript</csymbol><apply id="S2.Ex1.m1.6.6.1.1.3.3.1.2.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.1"><csymbol cd="ambiguous" id="S2.Ex1.m1.6.6.1.1.3.3.1.2.1.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.1">subscript</csymbol><sum id="S2.Ex1.m1.6.6.1.1.3.3.1.2.2.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.1.2.2"></sum><apply id="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.1.2.3"><eq id="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.1.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.1"></eq><ci id="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.2.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.2">𝑖</ci><cn type="integer" id="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.3.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.1.2.3.3">1</cn></apply></apply><ci id="S2.Ex1.m1.6.6.1.1.3.3.1.3.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.1.3">𝑁</ci></apply><apply id="S2.Ex1.m1.6.6.1.1.3.3.2.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.2"><csymbol cd="ambiguous" id="S2.Ex1.m1.6.6.1.1.3.3.2.1.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.2">subscript</csymbol><apply id="S2.Ex1.m1.6.6.1.1.3.3.2.2.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.2"><csymbol cd="ambiguous" id="S2.Ex1.m1.6.6.1.1.3.3.2.2.1.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.2">superscript</csymbol><ci id="S2.Ex1.m1.6.6.1.1.3.3.2.2.2.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.2.2.2">𝛼</ci><ci id="S2.Ex1.m1.6.6.1.1.3.3.2.2.3.cmml" xref="S2.Ex1.m1.6.6.1.1.3.3.2.2.3">𝑙</ci></apply><list id="S2.Ex1.m1.4.4.2.3.cmml" xref="S2.Ex1.m1.4.4.2.4"><ci id="S2.Ex1.m1.3.3.1.1.cmml" xref="S2.Ex1.m1.3.3.1.1">ℎ</ci><ci id="S2.Ex1.m1.4.4.2.2.cmml" xref="S2.Ex1.m1.4.4.2.2">𝑖</ci></list></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Ex1.m1.6c">A^{l,h}_{t}(\text{context})=\frac{1}{N}\sum_{i=1}^{N}\alpha^{l}_{h,i},</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<table id="S2.Ex2" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.Ex2.m1.6" class="ltx_Math" alttext="A^{l,h}_{t}(\text{new})=\frac{1}{t-1}\sum_{j=N+1}^{N+t-1}\alpha^{l}_{h,j}," display="block"><semantics id="S2.Ex2.m1.6a"><mrow id="S2.Ex2.m1.6.6.1" xref="S2.Ex2.m1.6.6.1.1.cmml"><mrow id="S2.Ex2.m1.6.6.1.1" xref="S2.Ex2.m1.6.6.1.1.cmml"><mrow id="S2.Ex2.m1.6.6.1.1.2" xref="S2.Ex2.m1.6.6.1.1.2.cmml"><msubsup id="S2.Ex2.m1.6.6.1.1.2.2" xref="S2.Ex2.m1.6.6.1.1.2.2.cmml"><mi id="S2.Ex2.m1.6.6.1.1.2.2.2.2" xref="S2.Ex2.m1.6.6.1.1.2.2.2.2.cmml">A</mi><mi id="S2.Ex2.m1.6.6.1.1.2.2.3" xref="S2.Ex2.m1.6.6.1.1.2.2.3.cmml">t</mi><mrow id="S2.Ex2.m1.2.2.2.4" xref="S2.Ex2.m1.2.2.2.3.cmml"><mi id="S2.Ex2.m1.1.1.1.1" xref="S2.Ex2.m1.1.1.1.1.cmml">l</mi><mo id="S2.Ex2.m1.2.2.2.4.1" xref="S2.Ex2.m1.2.2.2.3.cmml">,</mo><mi id="S2.Ex2.m1.2.2.2.2" xref="S2.Ex2.m1.2.2.2.2.cmml">h</mi></mrow></msubsup><mo lspace="0em" rspace="0em" id="S2.Ex2.m1.6.6.1.1.2.1" xref="S2.Ex2.m1.6.6.1.1.2.1.cmml">​</mo><mrow id="S2.Ex2.m1.6.6.1.1.2.3.2" xref="S2.Ex2.m1.5.5a.cmml"><mo stretchy="false" id="S2.Ex2.m1.6.6.1.1.2.3.2.1" xref="S2.Ex2.m1.5.5a.cmml">(</mo><mtext id="S2.Ex2.m1.5.5" xref="S2.Ex2.m1.5.5.cmml">new</mtext><mo stretchy="false" id="S2.Ex2.m1.6.6.1.1.2.3.2.2" xref="S2.Ex2.m1.5.5a.cmml">)</mo></mrow></mrow><mo id="S2.Ex2.m1.6.6.1.1.1" xref="S2.Ex2.m1.6.6.1.1.1.cmml">=</mo><mrow id="S2.Ex2.m1.6.6.1.1.3" xref="S2.Ex2.m1.6.6.1.1.3.cmml"><mfrac id="S2.Ex2.m1.6.6.1.1.3.2" xref="S2.Ex2.m1.6.6.1.1.3.2.cmml"><mn id="S2.Ex2.m1.6.6.1.1.3.2.2" xref="S2.Ex2.m1.6.6.1.1.3.2.2.cmml">1</mn><mrow id="S2.Ex2.m1.6.6.1.1.3.2.3" xref="S2.Ex2.m1.6.6.1.1.3.2.3.cmml"><mi id="S2.Ex2.m1.6.6.1.1.3.2.3.2" xref="S2.Ex2.m1.6.6.1.1.3.2.3.2.cmml">t</mi><mo id="S2.Ex2.m1.6.6.1.1.3.2.3.1" xref="S2.Ex2.m1.6.6.1.1.3.2.3.1.cmml">−</mo><mn id="S2.Ex2.m1.6.6.1.1.3.2.3.3" xref="S2.Ex2.m1.6.6.1.1.3.2.3.3.cmml">1</mn></mrow></mfrac><mo lspace="0em" rspace="0em" id="S2.Ex2.m1.6.6.1.1.3.1" xref="S2.Ex2.m1.6.6.1.1.3.1.cmml">​</mo><mrow id="S2.Ex2.m1.6.6.1.1.3.3" xref="S2.Ex2.m1.6.6.1.1.3.3.cmml"><munderover id="S2.Ex2.m1.6.6.1.1.3.3.1" xref="S2.Ex2.m1.6.6.1.1.3.3.1.cmml"><mo movablelimits="false" id="S2.Ex2.m1.6.6.1.1.3.3.1.2.2" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.2.cmml">∑</mo><mrow id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.cmml"><mi id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.2" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.2.cmml">j</mi><mo id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.1" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.1.cmml">=</mo><mrow id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.cmml"><mi id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.2" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.2.cmml">N</mi><mo id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.1" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.1.cmml">+</mo><mn id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.3" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.3.cmml">1</mn></mrow></mrow><mrow id="S2.Ex2.m1.6.6.1.1.3.3.1.3" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.cmml"><mrow id="S2.Ex2.m1.6.6.1.1.3.3.1.3.2" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.cmml"><mi id="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.2" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.2.cmml">N</mi><mo id="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.1" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.1.cmml">+</mo><mi id="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.3" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.3.cmml">t</mi></mrow><mo id="S2.Ex2.m1.6.6.1.1.3.3.1.3.1" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.1.cmml">−</mo><mn id="S2.Ex2.m1.6.6.1.1.3.3.1.3.3" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.3.cmml">1</mn></mrow></munderover><msubsup id="S2.Ex2.m1.6.6.1.1.3.3.2" xref="S2.Ex2.m1.6.6.1.1.3.3.2.cmml"><mi id="S2.Ex2.m1.6.6.1.1.3.3.2.2.2" xref="S2.Ex2.m1.6.6.1.1.3.3.2.2.2.cmml">α</mi><mrow id="S2.Ex2.m1.4.4.2.4" xref="S2.Ex2.m1.4.4.2.3.cmml"><mi id="S2.Ex2.m1.3.3.1.1" xref="S2.Ex2.m1.3.3.1.1.cmml">h</mi><mo id="S2.Ex2.m1.4.4.2.4.1" xref="S2.Ex2.m1.4.4.2.3.cmml">,</mo><mi id="S2.Ex2.m1.4.4.2.2" xref="S2.Ex2.m1.4.4.2.2.cmml">j</mi></mrow><mi id="S2.Ex2.m1.6.6.1.1.3.3.2.2.3" xref="S2.Ex2.m1.6.6.1.1.3.3.2.2.3.cmml">l</mi></msubsup></mrow></mrow></mrow><mo id="S2.Ex2.m1.6.6.1.2" xref="S2.Ex2.m1.6.6.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.Ex2.m1.6b"><apply id="S2.Ex2.m1.6.6.1.1.cmml" xref="S2.Ex2.m1.6.6.1"><eq id="S2.Ex2.m1.6.6.1.1.1.cmml" xref="S2.Ex2.m1.6.6.1.1.1"></eq><apply id="S2.Ex2.m1.6.6.1.1.2.cmml" xref="S2.Ex2.m1.6.6.1.1.2"><times id="S2.Ex2.m1.6.6.1.1.2.1.cmml" xref="S2.Ex2.m1.6.6.1.1.2.1"></times><apply id="S2.Ex2.m1.6.6.1.1.2.2.cmml" xref="S2.Ex2.m1.6.6.1.1.2.2"><csymbol cd="ambiguous" id="S2.Ex2.m1.6.6.1.1.2.2.1.cmml" xref="S2.Ex2.m1.6.6.1.1.2.2">subscript</csymbol><apply id="S2.Ex2.m1.6.6.1.1.2.2.2.cmml" xref="S2.Ex2.m1.6.6.1.1.2.2"><csymbol cd="ambiguous" id="S2.Ex2.m1.6.6.1.1.2.2.2.1.cmml" xref="S2.Ex2.m1.6.6.1.1.2.2">superscript</csymbol><ci id="S2.Ex2.m1.6.6.1.1.2.2.2.2.cmml" xref="S2.Ex2.m1.6.6.1.1.2.2.2.2">𝐴</ci><list id="S2.Ex2.m1.2.2.2.3.cmml" xref="S2.Ex2.m1.2.2.2.4"><ci id="S2.Ex2.m1.1.1.1.1.cmml" xref="S2.Ex2.m1.1.1.1.1">𝑙</ci><ci id="S2.Ex2.m1.2.2.2.2.cmml" xref="S2.Ex2.m1.2.2.2.2">ℎ</ci></list></apply><ci id="S2.Ex2.m1.6.6.1.1.2.2.3.cmml" xref="S2.Ex2.m1.6.6.1.1.2.2.3">𝑡</ci></apply><ci id="S2.Ex2.m1.5.5a.cmml" xref="S2.Ex2.m1.6.6.1.1.2.3.2"><mtext id="S2.Ex2.m1.5.5.cmml" xref="S2.Ex2.m1.5.5">new</mtext></ci></apply><apply id="S2.Ex2.m1.6.6.1.1.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3"><times id="S2.Ex2.m1.6.6.1.1.3.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.1"></times><apply id="S2.Ex2.m1.6.6.1.1.3.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.2"><divide id="S2.Ex2.m1.6.6.1.1.3.2.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.2"></divide><cn type="integer" id="S2.Ex2.m1.6.6.1.1.3.2.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.2.2">1</cn><apply id="S2.Ex2.m1.6.6.1.1.3.2.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3.2.3"><minus id="S2.Ex2.m1.6.6.1.1.3.2.3.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.2.3.1"></minus><ci id="S2.Ex2.m1.6.6.1.1.3.2.3.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.2.3.2">𝑡</ci><cn type="integer" id="S2.Ex2.m1.6.6.1.1.3.2.3.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3.2.3.3">1</cn></apply></apply><apply id="S2.Ex2.m1.6.6.1.1.3.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3"><apply id="S2.Ex2.m1.6.6.1.1.3.3.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1"><csymbol cd="ambiguous" id="S2.Ex2.m1.6.6.1.1.3.3.1.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1">superscript</csymbol><apply id="S2.Ex2.m1.6.6.1.1.3.3.1.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1"><csymbol cd="ambiguous" id="S2.Ex2.m1.6.6.1.1.3.3.1.2.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1">subscript</csymbol><sum id="S2.Ex2.m1.6.6.1.1.3.3.1.2.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.2"></sum><apply id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3"><eq id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.1"></eq><ci id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.2">𝑗</ci><apply id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3"><plus id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.1"></plus><ci id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.2">𝑁</ci><cn type="integer" id="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.2.3.3.3">1</cn></apply></apply></apply><apply id="S2.Ex2.m1.6.6.1.1.3.3.1.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3"><minus id="S2.Ex2.m1.6.6.1.1.3.3.1.3.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.1"></minus><apply id="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.2"><plus id="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.1"></plus><ci id="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.2">𝑁</ci><ci id="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.2.3">𝑡</ci></apply><cn type="integer" id="S2.Ex2.m1.6.6.1.1.3.3.1.3.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.1.3.3">1</cn></apply></apply><apply id="S2.Ex2.m1.6.6.1.1.3.3.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.2"><csymbol cd="ambiguous" id="S2.Ex2.m1.6.6.1.1.3.3.2.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.2">subscript</csymbol><apply id="S2.Ex2.m1.6.6.1.1.3.3.2.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.2"><csymbol cd="ambiguous" id="S2.Ex2.m1.6.6.1.1.3.3.2.2.1.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.2">superscript</csymbol><ci id="S2.Ex2.m1.6.6.1.1.3.3.2.2.2.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.2.2.2">𝛼</ci><ci id="S2.Ex2.m1.6.6.1.1.3.3.2.2.3.cmml" xref="S2.Ex2.m1.6.6.1.1.3.3.2.2.3">𝑙</ci></apply><list id="S2.Ex2.m1.4.4.2.3.cmml" xref="S2.Ex2.m1.4.4.2.4"><ci id="S2.Ex2.m1.3.3.1.1.cmml" xref="S2.Ex2.m1.3.3.1.1">ℎ</ci><ci id="S2.Ex2.m1.4.4.2.2.cmml" xref="S2.Ex2.m1.4.4.2.2">𝑗</ci></list></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Ex2.m1.6c">A^{l,h}_{t}(\text{new})=\frac{1}{t-1}\sum_{j=N+1}^{N+t-1}\alpha^{l}_{h,j},</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S2.SS1.p1.17" class="ltx_p">where <math id="S2.SS1.p1.10.m1.2" class="ltx_Math" alttext="\alpha^{l}_{h,i}" display="inline"><semantics id="S2.SS1.p1.10.m1.2a"><msubsup id="S2.SS1.p1.10.m1.2.3" xref="S2.SS1.p1.10.m1.2.3.cmml"><mi id="S2.SS1.p1.10.m1.2.3.2.2" xref="S2.SS1.p1.10.m1.2.3.2.2.cmml">α</mi><mrow id="S2.SS1.p1.10.m1.2.2.2.4" xref="S2.SS1.p1.10.m1.2.2.2.3.cmml"><mi id="S2.SS1.p1.10.m1.1.1.1.1" xref="S2.SS1.p1.10.m1.1.1.1.1.cmml">h</mi><mo id="S2.SS1.p1.10.m1.2.2.2.4.1" xref="S2.SS1.p1.10.m1.2.2.2.3.cmml">,</mo><mi id="S2.SS1.p1.10.m1.2.2.2.2" xref="S2.SS1.p1.10.m1.2.2.2.2.cmml">i</mi></mrow><mi id="S2.SS1.p1.10.m1.2.3.2.3" xref="S2.SS1.p1.10.m1.2.3.2.3.cmml">l</mi></msubsup><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.10.m1.2b"><apply id="S2.SS1.p1.10.m1.2.3.cmml" xref="S2.SS1.p1.10.m1.2.3"><csymbol cd="ambiguous" id="S2.SS1.p1.10.m1.2.3.1.cmml" xref="S2.SS1.p1.10.m1.2.3">subscript</csymbol><apply id="S2.SS1.p1.10.m1.2.3.2.cmml" xref="S2.SS1.p1.10.m1.2.3"><csymbol cd="ambiguous" id="S2.SS1.p1.10.m1.2.3.2.1.cmml" xref="S2.SS1.p1.10.m1.2.3">superscript</csymbol><ci id="S2.SS1.p1.10.m1.2.3.2.2.cmml" xref="S2.SS1.p1.10.m1.2.3.2.2">𝛼</ci><ci id="S2.SS1.p1.10.m1.2.3.2.3.cmml" xref="S2.SS1.p1.10.m1.2.3.2.3">𝑙</ci></apply><list id="S2.SS1.p1.10.m1.2.2.2.3.cmml" xref="S2.SS1.p1.10.m1.2.2.2.4"><ci id="S2.SS1.p1.10.m1.1.1.1.1.cmml" xref="S2.SS1.p1.10.m1.1.1.1.1">ℎ</ci><ci id="S2.SS1.p1.10.m1.2.2.2.2.cmml" xref="S2.SS1.p1.10.m1.2.2.2.2">𝑖</ci></list></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.10.m1.2c">\alpha^{l}_{h,i}</annotation></semantics></math> and <math id="S2.SS1.p1.11.m2.2" class="ltx_Math" alttext="\alpha^{l}_{h,j}" display="inline"><semantics id="S2.SS1.p1.11.m2.2a"><msubsup id="S2.SS1.p1.11.m2.2.3" xref="S2.SS1.p1.11.m2.2.3.cmml"><mi id="S2.SS1.p1.11.m2.2.3.2.2" xref="S2.SS1.p1.11.m2.2.3.2.2.cmml">α</mi><mrow id="S2.SS1.p1.11.m2.2.2.2.4" xref="S2.SS1.p1.11.m2.2.2.2.3.cmml"><mi id="S2.SS1.p1.11.m2.1.1.1.1" xref="S2.SS1.p1.11.m2.1.1.1.1.cmml">h</mi><mo id="S2.SS1.p1.11.m2.2.2.2.4.1" xref="S2.SS1.p1.11.m2.2.2.2.3.cmml">,</mo><mi id="S2.SS1.p1.11.m2.2.2.2.2" xref="S2.SS1.p1.11.m2.2.2.2.2.cmml">j</mi></mrow><mi id="S2.SS1.p1.11.m2.2.3.2.3" xref="S2.SS1.p1.11.m2.2.3.2.3.cmml">l</mi></msubsup><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.11.m2.2b"><apply id="S2.SS1.p1.11.m2.2.3.cmml" xref="S2.SS1.p1.11.m2.2.3"><csymbol cd="ambiguous" id="S2.SS1.p1.11.m2.2.3.1.cmml" xref="S2.SS1.p1.11.m2.2.3">subscript</csymbol><apply id="S2.SS1.p1.11.m2.2.3.2.cmml" xref="S2.SS1.p1.11.m2.2.3"><csymbol cd="ambiguous" id="S2.SS1.p1.11.m2.2.3.2.1.cmml" xref="S2.SS1.p1.11.m2.2.3">superscript</csymbol><ci id="S2.SS1.p1.11.m2.2.3.2.2.cmml" xref="S2.SS1.p1.11.m2.2.3.2.2">𝛼</ci><ci id="S2.SS1.p1.11.m2.2.3.2.3.cmml" xref="S2.SS1.p1.11.m2.2.3.2.3">𝑙</ci></apply><list id="S2.SS1.p1.11.m2.2.2.2.3.cmml" xref="S2.SS1.p1.11.m2.2.2.2.4"><ci id="S2.SS1.p1.11.m2.1.1.1.1.cmml" xref="S2.SS1.p1.11.m2.1.1.1.1">ℎ</ci><ci id="S2.SS1.p1.11.m2.2.2.2.2.cmml" xref="S2.SS1.p1.11.m2.2.2.2.2">𝑗</ci></list></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.11.m2.2c">\alpha^{l}_{h,j}</annotation></semantics></math> are softmax-ed attention weights assigned to context tokens <math id="S2.SS1.p1.12.m3.1" class="ltx_Math" alttext="X" display="inline"><semantics id="S2.SS1.p1.12.m3.1a"><mi id="S2.SS1.p1.12.m3.1.1" xref="S2.SS1.p1.12.m3.1.1.cmml">X</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.12.m3.1b"><ci id="S2.SS1.p1.12.m3.1.1.cmml" xref="S2.SS1.p1.12.m3.1.1">𝑋</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.12.m3.1c">X</annotation></semantics></math> and new tokens <math id="S2.SS1.p1.13.m4.1" class="ltx_Math" alttext="Y" display="inline"><semantics id="S2.SS1.p1.13.m4.1a"><mi id="S2.SS1.p1.13.m4.1.1" xref="S2.SS1.p1.13.m4.1.1.cmml">Y</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.13.m4.1b"><ci id="S2.SS1.p1.13.m4.1.1.cmml" xref="S2.SS1.p1.13.m4.1.1">𝑌</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.13.m4.1c">Y</annotation></semantics></math> respectively.
The lookback ratio <math id="S2.SS1.p1.14.m5.2" class="ltx_Math" alttext="\text{LR}^{l,h}_{t}" display="inline"><semantics id="S2.SS1.p1.14.m5.2a"><msubsup id="S2.SS1.p1.14.m5.2.3" xref="S2.SS1.p1.14.m5.2.3.cmml"><mtext id="S2.SS1.p1.14.m5.2.3.2.2" xref="S2.SS1.p1.14.m5.2.3.2.2a.cmml">LR</mtext><mi id="S2.SS1.p1.14.m5.2.3.3" xref="S2.SS1.p1.14.m5.2.3.3.cmml">t</mi><mrow id="S2.SS1.p1.14.m5.2.2.2.4" xref="S2.SS1.p1.14.m5.2.2.2.3.cmml"><mi id="S2.SS1.p1.14.m5.1.1.1.1" xref="S2.SS1.p1.14.m5.1.1.1.1.cmml">l</mi><mo id="S2.SS1.p1.14.m5.2.2.2.4.1" xref="S2.SS1.p1.14.m5.2.2.2.3.cmml">,</mo><mi id="S2.SS1.p1.14.m5.2.2.2.2" xref="S2.SS1.p1.14.m5.2.2.2.2.cmml">h</mi></mrow></msubsup><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.14.m5.2b"><apply id="S2.SS1.p1.14.m5.2.3.cmml" xref="S2.SS1.p1.14.m5.2.3"><csymbol cd="ambiguous" id="S2.SS1.p1.14.m5.2.3.1.cmml" xref="S2.SS1.p1.14.m5.2.3">subscript</csymbol><apply id="S2.SS1.p1.14.m5.2.3.2.cmml" xref="S2.SS1.p1.14.m5.2.3"><csymbol cd="ambiguous" id="S2.SS1.p1.14.m5.2.3.2.1.cmml" xref="S2.SS1.p1.14.m5.2.3">superscript</csymbol><ci id="S2.SS1.p1.14.m5.2.3.2.2a.cmml" xref="S2.SS1.p1.14.m5.2.3.2.2"><mtext id="S2.SS1.p1.14.m5.2.3.2.2.cmml" xref="S2.SS1.p1.14.m5.2.3.2.2">LR</mtext></ci><list id="S2.SS1.p1.14.m5.2.2.2.3.cmml" xref="S2.SS1.p1.14.m5.2.2.2.4"><ci id="S2.SS1.p1.14.m5.1.1.1.1.cmml" xref="S2.SS1.p1.14.m5.1.1.1.1">𝑙</ci><ci id="S2.SS1.p1.14.m5.2.2.2.2.cmml" xref="S2.SS1.p1.14.m5.2.2.2.2">ℎ</ci></list></apply><ci id="S2.SS1.p1.14.m5.2.3.3.cmml" xref="S2.SS1.p1.14.m5.2.3.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.14.m5.2c">\text{LR}^{l,h}_{t}</annotation></semantics></math> for head <math id="S2.SS1.p1.15.m6.1" class="ltx_Math" alttext="h" display="inline"><semantics id="S2.SS1.p1.15.m6.1a"><mi id="S2.SS1.p1.15.m6.1.1" xref="S2.SS1.p1.15.m6.1.1.cmml">h</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.15.m6.1b"><ci id="S2.SS1.p1.15.m6.1.1.cmml" xref="S2.SS1.p1.15.m6.1.1">ℎ</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.15.m6.1c">h</annotation></semantics></math> in layer <math id="S2.SS1.p1.16.m7.1" class="ltx_Math" alttext="l" display="inline"><semantics id="S2.SS1.p1.16.m7.1a"><mi id="S2.SS1.p1.16.m7.1.1" xref="S2.SS1.p1.16.m7.1.1.cmml">l</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.16.m7.1b"><ci id="S2.SS1.p1.16.m7.1.1.cmml" xref="S2.SS1.p1.16.m7.1.1">𝑙</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.16.m7.1c">l</annotation></semantics></math> at time step <math id="S2.SS1.p1.17.m8.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S2.SS1.p1.17.m8.1a"><mi id="S2.SS1.p1.17.m8.1.1" xref="S2.SS1.p1.17.m8.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.17.m8.1b"><ci id="S2.SS1.p1.17.m8.1.1.cmml" xref="S2.SS1.p1.17.m8.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.17.m8.1c">t</annotation></semantics></math> is then calculated as:</p>
<table id="S2.Ex3" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.Ex3.m1.12" class="ltx_Math" alttext="\text{LR}^{l,h}_{t}=\frac{A^{l,h}_{t}(\text{context})}{A^{l,h}_{t}(\text{context})+A^{l,h}_{t}(\text{new})}." display="block"><semantics id="S2.Ex3.m1.12a"><mrow id="S2.Ex3.m1.12.12.1" xref="S2.Ex3.m1.12.12.1.1.cmml"><mrow id="S2.Ex3.m1.12.12.1.1" xref="S2.Ex3.m1.12.12.1.1.cmml"><msubsup id="S2.Ex3.m1.12.12.1.1.2" xref="S2.Ex3.m1.12.12.1.1.2.cmml"><mtext id="S2.Ex3.m1.12.12.1.1.2.2.2" xref="S2.Ex3.m1.12.12.1.1.2.2.2a.cmml">LR</mtext><mi id="S2.Ex3.m1.12.12.1.1.2.3" xref="S2.Ex3.m1.12.12.1.1.2.3.cmml">t</mi><mrow id="S2.Ex3.m1.2.2.2.4" xref="S2.Ex3.m1.2.2.2.3.cmml"><mi id="S2.Ex3.m1.1.1.1.1" xref="S2.Ex3.m1.1.1.1.1.cmml">l</mi><mo id="S2.Ex3.m1.2.2.2.4.1" xref="S2.Ex3.m1.2.2.2.3.cmml">,</mo><mi id="S2.Ex3.m1.2.2.2.2" xref="S2.Ex3.m1.2.2.2.2.cmml">h</mi></mrow></msubsup><mo id="S2.Ex3.m1.12.12.1.1.1" xref="S2.Ex3.m1.12.12.1.1.1.cmml">=</mo><mfrac id="S2.Ex3.m1.11.11" xref="S2.Ex3.m1.11.11.cmml"><mrow id="S2.Ex3.m1.5.5.3" xref="S2.Ex3.m1.5.5.3.cmml"><msubsup id="S2.Ex3.m1.5.5.3.5" xref="S2.Ex3.m1.5.5.3.5.cmml"><mi id="S2.Ex3.m1.5.5.3.5.2.2" xref="S2.Ex3.m1.5.5.3.5.2.2.cmml">A</mi><mi id="S2.Ex3.m1.5.5.3.5.3" xref="S2.Ex3.m1.5.5.3.5.3.cmml">t</mi><mrow id="S2.Ex3.m1.4.4.2.2.2.4" xref="S2.Ex3.m1.4.4.2.2.2.3.cmml"><mi id="S2.Ex3.m1.3.3.1.1.1.1" xref="S2.Ex3.m1.3.3.1.1.1.1.cmml">l</mi><mo id="S2.Ex3.m1.4.4.2.2.2.4.1" xref="S2.Ex3.m1.4.4.2.2.2.3.cmml">,</mo><mi id="S2.Ex3.m1.4.4.2.2.2.2" xref="S2.Ex3.m1.4.4.2.2.2.2.cmml">h</mi></mrow></msubsup><mo lspace="0em" rspace="0em" id="S2.Ex3.m1.5.5.3.4" xref="S2.Ex3.m1.5.5.3.4.cmml">​</mo><mrow id="S2.Ex3.m1.5.5.3.6.2" xref="S2.Ex3.m1.5.5.3.3a.cmml"><mo stretchy="false" id="S2.Ex3.m1.5.5.3.6.2.1" xref="S2.Ex3.m1.5.5.3.3a.cmml">(</mo><mtext id="S2.Ex3.m1.5.5.3.3" xref="S2.Ex3.m1.5.5.3.3.cmml">context</mtext><mo stretchy="false" id="S2.Ex3.m1.5.5.3.6.2.2" xref="S2.Ex3.m1.5.5.3.3a.cmml">)</mo></mrow></mrow><mrow id="S2.Ex3.m1.11.11.9" xref="S2.Ex3.m1.11.11.9.cmml"><mrow id="S2.Ex3.m1.11.11.9.8" xref="S2.Ex3.m1.11.11.9.8.cmml"><msubsup id="S2.Ex3.m1.11.11.9.8.2" xref="S2.Ex3.m1.11.11.9.8.2.cmml"><mi id="S2.Ex3.m1.11.11.9.8.2.2.2" xref="S2.Ex3.m1.11.11.9.8.2.2.2.cmml">A</mi><mi id="S2.Ex3.m1.11.11.9.8.2.3" xref="S2.Ex3.m1.11.11.9.8.2.3.cmml">t</mi><mrow id="S2.Ex3.m1.7.7.5.2.2.4" xref="S2.Ex3.m1.7.7.5.2.2.3.cmml"><mi id="S2.Ex3.m1.6.6.4.1.1.1" xref="S2.Ex3.m1.6.6.4.1.1.1.cmml">l</mi><mo id="S2.Ex3.m1.7.7.5.2.2.4.1" xref="S2.Ex3.m1.7.7.5.2.2.3.cmml">,</mo><mi id="S2.Ex3.m1.7.7.5.2.2.2" xref="S2.Ex3.m1.7.7.5.2.2.2.cmml">h</mi></mrow></msubsup><mo lspace="0em" rspace="0em" id="S2.Ex3.m1.11.11.9.8.1" xref="S2.Ex3.m1.11.11.9.8.1.cmml">​</mo><mrow id="S2.Ex3.m1.11.11.9.8.3.2" xref="S2.Ex3.m1.10.10.8.5a.cmml"><mo stretchy="false" id="S2.Ex3.m1.11.11.9.8.3.2.1" xref="S2.Ex3.m1.10.10.8.5a.cmml">(</mo><mtext id="S2.Ex3.m1.10.10.8.5" xref="S2.Ex3.m1.10.10.8.5.cmml">context</mtext><mo stretchy="false" id="S2.Ex3.m1.11.11.9.8.3.2.2" xref="S2.Ex3.m1.10.10.8.5a.cmml">)</mo></mrow></mrow><mo id="S2.Ex3.m1.11.11.9.7" xref="S2.Ex3.m1.11.11.9.7.cmml">+</mo><mrow id="S2.Ex3.m1.11.11.9.9" xref="S2.Ex3.m1.11.11.9.9.cmml"><msubsup id="S2.Ex3.m1.11.11.9.9.2" xref="S2.Ex3.m1.11.11.9.9.2.cmml"><mi id="S2.Ex3.m1.11.11.9.9.2.2.2" xref="S2.Ex3.m1.11.11.9.9.2.2.2.cmml">A</mi><mi id="S2.Ex3.m1.11.11.9.9.2.3" xref="S2.Ex3.m1.11.11.9.9.2.3.cmml">t</mi><mrow id="S2.Ex3.m1.9.9.7.4.2.4" xref="S2.Ex3.m1.9.9.7.4.2.3.cmml"><mi id="S2.Ex3.m1.8.8.6.3.1.1" xref="S2.Ex3.m1.8.8.6.3.1.1.cmml">l</mi><mo id="S2.Ex3.m1.9.9.7.4.2.4.1" xref="S2.Ex3.m1.9.9.7.4.2.3.cmml">,</mo><mi id="S2.Ex3.m1.9.9.7.4.2.2" xref="S2.Ex3.m1.9.9.7.4.2.2.cmml">h</mi></mrow></msubsup><mo lspace="0em" rspace="0em" id="S2.Ex3.m1.11.11.9.9.1" xref="S2.Ex3.m1.11.11.9.9.1.cmml">​</mo><mrow id="S2.Ex3.m1.11.11.9.9.3.2" xref="S2.Ex3.m1.11.11.9.6a.cmml"><mo stretchy="false" id="S2.Ex3.m1.11.11.9.9.3.2.1" xref="S2.Ex3.m1.11.11.9.6a.cmml">(</mo><mtext id="S2.Ex3.m1.11.11.9.6" xref="S2.Ex3.m1.11.11.9.6.cmml">new</mtext><mo stretchy="false" id="S2.Ex3.m1.11.11.9.9.3.2.2" xref="S2.Ex3.m1.11.11.9.6a.cmml">)</mo></mrow></mrow></mrow></mfrac></mrow><mo lspace="0em" id="S2.Ex3.m1.12.12.1.2" xref="S2.Ex3.m1.12.12.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.Ex3.m1.12b"><apply id="S2.Ex3.m1.12.12.1.1.cmml" xref="S2.Ex3.m1.12.12.1"><eq id="S2.Ex3.m1.12.12.1.1.1.cmml" xref="S2.Ex3.m1.12.12.1.1.1"></eq><apply id="S2.Ex3.m1.12.12.1.1.2.cmml" xref="S2.Ex3.m1.12.12.1.1.2"><csymbol cd="ambiguous" id="S2.Ex3.m1.12.12.1.1.2.1.cmml" xref="S2.Ex3.m1.12.12.1.1.2">subscript</csymbol><apply id="S2.Ex3.m1.12.12.1.1.2.2.cmml" xref="S2.Ex3.m1.12.12.1.1.2"><csymbol cd="ambiguous" id="S2.Ex3.m1.12.12.1.1.2.2.1.cmml" xref="S2.Ex3.m1.12.12.1.1.2">superscript</csymbol><ci id="S2.Ex3.m1.12.12.1.1.2.2.2a.cmml" xref="S2.Ex3.m1.12.12.1.1.2.2.2"><mtext id="S2.Ex3.m1.12.12.1.1.2.2.2.cmml" xref="S2.Ex3.m1.12.12.1.1.2.2.2">LR</mtext></ci><list id="S2.Ex3.m1.2.2.2.3.cmml" xref="S2.Ex3.m1.2.2.2.4"><ci id="S2.Ex3.m1.1.1.1.1.cmml" xref="S2.Ex3.m1.1.1.1.1">𝑙</ci><ci id="S2.Ex3.m1.2.2.2.2.cmml" xref="S2.Ex3.m1.2.2.2.2">ℎ</ci></list></apply><ci id="S2.Ex3.m1.12.12.1.1.2.3.cmml" xref="S2.Ex3.m1.12.12.1.1.2.3">𝑡</ci></apply><apply id="S2.Ex3.m1.11.11.cmml" xref="S2.Ex3.m1.11.11"><divide id="S2.Ex3.m1.11.11.10.cmml" xref="S2.Ex3.m1.11.11"></divide><apply id="S2.Ex3.m1.5.5.3.cmml" xref="S2.Ex3.m1.5.5.3"><times id="S2.Ex3.m1.5.5.3.4.cmml" xref="S2.Ex3.m1.5.5.3.4"></times><apply id="S2.Ex3.m1.5.5.3.5.cmml" xref="S2.Ex3.m1.5.5.3.5"><csymbol cd="ambiguous" id="S2.Ex3.m1.5.5.3.5.1.cmml" xref="S2.Ex3.m1.5.5.3.5">subscript</csymbol><apply id="S2.Ex3.m1.5.5.3.5.2.cmml" xref="S2.Ex3.m1.5.5.3.5"><csymbol cd="ambiguous" id="S2.Ex3.m1.5.5.3.5.2.1.cmml" xref="S2.Ex3.m1.5.5.3.5">superscript</csymbol><ci id="S2.Ex3.m1.5.5.3.5.2.2.cmml" xref="S2.Ex3.m1.5.5.3.5.2.2">𝐴</ci><list id="S2.Ex3.m1.4.4.2.2.2.3.cmml" xref="S2.Ex3.m1.4.4.2.2.2.4"><ci id="S2.Ex3.m1.3.3.1.1.1.1.cmml" xref="S2.Ex3.m1.3.3.1.1.1.1">𝑙</ci><ci id="S2.Ex3.m1.4.4.2.2.2.2.cmml" xref="S2.Ex3.m1.4.4.2.2.2.2">ℎ</ci></list></apply><ci id="S2.Ex3.m1.5.5.3.5.3.cmml" xref="S2.Ex3.m1.5.5.3.5.3">𝑡</ci></apply><ci id="S2.Ex3.m1.5.5.3.3a.cmml" xref="S2.Ex3.m1.5.5.3.6.2"><mtext id="S2.Ex3.m1.5.5.3.3.cmml" xref="S2.Ex3.m1.5.5.3.3">context</mtext></ci></apply><apply id="S2.Ex3.m1.11.11.9.cmml" xref="S2.Ex3.m1.11.11.9"><plus id="S2.Ex3.m1.11.11.9.7.cmml" xref="S2.Ex3.m1.11.11.9.7"></plus><apply id="S2.Ex3.m1.11.11.9.8.cmml" xref="S2.Ex3.m1.11.11.9.8"><times id="S2.Ex3.m1.11.11.9.8.1.cmml" xref="S2.Ex3.m1.11.11.9.8.1"></times><apply id="S2.Ex3.m1.11.11.9.8.2.cmml" xref="S2.Ex3.m1.11.11.9.8.2"><csymbol cd="ambiguous" id="S2.Ex3.m1.11.11.9.8.2.1.cmml" xref="S2.Ex3.m1.11.11.9.8.2">subscript</csymbol><apply id="S2.Ex3.m1.11.11.9.8.2.2.cmml" xref="S2.Ex3.m1.11.11.9.8.2"><csymbol cd="ambiguous" id="S2.Ex3.m1.11.11.9.8.2.2.1.cmml" xref="S2.Ex3.m1.11.11.9.8.2">superscript</csymbol><ci id="S2.Ex3.m1.11.11.9.8.2.2.2.cmml" xref="S2.Ex3.m1.11.11.9.8.2.2.2">𝐴</ci><list id="S2.Ex3.m1.7.7.5.2.2.3.cmml" xref="S2.Ex3.m1.7.7.5.2.2.4"><ci id="S2.Ex3.m1.6.6.4.1.1.1.cmml" xref="S2.Ex3.m1.6.6.4.1.1.1">𝑙</ci><ci id="S2.Ex3.m1.7.7.5.2.2.2.cmml" xref="S2.Ex3.m1.7.7.5.2.2.2">ℎ</ci></list></apply><ci id="S2.Ex3.m1.11.11.9.8.2.3.cmml" xref="S2.Ex3.m1.11.11.9.8.2.3">𝑡</ci></apply><ci id="S2.Ex3.m1.10.10.8.5a.cmml" xref="S2.Ex3.m1.11.11.9.8.3.2"><mtext id="S2.Ex3.m1.10.10.8.5.cmml" xref="S2.Ex3.m1.10.10.8.5">context</mtext></ci></apply><apply id="S2.Ex3.m1.11.11.9.9.cmml" xref="S2.Ex3.m1.11.11.9.9"><times id="S2.Ex3.m1.11.11.9.9.1.cmml" xref="S2.Ex3.m1.11.11.9.9.1"></times><apply id="S2.Ex3.m1.11.11.9.9.2.cmml" xref="S2.Ex3.m1.11.11.9.9.2"><csymbol cd="ambiguous" id="S2.Ex3.m1.11.11.9.9.2.1.cmml" xref="S2.Ex3.m1.11.11.9.9.2">subscript</csymbol><apply id="S2.Ex3.m1.11.11.9.9.2.2.cmml" xref="S2.Ex3.m1.11.11.9.9.2"><csymbol cd="ambiguous" id="S2.Ex3.m1.11.11.9.9.2.2.1.cmml" xref="S2.Ex3.m1.11.11.9.9.2">superscript</csymbol><ci id="S2.Ex3.m1.11.11.9.9.2.2.2.cmml" xref="S2.Ex3.m1.11.11.9.9.2.2.2">𝐴</ci><list id="S2.Ex3.m1.9.9.7.4.2.3.cmml" xref="S2.Ex3.m1.9.9.7.4.2.4"><ci id="S2.Ex3.m1.8.8.6.3.1.1.cmml" xref="S2.Ex3.m1.8.8.6.3.1.1">𝑙</ci><ci id="S2.Ex3.m1.9.9.7.4.2.2.cmml" xref="S2.Ex3.m1.9.9.7.4.2.2">ℎ</ci></list></apply><ci id="S2.Ex3.m1.11.11.9.9.2.3.cmml" xref="S2.Ex3.m1.11.11.9.9.2.3">𝑡</ci></apply><ci id="S2.Ex3.m1.11.11.9.6a.cmml" xref="S2.Ex3.m1.11.11.9.9.3.2"><mtext id="S2.Ex3.m1.11.11.9.6.cmml" xref="S2.Ex3.m1.11.11.9.6">new</mtext></ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Ex3.m1.12c">\text{LR}^{l,h}_{t}=\frac{A^{l,h}_{t}(\text{context})}{A^{l,h}_{t}(\text{context})+A^{l,h}_{t}(\text{new})}.</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
</div>
<div id="S2.SS1.p2" class="ltx_para">
<p id="S2.SS1.p2.1" class="ltx_p">To utilize these lookback ratios as input features in detecting hallucinations, we concatenate the lookback ratios across all heads and layers into a feature vector for the time step <math id="S2.SS1.p2.1.m1.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S2.SS1.p2.1.m1.1a"><mi id="S2.SS1.p2.1.m1.1.1" xref="S2.SS1.p2.1.m1.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.1.m1.1b"><ci id="S2.SS1.p2.1.m1.1.1.cmml" xref="S2.SS1.p2.1.m1.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.1.m1.1c">t</annotation></semantics></math>:</p>
<table id="S2.Ex4" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.Ex4.m1.8" class="ltx_Math" alttext="\mathbf{v}_{t}=[\text{LR}^{1,1}_{t},\text{LR}^{1,2}_{t},\dots,\text{LR}^{L,H}_{t}]." display="block"><semantics id="S2.Ex4.m1.8a"><mrow id="S2.Ex4.m1.8.8.1" xref="S2.Ex4.m1.8.8.1.1.cmml"><mrow id="S2.Ex4.m1.8.8.1.1" xref="S2.Ex4.m1.8.8.1.1.cmml"><msub id="S2.Ex4.m1.8.8.1.1.5" xref="S2.Ex4.m1.8.8.1.1.5.cmml"><mi id="S2.Ex4.m1.8.8.1.1.5.2" xref="S2.Ex4.m1.8.8.1.1.5.2.cmml">𝐯</mi><mi id="S2.Ex4.m1.8.8.1.1.5.3" xref="S2.Ex4.m1.8.8.1.1.5.3.cmml">t</mi></msub><mo id="S2.Ex4.m1.8.8.1.1.4" xref="S2.Ex4.m1.8.8.1.1.4.cmml">=</mo><mrow id="S2.Ex4.m1.8.8.1.1.3.3" xref="S2.Ex4.m1.8.8.1.1.3.4.cmml"><mo stretchy="false" id="S2.Ex4.m1.8.8.1.1.3.3.4" xref="S2.Ex4.m1.8.8.1.1.3.4.cmml">[</mo><msubsup id="S2.Ex4.m1.8.8.1.1.1.1.1" xref="S2.Ex4.m1.8.8.1.1.1.1.1.cmml"><mtext id="S2.Ex4.m1.8.8.1.1.1.1.1.2.2" xref="S2.Ex4.m1.8.8.1.1.1.1.1.2.2a.cmml">LR</mtext><mi id="S2.Ex4.m1.8.8.1.1.1.1.1.3" xref="S2.Ex4.m1.8.8.1.1.1.1.1.3.cmml">t</mi><mrow id="S2.Ex4.m1.2.2.2.4" xref="S2.Ex4.m1.2.2.2.3.cmml"><mn id="S2.Ex4.m1.1.1.1.1" xref="S2.Ex4.m1.1.1.1.1.cmml">1</mn><mo id="S2.Ex4.m1.2.2.2.4.1" xref="S2.Ex4.m1.2.2.2.3.cmml">,</mo><mn id="S2.Ex4.m1.2.2.2.2" xref="S2.Ex4.m1.2.2.2.2.cmml">1</mn></mrow></msubsup><mo id="S2.Ex4.m1.8.8.1.1.3.3.5" xref="S2.Ex4.m1.8.8.1.1.3.4.cmml">,</mo><msubsup id="S2.Ex4.m1.8.8.1.1.2.2.2" xref="S2.Ex4.m1.8.8.1.1.2.2.2.cmml"><mtext id="S2.Ex4.m1.8.8.1.1.2.2.2.2.2" xref="S2.Ex4.m1.8.8.1.1.2.2.2.2.2a.cmml">LR</mtext><mi id="S2.Ex4.m1.8.8.1.1.2.2.2.3" xref="S2.Ex4.m1.8.8.1.1.2.2.2.3.cmml">t</mi><mrow id="S2.Ex4.m1.4.4.2.4" xref="S2.Ex4.m1.4.4.2.3.cmml"><mn id="S2.Ex4.m1.3.3.1.1" xref="S2.Ex4.m1.3.3.1.1.cmml">1</mn><mo id="S2.Ex4.m1.4.4.2.4.1" xref="S2.Ex4.m1.4.4.2.3.cmml">,</mo><mn id="S2.Ex4.m1.4.4.2.2" xref="S2.Ex4.m1.4.4.2.2.cmml">2</mn></mrow></msubsup><mo id="S2.Ex4.m1.8.8.1.1.3.3.6" xref="S2.Ex4.m1.8.8.1.1.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.Ex4.m1.7.7" xref="S2.Ex4.m1.7.7.cmml">…</mi><mo id="S2.Ex4.m1.8.8.1.1.3.3.7" xref="S2.Ex4.m1.8.8.1.1.3.4.cmml">,</mo><msubsup id="S2.Ex4.m1.8.8.1.1.3.3.3" xref="S2.Ex4.m1.8.8.1.1.3.3.3.cmml"><mtext id="S2.Ex4.m1.8.8.1.1.3.3.3.2.2" xref="S2.Ex4.m1.8.8.1.1.3.3.3.2.2a.cmml">LR</mtext><mi id="S2.Ex4.m1.8.8.1.1.3.3.3.3" xref="S2.Ex4.m1.8.8.1.1.3.3.3.3.cmml">t</mi><mrow id="S2.Ex4.m1.6.6.2.4" xref="S2.Ex4.m1.6.6.2.3.cmml"><mi id="S2.Ex4.m1.5.5.1.1" xref="S2.Ex4.m1.5.5.1.1.cmml">L</mi><mo id="S2.Ex4.m1.6.6.2.4.1" xref="S2.Ex4.m1.6.6.2.3.cmml">,</mo><mi id="S2.Ex4.m1.6.6.2.2" xref="S2.Ex4.m1.6.6.2.2.cmml">H</mi></mrow></msubsup><mo stretchy="false" id="S2.Ex4.m1.8.8.1.1.3.3.8" xref="S2.Ex4.m1.8.8.1.1.3.4.cmml">]</mo></mrow></mrow><mo lspace="0em" id="S2.Ex4.m1.8.8.1.2" xref="S2.Ex4.m1.8.8.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.Ex4.m1.8b"><apply id="S2.Ex4.m1.8.8.1.1.cmml" xref="S2.Ex4.m1.8.8.1"><eq id="S2.Ex4.m1.8.8.1.1.4.cmml" xref="S2.Ex4.m1.8.8.1.1.4"></eq><apply id="S2.Ex4.m1.8.8.1.1.5.cmml" xref="S2.Ex4.m1.8.8.1.1.5"><csymbol cd="ambiguous" id="S2.Ex4.m1.8.8.1.1.5.1.cmml" xref="S2.Ex4.m1.8.8.1.1.5">subscript</csymbol><ci id="S2.Ex4.m1.8.8.1.1.5.2.cmml" xref="S2.Ex4.m1.8.8.1.1.5.2">𝐯</ci><ci id="S2.Ex4.m1.8.8.1.1.5.3.cmml" xref="S2.Ex4.m1.8.8.1.1.5.3">𝑡</ci></apply><list id="S2.Ex4.m1.8.8.1.1.3.4.cmml" xref="S2.Ex4.m1.8.8.1.1.3.3"><apply id="S2.Ex4.m1.8.8.1.1.1.1.1.cmml" xref="S2.Ex4.m1.8.8.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.Ex4.m1.8.8.1.1.1.1.1.1.cmml" xref="S2.Ex4.m1.8.8.1.1.1.1.1">subscript</csymbol><apply id="S2.Ex4.m1.8.8.1.1.1.1.1.2.cmml" xref="S2.Ex4.m1.8.8.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.Ex4.m1.8.8.1.1.1.1.1.2.1.cmml" xref="S2.Ex4.m1.8.8.1.1.1.1.1">superscript</csymbol><ci id="S2.Ex4.m1.8.8.1.1.1.1.1.2.2a.cmml" xref="S2.Ex4.m1.8.8.1.1.1.1.1.2.2"><mtext id="S2.Ex4.m1.8.8.1.1.1.1.1.2.2.cmml" xref="S2.Ex4.m1.8.8.1.1.1.1.1.2.2">LR</mtext></ci><list id="S2.Ex4.m1.2.2.2.3.cmml" xref="S2.Ex4.m1.2.2.2.4"><cn type="integer" id="S2.Ex4.m1.1.1.1.1.cmml" xref="S2.Ex4.m1.1.1.1.1">1</cn><cn type="integer" id="S2.Ex4.m1.2.2.2.2.cmml" xref="S2.Ex4.m1.2.2.2.2">1</cn></list></apply><ci id="S2.Ex4.m1.8.8.1.1.1.1.1.3.cmml" xref="S2.Ex4.m1.8.8.1.1.1.1.1.3">𝑡</ci></apply><apply id="S2.Ex4.m1.8.8.1.1.2.2.2.cmml" xref="S2.Ex4.m1.8.8.1.1.2.2.2"><csymbol cd="ambiguous" id="S2.Ex4.m1.8.8.1.1.2.2.2.1.cmml" xref="S2.Ex4.m1.8.8.1.1.2.2.2">subscript</csymbol><apply id="S2.Ex4.m1.8.8.1.1.2.2.2.2.cmml" xref="S2.Ex4.m1.8.8.1.1.2.2.2"><csymbol cd="ambiguous" id="S2.Ex4.m1.8.8.1.1.2.2.2.2.1.cmml" xref="S2.Ex4.m1.8.8.1.1.2.2.2">superscript</csymbol><ci id="S2.Ex4.m1.8.8.1.1.2.2.2.2.2a.cmml" xref="S2.Ex4.m1.8.8.1.1.2.2.2.2.2"><mtext id="S2.Ex4.m1.8.8.1.1.2.2.2.2.2.cmml" xref="S2.Ex4.m1.8.8.1.1.2.2.2.2.2">LR</mtext></ci><list id="S2.Ex4.m1.4.4.2.3.cmml" xref="S2.Ex4.m1.4.4.2.4"><cn type="integer" id="S2.Ex4.m1.3.3.1.1.cmml" xref="S2.Ex4.m1.3.3.1.1">1</cn><cn type="integer" id="S2.Ex4.m1.4.4.2.2.cmml" xref="S2.Ex4.m1.4.4.2.2">2</cn></list></apply><ci id="S2.Ex4.m1.8.8.1.1.2.2.2.3.cmml" xref="S2.Ex4.m1.8.8.1.1.2.2.2.3">𝑡</ci></apply><ci id="S2.Ex4.m1.7.7.cmml" xref="S2.Ex4.m1.7.7">…</ci><apply id="S2.Ex4.m1.8.8.1.1.3.3.3.cmml" xref="S2.Ex4.m1.8.8.1.1.3.3.3"><csymbol cd="ambiguous" id="S2.Ex4.m1.8.8.1.1.3.3.3.1.cmml" xref="S2.Ex4.m1.8.8.1.1.3.3.3">subscript</csymbol><apply id="S2.Ex4.m1.8.8.1.1.3.3.3.2.cmml" xref="S2.Ex4.m1.8.8.1.1.3.3.3"><csymbol cd="ambiguous" id="S2.Ex4.m1.8.8.1.1.3.3.3.2.1.cmml" xref="S2.Ex4.m1.8.8.1.1.3.3.3">superscript</csymbol><ci id="S2.Ex4.m1.8.8.1.1.3.3.3.2.2a.cmml" xref="S2.Ex4.m1.8.8.1.1.3.3.3.2.2"><mtext id="S2.Ex4.m1.8.8.1.1.3.3.3.2.2.cmml" xref="S2.Ex4.m1.8.8.1.1.3.3.3.2.2">LR</mtext></ci><list id="S2.Ex4.m1.6.6.2.3.cmml" xref="S2.Ex4.m1.6.6.2.4"><ci id="S2.Ex4.m1.5.5.1.1.cmml" xref="S2.Ex4.m1.5.5.1.1">𝐿</ci><ci id="S2.Ex4.m1.6.6.2.2.cmml" xref="S2.Ex4.m1.6.6.2.2">𝐻</ci></list></apply><ci id="S2.Ex4.m1.8.8.1.1.3.3.3.3.cmml" xref="S2.Ex4.m1.8.8.1.1.3.3.3.3">𝑡</ci></apply></list></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Ex4.m1.8c">\mathbf{v}_{t}=[\text{LR}^{1,1}_{t},\text{LR}^{1,2}_{t},\dots,\text{LR}^{L,H}_{t}].</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S2.SS1.p2.5" class="ltx_p">Given a text span of interest <math id="S2.SS1.p2.2.m1.4" class="ltx_Math" alttext="\{y_{t},y_{t+1},...,y_{t+T-1}\}" display="inline"><semantics id="S2.SS1.p2.2.m1.4a"><mrow id="S2.SS1.p2.2.m1.4.4.3" xref="S2.SS1.p2.2.m1.4.4.4.cmml"><mo stretchy="false" id="S2.SS1.p2.2.m1.4.4.3.4" xref="S2.SS1.p2.2.m1.4.4.4.cmml">{</mo><msub id="S2.SS1.p2.2.m1.2.2.1.1" xref="S2.SS1.p2.2.m1.2.2.1.1.cmml"><mi id="S2.SS1.p2.2.m1.2.2.1.1.2" xref="S2.SS1.p2.2.m1.2.2.1.1.2.cmml">y</mi><mi id="S2.SS1.p2.2.m1.2.2.1.1.3" xref="S2.SS1.p2.2.m1.2.2.1.1.3.cmml">t</mi></msub><mo id="S2.SS1.p2.2.m1.4.4.3.5" xref="S2.SS1.p2.2.m1.4.4.4.cmml">,</mo><msub id="S2.SS1.p2.2.m1.3.3.2.2" xref="S2.SS1.p2.2.m1.3.3.2.2.cmml"><mi id="S2.SS1.p2.2.m1.3.3.2.2.2" xref="S2.SS1.p2.2.m1.3.3.2.2.2.cmml">y</mi><mrow id="S2.SS1.p2.2.m1.3.3.2.2.3" xref="S2.SS1.p2.2.m1.3.3.2.2.3.cmml"><mi id="S2.SS1.p2.2.m1.3.3.2.2.3.2" xref="S2.SS1.p2.2.m1.3.3.2.2.3.2.cmml">t</mi><mo id="S2.SS1.p2.2.m1.3.3.2.2.3.1" xref="S2.SS1.p2.2.m1.3.3.2.2.3.1.cmml">+</mo><mn id="S2.SS1.p2.2.m1.3.3.2.2.3.3" xref="S2.SS1.p2.2.m1.3.3.2.2.3.3.cmml">1</mn></mrow></msub><mo id="S2.SS1.p2.2.m1.4.4.3.6" xref="S2.SS1.p2.2.m1.4.4.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS1.p2.2.m1.1.1" xref="S2.SS1.p2.2.m1.1.1.cmml">…</mi><mo id="S2.SS1.p2.2.m1.4.4.3.7" xref="S2.SS1.p2.2.m1.4.4.4.cmml">,</mo><msub id="S2.SS1.p2.2.m1.4.4.3.3" xref="S2.SS1.p2.2.m1.4.4.3.3.cmml"><mi id="S2.SS1.p2.2.m1.4.4.3.3.2" xref="S2.SS1.p2.2.m1.4.4.3.3.2.cmml">y</mi><mrow id="S2.SS1.p2.2.m1.4.4.3.3.3" xref="S2.SS1.p2.2.m1.4.4.3.3.3.cmml"><mrow id="S2.SS1.p2.2.m1.4.4.3.3.3.2" xref="S2.SS1.p2.2.m1.4.4.3.3.3.2.cmml"><mi id="S2.SS1.p2.2.m1.4.4.3.3.3.2.2" xref="S2.SS1.p2.2.m1.4.4.3.3.3.2.2.cmml">t</mi><mo id="S2.SS1.p2.2.m1.4.4.3.3.3.2.1" xref="S2.SS1.p2.2.m1.4.4.3.3.3.2.1.cmml">+</mo><mi id="S2.SS1.p2.2.m1.4.4.3.3.3.2.3" xref="S2.SS1.p2.2.m1.4.4.3.3.3.2.3.cmml">T</mi></mrow><mo id="S2.SS1.p2.2.m1.4.4.3.3.3.1" xref="S2.SS1.p2.2.m1.4.4.3.3.3.1.cmml">−</mo><mn id="S2.SS1.p2.2.m1.4.4.3.3.3.3" xref="S2.SS1.p2.2.m1.4.4.3.3.3.3.cmml">1</mn></mrow></msub><mo stretchy="false" id="S2.SS1.p2.2.m1.4.4.3.8" xref="S2.SS1.p2.2.m1.4.4.4.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.2.m1.4b"><set id="S2.SS1.p2.2.m1.4.4.4.cmml" xref="S2.SS1.p2.2.m1.4.4.3"><apply id="S2.SS1.p2.2.m1.2.2.1.1.cmml" xref="S2.SS1.p2.2.m1.2.2.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.2.m1.2.2.1.1.1.cmml" xref="S2.SS1.p2.2.m1.2.2.1.1">subscript</csymbol><ci id="S2.SS1.p2.2.m1.2.2.1.1.2.cmml" xref="S2.SS1.p2.2.m1.2.2.1.1.2">𝑦</ci><ci id="S2.SS1.p2.2.m1.2.2.1.1.3.cmml" xref="S2.SS1.p2.2.m1.2.2.1.1.3">𝑡</ci></apply><apply id="S2.SS1.p2.2.m1.3.3.2.2.cmml" xref="S2.SS1.p2.2.m1.3.3.2.2"><csymbol cd="ambiguous" id="S2.SS1.p2.2.m1.3.3.2.2.1.cmml" xref="S2.SS1.p2.2.m1.3.3.2.2">subscript</csymbol><ci id="S2.SS1.p2.2.m1.3.3.2.2.2.cmml" xref="S2.SS1.p2.2.m1.3.3.2.2.2">𝑦</ci><apply id="S2.SS1.p2.2.m1.3.3.2.2.3.cmml" xref="S2.SS1.p2.2.m1.3.3.2.2.3"><plus id="S2.SS1.p2.2.m1.3.3.2.2.3.1.cmml" xref="S2.SS1.p2.2.m1.3.3.2.2.3.1"></plus><ci id="S2.SS1.p2.2.m1.3.3.2.2.3.2.cmml" xref="S2.SS1.p2.2.m1.3.3.2.2.3.2">𝑡</ci><cn type="integer" id="S2.SS1.p2.2.m1.3.3.2.2.3.3.cmml" xref="S2.SS1.p2.2.m1.3.3.2.2.3.3">1</cn></apply></apply><ci id="S2.SS1.p2.2.m1.1.1.cmml" xref="S2.SS1.p2.2.m1.1.1">…</ci><apply id="S2.SS1.p2.2.m1.4.4.3.3.cmml" xref="S2.SS1.p2.2.m1.4.4.3.3"><csymbol cd="ambiguous" id="S2.SS1.p2.2.m1.4.4.3.3.1.cmml" xref="S2.SS1.p2.2.m1.4.4.3.3">subscript</csymbol><ci id="S2.SS1.p2.2.m1.4.4.3.3.2.cmml" xref="S2.SS1.p2.2.m1.4.4.3.3.2">𝑦</ci><apply id="S2.SS1.p2.2.m1.4.4.3.3.3.cmml" xref="S2.SS1.p2.2.m1.4.4.3.3.3"><minus id="S2.SS1.p2.2.m1.4.4.3.3.3.1.cmml" xref="S2.SS1.p2.2.m1.4.4.3.3.3.1"></minus><apply id="S2.SS1.p2.2.m1.4.4.3.3.3.2.cmml" xref="S2.SS1.p2.2.m1.4.4.3.3.3.2"><plus id="S2.SS1.p2.2.m1.4.4.3.3.3.2.1.cmml" xref="S2.SS1.p2.2.m1.4.4.3.3.3.2.1"></plus><ci id="S2.SS1.p2.2.m1.4.4.3.3.3.2.2.cmml" xref="S2.SS1.p2.2.m1.4.4.3.3.3.2.2">𝑡</ci><ci id="S2.SS1.p2.2.m1.4.4.3.3.3.2.3.cmml" xref="S2.SS1.p2.2.m1.4.4.3.3.3.2.3">𝑇</ci></apply><cn type="integer" id="S2.SS1.p2.2.m1.4.4.3.3.3.3.cmml" xref="S2.SS1.p2.2.m1.4.4.3.3.3.3">1</cn></apply></apply></set></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.2.m1.4c">\{y_{t},y_{t+1},...,y_{t+T-1}\}</annotation></semantics></math>, we average the corresponding lookback ratio vectors <math id="S2.SS1.p2.3.m2.4" class="ltx_Math" alttext="\{\mathbf{v}_{t},\mathbf{v}_{t+1},...,\mathbf{v}_{t+T-1}\}" display="inline"><semantics id="S2.SS1.p2.3.m2.4a"><mrow id="S2.SS1.p2.3.m2.4.4.3" xref="S2.SS1.p2.3.m2.4.4.4.cmml"><mo stretchy="false" id="S2.SS1.p2.3.m2.4.4.3.4" xref="S2.SS1.p2.3.m2.4.4.4.cmml">{</mo><msub id="S2.SS1.p2.3.m2.2.2.1.1" xref="S2.SS1.p2.3.m2.2.2.1.1.cmml"><mi id="S2.SS1.p2.3.m2.2.2.1.1.2" xref="S2.SS1.p2.3.m2.2.2.1.1.2.cmml">𝐯</mi><mi id="S2.SS1.p2.3.m2.2.2.1.1.3" xref="S2.SS1.p2.3.m2.2.2.1.1.3.cmml">t</mi></msub><mo id="S2.SS1.p2.3.m2.4.4.3.5" xref="S2.SS1.p2.3.m2.4.4.4.cmml">,</mo><msub id="S2.SS1.p2.3.m2.3.3.2.2" xref="S2.SS1.p2.3.m2.3.3.2.2.cmml"><mi id="S2.SS1.p2.3.m2.3.3.2.2.2" xref="S2.SS1.p2.3.m2.3.3.2.2.2.cmml">𝐯</mi><mrow id="S2.SS1.p2.3.m2.3.3.2.2.3" xref="S2.SS1.p2.3.m2.3.3.2.2.3.cmml"><mi id="S2.SS1.p2.3.m2.3.3.2.2.3.2" xref="S2.SS1.p2.3.m2.3.3.2.2.3.2.cmml">t</mi><mo id="S2.SS1.p2.3.m2.3.3.2.2.3.1" xref="S2.SS1.p2.3.m2.3.3.2.2.3.1.cmml">+</mo><mn id="S2.SS1.p2.3.m2.3.3.2.2.3.3" xref="S2.SS1.p2.3.m2.3.3.2.2.3.3.cmml">1</mn></mrow></msub><mo id="S2.SS1.p2.3.m2.4.4.3.6" xref="S2.SS1.p2.3.m2.4.4.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS1.p2.3.m2.1.1" xref="S2.SS1.p2.3.m2.1.1.cmml">…</mi><mo id="S2.SS1.p2.3.m2.4.4.3.7" xref="S2.SS1.p2.3.m2.4.4.4.cmml">,</mo><msub id="S2.SS1.p2.3.m2.4.4.3.3" xref="S2.SS1.p2.3.m2.4.4.3.3.cmml"><mi id="S2.SS1.p2.3.m2.4.4.3.3.2" xref="S2.SS1.p2.3.m2.4.4.3.3.2.cmml">𝐯</mi><mrow id="S2.SS1.p2.3.m2.4.4.3.3.3" xref="S2.SS1.p2.3.m2.4.4.3.3.3.cmml"><mrow id="S2.SS1.p2.3.m2.4.4.3.3.3.2" xref="S2.SS1.p2.3.m2.4.4.3.3.3.2.cmml"><mi id="S2.SS1.p2.3.m2.4.4.3.3.3.2.2" xref="S2.SS1.p2.3.m2.4.4.3.3.3.2.2.cmml">t</mi><mo id="S2.SS1.p2.3.m2.4.4.3.3.3.2.1" xref="S2.SS1.p2.3.m2.4.4.3.3.3.2.1.cmml">+</mo><mi id="S2.SS1.p2.3.m2.4.4.3.3.3.2.3" xref="S2.SS1.p2.3.m2.4.4.3.3.3.2.3.cmml">T</mi></mrow><mo id="S2.SS1.p2.3.m2.4.4.3.3.3.1" xref="S2.SS1.p2.3.m2.4.4.3.3.3.1.cmml">−</mo><mn id="S2.SS1.p2.3.m2.4.4.3.3.3.3" xref="S2.SS1.p2.3.m2.4.4.3.3.3.3.cmml">1</mn></mrow></msub><mo stretchy="false" id="S2.SS1.p2.3.m2.4.4.3.8" xref="S2.SS1.p2.3.m2.4.4.4.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.3.m2.4b"><set id="S2.SS1.p2.3.m2.4.4.4.cmml" xref="S2.SS1.p2.3.m2.4.4.3"><apply id="S2.SS1.p2.3.m2.2.2.1.1.cmml" xref="S2.SS1.p2.3.m2.2.2.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.3.m2.2.2.1.1.1.cmml" xref="S2.SS1.p2.3.m2.2.2.1.1">subscript</csymbol><ci id="S2.SS1.p2.3.m2.2.2.1.1.2.cmml" xref="S2.SS1.p2.3.m2.2.2.1.1.2">𝐯</ci><ci id="S2.SS1.p2.3.m2.2.2.1.1.3.cmml" xref="S2.SS1.p2.3.m2.2.2.1.1.3">𝑡</ci></apply><apply id="S2.SS1.p2.3.m2.3.3.2.2.cmml" xref="S2.SS1.p2.3.m2.3.3.2.2"><csymbol cd="ambiguous" id="S2.SS1.p2.3.m2.3.3.2.2.1.cmml" xref="S2.SS1.p2.3.m2.3.3.2.2">subscript</csymbol><ci id="S2.SS1.p2.3.m2.3.3.2.2.2.cmml" xref="S2.SS1.p2.3.m2.3.3.2.2.2">𝐯</ci><apply id="S2.SS1.p2.3.m2.3.3.2.2.3.cmml" xref="S2.SS1.p2.3.m2.3.3.2.2.3"><plus id="S2.SS1.p2.3.m2.3.3.2.2.3.1.cmml" xref="S2.SS1.p2.3.m2.3.3.2.2.3.1"></plus><ci id="S2.SS1.p2.3.m2.3.3.2.2.3.2.cmml" xref="S2.SS1.p2.3.m2.3.3.2.2.3.2">𝑡</ci><cn type="integer" id="S2.SS1.p2.3.m2.3.3.2.2.3.3.cmml" xref="S2.SS1.p2.3.m2.3.3.2.2.3.3">1</cn></apply></apply><ci id="S2.SS1.p2.3.m2.1.1.cmml" xref="S2.SS1.p2.3.m2.1.1">…</ci><apply id="S2.SS1.p2.3.m2.4.4.3.3.cmml" xref="S2.SS1.p2.3.m2.4.4.3.3"><csymbol cd="ambiguous" id="S2.SS1.p2.3.m2.4.4.3.3.1.cmml" xref="S2.SS1.p2.3.m2.4.4.3.3">subscript</csymbol><ci id="S2.SS1.p2.3.m2.4.4.3.3.2.cmml" xref="S2.SS1.p2.3.m2.4.4.3.3.2">𝐯</ci><apply id="S2.SS1.p2.3.m2.4.4.3.3.3.cmml" xref="S2.SS1.p2.3.m2.4.4.3.3.3"><minus id="S2.SS1.p2.3.m2.4.4.3.3.3.1.cmml" xref="S2.SS1.p2.3.m2.4.4.3.3.3.1"></minus><apply id="S2.SS1.p2.3.m2.4.4.3.3.3.2.cmml" xref="S2.SS1.p2.3.m2.4.4.3.3.3.2"><plus id="S2.SS1.p2.3.m2.4.4.3.3.3.2.1.cmml" xref="S2.SS1.p2.3.m2.4.4.3.3.3.2.1"></plus><ci id="S2.SS1.p2.3.m2.4.4.3.3.3.2.2.cmml" xref="S2.SS1.p2.3.m2.4.4.3.3.3.2.2">𝑡</ci><ci id="S2.SS1.p2.3.m2.4.4.3.3.3.2.3.cmml" xref="S2.SS1.p2.3.m2.4.4.3.3.3.2.3">𝑇</ci></apply><cn type="integer" id="S2.SS1.p2.3.m2.4.4.3.3.3.3.cmml" xref="S2.SS1.p2.3.m2.4.4.3.3.3.3">1</cn></apply></apply></set></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.3.m2.4c">\{\mathbf{v}_{t},\mathbf{v}_{t+1},...,\mathbf{v}_{t+T-1}\}</annotation></semantics></math> into a single vector <math id="S2.SS1.p2.4.m3.1" class="ltx_Math" alttext="\bar{\mathbf{v}}" display="inline"><semantics id="S2.SS1.p2.4.m3.1a"><mover accent="true" id="S2.SS1.p2.4.m3.1.1" xref="S2.SS1.p2.4.m3.1.1.cmml"><mi id="S2.SS1.p2.4.m3.1.1.2" xref="S2.SS1.p2.4.m3.1.1.2.cmml">𝐯</mi><mo id="S2.SS1.p2.4.m3.1.1.1" xref="S2.SS1.p2.4.m3.1.1.1.cmml">¯</mo></mover><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.4.m3.1b"><apply id="S2.SS1.p2.4.m3.1.1.cmml" xref="S2.SS1.p2.4.m3.1.1"><ci id="S2.SS1.p2.4.m3.1.1.1.cmml" xref="S2.SS1.p2.4.m3.1.1.1">¯</ci><ci id="S2.SS1.p2.4.m3.1.1.2.cmml" xref="S2.SS1.p2.4.m3.1.1.2">𝐯</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.4.m3.1c">\bar{\mathbf{v}}</annotation></semantics></math>.
We then employ a logistic regression classifier <math id="S2.SS1.p2.5.m4.1" class="ltx_Math" alttext="\mathcal{F}" display="inline"><semantics id="S2.SS1.p2.5.m4.1a"><mi class="ltx_font_mathcaligraphic" id="S2.SS1.p2.5.m4.1.1" xref="S2.SS1.p2.5.m4.1.1.cmml">ℱ</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.5.m4.1b"><ci id="S2.SS1.p2.5.m4.1.1.cmml" xref="S2.SS1.p2.5.m4.1.1">ℱ</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.5.m4.1c">\mathcal{F}</annotation></semantics></math> to predict if the span is factual (1) or hallucinated (0) based on the averaged lookback ratio vector.</p>
<table id="S2.Ex5" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.Ex5.m1.2" class="ltx_Math" alttext="P(y=1|\mathbf{\bar{v}})=\mathcal{F}(\mathbf{\bar{v}})=\sigma(\mathbf{w}^{\top}\mathbf{\bar{v}}+b)," display="block"><semantics id="S2.Ex5.m1.2a"><mrow id="S2.Ex5.m1.2.2.1" xref="S2.Ex5.m1.2.2.1.1.cmml"><mrow id="S2.Ex5.m1.2.2.1.1" xref="S2.Ex5.m1.2.2.1.1.cmml"><mrow id="S2.Ex5.m1.2.2.1.1.1" xref="S2.Ex5.m1.2.2.1.1.1.cmml"><mi id="S2.Ex5.m1.2.2.1.1.1.3" xref="S2.Ex5.m1.2.2.1.1.1.3.cmml">P</mi><mo lspace="0em" rspace="0em" id="S2.Ex5.m1.2.2.1.1.1.2" xref="S2.Ex5.m1.2.2.1.1.1.2.cmml">​</mo><mrow id="S2.Ex5.m1.2.2.1.1.1.1.1" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S2.Ex5.m1.2.2.1.1.1.1.1.2" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.cmml">(</mo><mrow id="S2.Ex5.m1.2.2.1.1.1.1.1.1" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.cmml"><mi id="S2.Ex5.m1.2.2.1.1.1.1.1.1.2" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.2.cmml">y</mi><mo id="S2.Ex5.m1.2.2.1.1.1.1.1.1.1" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.1.cmml">=</mo><mrow id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.cmml"><mn id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.2" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.2.cmml">1</mn><mo fence="false" id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.1" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.1.cmml">|</mo><mover accent="true" id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3.cmml"><mi id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3.2" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3.2.cmml">𝐯</mi><mo id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3.1" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3.1.cmml">¯</mo></mover></mrow></mrow><mo stretchy="false" id="S2.Ex5.m1.2.2.1.1.1.1.1.3" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.cmml">)</mo></mrow></mrow><mo id="S2.Ex5.m1.2.2.1.1.4" xref="S2.Ex5.m1.2.2.1.1.4.cmml">=</mo><mrow id="S2.Ex5.m1.2.2.1.1.5" xref="S2.Ex5.m1.2.2.1.1.5.cmml"><mi class="ltx_font_mathcaligraphic" id="S2.Ex5.m1.2.2.1.1.5.2" xref="S2.Ex5.m1.2.2.1.1.5.2.cmml">ℱ</mi><mo lspace="0em" rspace="0em" id="S2.Ex5.m1.2.2.1.1.5.1" xref="S2.Ex5.m1.2.2.1.1.5.1.cmml">​</mo><mrow id="S2.Ex5.m1.2.2.1.1.5.3.2" xref="S2.Ex5.m1.1.1.cmml"><mo stretchy="false" id="S2.Ex5.m1.2.2.1.1.5.3.2.1" xref="S2.Ex5.m1.1.1.cmml">(</mo><mover accent="true" id="S2.Ex5.m1.1.1" xref="S2.Ex5.m1.1.1.cmml"><mi id="S2.Ex5.m1.1.1.2" xref="S2.Ex5.m1.1.1.2.cmml">𝐯</mi><mo id="S2.Ex5.m1.1.1.1" xref="S2.Ex5.m1.1.1.1.cmml">¯</mo></mover><mo stretchy="false" id="S2.Ex5.m1.2.2.1.1.5.3.2.2" xref="S2.Ex5.m1.1.1.cmml">)</mo></mrow></mrow><mo id="S2.Ex5.m1.2.2.1.1.6" xref="S2.Ex5.m1.2.2.1.1.6.cmml">=</mo><mrow id="S2.Ex5.m1.2.2.1.1.2" xref="S2.Ex5.m1.2.2.1.1.2.cmml"><mi id="S2.Ex5.m1.2.2.1.1.2.3" xref="S2.Ex5.m1.2.2.1.1.2.3.cmml">σ</mi><mo lspace="0em" rspace="0em" id="S2.Ex5.m1.2.2.1.1.2.2" xref="S2.Ex5.m1.2.2.1.1.2.2.cmml">​</mo><mrow id="S2.Ex5.m1.2.2.1.1.2.1.1" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.cmml"><mo stretchy="false" id="S2.Ex5.m1.2.2.1.1.2.1.1.2" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.cmml">(</mo><mrow id="S2.Ex5.m1.2.2.1.1.2.1.1.1" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.cmml"><mrow id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.cmml"><msup id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.cmml"><mi id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.2" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.2.cmml">𝐰</mi><mo id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.3" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.3.cmml">⊤</mo></msup><mo lspace="0em" rspace="0em" id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.1" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.1.cmml">​</mo><mover accent="true" id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3.cmml"><mi id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3.2" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3.2.cmml">𝐯</mi><mo id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3.1" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3.1.cmml">¯</mo></mover></mrow><mo id="S2.Ex5.m1.2.2.1.1.2.1.1.1.1" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.1.cmml">+</mo><mi id="S2.Ex5.m1.2.2.1.1.2.1.1.1.3" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.3.cmml">b</mi></mrow><mo stretchy="false" id="S2.Ex5.m1.2.2.1.1.2.1.1.3" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo id="S2.Ex5.m1.2.2.1.2" xref="S2.Ex5.m1.2.2.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.Ex5.m1.2b"><apply id="S2.Ex5.m1.2.2.1.1.cmml" xref="S2.Ex5.m1.2.2.1"><and id="S2.Ex5.m1.2.2.1.1a.cmml" xref="S2.Ex5.m1.2.2.1"></and><apply id="S2.Ex5.m1.2.2.1.1b.cmml" xref="S2.Ex5.m1.2.2.1"><eq id="S2.Ex5.m1.2.2.1.1.4.cmml" xref="S2.Ex5.m1.2.2.1.1.4"></eq><apply id="S2.Ex5.m1.2.2.1.1.1.cmml" xref="S2.Ex5.m1.2.2.1.1.1"><times id="S2.Ex5.m1.2.2.1.1.1.2.cmml" xref="S2.Ex5.m1.2.2.1.1.1.2"></times><ci id="S2.Ex5.m1.2.2.1.1.1.3.cmml" xref="S2.Ex5.m1.2.2.1.1.1.3">𝑃</ci><apply id="S2.Ex5.m1.2.2.1.1.1.1.1.1.cmml" xref="S2.Ex5.m1.2.2.1.1.1.1.1"><eq id="S2.Ex5.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.1"></eq><ci id="S2.Ex5.m1.2.2.1.1.1.1.1.1.2.cmml" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.2">𝑦</ci><apply id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.cmml" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3"><csymbol cd="latexml" id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.1.cmml" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.1">conditional</csymbol><cn type="integer" id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.2.cmml" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.2">1</cn><apply id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3.cmml" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3"><ci id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3.1.cmml" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3.1">¯</ci><ci id="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3.2.cmml" xref="S2.Ex5.m1.2.2.1.1.1.1.1.1.3.3.2">𝐯</ci></apply></apply></apply></apply><apply id="S2.Ex5.m1.2.2.1.1.5.cmml" xref="S2.Ex5.m1.2.2.1.1.5"><times id="S2.Ex5.m1.2.2.1.1.5.1.cmml" xref="S2.Ex5.m1.2.2.1.1.5.1"></times><ci id="S2.Ex5.m1.2.2.1.1.5.2.cmml" xref="S2.Ex5.m1.2.2.1.1.5.2">ℱ</ci><apply id="S2.Ex5.m1.1.1.cmml" xref="S2.Ex5.m1.2.2.1.1.5.3.2"><ci id="S2.Ex5.m1.1.1.1.cmml" xref="S2.Ex5.m1.1.1.1">¯</ci><ci id="S2.Ex5.m1.1.1.2.cmml" xref="S2.Ex5.m1.1.1.2">𝐯</ci></apply></apply></apply><apply id="S2.Ex5.m1.2.2.1.1c.cmml" xref="S2.Ex5.m1.2.2.1"><eq id="S2.Ex5.m1.2.2.1.1.6.cmml" xref="S2.Ex5.m1.2.2.1.1.6"></eq><share href="#S2.Ex5.m1.2.2.1.1.5.cmml" id="S2.Ex5.m1.2.2.1.1d.cmml" xref="S2.Ex5.m1.2.2.1"></share><apply id="S2.Ex5.m1.2.2.1.1.2.cmml" xref="S2.Ex5.m1.2.2.1.1.2"><times id="S2.Ex5.m1.2.2.1.1.2.2.cmml" xref="S2.Ex5.m1.2.2.1.1.2.2"></times><ci id="S2.Ex5.m1.2.2.1.1.2.3.cmml" xref="S2.Ex5.m1.2.2.1.1.2.3">𝜎</ci><apply id="S2.Ex5.m1.2.2.1.1.2.1.1.1.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1"><plus id="S2.Ex5.m1.2.2.1.1.2.1.1.1.1.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.1"></plus><apply id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2"><times id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.1.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.1"></times><apply id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2"><csymbol cd="ambiguous" id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.1.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2">superscript</csymbol><ci id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.2.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.2">𝐰</ci><csymbol cd="latexml" id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.3.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.2.3">top</csymbol></apply><apply id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3"><ci id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3.1.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3.1">¯</ci><ci id="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3.2.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.2.3.2">𝐯</ci></apply></apply><ci id="S2.Ex5.m1.2.2.1.1.2.1.1.1.3.cmml" xref="S2.Ex5.m1.2.2.1.1.2.1.1.1.3">𝑏</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Ex5.m1.2c">P(y=1|\mathbf{\bar{v}})=\mathcal{F}(\mathbf{\bar{v}})=\sigma(\mathbf{w}^{\top}\mathbf{\bar{v}}+b),</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S2.SS1.p2.8" class="ltx_p">where <math id="S2.SS1.p2.6.m1.1" class="ltx_Math" alttext="\sigma" display="inline"><semantics id="S2.SS1.p2.6.m1.1a"><mi id="S2.SS1.p2.6.m1.1.1" xref="S2.SS1.p2.6.m1.1.1.cmml">σ</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.6.m1.1b"><ci id="S2.SS1.p2.6.m1.1.1.cmml" xref="S2.SS1.p2.6.m1.1.1">𝜎</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.6.m1.1c">\sigma</annotation></semantics></math> denotes the sigmoid function, <math id="S2.SS1.p2.7.m2.1" class="ltx_Math" alttext="\mathbf{w}" display="inline"><semantics id="S2.SS1.p2.7.m2.1a"><mi id="S2.SS1.p2.7.m2.1.1" xref="S2.SS1.p2.7.m2.1.1.cmml">𝐰</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.7.m2.1b"><ci id="S2.SS1.p2.7.m2.1.1.cmml" xref="S2.SS1.p2.7.m2.1.1">𝐰</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.7.m2.1c">\mathbf{w}</annotation></semantics></math> is the weight vector, and <math id="S2.SS1.p2.8.m3.1" class="ltx_Math" alttext="b" display="inline"><semantics id="S2.SS1.p2.8.m3.1a"><mi id="S2.SS1.p2.8.m3.1.1" xref="S2.SS1.p2.8.m3.1.1.cmml">b</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.8.m3.1b"><ci id="S2.SS1.p2.8.m3.1.1.cmml" xref="S2.SS1.p2.8.m3.1.1">𝑏</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.8.m3.1c">b</annotation></semantics></math> is the bias term of the classifier.</p>
</div>
<section id="S2.SS1.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Defining Span</h4>

<div id="S2.SS1.SSS0.Px1.p1" class="ltx_para">
<p id="S2.SS1.SSS0.Px1.p1.1" class="ltx_p">The <span id="S2.SS1.SSS0.Px1.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> predicts the probability of hallucinations over spans. We consider two ways to obtain spans for a given sequence: <em id="S2.SS1.SSS0.Px1.p1.1.2" class="ltx_emph ltx_font_italic">predefined spans</em> or <em id="S2.SS1.SSS0.Px1.p1.1.3" class="ltx_emph ltx_font_italic">sliding window</em>.</p>
</div>
<div id="S2.SS1.SSS0.Px1.p2" class="ltx_para">
<p id="S2.SS1.SSS0.Px1.p2.1" class="ltx_p"><span id="S2.SS1.SSS0.Px1.p2.1.1" class="ltx_text ltx_font_bold">1) Predefined Spans:</span>
When the hallucinated and non-hallucinated span annotations are available, we directly train the classifier to differentiate between them. This is a clean setting where all spans are either hallucinated or non-hallucinated.</p>
</div>
<div id="S2.SS1.SSS0.Px1.p3" class="ltx_para">
<p id="S2.SS1.SSS0.Px1.p3.1" class="ltx_p"><span id="S2.SS1.SSS0.Px1.p3.1.1" class="ltx_text ltx_font_bold">2) Sliding Window:</span>
In practice, we do not have any predefined spans during decoding, thus we need a sliding window setup that iterates over all possible spans.
Specifically, we process the sentences into fixed-sized chunks and train the classifier to predict a label of 0 if any hallucinated content exists within a chunk, and 1 otherwise. Here, the annotated data is only used for creating labels, not for the span segmentation. This is more realistic for classifier-guided decoding, but it presents greater challenges because a chunk can contain both hallucinated and non-hallucinated content.</p>
</div>
</section>
</section>
<section id="S2.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.2 </span>Experimental Setup</h3>

<section id="S2.SS2.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Data</h4>

<div id="S2.SS2.SSS0.Px1.p1" class="ltx_para">
<p id="S2.SS2.SSS0.Px1.p1.1" class="ltx_p">Training the <span id="S2.SS2.SSS0.Px1.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> requires labels for hallucinated and non-hallucinated examples. To obtain these examples, we first prompt LLaMA-2-7B-Chat <cite class="ltx_cite ltx_citemacro_citep">(Touvron et al., <a href="#bib.bib40" title="" class="ltx_ref">2023</a>)</cite> to greedy decode responses for 1,000 summarization examples from the CNN/DM dataset <cite class="ltx_cite ltx_citemacro_citep">(See et al., <a href="#bib.bib36" title="" class="ltx_ref">2017</a>)</cite> and 2,655 QA examples from the Natural Questions <cite class="ltx_cite ltx_citemacro_citep">(Kwiatkowski et al., <a href="#bib.bib18" title="" class="ltx_ref">2019</a>)</cite> following the setup of <cite class="ltx_cite ltx_citemacro_citet">Liu et al. (<a href="#bib.bib24" title="" class="ltx_ref">2024</a>)</cite>. More details are shown in Appendix <a href="#A1" title="Appendix A Data Creation for Lookback Lens ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">A</span></a>.
Although being prompted to generate correct responses, the decoded responses will contain both hallucinated and non-hallucinated information as the LLaMA model is still not perfect.
Then, we employed GPT-4o <cite class="ltx_cite ltx_citemacro_citep">(OpenAI, <a href="#bib.bib34" title="" class="ltx_ref">2024</a>)</cite> to verify the truthfulness of these responses and provide span-level annotations on hallucinated segments (detailed prompts in Appendix <a href="#A2" title="Appendix B Evaluation Prompt for GPT-4o ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">B</span></a>). Additionally, we performed a pilot study of human annotation on a subset of 70 examples of the summarization task (details in Appendix <a href="#A3" title="Appendix C Pilot Study of Human Evaluation on GPT-4o Evaluation ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">C</span></a>), confirming a 97% consistency rate between GPT-4o annotations and human judgments, and validating the reliability of the automated annotations.
We show LLaMA-2-7B-Chat’s results on both tasks, as evaluated by GPT-4o, in Table <a href="#S2.T1" title="Table 1 ‣ Data ‣ 2.2 Experimental Setup ‣ 2 Contextual Hallucinations Detection ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>.
The results show that the generated summaries from LLaMA-2-7B-Chat still exhibit hallucinations about half of the time, highlighting the challenge of summarization tasks.</p>
</div>
<figure id="S2.T1" class="ltx_table">
<table id="S2.T1.1" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S2.T1.1.1.1" class="ltx_tr">
<th id="S2.T1.1.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt"><span id="S2.T1.1.1.1.1.1" class="ltx_text ltx_font_bold">Dataset</span></th>
<th id="S2.T1.1.1.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="S2.T1.1.1.1.2.1" class="ltx_text ltx_font_bold">Examples</span></th>
<th id="S2.T1.1.1.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="S2.T1.1.1.1.3.1" class="ltx_text ltx_font_bold">Correct</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S2.T1.1.2.1" class="ltx_tr">
<th id="S2.T1.1.2.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">CNN/DM</th>
<td id="S2.T1.1.2.1.2" class="ltx_td ltx_align_center ltx_border_t">1000</td>
<td id="S2.T1.1.2.1.3" class="ltx_td ltx_align_center ltx_border_t">49.6%</td>
</tr>
<tr id="S2.T1.1.3.2" class="ltx_tr">
<th id="S2.T1.1.3.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb">NQ</th>
<td id="S2.T1.1.3.2.2" class="ltx_td ltx_align_center ltx_border_bb">2655</td>
<td id="S2.T1.1.3.2.3" class="ltx_td ltx_align_center ltx_border_bb">67.8%</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 1: </span>Dataset statistics and GPT-4o evaluation results on responses greedy decoded by LLaMA-2-7B-chat.</figcaption>
</figure>
</section>
<section id="S2.SS2.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Baselines</h4>

<div id="S2.SS2.SSS0.Px2.p1" class="ltx_para">
<p id="S2.SS2.SSS0.Px2.p1.1" class="ltx_p">We compare our detection method against several baselines:
<span id="S2.SS2.SSS0.Px2.p1.1.1" class="ltx_text ltx_font_bold">1) Text-based entailment classifier:</span> We fine-tune the DeBERTa-v3-base <cite class="ltx_cite ltx_citemacro_citep">(He et al., <a href="#bib.bib14" title="" class="ltx_ref">2021</a>)</cite> model on the same dataset of CNN/DM and NQ as a natural language entailment (NLI) task. Additionally, we include the results from a state-of-the-art entailment model <cite class="ltx_cite ltx_citemacro_citep">(Vectara, <a href="#bib.bib43" title="" class="ltx_ref">2023</a>)</cite> trained on a huge amount of annotated NLI data (see details in Appendix <a href="#A5" title="Appendix E Model Details ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">E</span></a>.
<span id="S2.SS2.SSS0.Px2.p1.1.2" class="ltx_text ltx_font_bold">2) Hidden states-based classifier:</span> We train classifiers using the same setting as the <span id="S2.SS2.SSS0.Px2.p1.1.3" class="ltx_text ltx_font_italic">Lookback Lens</span> but used input features from the hidden states of LLaMA-2-7B-Chat from its 24th, 28th, and 32nd layers instead of the lookback ratio. This baseline resembles a broad range of existing methods in the literature <cite class="ltx_cite ltx_citemacro_citep">(Azaria and Mitchell, <a href="#bib.bib1" title="" class="ltx_ref">2023</a>; Simhi et al., <a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite>.</p>
</div>
<figure id="S2.T2" class="ltx_table">
<table id="S2.T2.2" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S2.T2.2.3.1" class="ltx_tr">
<th id="S2.T2.2.3.1.1" class="ltx_td ltx_th ltx_th_row ltx_border_tt"></th>
<td id="S2.T2.2.3.1.2" class="ltx_td ltx_border_tt"></td>
<td id="S2.T2.2.3.1.3" class="ltx_td ltx_border_tt"></td>
<td id="S2.T2.2.3.1.4" class="ltx_td ltx_align_center ltx_border_tt" colspan="3"><span id="S2.T2.2.3.1.4.1" class="ltx_text ltx_font_bold">Predefined Span</span></td>
<td id="S2.T2.2.3.1.5" class="ltx_td ltx_align_center ltx_border_tt" colspan="3"><span id="S2.T2.2.3.1.5.1" class="ltx_text ltx_font_bold">Sliding Window = 8</span></td>
</tr>
<tr id="S2.T2.2.2" class="ltx_tr">
<th id="S2.T2.2.2.3" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" rowspan="2"><span id="S2.T2.2.2.3.1" class="ltx_text ltx_font_bold">Method</span></th>
<td id="S2.T2.2.2.4" class="ltx_td ltx_align_center ltx_border_t" rowspan="2"><span id="S2.T2.2.2.4.1" class="ltx_text ltx_font_bold">Source</span></td>
<td id="S2.T2.2.2.5" class="ltx_td ltx_align_center ltx_border_t" rowspan="2"><span id="S2.T2.2.2.5.1" class="ltx_text ltx_font_bold">Target</span></td>
<td id="S2.T2.1.1.1" class="ltx_td ltx_align_center ltx_border_t" colspan="3"><span id="S2.T2.1.1.1.1" class="ltx_text ltx_font_bold">Source <math id="S2.T2.1.1.1.1.m1.1" class="ltx_Math" alttext="\xrightarrow{\hskip 17.07182pt}" display="inline"><semantics id="S2.T2.1.1.1.1.m1.1a"><mover accent="true" id="S2.T2.1.1.1.1.m1.1.1" xref="S2.T2.1.1.1.1.m1.1.1.cmml"><mo stretchy="false" id="S2.T2.1.1.1.1.m1.1.1.2" xref="S2.T2.1.1.1.1.m1.1.1.2.cmml">→</mo><mspace width="1.71em" id="S2.T2.1.1.1.1.m1.1.1.1" xref="S2.T2.1.1.1.1.m1.1.1.cmml"></mspace></mover><annotation-xml encoding="MathML-Content" id="S2.T2.1.1.1.1.m1.1b"><apply id="S2.T2.1.1.1.1.m1.1.1.cmml" xref="S2.T2.1.1.1.1.m1.1.1"><ci id="S2.T2.1.1.1.1.m1.1.1.2.cmml" xref="S2.T2.1.1.1.1.m1.1.1.2">→</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.T2.1.1.1.1.m1.1c">\xrightarrow{\hskip 17.07182pt}</annotation></semantics></math> Target</span></td>
<td id="S2.T2.2.2.2" class="ltx_td ltx_align_center ltx_border_t" colspan="3"><span id="S2.T2.2.2.2.1" class="ltx_text ltx_font_bold">Source <math id="S2.T2.2.2.2.1.m1.1" class="ltx_Math" alttext="\xrightarrow{\hskip 17.07182pt}" display="inline"><semantics id="S2.T2.2.2.2.1.m1.1a"><mover accent="true" id="S2.T2.2.2.2.1.m1.1.1" xref="S2.T2.2.2.2.1.m1.1.1.cmml"><mo stretchy="false" id="S2.T2.2.2.2.1.m1.1.1.2" xref="S2.T2.2.2.2.1.m1.1.1.2.cmml">→</mo><mspace width="1.71em" id="S2.T2.2.2.2.1.m1.1.1.1" xref="S2.T2.2.2.2.1.m1.1.1.cmml"></mspace></mover><annotation-xml encoding="MathML-Content" id="S2.T2.2.2.2.1.m1.1b"><apply id="S2.T2.2.2.2.1.m1.1.1.cmml" xref="S2.T2.2.2.2.1.m1.1.1"><ci id="S2.T2.2.2.2.1.m1.1.1.2.cmml" xref="S2.T2.2.2.2.1.m1.1.1.2">→</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.T2.2.2.2.1.m1.1c">\xrightarrow{\hskip 17.07182pt}</annotation></semantics></math> Target</span></td>
</tr>
<tr id="S2.T2.2.4.2" class="ltx_tr">
<td id="S2.T2.2.4.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="S2.T2.2.4.2.1.1" class="ltx_text ltx_font_bold">Train</span></td>
<td id="S2.T2.2.4.2.2" class="ltx_td ltx_align_center ltx_border_t"><span id="S2.T2.2.4.2.2.1" class="ltx_text ltx_font_bold">Test</span></td>
<td id="S2.T2.2.4.2.3" class="ltx_td ltx_align_center ltx_border_t"><span id="S2.T2.2.4.2.3.1" class="ltx_text ltx_font_bold">Transfer</span></td>
<td id="S2.T2.2.4.2.4" class="ltx_td ltx_align_center ltx_border_t"><span id="S2.T2.2.4.2.4.1" class="ltx_text ltx_font_bold">Train</span></td>
<td id="S2.T2.2.4.2.5" class="ltx_td ltx_align_center ltx_border_t"><span id="S2.T2.2.4.2.5.1" class="ltx_text ltx_font_bold">Test</span></td>
<td id="S2.T2.2.4.2.6" class="ltx_td ltx_align_center ltx_border_t"><span id="S2.T2.2.4.2.6.1" class="ltx_text ltx_font_bold">Transfer</span></td>
</tr>
<tr id="S2.T2.2.5.3" class="ltx_tr">
<th id="S2.T2.2.5.3.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t" colspan="9"><span id="S2.T2.2.5.3.1.1" class="ltx_text ltx_font_italic">Text based NLI</span></th>
</tr>
<tr id="S2.T2.2.6.4" class="ltx_tr">
<th id="S2.T2.2.6.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">SoTA NLI</th>
<td id="S2.T2.2.6.4.2" class="ltx_td ltx_align_center ltx_border_t">–</td>
<td id="S2.T2.2.6.4.3" class="ltx_td ltx_align_center ltx_border_t">Sum.</td>
<td id="S2.T2.2.6.4.4" class="ltx_td ltx_align_center ltx_border_t">–</td>
<td id="S2.T2.2.6.4.5" class="ltx_td ltx_align_center ltx_border_t">–</td>
<td id="S2.T2.2.6.4.6" class="ltx_td ltx_align_center ltx_border_t">76.6</td>
<td id="S2.T2.2.6.4.7" class="ltx_td ltx_align_center ltx_border_t">–</td>
<td id="S2.T2.2.6.4.8" class="ltx_td ltx_align_center ltx_border_t">–</td>
<td id="S2.T2.2.6.4.9" class="ltx_td ltx_align_center ltx_border_t">57.1</td>
</tr>
<tr id="S2.T2.2.7.5" class="ltx_tr">
<th id="S2.T2.2.7.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">SoTA NLI</th>
<td id="S2.T2.2.7.5.2" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.7.5.3" class="ltx_td ltx_align_center">QA</td>
<td id="S2.T2.2.7.5.4" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.7.5.5" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.7.5.6" class="ltx_td ltx_align_center">58.6</td>
<td id="S2.T2.2.7.5.7" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.7.5.8" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.7.5.9" class="ltx_td ltx_align_center">61.8</td>
</tr>
<tr id="S2.T2.2.8.6" class="ltx_tr">
<th id="S2.T2.2.8.6.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">NLI (our impl.)</th>
<td id="S2.T2.2.8.6.2" class="ltx_td ltx_align_center">QA</td>
<td id="S2.T2.2.8.6.3" class="ltx_td ltx_align_center">Sum.</td>
<td id="S2.T2.2.8.6.4" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.8.6.5" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.8.6.6" class="ltx_td ltx_align_center">55.1</td>
<td id="S2.T2.2.8.6.7" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.8.6.8" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.8.6.9" class="ltx_td ltx_align_center">53.0</td>
</tr>
<tr id="S2.T2.2.9.7" class="ltx_tr">
<th id="S2.T2.2.9.7.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">NLI (our impl.)</th>
<td id="S2.T2.2.9.7.2" class="ltx_td ltx_align_center">Sum.</td>
<td id="S2.T2.2.9.7.3" class="ltx_td ltx_align_center">QA</td>
<td id="S2.T2.2.9.7.4" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.9.7.5" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.9.7.6" class="ltx_td ltx_align_center">71.0</td>
<td id="S2.T2.2.9.7.7" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.9.7.8" class="ltx_td ltx_align_center">–</td>
<td id="S2.T2.2.9.7.9" class="ltx_td ltx_align_center">64.9</td>
</tr>
<tr id="S2.T2.2.10.8" class="ltx_tr">
<th id="S2.T2.2.10.8.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t" colspan="9"><span id="S2.T2.2.10.8.1.1" class="ltx_text ltx_font_italic">Hidden states based</span></th>
</tr>
<tr id="S2.T2.2.11.9" class="ltx_tr">
<th id="S2.T2.2.11.9.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">32nd Layer</th>
<td id="S2.T2.2.11.9.2" class="ltx_td ltx_align_center ltx_border_t">QA</td>
<td id="S2.T2.2.11.9.3" class="ltx_td ltx_align_center ltx_border_t">Sum.</td>
<td id="S2.T2.2.11.9.4" class="ltx_td ltx_align_center ltx_border_t">100.0</td>
<td id="S2.T2.2.11.9.5" class="ltx_td ltx_align_center ltx_border_t">89.6</td>
<td id="S2.T2.2.11.9.6" class="ltx_td ltx_align_center ltx_border_t">79.4</td>
<td id="S2.T2.2.11.9.7" class="ltx_td ltx_align_center ltx_border_t">99.0</td>
<td id="S2.T2.2.11.9.8" class="ltx_td ltx_align_center ltx_border_t">97.1</td>
<td id="S2.T2.2.11.9.9" class="ltx_td ltx_align_center ltx_border_t">56.1</td>
</tr>
<tr id="S2.T2.2.12.10" class="ltx_tr">
<th id="S2.T2.2.12.10.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">32nd Layer</th>
<td id="S2.T2.2.12.10.2" class="ltx_td ltx_align_center">Sum.</td>
<td id="S2.T2.2.12.10.3" class="ltx_td ltx_align_center">QA</td>
<td id="S2.T2.2.12.10.4" class="ltx_td ltx_align_center">100.0</td>
<td id="S2.T2.2.12.10.5" class="ltx_td ltx_align_center">82.5</td>
<td id="S2.T2.2.12.10.6" class="ltx_td ltx_align_center">81.8</td>
<td id="S2.T2.2.12.10.7" class="ltx_td ltx_align_center">97.0</td>
<td id="S2.T2.2.12.10.8" class="ltx_td ltx_align_center">94.8</td>
<td id="S2.T2.2.12.10.9" class="ltx_td ltx_align_center">59.4</td>
</tr>
<tr id="S2.T2.2.13.11" class="ltx_tr">
<th id="S2.T2.2.13.11.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">28th Layer</th>
<td id="S2.T2.2.13.11.2" class="ltx_td ltx_align_center">QA</td>
<td id="S2.T2.2.13.11.3" class="ltx_td ltx_align_center">Sum.</td>
<td id="S2.T2.2.13.11.4" class="ltx_td ltx_align_center">100.0</td>
<td id="S2.T2.2.13.11.5" class="ltx_td ltx_align_center">91.4</td>
<td id="S2.T2.2.13.11.6" class="ltx_td ltx_align_center">83.6</td>
<td id="S2.T2.2.13.11.7" class="ltx_td ltx_align_center">99.2</td>
<td id="S2.T2.2.13.11.8" class="ltx_td ltx_align_center">97.3</td>
<td id="S2.T2.2.13.11.9" class="ltx_td ltx_align_center">57.7</td>
</tr>
<tr id="S2.T2.2.14.12" class="ltx_tr">
<th id="S2.T2.2.14.12.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">28th Layer</th>
<td id="S2.T2.2.14.12.2" class="ltx_td ltx_align_center">Sum.</td>
<td id="S2.T2.2.14.12.3" class="ltx_td ltx_align_center">QA</td>
<td id="S2.T2.2.14.12.4" class="ltx_td ltx_align_center">100.0</td>
<td id="S2.T2.2.14.12.5" class="ltx_td ltx_align_center">83.3</td>
<td id="S2.T2.2.14.12.6" class="ltx_td ltx_align_center">84.7</td>
<td id="S2.T2.2.14.12.7" class="ltx_td ltx_align_center">97.2</td>
<td id="S2.T2.2.14.12.8" class="ltx_td ltx_align_center">95.2</td>
<td id="S2.T2.2.14.12.9" class="ltx_td ltx_align_center">58.8</td>
</tr>
<tr id="S2.T2.2.15.13" class="ltx_tr">
<th id="S2.T2.2.15.13.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">24th Layer</th>
<td id="S2.T2.2.15.13.2" class="ltx_td ltx_align_center">QA</td>
<td id="S2.T2.2.15.13.3" class="ltx_td ltx_align_center">Sum.</td>
<td id="S2.T2.2.15.13.4" class="ltx_td ltx_align_center">100.0</td>
<td id="S2.T2.2.15.13.5" class="ltx_td ltx_align_center">92.0</td>
<td id="S2.T2.2.15.13.6" class="ltx_td ltx_align_center">81.3</td>
<td id="S2.T2.2.15.13.7" class="ltx_td ltx_align_center">99.2</td>
<td id="S2.T2.2.15.13.8" class="ltx_td ltx_align_center">97.4</td>
<td id="S2.T2.2.15.13.9" class="ltx_td ltx_align_center">58.3</td>
</tr>
<tr id="S2.T2.2.16.14" class="ltx_tr">
<th id="S2.T2.2.16.14.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">24th Layer</th>
<td id="S2.T2.2.16.14.2" class="ltx_td ltx_align_center">Sum.</td>
<td id="S2.T2.2.16.14.3" class="ltx_td ltx_align_center">QA</td>
<td id="S2.T2.2.16.14.4" class="ltx_td ltx_align_center">100.0</td>
<td id="S2.T2.2.16.14.5" class="ltx_td ltx_align_center">83.1</td>
<td id="S2.T2.2.16.14.6" class="ltx_td ltx_align_center">83.0</td>
<td id="S2.T2.2.16.14.7" class="ltx_td ltx_align_center">99.2</td>
<td id="S2.T2.2.16.14.8" class="ltx_td ltx_align_center">97.4</td>
<td id="S2.T2.2.16.14.9" class="ltx_td ltx_align_center">58.3</td>
</tr>
<tr id="S2.T2.2.17.15" class="ltx_tr">
<th id="S2.T2.2.17.15.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t" colspan="9"><span id="S2.T2.2.17.15.1.1" class="ltx_text ltx_font_italic">Attention maps based (Ours)</span></th>
</tr>
<tr id="S2.T2.2.18.16" class="ltx_tr">
<th id="S2.T2.2.18.16.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t"><span id="S2.T2.2.18.16.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span></th>
<td id="S2.T2.2.18.16.2" class="ltx_td ltx_align_center ltx_border_t">QA</td>
<td id="S2.T2.2.18.16.3" class="ltx_td ltx_align_center ltx_border_t">Sum.</td>
<td id="S2.T2.2.18.16.4" class="ltx_td ltx_align_center ltx_border_t">98.3</td>
<td id="S2.T2.2.18.16.5" class="ltx_td ltx_align_center ltx_border_t">91.2</td>
<td id="S2.T2.2.18.16.6" class="ltx_td ltx_align_center ltx_border_t">85.3</td>
<td id="S2.T2.2.18.16.7" class="ltx_td ltx_align_center ltx_border_t">88.3</td>
<td id="S2.T2.2.18.16.8" class="ltx_td ltx_align_center ltx_border_t">87.1</td>
<td id="S2.T2.2.18.16.9" class="ltx_td ltx_align_center ltx_border_t">66.1</td>
</tr>
<tr id="S2.T2.2.19.17" class="ltx_tr">
<th id="S2.T2.2.19.17.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb"><span id="S2.T2.2.19.17.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span></th>
<td id="S2.T2.2.19.17.2" class="ltx_td ltx_align_center ltx_border_bb">Sum.</td>
<td id="S2.T2.2.19.17.3" class="ltx_td ltx_align_center ltx_border_bb">QA</td>
<td id="S2.T2.2.19.17.4" class="ltx_td ltx_align_center ltx_border_bb">97.7</td>
<td id="S2.T2.2.19.17.5" class="ltx_td ltx_align_center ltx_border_bb">88.8</td>
<td id="S2.T2.2.19.17.6" class="ltx_td ltx_align_center ltx_border_bb">82.0</td>
<td id="S2.T2.2.19.17.7" class="ltx_td ltx_align_center ltx_border_bb">86.2</td>
<td id="S2.T2.2.19.17.8" class="ltx_td ltx_align_center ltx_border_bb">85.3</td>
<td id="S2.T2.2.19.17.9" class="ltx_td ltx_align_center ltx_border_bb">66.0</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 2: </span>AUROC of the classification tasks using predefined span segmentation and sliding window (size = 8) on NQ (QA) and CNN/DM (Sum.). The source task scores (Train/Test) are averaged over two-fold validation.</figcaption>
</figure>
</section>
</section>
<section id="S2.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.3 </span>Results</h3>

<div id="S2.SS3.p1" class="ltx_para">
<p id="S2.SS3.p1.1" class="ltx_p">Our results are presented in Table <a href="#S2.T2" title="Table 2 ‣ Baselines ‣ 2.2 Experimental Setup ‣ 2 Contextual Hallucinations Detection ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>. We consider both predefined span segmentation and sliding window with a window size of 8.
We include the two-fold validation setting on the source task and the out-of-domain transfer setting on the target task, with the tasks either question answering (QA) or summarization (Sum.). We find that the <span id="S2.SS3.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> achieves slightly better performance than the hidden states-based classifier and significantly outperforms the NLI models (SoTA and our impl.). The advantage of the <span id="S2.SS3.p1.1.2" class="ltx_text ltx_font_italic">Lookback Lens</span> over the hidden states-based classifier is more significant in the sliding window settings, as shown in the right-hand side of Table <a href="#S2.T2" title="Table 2 ‣ Baselines ‣ 2.2 Experimental Setup ‣ 2 Contextual Hallucinations Detection ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>.</p>
</div>
<div id="S2.SS3.p2" class="ltx_para">
<p id="S2.SS3.p2.1" class="ltx_p">Additionally, we observe that the hidden states-based classifier tends to overfit the training sets during the two-fold validation, and present a substantial performance drop when transferred to out-of-domain tasks. In contrast, <span id="S2.SS3.p2.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span>, while not always fitting the training set perfectly, consistently exhibits better performance when applied to out-of-domain tasks. This contrast highlights the effectiveness and generalizability of the lookback ratio features we extract from the attention maps.</p>
</div>
</section>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>Contextual Hallucinations Mitigation</h2>

<figure id="S3.F2" class="ltx_figure"><img src="/html/2407.07071/assets/x2.png" id="S3.F2.1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="72" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 2: </span><span id="S3.F2.4.1" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span>: sample multiple chunk candidates, compute lookback ratios from attention maps to be scored by <span id="S3.F2.5.2" class="ltx_text ltx_font_italic">Lookback Lens</span>, and select the best candidate that is less likely to be hallucinations.</figcaption>
</figure>
<section id="S3.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.1 </span>Lookback Lens Guided Decoding</h3>

<div id="S3.SS1.p1" class="ltx_para">
<p id="S3.SS1.p1.1" class="ltx_p">To mitigate the impact of contextual hallucinations identified by the <span id="S3.SS1.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span>, we introduce a classifier-guided decoding strategy to guide the generation toward more contextually accurate outputs.
This approach serves as a robustness test of the <span id="S3.SS1.p1.1.2" class="ltx_text ltx_font_italic">Lookback Lens</span>’ ability to handle various text generation scenarios.
While prior studies on controllable text generation adjust the output probabilities using classifiers based on the output tokens <cite class="ltx_cite ltx_citemacro_citep">(Yang and Klein, <a href="#bib.bib44" title="" class="ltx_ref">2021</a>)</cite>, our method fundamentally differs by not using the tokens themselves but rather their attention maps during generation.</p>
</div>
<div id="S3.SS1.p2" class="ltx_para">
<p id="S3.SS1.p2.3" class="ltx_p">We propose <span id="S3.SS1.p2.3.1" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span>, which incorporates <span id="S3.SS1.p2.3.2" class="ltx_text ltx_font_italic">Lookback Lens</span> (<math id="S3.SS1.p2.1.m1.1" class="ltx_Math" alttext="\mathcal{F}" display="inline"><semantics id="S3.SS1.p2.1.m1.1a"><mi class="ltx_font_mathcaligraphic" id="S3.SS1.p2.1.m1.1.1" xref="S3.SS1.p2.1.m1.1.1.cmml">ℱ</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.1.m1.1b"><ci id="S3.SS1.p2.1.m1.1.1.cmml" xref="S3.SS1.p2.1.m1.1.1">ℱ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.1.m1.1c">\mathcal{F}</annotation></semantics></math>) into the decoding process.
Since all tokens in the vocabulary share the same attention pattern during one decoding step, <math id="S3.SS1.p2.2.m2.1" class="ltx_Math" alttext="\mathcal{F}" display="inline"><semantics id="S3.SS1.p2.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="S3.SS1.p2.2.m2.1.1" xref="S3.SS1.p2.2.m2.1.1.cmml">ℱ</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.2.m2.1b"><ci id="S3.SS1.p2.2.m2.1.1.cmml" xref="S3.SS1.p2.2.m2.1.1">ℱ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.2.m2.1c">\mathcal{F}</annotation></semantics></math> cannot directly influence one-step token choice.
Instead, <math id="S3.SS1.p2.3.m3.1" class="ltx_Math" alttext="\mathcal{F}" display="inline"><semantics id="S3.SS1.p2.3.m3.1a"><mi class="ltx_font_mathcaligraphic" id="S3.SS1.p2.3.m3.1.1" xref="S3.SS1.p2.3.m3.1.1.cmml">ℱ</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.3.m3.1b"><ci id="S3.SS1.p2.3.m3.1.1.cmml" xref="S3.SS1.p2.3.m3.1.1">ℱ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.3.m3.1c">\mathcal{F}</annotation></semantics></math> can evaluate multiple-token chunks, as each chunk causes different attention patterns in multiple decoding steps.</p>
</div>
<div id="S3.SS1.p3" class="ltx_para">
<p id="S3.SS1.p3.7" class="ltx_p">Given the context and partially generated text, we independently sample a set of <math id="S3.SS1.p3.1.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S3.SS1.p3.1.m1.1a"><mi id="S3.SS1.p3.1.m1.1.1" xref="S3.SS1.p3.1.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.1.m1.1b"><ci id="S3.SS1.p3.1.m1.1.1.cmml" xref="S3.SS1.p3.1.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.1.m1.1c">k</annotation></semantics></math> candidate chunks <math id="S3.SS1.p3.2.m2.4" class="ltx_Math" alttext="\{C_{1},C_{2},\dots,C_{k}\}" display="inline"><semantics id="S3.SS1.p3.2.m2.4a"><mrow id="S3.SS1.p3.2.m2.4.4.3" xref="S3.SS1.p3.2.m2.4.4.4.cmml"><mo stretchy="false" id="S3.SS1.p3.2.m2.4.4.3.4" xref="S3.SS1.p3.2.m2.4.4.4.cmml">{</mo><msub id="S3.SS1.p3.2.m2.2.2.1.1" xref="S3.SS1.p3.2.m2.2.2.1.1.cmml"><mi id="S3.SS1.p3.2.m2.2.2.1.1.2" xref="S3.SS1.p3.2.m2.2.2.1.1.2.cmml">C</mi><mn id="S3.SS1.p3.2.m2.2.2.1.1.3" xref="S3.SS1.p3.2.m2.2.2.1.1.3.cmml">1</mn></msub><mo id="S3.SS1.p3.2.m2.4.4.3.5" xref="S3.SS1.p3.2.m2.4.4.4.cmml">,</mo><msub id="S3.SS1.p3.2.m2.3.3.2.2" xref="S3.SS1.p3.2.m2.3.3.2.2.cmml"><mi id="S3.SS1.p3.2.m2.3.3.2.2.2" xref="S3.SS1.p3.2.m2.3.3.2.2.2.cmml">C</mi><mn id="S3.SS1.p3.2.m2.3.3.2.2.3" xref="S3.SS1.p3.2.m2.3.3.2.2.3.cmml">2</mn></msub><mo id="S3.SS1.p3.2.m2.4.4.3.6" xref="S3.SS1.p3.2.m2.4.4.4.cmml">,</mo><mi mathvariant="normal" id="S3.SS1.p3.2.m2.1.1" xref="S3.SS1.p3.2.m2.1.1.cmml">…</mi><mo id="S3.SS1.p3.2.m2.4.4.3.7" xref="S3.SS1.p3.2.m2.4.4.4.cmml">,</mo><msub id="S3.SS1.p3.2.m2.4.4.3.3" xref="S3.SS1.p3.2.m2.4.4.3.3.cmml"><mi id="S3.SS1.p3.2.m2.4.4.3.3.2" xref="S3.SS1.p3.2.m2.4.4.3.3.2.cmml">C</mi><mi id="S3.SS1.p3.2.m2.4.4.3.3.3" xref="S3.SS1.p3.2.m2.4.4.3.3.3.cmml">k</mi></msub><mo stretchy="false" id="S3.SS1.p3.2.m2.4.4.3.8" xref="S3.SS1.p3.2.m2.4.4.4.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.2.m2.4b"><set id="S3.SS1.p3.2.m2.4.4.4.cmml" xref="S3.SS1.p3.2.m2.4.4.3"><apply id="S3.SS1.p3.2.m2.2.2.1.1.cmml" xref="S3.SS1.p3.2.m2.2.2.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.2.m2.2.2.1.1.1.cmml" xref="S3.SS1.p3.2.m2.2.2.1.1">subscript</csymbol><ci id="S3.SS1.p3.2.m2.2.2.1.1.2.cmml" xref="S3.SS1.p3.2.m2.2.2.1.1.2">𝐶</ci><cn type="integer" id="S3.SS1.p3.2.m2.2.2.1.1.3.cmml" xref="S3.SS1.p3.2.m2.2.2.1.1.3">1</cn></apply><apply id="S3.SS1.p3.2.m2.3.3.2.2.cmml" xref="S3.SS1.p3.2.m2.3.3.2.2"><csymbol cd="ambiguous" id="S3.SS1.p3.2.m2.3.3.2.2.1.cmml" xref="S3.SS1.p3.2.m2.3.3.2.2">subscript</csymbol><ci id="S3.SS1.p3.2.m2.3.3.2.2.2.cmml" xref="S3.SS1.p3.2.m2.3.3.2.2.2">𝐶</ci><cn type="integer" id="S3.SS1.p3.2.m2.3.3.2.2.3.cmml" xref="S3.SS1.p3.2.m2.3.3.2.2.3">2</cn></apply><ci id="S3.SS1.p3.2.m2.1.1.cmml" xref="S3.SS1.p3.2.m2.1.1">…</ci><apply id="S3.SS1.p3.2.m2.4.4.3.3.cmml" xref="S3.SS1.p3.2.m2.4.4.3.3"><csymbol cd="ambiguous" id="S3.SS1.p3.2.m2.4.4.3.3.1.cmml" xref="S3.SS1.p3.2.m2.4.4.3.3">subscript</csymbol><ci id="S3.SS1.p3.2.m2.4.4.3.3.2.cmml" xref="S3.SS1.p3.2.m2.4.4.3.3.2">𝐶</ci><ci id="S3.SS1.p3.2.m2.4.4.3.3.3.cmml" xref="S3.SS1.p3.2.m2.4.4.3.3.3">𝑘</ci></apply></set></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.2.m2.4c">\{C_{1},C_{2},\dots,C_{k}\}</annotation></semantics></math> at the same decoding step <math id="S3.SS1.p3.3.m3.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S3.SS1.p3.3.m3.1a"><mi id="S3.SS1.p3.3.m3.1.1" xref="S3.SS1.p3.3.m3.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.3.m3.1b"><ci id="S3.SS1.p3.3.m3.1.1.cmml" xref="S3.SS1.p3.3.m3.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.3.m3.1c">t</annotation></semantics></math>.
For each chunk <math id="S3.SS1.p3.4.m4.1" class="ltx_Math" alttext="C_{j}" display="inline"><semantics id="S3.SS1.p3.4.m4.1a"><msub id="S3.SS1.p3.4.m4.1.1" xref="S3.SS1.p3.4.m4.1.1.cmml"><mi id="S3.SS1.p3.4.m4.1.1.2" xref="S3.SS1.p3.4.m4.1.1.2.cmml">C</mi><mi id="S3.SS1.p3.4.m4.1.1.3" xref="S3.SS1.p3.4.m4.1.1.3.cmml">j</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.4.m4.1b"><apply id="S3.SS1.p3.4.m4.1.1.cmml" xref="S3.SS1.p3.4.m4.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.4.m4.1.1.1.cmml" xref="S3.SS1.p3.4.m4.1.1">subscript</csymbol><ci id="S3.SS1.p3.4.m4.1.1.2.cmml" xref="S3.SS1.p3.4.m4.1.1.2">𝐶</ci><ci id="S3.SS1.p3.4.m4.1.1.3.cmml" xref="S3.SS1.p3.4.m4.1.1.3">𝑗</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.4.m4.1c">C_{j}</annotation></semantics></math>, the associated lookback ratios are averaged to form a feature vector <math id="S3.SS1.p3.5.m5.1" class="ltx_Math" alttext="\mathbf{\bar{v}}^{j}" display="inline"><semantics id="S3.SS1.p3.5.m5.1a"><msup id="S3.SS1.p3.5.m5.1.1" xref="S3.SS1.p3.5.m5.1.1.cmml"><mover accent="true" id="S3.SS1.p3.5.m5.1.1.2" xref="S3.SS1.p3.5.m5.1.1.2.cmml"><mi id="S3.SS1.p3.5.m5.1.1.2.2" xref="S3.SS1.p3.5.m5.1.1.2.2.cmml">𝐯</mi><mo id="S3.SS1.p3.5.m5.1.1.2.1" xref="S3.SS1.p3.5.m5.1.1.2.1.cmml">¯</mo></mover><mi id="S3.SS1.p3.5.m5.1.1.3" xref="S3.SS1.p3.5.m5.1.1.3.cmml">j</mi></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.5.m5.1b"><apply id="S3.SS1.p3.5.m5.1.1.cmml" xref="S3.SS1.p3.5.m5.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.5.m5.1.1.1.cmml" xref="S3.SS1.p3.5.m5.1.1">superscript</csymbol><apply id="S3.SS1.p3.5.m5.1.1.2.cmml" xref="S3.SS1.p3.5.m5.1.1.2"><ci id="S3.SS1.p3.5.m5.1.1.2.1.cmml" xref="S3.SS1.p3.5.m5.1.1.2.1">¯</ci><ci id="S3.SS1.p3.5.m5.1.1.2.2.cmml" xref="S3.SS1.p3.5.m5.1.1.2.2">𝐯</ci></apply><ci id="S3.SS1.p3.5.m5.1.1.3.cmml" xref="S3.SS1.p3.5.m5.1.1.3">𝑗</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.5.m5.1c">\mathbf{\bar{v}}^{j}</annotation></semantics></math>. As shown in Figure <a href="#S3.F2" title="Figure 2 ‣ 3 Contextual Hallucinations Mitigation ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>, we select the best candidate <math id="S3.SS1.p3.6.m6.1" class="ltx_Math" alttext="C^{*}" display="inline"><semantics id="S3.SS1.p3.6.m6.1a"><msup id="S3.SS1.p3.6.m6.1.1" xref="S3.SS1.p3.6.m6.1.1.cmml"><mi id="S3.SS1.p3.6.m6.1.1.2" xref="S3.SS1.p3.6.m6.1.1.2.cmml">C</mi><mo id="S3.SS1.p3.6.m6.1.1.3" xref="S3.SS1.p3.6.m6.1.1.3.cmml">∗</mo></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.6.m6.1b"><apply id="S3.SS1.p3.6.m6.1.1.cmml" xref="S3.SS1.p3.6.m6.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.6.m6.1.1.1.cmml" xref="S3.SS1.p3.6.m6.1.1">superscript</csymbol><ci id="S3.SS1.p3.6.m6.1.1.2.cmml" xref="S3.SS1.p3.6.m6.1.1.2">𝐶</ci><times id="S3.SS1.p3.6.m6.1.1.3.cmml" xref="S3.SS1.p3.6.m6.1.1.3"></times></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.6.m6.1c">C^{*}</annotation></semantics></math> predicted by <math id="S3.SS1.p3.7.m7.1" class="ltx_Math" alttext="\mathcal{F}" display="inline"><semantics id="S3.SS1.p3.7.m7.1a"><mi class="ltx_font_mathcaligraphic" id="S3.SS1.p3.7.m7.1.1" xref="S3.SS1.p3.7.m7.1.1.cmml">ℱ</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.7.m7.1b"><ci id="S3.SS1.p3.7.m7.1.1.cmml" xref="S3.SS1.p3.7.m7.1.1">ℱ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.7.m7.1c">\mathcal{F}</annotation></semantics></math> and append to the generation,</p>
<table id="S3.Ex6" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S3.Ex6.m1.5" class="ltx_Math" alttext="C^{*}=\underset{C_{j}\in\{C_{1},C_{2},\dots,C_{k}\}}{\arg\max}\mathcal{F}(\mathbf{\bar{v}}^{j})." display="block"><semantics id="S3.Ex6.m1.5a"><mrow id="S3.Ex6.m1.5.5.1" xref="S3.Ex6.m1.5.5.1.1.cmml"><mrow id="S3.Ex6.m1.5.5.1.1" xref="S3.Ex6.m1.5.5.1.1.cmml"><msup id="S3.Ex6.m1.5.5.1.1.3" xref="S3.Ex6.m1.5.5.1.1.3.cmml"><mi id="S3.Ex6.m1.5.5.1.1.3.2" xref="S3.Ex6.m1.5.5.1.1.3.2.cmml">C</mi><mo id="S3.Ex6.m1.5.5.1.1.3.3" xref="S3.Ex6.m1.5.5.1.1.3.3.cmml">∗</mo></msup><mo id="S3.Ex6.m1.5.5.1.1.2" xref="S3.Ex6.m1.5.5.1.1.2.cmml">=</mo><mrow id="S3.Ex6.m1.5.5.1.1.1" xref="S3.Ex6.m1.5.5.1.1.1.cmml"><munder accentunder="true" id="S3.Ex6.m1.4.4" xref="S3.Ex6.m1.4.4.cmml"><mrow id="S3.Ex6.m1.4.4.5" xref="S3.Ex6.m1.4.4.5.cmml"><mi id="S3.Ex6.m1.4.4.5.1" xref="S3.Ex6.m1.4.4.5.1.cmml">arg</mi><mo lspace="0.167em" id="S3.Ex6.m1.4.4.5a" xref="S3.Ex6.m1.4.4.5.cmml">⁡</mo><mi id="S3.Ex6.m1.4.4.5.2" xref="S3.Ex6.m1.4.4.5.2.cmml">max</mi></mrow><mrow id="S3.Ex6.m1.4.4.4" xref="S3.Ex6.m1.4.4.4.cmml"><msub id="S3.Ex6.m1.4.4.4.6" xref="S3.Ex6.m1.4.4.4.6.cmml"><mi id="S3.Ex6.m1.4.4.4.6.2" xref="S3.Ex6.m1.4.4.4.6.2.cmml">C</mi><mi id="S3.Ex6.m1.4.4.4.6.3" xref="S3.Ex6.m1.4.4.4.6.3.cmml">j</mi></msub><mo id="S3.Ex6.m1.4.4.4.5" xref="S3.Ex6.m1.4.4.4.5.cmml">∈</mo><mrow id="S3.Ex6.m1.4.4.4.4.3" xref="S3.Ex6.m1.4.4.4.4.4.cmml"><mo stretchy="false" id="S3.Ex6.m1.4.4.4.4.3.4" xref="S3.Ex6.m1.4.4.4.4.4.cmml">{</mo><msub id="S3.Ex6.m1.2.2.2.2.1.1" xref="S3.Ex6.m1.2.2.2.2.1.1.cmml"><mi id="S3.Ex6.m1.2.2.2.2.1.1.2" xref="S3.Ex6.m1.2.2.2.2.1.1.2.cmml">C</mi><mn id="S3.Ex6.m1.2.2.2.2.1.1.3" xref="S3.Ex6.m1.2.2.2.2.1.1.3.cmml">1</mn></msub><mo id="S3.Ex6.m1.4.4.4.4.3.5" xref="S3.Ex6.m1.4.4.4.4.4.cmml">,</mo><msub id="S3.Ex6.m1.3.3.3.3.2.2" xref="S3.Ex6.m1.3.3.3.3.2.2.cmml"><mi id="S3.Ex6.m1.3.3.3.3.2.2.2" xref="S3.Ex6.m1.3.3.3.3.2.2.2.cmml">C</mi><mn id="S3.Ex6.m1.3.3.3.3.2.2.3" xref="S3.Ex6.m1.3.3.3.3.2.2.3.cmml">2</mn></msub><mo id="S3.Ex6.m1.4.4.4.4.3.6" xref="S3.Ex6.m1.4.4.4.4.4.cmml">,</mo><mi mathvariant="normal" id="S3.Ex6.m1.1.1.1.1" xref="S3.Ex6.m1.1.1.1.1.cmml">…</mi><mo id="S3.Ex6.m1.4.4.4.4.3.7" xref="S3.Ex6.m1.4.4.4.4.4.cmml">,</mo><msub id="S3.Ex6.m1.4.4.4.4.3.3" xref="S3.Ex6.m1.4.4.4.4.3.3.cmml"><mi id="S3.Ex6.m1.4.4.4.4.3.3.2" xref="S3.Ex6.m1.4.4.4.4.3.3.2.cmml">C</mi><mi id="S3.Ex6.m1.4.4.4.4.3.3.3" xref="S3.Ex6.m1.4.4.4.4.3.3.3.cmml">k</mi></msub><mo stretchy="false" id="S3.Ex6.m1.4.4.4.4.3.8" xref="S3.Ex6.m1.4.4.4.4.4.cmml">}</mo></mrow></mrow></munder><mo lspace="0.167em" rspace="0em" id="S3.Ex6.m1.5.5.1.1.1.2" xref="S3.Ex6.m1.5.5.1.1.1.2.cmml">​</mo><mi class="ltx_font_mathcaligraphic" id="S3.Ex6.m1.5.5.1.1.1.3" xref="S3.Ex6.m1.5.5.1.1.1.3.cmml">ℱ</mi><mo lspace="0em" rspace="0em" id="S3.Ex6.m1.5.5.1.1.1.2a" xref="S3.Ex6.m1.5.5.1.1.1.2.cmml">​</mo><mrow id="S3.Ex6.m1.5.5.1.1.1.1.1" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S3.Ex6.m1.5.5.1.1.1.1.1.2" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.cmml">(</mo><msup id="S3.Ex6.m1.5.5.1.1.1.1.1.1" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.cmml"><mover accent="true" id="S3.Ex6.m1.5.5.1.1.1.1.1.1.2" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.2.cmml"><mi id="S3.Ex6.m1.5.5.1.1.1.1.1.1.2.2" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.2.2.cmml">𝐯</mi><mo id="S3.Ex6.m1.5.5.1.1.1.1.1.1.2.1" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.2.1.cmml">¯</mo></mover><mi id="S3.Ex6.m1.5.5.1.1.1.1.1.1.3" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.3.cmml">j</mi></msup><mo stretchy="false" id="S3.Ex6.m1.5.5.1.1.1.1.1.3" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo lspace="0em" id="S3.Ex6.m1.5.5.1.2" xref="S3.Ex6.m1.5.5.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.Ex6.m1.5b"><apply id="S3.Ex6.m1.5.5.1.1.cmml" xref="S3.Ex6.m1.5.5.1"><eq id="S3.Ex6.m1.5.5.1.1.2.cmml" xref="S3.Ex6.m1.5.5.1.1.2"></eq><apply id="S3.Ex6.m1.5.5.1.1.3.cmml" xref="S3.Ex6.m1.5.5.1.1.3"><csymbol cd="ambiguous" id="S3.Ex6.m1.5.5.1.1.3.1.cmml" xref="S3.Ex6.m1.5.5.1.1.3">superscript</csymbol><ci id="S3.Ex6.m1.5.5.1.1.3.2.cmml" xref="S3.Ex6.m1.5.5.1.1.3.2">𝐶</ci><times id="S3.Ex6.m1.5.5.1.1.3.3.cmml" xref="S3.Ex6.m1.5.5.1.1.3.3"></times></apply><apply id="S3.Ex6.m1.5.5.1.1.1.cmml" xref="S3.Ex6.m1.5.5.1.1.1"><times id="S3.Ex6.m1.5.5.1.1.1.2.cmml" xref="S3.Ex6.m1.5.5.1.1.1.2"></times><apply id="S3.Ex6.m1.4.4.cmml" xref="S3.Ex6.m1.4.4"><apply id="S3.Ex6.m1.4.4.4.cmml" xref="S3.Ex6.m1.4.4.4"><in id="S3.Ex6.m1.4.4.4.5.cmml" xref="S3.Ex6.m1.4.4.4.5"></in><apply id="S3.Ex6.m1.4.4.4.6.cmml" xref="S3.Ex6.m1.4.4.4.6"><csymbol cd="ambiguous" id="S3.Ex6.m1.4.4.4.6.1.cmml" xref="S3.Ex6.m1.4.4.4.6">subscript</csymbol><ci id="S3.Ex6.m1.4.4.4.6.2.cmml" xref="S3.Ex6.m1.4.4.4.6.2">𝐶</ci><ci id="S3.Ex6.m1.4.4.4.6.3.cmml" xref="S3.Ex6.m1.4.4.4.6.3">𝑗</ci></apply><set id="S3.Ex6.m1.4.4.4.4.4.cmml" xref="S3.Ex6.m1.4.4.4.4.3"><apply id="S3.Ex6.m1.2.2.2.2.1.1.cmml" xref="S3.Ex6.m1.2.2.2.2.1.1"><csymbol cd="ambiguous" id="S3.Ex6.m1.2.2.2.2.1.1.1.cmml" xref="S3.Ex6.m1.2.2.2.2.1.1">subscript</csymbol><ci id="S3.Ex6.m1.2.2.2.2.1.1.2.cmml" xref="S3.Ex6.m1.2.2.2.2.1.1.2">𝐶</ci><cn type="integer" id="S3.Ex6.m1.2.2.2.2.1.1.3.cmml" xref="S3.Ex6.m1.2.2.2.2.1.1.3">1</cn></apply><apply id="S3.Ex6.m1.3.3.3.3.2.2.cmml" xref="S3.Ex6.m1.3.3.3.3.2.2"><csymbol cd="ambiguous" id="S3.Ex6.m1.3.3.3.3.2.2.1.cmml" xref="S3.Ex6.m1.3.3.3.3.2.2">subscript</csymbol><ci id="S3.Ex6.m1.3.3.3.3.2.2.2.cmml" xref="S3.Ex6.m1.3.3.3.3.2.2.2">𝐶</ci><cn type="integer" id="S3.Ex6.m1.3.3.3.3.2.2.3.cmml" xref="S3.Ex6.m1.3.3.3.3.2.2.3">2</cn></apply><ci id="S3.Ex6.m1.1.1.1.1.cmml" xref="S3.Ex6.m1.1.1.1.1">…</ci><apply id="S3.Ex6.m1.4.4.4.4.3.3.cmml" xref="S3.Ex6.m1.4.4.4.4.3.3"><csymbol cd="ambiguous" id="S3.Ex6.m1.4.4.4.4.3.3.1.cmml" xref="S3.Ex6.m1.4.4.4.4.3.3">subscript</csymbol><ci id="S3.Ex6.m1.4.4.4.4.3.3.2.cmml" xref="S3.Ex6.m1.4.4.4.4.3.3.2">𝐶</ci><ci id="S3.Ex6.m1.4.4.4.4.3.3.3.cmml" xref="S3.Ex6.m1.4.4.4.4.3.3.3">𝑘</ci></apply></set></apply><apply id="S3.Ex6.m1.4.4.5.cmml" xref="S3.Ex6.m1.4.4.5"><arg id="S3.Ex6.m1.4.4.5.1.cmml" xref="S3.Ex6.m1.4.4.5.1"></arg><max id="S3.Ex6.m1.4.4.5.2.cmml" xref="S3.Ex6.m1.4.4.5.2"></max></apply></apply><ci id="S3.Ex6.m1.5.5.1.1.1.3.cmml" xref="S3.Ex6.m1.5.5.1.1.1.3">ℱ</ci><apply id="S3.Ex6.m1.5.5.1.1.1.1.1.1.cmml" xref="S3.Ex6.m1.5.5.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.Ex6.m1.5.5.1.1.1.1.1.1.1.cmml" xref="S3.Ex6.m1.5.5.1.1.1.1.1">superscript</csymbol><apply id="S3.Ex6.m1.5.5.1.1.1.1.1.1.2.cmml" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.2"><ci id="S3.Ex6.m1.5.5.1.1.1.1.1.1.2.1.cmml" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.2.1">¯</ci><ci id="S3.Ex6.m1.5.5.1.1.1.1.1.1.2.2.cmml" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.2.2">𝐯</ci></apply><ci id="S3.Ex6.m1.5.5.1.1.1.1.1.1.3.cmml" xref="S3.Ex6.m1.5.5.1.1.1.1.1.1.3">𝑗</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.Ex6.m1.5c">C^{*}=\underset{C_{j}\in\{C_{1},C_{2},\dots,C_{k}\}}{\arg\max}\mathcal{F}(\mathbf{\bar{v}}^{j}).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
</div>
<div id="S3.SS1.p4" class="ltx_para">
<p id="S3.SS1.p4.1" class="ltx_p">We repeat this process until it generates the EOS token or reaches the maximum length.</p>
</div>
</section>
<section id="S3.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.2 </span>Experimental Setup</h3>

<div id="S3.SS2.p1" class="ltx_para">
<p id="S3.SS2.p1.1" class="ltx_p">We evaluate <span id="S3.SS2.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span> on three tasks that involve generating texts conditioned on given contexts, including summarization with XSum <cite class="ltx_cite ltx_citemacro_citep">(Narayan et al., <a href="#bib.bib30" title="" class="ltx_ref">2018</a>)</cite>, QA with NQ <cite class="ltx_cite ltx_citemacro_citep">(Kwiatkowski et al., <a href="#bib.bib18" title="" class="ltx_ref">2019</a>)</cite>, and multi-turn conversations with MT-bench <cite class="ltx_cite ltx_citemacro_citep">(Zheng et al., <a href="#bib.bib48" title="" class="ltx_ref">2024</a>)</cite>.</p>
</div>
<div id="S3.SS2.p2" class="ltx_para">
<p id="S3.SS2.p2.1" class="ltx_p">For testing the generalization ability of the <span id="S3.SS2.p2.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span>, we only train it with the CNN/DM summarization dataset from the detection task in Section <a href="#S2.SS2" title="2.2 Experimental Setup ‣ 2 Contextual Hallucinations Detection ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.2</span></a>. Thus, only the XSum dataset will be the same-task transfer setting, while NQ and MT-bench will be cross-task transfer setting.</p>
</div>
<section id="S3.SS2.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">XSum</h4>

<div id="S3.SS2.SSS0.Px1.p1" class="ltx_para">
<p id="S3.SS2.SSS0.Px1.p1.1" class="ltx_p">To test the <span id="S3.SS2.SSS0.Px1.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span>’s effectiveness at transferring across data distributions for the same task (summarization), we use 1,000 examples sampled from the testing set of XSum. Prior studies <cite class="ltx_cite ltx_citemacro_citep">(Maynez et al., <a href="#bib.bib28" title="" class="ltx_ref">2020</a>)</cite> indicate that traditional evaluation metrics such as ROUGE <cite class="ltx_cite ltx_citemacro_citep">(Lin, <a href="#bib.bib23" title="" class="ltx_ref">2004</a>)</cite> or BERTScore <cite class="ltx_cite ltx_citemacro_citep">(Zhang et al., <a href="#bib.bib46" title="" class="ltx_ref">2019a</a>)</cite> correlated poorly with human evaluation on faithfulness and factuality. Recent studies <cite class="ltx_cite ltx_citemacro_citep">(Chiang and Lee, <a href="#bib.bib8" title="" class="ltx_ref">2023</a>; Liu et al., <a href="#bib.bib25" title="" class="ltx_ref">2023</a>)</cite> also show a strong correlation between GPT-4 <cite class="ltx_cite ltx_citemacro_citep">(OpenAI, <a href="#bib.bib33" title="" class="ltx_ref">2023</a>)</cite> evaluation and human evaluation. Thus, we report the averaged accuracy from the binary judgments of GPT-4o, with the prompts in Appendix <a href="#A2" title="Appendix B Evaluation Prompt for GPT-4o ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">B</span></a>. We also conduct a pilot study for human evaluation on GPT-4o’s judgment in Appendix <a href="#A3" title="Appendix C Pilot Study of Human Evaluation on GPT-4o Evaluation ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">C</span></a>, finding that 97% of the GPT-4o judgments are consistent with human judgment.</p>
</div>
</section>
<section id="S3.SS2.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Natural Questions</h4>

<div id="S3.SS2.SSS0.Px2.p1" class="ltx_para">
<p id="S3.SS2.SSS0.Px2.p1.1" class="ltx_p">We use the NQ data from the setup of <cite class="ltx_cite ltx_citemacro_citet">Liu et al. (<a href="#bib.bib24" title="" class="ltx_ref">2024</a>)</cite> we describe in Appendix <a href="#A7" title="Appendix G Dataset Details ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">G</span></a> and evaluate the best span exact match following <cite class="ltx_cite ltx_citemacro_citet">Kandpal et al. (<a href="#bib.bib17" title="" class="ltx_ref">2023</a>); Mallen et al. (<a href="#bib.bib27" title="" class="ltx_ref">2023</a>)</cite>.</p>
</div>
</section>
<section id="S3.SS2.SSS0.Px3" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">MT-Bench</h4>

<div id="S3.SS2.SSS0.Px3.p1" class="ltx_para">
<p id="S3.SS2.SSS0.Px3.p1.1" class="ltx_p">We consider a multi-turn conversations setup where the model needs to follow previous chat history. We use MT-bench <cite class="ltx_cite ltx_citemacro_citep">(Zheng et al., <a href="#bib.bib48" title="" class="ltx_ref">2024</a>)</cite>, a multi-turn instruction-following benchmark covering 8 categories. We focus exclusively on generating responses for the second turn and use GPT-3.5’s responses as the default for the first turn. We use GPT-4 to score the model’s answers on a scale of 1 to 10 based on various factors, including <em id="S3.SS2.SSS0.Px3.p1.1.1" class="ltx_emph ltx_font_italic">helpfulness, relevance, accuracy, depth, creativity, and level of detail of the response</em>.</p>
</div>
<div id="S3.SS2.SSS0.Px3.p2" class="ltx_para">
<p id="S3.SS2.SSS0.Px3.p2.1" class="ltx_p">Additionally, since we are particularly interested in mitigating contextual hallucinations, we further exclude math questions and evaluate the remaining 50 general questions. We specifically instruct GPT-4o to focus on whether the responses are faithful to the chat history (see prompt in Appendix <a href="#A2" title="Appendix B Evaluation Prompt for GPT-4o ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">B</span></a>). We refer to this setup as MT-Bench (hallu.).</p>
</div>
</section>
<section id="S3.SS2.SSS0.Px4" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Baselines</h4>

<div id="S3.SS2.SSS0.Px4.p1" class="ltx_para">
<p id="S3.SS2.SSS0.Px4.p1.1" class="ltx_p">To evaluate the performance of our proposed method, we compared it against the following baselines: <span id="S3.SS2.SSS0.Px4.p1.1.1" class="ltx_text ltx_font_bold">1) Greedy Decoding:</span> generating responses using the LLaMA-2-7B-Chat model <cite class="ltx_cite ltx_citemacro_citep">(Touvron et al., <a href="#bib.bib40" title="" class="ltx_ref">2023</a>)</cite> through greedy decoding. <span id="S3.SS2.SSS0.Px4.p1.1.2" class="ltx_text ltx_font_bold">2) Other Classifier-Guided Decoding:</span> using exactly the same setting but with different classifiers introduced in Section <a href="#S2.SS2" title="2.2 Experimental Setup ‣ 2 Contextual Hallucinations Detection ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.2</span></a>, including
text-based entailment classifiers and hidden states-based classifiers.</p>
</div>
</section>
</section>
<section id="S3.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.3 </span>Main Results</h3>

<figure id="S3.T3" class="ltx_table">
<table id="S3.T3.1" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S3.T3.1.2.1" class="ltx_tr">
<th id="S3.T3.1.2.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt" rowspan="2"><span id="S3.T3.1.2.1.1.1" class="ltx_text ltx_font_bold">Method</span></th>
<td id="S3.T3.1.2.1.2" class="ltx_td ltx_align_center ltx_border_tt" rowspan="2"><span id="S3.T3.1.2.1.2.1" class="ltx_text ltx_font_bold">XSum</span></td>
<td id="S3.T3.1.2.1.3" class="ltx_td ltx_align_center ltx_border_tt" rowspan="2"><span id="S3.T3.1.2.1.3.1" class="ltx_text ltx_font_bold">NQ</span></td>
<td id="S3.T3.1.2.1.4" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="S3.T3.1.2.1.4.1" class="ltx_text ltx_font_bold">MT-Bench</span></td>
</tr>
<tr id="S3.T3.1.3.2" class="ltx_tr">
<td id="S3.T3.1.3.2.1" class="ltx_td ltx_align_center ltx_border_t">Hallu.</td>
<td id="S3.T3.1.3.2.2" class="ltx_td ltx_align_center ltx_border_t">Ori.</td>
</tr>
<tr id="S3.T3.1.4.3" class="ltx_tr">
<th id="S3.T3.1.4.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">Greedy Decoding</th>
<td id="S3.T3.1.4.3.2" class="ltx_td ltx_align_center ltx_border_t">49.0</td>
<td id="S3.T3.1.4.3.3" class="ltx_td ltx_align_center ltx_border_t">71.2</td>
<td id="S3.T3.1.4.3.4" class="ltx_td ltx_align_center ltx_border_t">6.08</td>
<td id="S3.T3.1.4.3.5" class="ltx_td ltx_align_center ltx_border_t">5.10</td>
</tr>
<tr id="S3.T3.1.5.4" class="ltx_tr">
<th id="S3.T3.1.5.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="5"><span id="S3.T3.1.5.4.1.1" class="ltx_text ltx_font_italic">Text-based classifier guided decoding</span></th>
</tr>
<tr id="S3.T3.1.1" class="ltx_tr">
<th id="S3.T3.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">SoTA NLI<sup id="S3.T3.1.1.1.1" class="ltx_sup">†</sup>
</th>
<td id="S3.T3.1.1.2" class="ltx_td ltx_align_center ltx_border_t">59.0</td>
<td id="S3.T3.1.1.3" class="ltx_td ltx_align_center ltx_border_t">74.2</td>
<td id="S3.T3.1.1.4" class="ltx_td ltx_align_center ltx_border_t">6.12</td>
<td id="S3.T3.1.1.5" class="ltx_td ltx_align_center ltx_border_t">5.03</td>
</tr>
<tr id="S3.T3.1.6.5" class="ltx_tr">
<th id="S3.T3.1.6.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">NLI (our impl.)</th>
<td id="S3.T3.1.6.5.2" class="ltx_td ltx_align_center">44.1</td>
<td id="S3.T3.1.6.5.3" class="ltx_td ltx_align_center">72.5</td>
<td id="S3.T3.1.6.5.4" class="ltx_td ltx_align_center">5.72</td>
<td id="S3.T3.1.6.5.5" class="ltx_td ltx_align_center">4.99</td>
</tr>
<tr id="S3.T3.1.7.6" class="ltx_tr">
<th id="S3.T3.1.7.6.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="5"><span id="S3.T3.1.7.6.1.1" class="ltx_text ltx_font_italic">Hidden states based classifier guided decoding</span></th>
</tr>
<tr id="S3.T3.1.8.7" class="ltx_tr">
<th id="S3.T3.1.8.7.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">32nd layer</th>
<td id="S3.T3.1.8.7.2" class="ltx_td ltx_align_center ltx_border_t">48.3</td>
<td id="S3.T3.1.8.7.3" class="ltx_td ltx_align_center ltx_border_t">73.9</td>
<td id="S3.T3.1.8.7.4" class="ltx_td ltx_align_center ltx_border_t">5.49</td>
<td id="S3.T3.1.8.7.5" class="ltx_td ltx_align_center ltx_border_t">4.91</td>
</tr>
<tr id="S3.T3.1.9.8" class="ltx_tr">
<th id="S3.T3.1.9.8.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">28th layer</th>
<td id="S3.T3.1.9.8.2" class="ltx_td ltx_align_center">48.9</td>
<td id="S3.T3.1.9.8.3" class="ltx_td ltx_align_center">73.0</td>
<td id="S3.T3.1.9.8.4" class="ltx_td ltx_align_center">5.71</td>
<td id="S3.T3.1.9.8.5" class="ltx_td ltx_align_center">5.06</td>
</tr>
<tr id="S3.T3.1.10.9" class="ltx_tr">
<th id="S3.T3.1.10.9.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">24th layer</th>
<td id="S3.T3.1.10.9.2" class="ltx_td ltx_align_center">47.5</td>
<td id="S3.T3.1.10.9.3" class="ltx_td ltx_align_center">73.9</td>
<td id="S3.T3.1.10.9.4" class="ltx_td ltx_align_center">5.65</td>
<td id="S3.T3.1.10.9.5" class="ltx_td ltx_align_center">5.16</td>
</tr>
<tr id="S3.T3.1.11.10" class="ltx_tr">
<th id="S3.T3.1.11.10.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="5"><span id="S3.T3.1.11.10.1.1" class="ltx_text ltx_font_italic">Lookback Lens guided decoding</span></th>
</tr>
<tr id="S3.T3.1.12.11" class="ltx_tr">
<th id="S3.T3.1.12.11.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb ltx_border_t">Ours</th>
<td id="S3.T3.1.12.11.2" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">58.6</td>
<td id="S3.T3.1.12.11.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">74.2</td>
<td id="S3.T3.1.12.11.4" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">6.27</td>
<td id="S3.T3.1.12.11.5" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">5.10</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 3: </span>Decoding results using 8 candidates per chunk in a chunk size of 8. We compare our methods with greedy decoding and classifier-guided decoding using the NLI models, and hidden state representations of different layers. <sup id="S3.T3.5.1" class="ltx_sup">†</sup>The SoTA NLI is trained on 731k examples so it may not be directly comparable.</figcaption>
</figure>
<div id="S3.SS3.p1" class="ltx_para">
<p id="S3.SS3.p1.1" class="ltx_p">We show our results using 8 candidates per chunk in a chunk size of 8 in Table <a href="#S3.T3" title="Table 3 ‣ 3.3 Main Results ‣ 3 Contextual Hallucinations Mitigation ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>, and the ablation with different chunk sizes is shown in Table <a href="#S5.T6" title="Table 6 ‣ Effect of Chunk Size ‣ 5 Discussions and Ablations ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">6</span></a>. <span id="S3.SS3.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span> can improve the performance on both in-domain task (XSum, by 9.6%) and out-of-domain tasks (NQ, by 3%). The original greedy decoding results on XSum achieved 49.0% correct
which means 510 examples were hallucinated. Our decoding method significantly reduced the number of hallucinated examples from 510 to 414, resulting in an 18.8% reduction in the hallucinated examples. This result is on par with using SoTA NLI to guide the decoding, where SoTA NLI is trained on roughly 731k annotated summarization examples, which is 700<math id="S3.SS3.p1.1.m1.1" class="ltx_Math" alttext="\times" display="inline"><semantics id="S3.SS3.p1.1.m1.1a"><mo id="S3.SS3.p1.1.m1.1.1" xref="S3.SS3.p1.1.m1.1.1.cmml">×</mo><annotation-xml encoding="MathML-Content" id="S3.SS3.p1.1.m1.1b"><times id="S3.SS3.p1.1.m1.1.1.cmml" xref="S3.SS3.p1.1.m1.1.1"></times></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p1.1.m1.1c">\times</annotation></semantics></math> larger compared to our 1k training set. (See Appendix <a href="#A5" title="Appendix E Model Details ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">E</span></a>.) In contrast, decoding guided by hidden states-based or the NLI (our implementation) classifiers, both trained on the same data of our method, can only slightly improve the performance on NQ, but not for XSum, probably due to the issue of distribution shift, highlighting the advantages of <span id="S3.SS3.p1.1.2" class="ltx_text ltx_font_italic">Lookback Lens</span> in generalization ability.</p>
</div>
<div id="S3.SS3.p2" class="ltx_para">
<p id="S3.SS3.p2.1" class="ltx_p">For MT-bench, we evaluate both settings: the original setting (ori.) and the setting that is specifically for judging contextual hallucinations (hallu.).
We do not expect our method can improve on the original setting, because it evaluates many factors such as helpfulness, relevance, etc. But we expect to see an improvement on the hallucination setting.
The results shown in Table <a href="#S3.T3" title="Table 3 ‣ 3.3 Main Results ‣ 3 Contextual Hallucinations Mitigation ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a> suggest that our decoding method can boost the performance on the hallucination setting while maintaining the same performance in the original setting, which shows that our decoding method is effective in reducing hallucinations without compromising the overall generation quality.</p>
</div>
</section>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>Cross-model Transfer</h2>

<div id="S4.p1" class="ltx_para">
<p id="S4.p1.1" class="ltx_p">One benefit of using the lookback ratio to capture higher-level model patterns for hallucination detection is its potential to better transfer across models. A classifier trained with one model’s lookback ratio could potentially be applied to another model <em id="S4.p1.1.1" class="ltx_emph ltx_font_italic">without</em> retraining, provided correlation between the target model’s attention pattern and that of the original model.
Here, we show that we can transfer a <span id="S4.p1.1.2" class="ltx_text ltx_font_italic">Lookback Lens</span> trained on attention maps from LLaMA-2-7B-Chat to LLaMA-2-13B-Chat without any retraining.</p>
</div>
<figure id="S4.T4" class="ltx_table">
<table id="S4.T4.2" class="ltx_tabular ltx_centering ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S4.T4.2.3.1" class="ltx_tr">
<td id="S4.T4.2.3.1.1" class="ltx_td ltx_align_center ltx_border_tt" rowspan="2"><span id="S4.T4.2.3.1.1.1" class="ltx_text ltx_font_bold">Source</span></td>
<td id="S4.T4.2.3.1.2" class="ltx_td ltx_align_center ltx_border_tt" rowspan="2"><span id="S4.T4.2.3.1.2.1" class="ltx_text ltx_font_bold">Target</span></td>
<td id="S4.T4.2.3.1.3" class="ltx_td ltx_align_center ltx_border_tt"><span id="S4.T4.2.3.1.3.1" class="ltx_text ltx_font_bold">Predefined</span></td>
<td id="S4.T4.2.3.1.4" class="ltx_td ltx_align_center ltx_border_tt"><span id="S4.T4.2.3.1.4.1" class="ltx_text ltx_font_bold">Sliding</span></td>
</tr>
<tr id="S4.T4.2.4.2" class="ltx_tr">
<td id="S4.T4.2.4.2.1" class="ltx_td ltx_align_center"><span id="S4.T4.2.4.2.1.1" class="ltx_text ltx_font_bold">Span</span></td>
<td id="S4.T4.2.4.2.2" class="ltx_td ltx_align_center"><span id="S4.T4.2.4.2.2.1" class="ltx_text ltx_font_bold">Window</span></td>
</tr>
<tr id="S4.T4.1.1" class="ltx_tr">
<td id="S4.T4.1.1.1" class="ltx_td ltx_align_left ltx_border_t" colspan="4"><span id="S4.T4.1.1.1.1" class="ltx_text ltx_font_italic">Lookback Lens: Train 13B <math id="S4.T4.1.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S4.T4.1.1.1.1.m1.1a"><mo stretchy="false" id="S4.T4.1.1.1.1.m1.1.1" xref="S4.T4.1.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S4.T4.1.1.1.1.m1.1b"><ci id="S4.T4.1.1.1.1.m1.1.1.cmml" xref="S4.T4.1.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T4.1.1.1.1.m1.1c">\rightarrow</annotation></semantics></math> Test 13B</span></td>
</tr>
<tr id="S4.T4.2.5.3" class="ltx_tr">
<td id="S4.T4.2.5.3.1" class="ltx_td ltx_align_center ltx_border_t">QA</td>
<td id="S4.T4.2.5.3.2" class="ltx_td ltx_align_center ltx_border_t">Sum.</td>
<td id="S4.T4.2.5.3.3" class="ltx_td ltx_align_center ltx_border_t">84.0</td>
<td id="S4.T4.2.5.3.4" class="ltx_td ltx_align_center ltx_border_t">60.4</td>
</tr>
<tr id="S4.T4.2.6.4" class="ltx_tr">
<td id="S4.T4.2.6.4.1" class="ltx_td ltx_align_center">Sum.</td>
<td id="S4.T4.2.6.4.2" class="ltx_td ltx_align_center">QA</td>
<td id="S4.T4.2.6.4.3" class="ltx_td ltx_align_center">84.3</td>
<td id="S4.T4.2.6.4.4" class="ltx_td ltx_align_center">60.8</td>
</tr>
<tr id="S4.T4.2.7.5" class="ltx_tr">
<td id="S4.T4.2.7.5.1" class="ltx_td ltx_align_center">QA-train</td>
<td id="S4.T4.2.7.5.2" class="ltx_td ltx_align_center">QA</td>
<td id="S4.T4.2.7.5.3" class="ltx_td ltx_align_center">93.3</td>
<td id="S4.T4.2.7.5.4" class="ltx_td ltx_align_center">63.7</td>
</tr>
<tr id="S4.T4.2.2" class="ltx_tr">
<td id="S4.T4.2.2.1" class="ltx_td ltx_align_left ltx_border_t" colspan="4"><span id="S4.T4.2.2.1.1" class="ltx_text ltx_font_italic">Lookback Lens: Train 7B <math id="S4.T4.2.2.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S4.T4.2.2.1.1.m1.1a"><mo stretchy="false" id="S4.T4.2.2.1.1.m1.1.1" xref="S4.T4.2.2.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S4.T4.2.2.1.1.m1.1b"><ci id="S4.T4.2.2.1.1.m1.1.1.cmml" xref="S4.T4.2.2.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T4.2.2.1.1.m1.1c">\rightarrow</annotation></semantics></math> Test 13B</span></td>
</tr>
<tr id="S4.T4.2.8.6" class="ltx_tr">
<td id="S4.T4.2.8.6.1" class="ltx_td ltx_align_center ltx_border_t">QA</td>
<td id="S4.T4.2.8.6.2" class="ltx_td ltx_align_center ltx_border_t">Sum.</td>
<td id="S4.T4.2.8.6.3" class="ltx_td ltx_align_center ltx_border_t">73.5</td>
<td id="S4.T4.2.8.6.4" class="ltx_td ltx_align_center ltx_border_t">58.8</td>
</tr>
<tr id="S4.T4.2.9.7" class="ltx_tr">
<td id="S4.T4.2.9.7.1" class="ltx_td ltx_align_center">Sum.</td>
<td id="S4.T4.2.9.7.2" class="ltx_td ltx_align_center">QA</td>
<td id="S4.T4.2.9.7.3" class="ltx_td ltx_align_center">78.2</td>
<td id="S4.T4.2.9.7.4" class="ltx_td ltx_align_center">60.5</td>
</tr>
<tr id="S4.T4.2.10.8" class="ltx_tr">
<td id="S4.T4.2.10.8.1" class="ltx_td ltx_align_center ltx_border_bb">QA-train</td>
<td id="S4.T4.2.10.8.2" class="ltx_td ltx_align_center ltx_border_bb">QA</td>
<td id="S4.T4.2.10.8.3" class="ltx_td ltx_align_center ltx_border_bb">80.6</td>
<td id="S4.T4.2.10.8.4" class="ltx_td ltx_align_center ltx_border_bb">62.4</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 4: </span>Cross model transfer results on detection tasks.</figcaption>
</figure>
<div id="S4.p2" class="ltx_para">
<p id="S4.p2.3" class="ltx_p">Since the total numbers of attention heads are different in 7B and 13B models, and there is no obvious one-to-one mapping between the heads, we use a linear regression model to map the heads from the 13B model to the heads in 7B model. Concretely, we have 1024 heads in 7B and 1600 heads in 13B. We extract the averaged lookback ratio per head for all the <math id="S4.p2.1.m1.1" class="ltx_Math" alttext="|D|" display="inline"><semantics id="S4.p2.1.m1.1a"><mrow id="S4.p2.1.m1.1.2.2" xref="S4.p2.1.m1.1.2.1.cmml"><mo stretchy="false" id="S4.p2.1.m1.1.2.2.1" xref="S4.p2.1.m1.1.2.1.1.cmml">|</mo><mi id="S4.p2.1.m1.1.1" xref="S4.p2.1.m1.1.1.cmml">D</mi><mo stretchy="false" id="S4.p2.1.m1.1.2.2.2" xref="S4.p2.1.m1.1.2.1.1.cmml">|</mo></mrow><annotation-xml encoding="MathML-Content" id="S4.p2.1.m1.1b"><apply id="S4.p2.1.m1.1.2.1.cmml" xref="S4.p2.1.m1.1.2.2"><abs id="S4.p2.1.m1.1.2.1.1.cmml" xref="S4.p2.1.m1.1.2.2.1"></abs><ci id="S4.p2.1.m1.1.1.cmml" xref="S4.p2.1.m1.1.1">𝐷</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p2.1.m1.1c">|D|</annotation></semantics></math> training examples, resulting in a <math id="S4.p2.2.m2.1" class="ltx_Math" alttext="1024\times|D|" display="inline"><semantics id="S4.p2.2.m2.1a"><mrow id="S4.p2.2.m2.1.2" xref="S4.p2.2.m2.1.2.cmml"><mn id="S4.p2.2.m2.1.2.2" xref="S4.p2.2.m2.1.2.2.cmml">1024</mn><mo lspace="0.222em" rspace="0.222em" id="S4.p2.2.m2.1.2.1" xref="S4.p2.2.m2.1.2.1.cmml">×</mo><mrow id="S4.p2.2.m2.1.2.3.2" xref="S4.p2.2.m2.1.2.3.1.cmml"><mo stretchy="false" id="S4.p2.2.m2.1.2.3.2.1" xref="S4.p2.2.m2.1.2.3.1.1.cmml">|</mo><mi id="S4.p2.2.m2.1.1" xref="S4.p2.2.m2.1.1.cmml">D</mi><mo stretchy="false" id="S4.p2.2.m2.1.2.3.2.2" xref="S4.p2.2.m2.1.2.3.1.1.cmml">|</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.p2.2.m2.1b"><apply id="S4.p2.2.m2.1.2.cmml" xref="S4.p2.2.m2.1.2"><times id="S4.p2.2.m2.1.2.1.cmml" xref="S4.p2.2.m2.1.2.1"></times><cn type="integer" id="S4.p2.2.m2.1.2.2.cmml" xref="S4.p2.2.m2.1.2.2">1024</cn><apply id="S4.p2.2.m2.1.2.3.1.cmml" xref="S4.p2.2.m2.1.2.3.2"><abs id="S4.p2.2.m2.1.2.3.1.1.cmml" xref="S4.p2.2.m2.1.2.3.2.1"></abs><ci id="S4.p2.2.m2.1.1.cmml" xref="S4.p2.2.m2.1.1">𝐷</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p2.2.m2.1c">1024\times|D|</annotation></semantics></math> matrix and a <math id="S4.p2.3.m3.1" class="ltx_Math" alttext="1600\times|D|" display="inline"><semantics id="S4.p2.3.m3.1a"><mrow id="S4.p2.3.m3.1.2" xref="S4.p2.3.m3.1.2.cmml"><mn id="S4.p2.3.m3.1.2.2" xref="S4.p2.3.m3.1.2.2.cmml">1600</mn><mo lspace="0.222em" rspace="0.222em" id="S4.p2.3.m3.1.2.1" xref="S4.p2.3.m3.1.2.1.cmml">×</mo><mrow id="S4.p2.3.m3.1.2.3.2" xref="S4.p2.3.m3.1.2.3.1.cmml"><mo stretchy="false" id="S4.p2.3.m3.1.2.3.2.1" xref="S4.p2.3.m3.1.2.3.1.1.cmml">|</mo><mi id="S4.p2.3.m3.1.1" xref="S4.p2.3.m3.1.1.cmml">D</mi><mo stretchy="false" id="S4.p2.3.m3.1.2.3.2.2" xref="S4.p2.3.m3.1.2.3.1.1.cmml">|</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.p2.3.m3.1b"><apply id="S4.p2.3.m3.1.2.cmml" xref="S4.p2.3.m3.1.2"><times id="S4.p2.3.m3.1.2.1.cmml" xref="S4.p2.3.m3.1.2.1"></times><cn type="integer" id="S4.p2.3.m3.1.2.2.cmml" xref="S4.p2.3.m3.1.2.2">1600</cn><apply id="S4.p2.3.m3.1.2.3.1.cmml" xref="S4.p2.3.m3.1.2.3.2"><abs id="S4.p2.3.m3.1.2.3.1.1.cmml" xref="S4.p2.3.m3.1.2.3.2.1"></abs><ci id="S4.p2.3.m3.1.1.cmml" xref="S4.p2.3.m3.1.1">𝐷</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p2.3.m3.1c">1600\times|D|</annotation></semantics></math> matrix.<span id="footnote2" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">2</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">2</sup><span class="ltx_tag ltx_tag_note">2</span>To ensure that two models are generating the same content when extracting lookback ratio, we decode from 7B and run the 13B model on the 7B outputs.</span></span></span>
We then fit a linear regression model to map the heads to reconstruct the 7B heads from 13B heads. After applying the linear transformation to the lookback ratio from 13B, the transformed heads can be directly used by 7B’s classifiers. See details in Appendix <a href="#A5" title="Appendix E Model Details ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">E</span></a>.</p>
</div>
<div id="S4.p3" class="ltx_para">
<p id="S4.p3.2" class="ltx_p">The detection results are shown in Table <a href="#S4.T4" title="Table 4 ‣ 4 Cross-model Transfer ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>. We first show the same-model (13B<math id="S4.p3.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S4.p3.1.m1.1a"><mo stretchy="false" id="S4.p3.1.m1.1.1" xref="S4.p3.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S4.p3.1.m1.1b"><ci id="S4.p3.1.m1.1.1.cmml" xref="S4.p3.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.p3.1.m1.1c">\rightarrow</annotation></semantics></math>13B) + cross-task transfer result, and the cross-model (7B<math id="S4.p3.2.m2.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S4.p3.2.m2.1a"><mo stretchy="false" id="S4.p3.2.m2.1.1" xref="S4.p3.2.m2.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S4.p3.2.m2.1b"><ci id="S4.p3.2.m2.1.1.cmml" xref="S4.p3.2.m2.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.p3.2.m2.1c">\rightarrow</annotation></semantics></math>13B) + cross-task transfer result. Although cross-model transfer yields slightly worse results compared to same-model transfer, the AUROC scores are still non-trivially high. Consider that doing cross-model + cross-task transfer at the same time may be tough to <span id="S4.p3.2.1" class="ltx_text ltx_font_italic">Lookback Lens</span>, we also include one more setting that does training on 2.5K examples of the NQ training set<span id="footnote3" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">3</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">3</sup><span class="ltx_tag ltx_tag_note">3</span>The NQ-train 2.5K data is annotated in the same method to annotate NQ testing set, as described in Section <a href="#S2.SS2" title="2.2 Experimental Setup ‣ 2 Contextual Hallucinations Detection ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.2</span></a>.</span></span></span> and then transfer to the NQ testing set. We see the cross-model same-task transfer results are even closer to the same-model transfer results.</p>
</div>
<div id="S4.p4" class="ltx_para">
<p id="S4.p4.1" class="ltx_p">Given promising results on detection tasks, we apply cross-model transfer to <span id="S4.p4.1.1" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span>. We conduct the same-task transfer setting: NQ-train (7B) to NQ (13B), and CNN/DM (7B) to XSum (13B).
In Table <a href="#S4.T5" title="Table 5 ‣ 4 Cross-model Transfer ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>, we observe a performance improvement similar to same-model transfer using 13B itself, or using the SoTA NLI model applied on the 13B decoding.
However, on cross-task + cross-model transfer settings: CNN/DM (7B) to NQ (13B), we do not observe significant improvements where we attribute to the larger distribution shift. We leave this challenging setting for future work.</p>
</div>
<figure id="S4.T5" class="ltx_table">
<table id="S4.T5.6" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S4.T5.6.7.1" class="ltx_tr">
<th id="S4.T5.6.7.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt"><span id="S4.T5.6.7.1.1.1" class="ltx_text ltx_font_bold">Method</span></th>
<td id="S4.T5.6.7.1.2" class="ltx_td ltx_align_center ltx_border_tt"><span id="S4.T5.6.7.1.2.1" class="ltx_text ltx_font_bold">XSum</span></td>
<td id="S4.T5.6.7.1.3" class="ltx_td ltx_align_center ltx_border_tt"><span id="S4.T5.6.7.1.3.1" class="ltx_text ltx_font_bold">NQ</span></td>
<td id="S4.T5.6.7.1.4" class="ltx_td ltx_border_tt"></td>
</tr>
<tr id="S4.T5.6.8.2" class="ltx_tr">
<th id="S4.T5.6.8.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">Greedy</th>
<td id="S4.T5.6.8.2.2" class="ltx_td ltx_align_center ltx_border_t">52.9</td>
<td id="S4.T5.6.8.2.3" class="ltx_td ltx_align_center ltx_border_t">74.0</td>
<td id="S4.T5.6.8.2.4" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="S4.T5.6.9.3" class="ltx_tr">
<th id="S4.T5.6.9.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="3"><span id="S4.T5.6.9.3.1.1" class="ltx_text ltx_font_italic">Text-based classifier guided decoding</span></th>
<td id="S4.T5.6.9.3.2" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="S4.T5.1.1" class="ltx_tr">
<th id="S4.T5.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">SoTA NLI<sup id="S4.T5.1.1.1.1" class="ltx_sup">†</sup>
</th>
<td id="S4.T5.1.1.2" class="ltx_td ltx_align_center ltx_border_t">59.6</td>
<td id="S4.T5.1.1.3" class="ltx_td ltx_align_center ltx_border_t">74.4</td>
<td id="S4.T5.1.1.4" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="S4.T5.6.10.4" class="ltx_tr">
<th id="S4.T5.6.10.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" rowspan="2"><span id="S4.T5.6.10.4.1.1" class="ltx_text ltx_font_bold">Method</span></th>
<td id="S4.T5.6.10.4.2" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T5.6.10.4.2.1" class="ltx_text ltx_font_bold">CNN/DM</span></td>
<td id="S4.T5.6.10.4.3" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T5.6.10.4.3.1" class="ltx_text ltx_font_bold">NQ-train</span></td>
<td id="S4.T5.6.10.4.4" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T5.6.10.4.4.1" class="ltx_text ltx_font_bold">CNN/DM</span></td>
</tr>
<tr id="S4.T5.4.4" class="ltx_tr">
<td id="S4.T5.2.2.1" class="ltx_td ltx_align_center">
<math id="S4.T5.2.2.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S4.T5.2.2.1.m1.1a"><mo stretchy="false" id="S4.T5.2.2.1.m1.1.1" xref="S4.T5.2.2.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S4.T5.2.2.1.m1.1b"><ci id="S4.T5.2.2.1.m1.1.1.cmml" xref="S4.T5.2.2.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T5.2.2.1.m1.1c">\rightarrow</annotation></semantics></math><span id="S4.T5.2.2.1.1" class="ltx_text ltx_font_bold">XSum</span>
</td>
<td id="S4.T5.3.3.2" class="ltx_td ltx_align_center">
<math id="S4.T5.3.3.2.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S4.T5.3.3.2.m1.1a"><mo stretchy="false" id="S4.T5.3.3.2.m1.1.1" xref="S4.T5.3.3.2.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S4.T5.3.3.2.m1.1b"><ci id="S4.T5.3.3.2.m1.1.1.cmml" xref="S4.T5.3.3.2.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T5.3.3.2.m1.1c">\rightarrow</annotation></semantics></math><span id="S4.T5.3.3.2.1" class="ltx_text ltx_font_bold">NQ</span>
</td>
<td id="S4.T5.4.4.3" class="ltx_td ltx_align_center">
<math id="S4.T5.4.4.3.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S4.T5.4.4.3.m1.1a"><mo stretchy="false" id="S4.T5.4.4.3.m1.1.1" xref="S4.T5.4.4.3.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S4.T5.4.4.3.m1.1b"><ci id="S4.T5.4.4.3.m1.1.1.cmml" xref="S4.T5.4.4.3.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T5.4.4.3.m1.1c">\rightarrow</annotation></semantics></math><span id="S4.T5.4.4.3.1" class="ltx_text ltx_font_bold">NQ</span>
</td>
</tr>
<tr id="S4.T5.6.11.5" class="ltx_tr">
<th id="S4.T5.6.11.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="3"><span id="S4.T5.6.11.5.1.1" class="ltx_text ltx_font_italic">Lookback Lens guided decoding</span></th>
<td id="S4.T5.6.11.5.2" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="S4.T5.5.5" class="ltx_tr">
<th id="S4.T5.5.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">13B <math id="S4.T5.5.5.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S4.T5.5.5.1.m1.1a"><mo stretchy="false" id="S4.T5.5.5.1.m1.1.1" xref="S4.T5.5.5.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S4.T5.5.5.1.m1.1b"><ci id="S4.T5.5.5.1.m1.1.1.cmml" xref="S4.T5.5.5.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T5.5.5.1.m1.1c">\rightarrow</annotation></semantics></math> 13B</th>
<td id="S4.T5.5.5.2" class="ltx_td ltx_align_center ltx_border_t">57.9</td>
<td id="S4.T5.5.5.3" class="ltx_td ltx_align_center ltx_border_t">75.6</td>
<td id="S4.T5.5.5.4" class="ltx_td ltx_align_center ltx_border_t">74.8</td>
</tr>
<tr id="S4.T5.6.6" class="ltx_tr">
<th id="S4.T5.6.6.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb">7B <math id="S4.T5.6.6.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S4.T5.6.6.1.m1.1a"><mo stretchy="false" id="S4.T5.6.6.1.m1.1.1" xref="S4.T5.6.6.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S4.T5.6.6.1.m1.1b"><ci id="S4.T5.6.6.1.m1.1.1.cmml" xref="S4.T5.6.6.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T5.6.6.1.m1.1c">\rightarrow</annotation></semantics></math> 13B</th>
<td id="S4.T5.6.6.2" class="ltx_td ltx_align_center ltx_border_bb">56.1</td>
<td id="S4.T5.6.6.3" class="ltx_td ltx_align_center ltx_border_bb">76.4</td>
<td id="S4.T5.6.6.4" class="ltx_td ltx_align_center ltx_border_bb">73.7</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 5: </span>Cross model transfer from LLaMA-2-7B-chat to LLaMA-2-13B-chat using greedy decoding and classifier guided sampling methods with chunk size 8.</figcaption>
</figure>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Discussions and Ablations</h2>

<div id="S5.p1" class="ltx_para">
<p id="S5.p1.1" class="ltx_p">In this section, we further conduct various experiments and ablation studies on the <span id="S5.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> and its corresponding classifier guided decoding.</p>
</div>
<section id="S5.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Effect of Chunk Size</h4>

<div id="S5.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="S5.SS0.SSS0.Px1.p1.1" class="ltx_p">In Section <a href="#S3.SS3" title="3.3 Main Results ‣ 3 Contextual Hallucinations Mitigation ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.3</span></a> (Table <a href="#S3.T3" title="Table 3 ‣ 3.3 Main Results ‣ 3 Contextual Hallucinations Mitigation ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>), we experiment with chunk size = 8. Here, we study the effect of varying chunk sizes, from 4, 8, to 16. We see that there is a slight trend that <span id="S5.SS0.SSS0.Px1.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> guided decoding prefers shorter chunk size for NQ and longer chunk size for XSum. However, in general the improvements are consistent across different chunk sizes, thus reducing the need to optimize for chunk sizes.</p>
</div>
<figure id="S5.T6" class="ltx_table">
<table id="S5.T6.1" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S5.T6.1.2.1" class="ltx_tr">
<th id="S5.T6.1.2.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt"><span id="S5.T6.1.2.1.1.1" class="ltx_text ltx_font_bold">Method</span></th>
<td id="S5.T6.1.2.1.2" class="ltx_td ltx_align_center ltx_border_tt" colspan="3"><span id="S5.T6.1.2.1.2.1" class="ltx_text ltx_font_bold">NQ</span></td>
<td id="S5.T6.1.2.1.3" class="ltx_td ltx_align_center ltx_border_tt" colspan="3"><span id="S5.T6.1.2.1.3.1" class="ltx_text ltx_font_bold">XSum</span></td>
</tr>
<tr id="S5.T6.1.3.2" class="ltx_tr">
<th id="S5.T6.1.3.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Chunk size=</th>
<td id="S5.T6.1.3.2.2" class="ltx_td ltx_align_center ltx_border_t">4</td>
<td id="S5.T6.1.3.2.3" class="ltx_td ltx_align_center ltx_border_t">8</td>
<td id="S5.T6.1.3.2.4" class="ltx_td ltx_align_center ltx_border_t">16</td>
<td id="S5.T6.1.3.2.5" class="ltx_td ltx_align_center ltx_border_t">4</td>
<td id="S5.T6.1.3.2.6" class="ltx_td ltx_align_center ltx_border_t">8</td>
<td id="S5.T6.1.3.2.7" class="ltx_td ltx_align_center ltx_border_t">16</td>
</tr>
<tr id="S5.T6.1.4.3" class="ltx_tr">
<th id="S5.T6.1.4.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">Greedy</th>
<td id="S5.T6.1.4.3.2" class="ltx_td ltx_align_center ltx_border_t" colspan="3">71.2</td>
<td id="S5.T6.1.4.3.3" class="ltx_td ltx_align_center ltx_border_t" colspan="3">49.0</td>
</tr>
<tr id="S5.T6.1.5.4" class="ltx_tr">
<th id="S5.T6.1.5.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="7"><span id="S5.T6.1.5.4.1.1" class="ltx_text ltx_font_italic">Text-based classifier guided decoding</span></th>
</tr>
<tr id="S5.T6.1.1" class="ltx_tr">
<th id="S5.T6.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">SoTA NLI<sup id="S5.T6.1.1.1.1" class="ltx_sup">†</sup>
</th>
<td id="S5.T6.1.1.2" class="ltx_td ltx_align_center ltx_border_t">73.7</td>
<td id="S5.T6.1.1.3" class="ltx_td ltx_align_center ltx_border_t">74.2</td>
<td id="S5.T6.1.1.4" class="ltx_td ltx_align_center ltx_border_t">74.4</td>
<td id="S5.T6.1.1.5" class="ltx_td ltx_align_center ltx_border_t">57.3</td>
<td id="S5.T6.1.1.6" class="ltx_td ltx_align_center ltx_border_t">59.0</td>
<td id="S5.T6.1.1.7" class="ltx_td ltx_align_center ltx_border_t">62.1</td>
</tr>
<tr id="S5.T6.1.6.5" class="ltx_tr">
<th id="S5.T6.1.6.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="7"><span id="S5.T6.1.6.5.1.1" class="ltx_text ltx_font_italic">Hidden states based classifier guided decoding</span></th>
</tr>
<tr id="S5.T6.1.7.6" class="ltx_tr">
<th id="S5.T6.1.7.6.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">32nd layer</th>
<td id="S5.T6.1.7.6.2" class="ltx_td ltx_align_center ltx_border_t">72.6</td>
<td id="S5.T6.1.7.6.3" class="ltx_td ltx_align_center ltx_border_t">73.9</td>
<td id="S5.T6.1.7.6.4" class="ltx_td ltx_align_center ltx_border_t">72.7</td>
<td id="S5.T6.1.7.6.5" class="ltx_td ltx_align_center ltx_border_t">48.9</td>
<td id="S5.T6.1.7.6.6" class="ltx_td ltx_align_center ltx_border_t">48.3</td>
<td id="S5.T6.1.7.6.7" class="ltx_td ltx_align_center ltx_border_t">48.3</td>
</tr>
<tr id="S5.T6.1.8.7" class="ltx_tr">
<th id="S5.T6.1.8.7.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">28th layer</th>
<td id="S5.T6.1.8.7.2" class="ltx_td ltx_align_center">72.9</td>
<td id="S5.T6.1.8.7.3" class="ltx_td ltx_align_center">73.0</td>
<td id="S5.T6.1.8.7.4" class="ltx_td ltx_align_center">74.1</td>
<td id="S5.T6.1.8.7.5" class="ltx_td ltx_align_center">47.2</td>
<td id="S5.T6.1.8.7.6" class="ltx_td ltx_align_center">48.9</td>
<td id="S5.T6.1.8.7.7" class="ltx_td ltx_align_center">47.1</td>
</tr>
<tr id="S5.T6.1.9.8" class="ltx_tr">
<th id="S5.T6.1.9.8.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">24th layer</th>
<td id="S5.T6.1.9.8.2" class="ltx_td ltx_align_center">75.0</td>
<td id="S5.T6.1.9.8.3" class="ltx_td ltx_align_center">73.9</td>
<td id="S5.T6.1.9.8.4" class="ltx_td ltx_align_center">72.5</td>
<td id="S5.T6.1.9.8.5" class="ltx_td ltx_align_center">47.6</td>
<td id="S5.T6.1.9.8.6" class="ltx_td ltx_align_center">47.5</td>
<td id="S5.T6.1.9.8.7" class="ltx_td ltx_align_center">51.2</td>
</tr>
<tr id="S5.T6.1.10.9" class="ltx_tr">
<th id="S5.T6.1.10.9.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="7"><span id="S5.T6.1.10.9.1.1" class="ltx_text ltx_font_italic">Lookback Lens guided decoding</span></th>
</tr>
<tr id="S5.T6.1.11.10" class="ltx_tr">
<th id="S5.T6.1.11.10.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb ltx_border_t">Ours</th>
<td id="S5.T6.1.11.10.2" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">75.4</td>
<td id="S5.T6.1.11.10.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">74.2</td>
<td id="S5.T6.1.11.10.4" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">74.3</td>
<td id="S5.T6.1.11.10.5" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">53.2</td>
<td id="S5.T6.1.11.10.6" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">58.6</td>
<td id="S5.T6.1.11.10.7" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">57.7</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 6: </span>Performance comparison on various datasets using different methods and chunk sizes.</figcaption>
</figure>
</section>
<section id="S5.SS0.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Predictive Power of Different Heads</h4>

<figure id="S5.T7" class="ltx_table">
<div id="S5.T7.4" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:258.8pt;height:144pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(0.0pt,0.0pt) scale(1,1) ;">
<p id="S5.T7.4.4" class="ltx_p"><span id="S5.T7.4.4.4" class="ltx_text">
<span id="S5.T7.4.4.4.4" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<span class="ltx_tbody">
<span id="S5.T7.4.4.4.4.5.1" class="ltx_tr">
<span id="S5.T7.4.4.4.4.5.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt ltx_rowspan ltx_rowspan_2"><span id="S5.T7.4.4.4.4.5.1.1.1" class="ltx_text ltx_font_bold">Method</span></span>
<span id="S5.T7.4.4.4.4.5.1.2" class="ltx_td ltx_align_center ltx_border_tt ltx_colspan ltx_colspan_6"><span id="S5.T7.4.4.4.4.5.1.2.1" class="ltx_text ltx_font_bold">Predefined Span</span></span></span>
<span id="S5.T7.2.2.2.2.2" class="ltx_tr">
<span id="S5.T7.1.1.1.1.1.1" class="ltx_td ltx_align_center ltx_border_t ltx_colspan ltx_colspan_3"><span id="S5.T7.1.1.1.1.1.1.1" class="ltx_text ltx_font_bold">QA <math id="S5.T7.1.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S5.T7.1.1.1.1.1.1.1.m1.1a"><mo stretchy="false" id="S5.T7.1.1.1.1.1.1.1.m1.1.1" xref="S5.T7.1.1.1.1.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S5.T7.1.1.1.1.1.1.1.m1.1b"><ci id="S5.T7.1.1.1.1.1.1.1.m1.1.1.cmml" xref="S5.T7.1.1.1.1.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.T7.1.1.1.1.1.1.1.m1.1c">\rightarrow</annotation></semantics></math> Sum.</span></span>
<span id="S5.T7.2.2.2.2.2.2" class="ltx_td ltx_align_center ltx_border_t ltx_colspan ltx_colspan_3"><span id="S5.T7.2.2.2.2.2.2.1" class="ltx_text ltx_font_bold">Sum. <math id="S5.T7.2.2.2.2.2.2.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S5.T7.2.2.2.2.2.2.1.m1.1a"><mo stretchy="false" id="S5.T7.2.2.2.2.2.2.1.m1.1.1" xref="S5.T7.2.2.2.2.2.2.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S5.T7.2.2.2.2.2.2.1.m1.1b"><ci id="S5.T7.2.2.2.2.2.2.1.m1.1.1.cmml" xref="S5.T7.2.2.2.2.2.2.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.T7.2.2.2.2.2.2.1.m1.1c">\rightarrow</annotation></semantics></math> QA</span></span></span>
<span id="S5.T7.4.4.4.4.6.2" class="ltx_tr">
<span id="S5.T7.4.4.4.4.6.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">All heads</span>
<span id="S5.T7.4.4.4.4.6.2.2" class="ltx_td ltx_align_center ltx_border_t ltx_colspan ltx_colspan_3">85.3</span>
<span id="S5.T7.4.4.4.4.6.2.3" class="ltx_td ltx_align_center ltx_border_t ltx_colspan ltx_colspan_3">82.0</span></span>
<span id="S5.T7.3.3.3.3.3" class="ltx_tr">
<span id="S5.T7.3.3.3.3.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t ltx_colspan ltx_colspan_7"><span id="S5.T7.3.3.3.3.3.1.1" class="ltx_text ltx_font_italic">Top-<math id="S5.T7.3.3.3.3.3.1.1.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.T7.3.3.3.3.3.1.1.m1.1a"><mi id="S5.T7.3.3.3.3.3.1.1.m1.1.1" xref="S5.T7.3.3.3.3.3.1.1.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.T7.3.3.3.3.3.1.1.m1.1b"><ci id="S5.T7.3.3.3.3.3.1.1.m1.1.1.cmml" xref="S5.T7.3.3.3.3.3.1.1.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.T7.3.3.3.3.3.1.1.m1.1c">k</annotation></semantics></math> heads only</span></span></span>
<span id="S5.T7.4.4.4.4.4" class="ltx_tr">
<span id="S5.T7.4.4.4.4.4.1" class="ltx_td ltx_align_right ltx_th ltx_th_row"><span id="S5.T7.4.4.4.4.4.1.1" class="ltx_text ltx_font_italic">with <math id="S5.T7.4.4.4.4.4.1.1.m1.1" class="ltx_Math" alttext="k=" display="inline"><semantics id="S5.T7.4.4.4.4.4.1.1.m1.1a"><mrow id="S5.T7.4.4.4.4.4.1.1.m1.1.1" xref="S5.T7.4.4.4.4.4.1.1.m1.1.1.cmml"><mi id="S5.T7.4.4.4.4.4.1.1.m1.1.1.2" xref="S5.T7.4.4.4.4.4.1.1.m1.1.1.2.cmml">k</mi><mo id="S5.T7.4.4.4.4.4.1.1.m1.1.1.1" xref="S5.T7.4.4.4.4.4.1.1.m1.1.1.1.cmml">=</mo><mi id="S5.T7.4.4.4.4.4.1.1.m1.1.1.3" xref="S5.T7.4.4.4.4.4.1.1.m1.1.1.3.cmml"></mi></mrow><annotation-xml encoding="MathML-Content" id="S5.T7.4.4.4.4.4.1.1.m1.1b"><apply id="S5.T7.4.4.4.4.4.1.1.m1.1.1.cmml" xref="S5.T7.4.4.4.4.4.1.1.m1.1.1"><eq id="S5.T7.4.4.4.4.4.1.1.m1.1.1.1.cmml" xref="S5.T7.4.4.4.4.4.1.1.m1.1.1.1"></eq><ci id="S5.T7.4.4.4.4.4.1.1.m1.1.1.2.cmml" xref="S5.T7.4.4.4.4.4.1.1.m1.1.1.2">𝑘</ci><csymbol cd="latexml" id="S5.T7.4.4.4.4.4.1.1.m1.1.1.3.cmml" xref="S5.T7.4.4.4.4.4.1.1.m1.1.1.3">absent</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T7.4.4.4.4.4.1.1.m1.1c">k=</annotation></semantics></math></span></span>
<span id="S5.T7.4.4.4.4.4.2" class="ltx_td ltx_align_center ltx_border_t">10</span>
<span id="S5.T7.4.4.4.4.4.3" class="ltx_td ltx_align_center ltx_border_t">50</span>
<span id="S5.T7.4.4.4.4.4.4" class="ltx_td ltx_align_center ltx_border_t">100</span>
<span id="S5.T7.4.4.4.4.4.5" class="ltx_td ltx_align_center ltx_border_t">10</span>
<span id="S5.T7.4.4.4.4.4.6" class="ltx_td ltx_align_center ltx_border_t">50</span>
<span id="S5.T7.4.4.4.4.4.7" class="ltx_td ltx_align_center ltx_border_t">100</span></span>
<span id="S5.T7.4.4.4.4.7.3" class="ltx_tr">
<span id="S5.T7.4.4.4.4.7.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">Largest mag.</span>
<span id="S5.T7.4.4.4.4.7.3.2" class="ltx_td ltx_align_center ltx_border_t">71.2</span>
<span id="S5.T7.4.4.4.4.7.3.3" class="ltx_td ltx_align_center ltx_border_t">82.3</span>
<span id="S5.T7.4.4.4.4.7.3.4" class="ltx_td ltx_align_center ltx_border_t">82.8</span>
<span id="S5.T7.4.4.4.4.7.3.5" class="ltx_td ltx_align_center ltx_border_t">79.2</span>
<span id="S5.T7.4.4.4.4.7.3.6" class="ltx_td ltx_align_center ltx_border_t">80.3</span>
<span id="S5.T7.4.4.4.4.7.3.7" class="ltx_td ltx_align_center ltx_border_t">81.1</span></span>
<span id="S5.T7.4.4.4.4.8.4" class="ltx_tr">
<span id="S5.T7.4.4.4.4.8.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Most positive</span>
<span id="S5.T7.4.4.4.4.8.4.2" class="ltx_td ltx_align_center">65.1</span>
<span id="S5.T7.4.4.4.4.8.4.3" class="ltx_td ltx_align_center">74.9</span>
<span id="S5.T7.4.4.4.4.8.4.4" class="ltx_td ltx_align_center">75.4</span>
<span id="S5.T7.4.4.4.4.8.4.5" class="ltx_td ltx_align_center">66.3</span>
<span id="S5.T7.4.4.4.4.8.4.6" class="ltx_td ltx_align_center">70.3</span>
<span id="S5.T7.4.4.4.4.8.4.7" class="ltx_td ltx_align_center">74.4</span></span>
<span id="S5.T7.4.4.4.4.9.5" class="ltx_tr">
<span id="S5.T7.4.4.4.4.9.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb">Most negative</span>
<span id="S5.T7.4.4.4.4.9.5.2" class="ltx_td ltx_align_center ltx_border_bb">59.5</span>
<span id="S5.T7.4.4.4.4.9.5.3" class="ltx_td ltx_align_center ltx_border_bb">67.5</span>
<span id="S5.T7.4.4.4.4.9.5.4" class="ltx_td ltx_align_center ltx_border_bb">74.4</span>
<span id="S5.T7.4.4.4.4.9.5.5" class="ltx_td ltx_align_center ltx_border_bb">66.4</span>
<span id="S5.T7.4.4.4.4.9.5.6" class="ltx_td ltx_align_center ltx_border_bb">70.2</span>
<span id="S5.T7.4.4.4.4.9.5.7" class="ltx_td ltx_align_center ltx_border_bb">73.0</span></span>
</span>
</span></span></p>
</span></div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 7: </span>Cross-task transfer AUROC using top-<math id="S5.T7.7.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.T7.7.m1.1b"><mi id="S5.T7.7.m1.1.1" xref="S5.T7.7.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.T7.7.m1.1c"><ci id="S5.T7.7.m1.1.1.cmml" xref="S5.T7.7.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.T7.7.m1.1d">k</annotation></semantics></math> attention heads selected according to: coefficients with the largest magnitude (largest mag.), most positive, and most negative. We consider <math id="S5.T7.8.m2.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.T7.8.m2.1b"><mi id="S5.T7.8.m2.1.1" xref="S5.T7.8.m2.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.T7.8.m2.1c"><ci id="S5.T7.8.m2.1.1.cmml" xref="S5.T7.8.m2.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.T7.8.m2.1d">k</annotation></semantics></math> = 10, 50, and 100.</figcaption>
</figure>
<div id="S5.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="S5.SS0.SSS0.Px2.p1.1" class="ltx_p">In the aforementioned experiments, we utilize all attention heads to train the <span id="S5.SS0.SSS0.Px2.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span>. We are thus interested in
how the predictive power is distributed among different heads in making predictions. That is, how much performance can we recover if we only utilize a subset of heads?
To answer this, we use the coefficients in the linear classifier of the <span id="S5.SS0.SSS0.Px2.p1.1.2" class="ltx_text ltx_font_italic">Lookback Lens</span> (in Section <a href="#S2" title="2 Contextual Hallucinations Detection ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>) to estimate the importance of each head in detecting hallucinations.</p>
</div>
<div id="S5.SS0.SSS0.Px2.p2" class="ltx_para">
<p id="S5.SS0.SSS0.Px2.p2.2" class="ltx_p">In Table <a href="#S5.T7" title="Table 7 ‣ Predictive Power of Different Heads ‣ 5 Discussions and Ablations ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a>, we show the results on detection tasks achieved by different detectors trained using only a subset of top-<math id="S5.SS0.SSS0.Px2.p2.1.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.SS0.SSS0.Px2.p2.1.m1.1a"><mi id="S5.SS0.SSS0.Px2.p2.1.m1.1.1" xref="S5.SS0.SSS0.Px2.p2.1.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.SS0.SSS0.Px2.p2.1.m1.1b"><ci id="S5.SS0.SSS0.Px2.p2.1.m1.1.1.cmml" xref="S5.SS0.SSS0.Px2.p2.1.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS0.SSS0.Px2.p2.1.m1.1c">k</annotation></semantics></math> heads with the largest magnitude of coefficients in the original <span id="S5.SS0.SSS0.Px2.p2.2.1" class="ltx_text ltx_font_italic">Lookback Lens</span> trained will all heads.
The results show that the predictive power is not concentrated only on a subset of heads.
Using only top-10 heads is worse than using all heads, and increasing <math id="S5.SS0.SSS0.Px2.p2.2.m2.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.SS0.SSS0.Px2.p2.2.m2.1a"><mi id="S5.SS0.SSS0.Px2.p2.2.m2.1.1" xref="S5.SS0.SSS0.Px2.p2.2.m2.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.SS0.SSS0.Px2.p2.2.m2.1b"><ci id="S5.SS0.SSS0.Px2.p2.2.m2.1.1.cmml" xref="S5.SS0.SSS0.Px2.p2.2.m2.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS0.SSS0.Px2.p2.2.m2.1c">k</annotation></semantics></math> consistently improves performance and top-100 heads largely recover the model’s performance using all heads.</p>
</div>
<div id="S5.SS0.SSS0.Px2.p3" class="ltx_para">
<p id="S5.SS0.SSS0.Px2.p3.2" class="ltx_p">More interestingly, we also include the results that only select the top-<math id="S5.SS0.SSS0.Px2.p3.1.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.SS0.SSS0.Px2.p3.1.m1.1a"><mi id="S5.SS0.SSS0.Px2.p3.1.m1.1.1" xref="S5.SS0.SSS0.Px2.p3.1.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.SS0.SSS0.Px2.p3.1.m1.1b"><ci id="S5.SS0.SSS0.Px2.p3.1.m1.1.1.cmml" xref="S5.SS0.SSS0.Px2.p3.1.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS0.SSS0.Px2.p3.1.m1.1c">k</annotation></semantics></math> heads among the heads with most positive/negative coefficients, which are positive/negatively correlated to factuality.
On the heads with positive coefficients, higher lookback ratio (i.e., when the heads attend at the context more) indicates higher factuality and less hallucination; conversely, heads with negative coefficients suggest a lower lookback ratio (i.e., attending to generated tokens more) is more likely to be truthful.
Table <a href="#S5.T7" title="Table 7 ‣ Predictive Power of Different Heads ‣ 5 Discussions and Ablations ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a> shows that none of positive or negative heads alone can be on par with using the top-<math id="S5.SS0.SSS0.Px2.p3.2.m2.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.SS0.SSS0.Px2.p3.2.m2.1a"><mi id="S5.SS0.SSS0.Px2.p3.2.m2.1.1" xref="S5.SS0.SSS0.Px2.p3.2.m2.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.SS0.SSS0.Px2.p3.2.m2.1b"><ci id="S5.SS0.SSS0.Px2.p3.2.m2.1.1.cmml" xref="S5.SS0.SSS0.Px2.p3.2.m2.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS0.SSS0.Px2.p3.2.m2.1c">k</annotation></semantics></math> largest magnitude heads.
This result implies that both positive and negative heads are critical for a model to generate factual responses. We conjecture that the positive heads may specialize at context grounding, and thus higher lookback ratio on these heads leads to more factual response. On the other hand, the negative heads may be critical at ensuring consistency in its own generation, and thus should attend to the generated tokens more.
We leave further investigation on this interesting balance for future work. Meanwhile, we visualize the lookback ratio of positive/negative heads in Appendix <a href="#A6" title="Appendix F Visualization ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">F</span></a>.</p>
</div>
<figure id="S5.F3" class="ltx_figure"><img src="/html/2407.07071/assets/x3.png" id="S5.F3.1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="226" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 3: </span>Qualitative example on XSum using the LLaMA-2-7B-Chat model with greedy decoding and <span id="S5.F3.4.1" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span>. The numbers in the parenthesis show the predicted scores from the <span id="S5.F3.5.2" class="ltx_text ltx_font_italic">Lookback Lens</span>.</figcaption>
</figure>
<figure id="S5.T8" class="ltx_table">
<table id="S5.T8.2" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S5.T8.2.3.1" class="ltx_tr">
<th id="S5.T8.2.3.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt" rowspan="2"><span id="S5.T8.2.3.1.1.1" class="ltx_text ltx_font_bold">Layers</span></th>
<th id="S5.T8.2.3.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt" colspan="2"><span id="S5.T8.2.3.1.2.1" class="ltx_text ltx_font_bold">Predefined Span</span></th>
</tr>
<tr id="S5.T8.2.2" class="ltx_tr">
<th id="S5.T8.1.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_t"><span id="S5.T8.1.1.1.1" class="ltx_text ltx_font_bold">QA <math id="S5.T8.1.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S5.T8.1.1.1.1.m1.1a"><mo stretchy="false" id="S5.T8.1.1.1.1.m1.1.1" xref="S5.T8.1.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S5.T8.1.1.1.1.m1.1b"><ci id="S5.T8.1.1.1.1.m1.1.1.cmml" xref="S5.T8.1.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.T8.1.1.1.1.m1.1c">\rightarrow</annotation></semantics></math> Sum.</span></th>
<th id="S5.T8.2.2.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_t"><span id="S5.T8.2.2.2.1" class="ltx_text ltx_font_bold">Sum. <math id="S5.T8.2.2.2.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="S5.T8.2.2.2.1.m1.1a"><mo stretchy="false" id="S5.T8.2.2.2.1.m1.1.1" xref="S5.T8.2.2.2.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="S5.T8.2.2.2.1.m1.1b"><ci id="S5.T8.2.2.2.1.m1.1.1.cmml" xref="S5.T8.2.2.2.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.T8.2.2.2.1.m1.1c">\rightarrow</annotation></semantics></math> QA</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S5.T8.2.4.1" class="ltx_tr">
<th id="S5.T8.2.4.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">Layer 1-4</th>
<td id="S5.T8.2.4.1.2" class="ltx_td ltx_align_center ltx_border_t">69.6</td>
<td id="S5.T8.2.4.1.3" class="ltx_td ltx_align_center ltx_border_t">64.0</td>
</tr>
<tr id="S5.T8.2.5.2" class="ltx_tr">
<th id="S5.T8.2.5.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Layer 5-8</th>
<td id="S5.T8.2.5.2.2" class="ltx_td ltx_align_center">75.6</td>
<td id="S5.T8.2.5.2.3" class="ltx_td ltx_align_center">70.1</td>
</tr>
<tr id="S5.T8.2.6.3" class="ltx_tr">
<th id="S5.T8.2.6.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Layer 9-12</th>
<td id="S5.T8.2.6.3.2" class="ltx_td ltx_align_center">75.4</td>
<td id="S5.T8.2.6.3.3" class="ltx_td ltx_align_center">68.3</td>
</tr>
<tr id="S5.T8.2.7.4" class="ltx_tr">
<th id="S5.T8.2.7.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Layer 13-16</th>
<td id="S5.T8.2.7.4.2" class="ltx_td ltx_align_center">81.2</td>
<td id="S5.T8.2.7.4.3" class="ltx_td ltx_align_center">78.2</td>
</tr>
<tr id="S5.T8.2.8.5" class="ltx_tr">
<th id="S5.T8.2.8.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Layer 17-20</th>
<td id="S5.T8.2.8.5.2" class="ltx_td ltx_align_center">80.8</td>
<td id="S5.T8.2.8.5.3" class="ltx_td ltx_align_center">78.2</td>
</tr>
<tr id="S5.T8.2.9.6" class="ltx_tr">
<th id="S5.T8.2.9.6.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Layer 21-24</th>
<td id="S5.T8.2.9.6.2" class="ltx_td ltx_align_center">64.4</td>
<td id="S5.T8.2.9.6.3" class="ltx_td ltx_align_center">73.1</td>
</tr>
<tr id="S5.T8.2.10.7" class="ltx_tr">
<th id="S5.T8.2.10.7.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Layer 25-28</th>
<td id="S5.T8.2.10.7.2" class="ltx_td ltx_align_center">66.0</td>
<td id="S5.T8.2.10.7.3" class="ltx_td ltx_align_center">74.4</td>
</tr>
<tr id="S5.T8.2.11.8" class="ltx_tr">
<th id="S5.T8.2.11.8.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Layer 29-32</th>
<td id="S5.T8.2.11.8.2" class="ltx_td ltx_align_center">66.4</td>
<td id="S5.T8.2.11.8.3" class="ltx_td ltx_align_center">71.4</td>
</tr>
<tr id="S5.T8.2.12.9" class="ltx_tr">
<th id="S5.T8.2.12.9.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb ltx_border_t">Layer 1-32</th>
<td id="S5.T8.2.12.9.2" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">85.3</td>
<td id="S5.T8.2.12.9.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">82.0</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 8: </span>Cross-task transfer AUROC among layers.</figcaption>
</figure>
</section>
<section id="S5.SS0.SSS0.Px3" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Reducing Number of Layers</h4>

<div id="S5.SS0.SSS0.Px3.p1" class="ltx_para">
<p id="S5.SS0.SSS0.Px3.p1.1" class="ltx_p">We experiment with using only a subset of layers for <span id="S5.SS0.SSS0.Px3.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span>, as shown in Table <a href="#S5.T8" title="Table 8 ‣ Predictive Power of Different Heads ‣ 5 Discussions and Ablations ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a>. We can see that the predictive power is not concentrated in any subset of layers, as none of them can recover the performance of the full model that uses all layers. However, we observe that the middle layers (13-16, 17-20) are slightly more useful than other layers.</p>
</div>
</section>
<section id="S5.SS0.SSS0.Px4" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Qualitative Study</h4>

<div id="S5.SS0.SSS0.Px4.p1" class="ltx_para">
<p id="S5.SS0.SSS0.Px4.p1.1" class="ltx_p">We show qualitative examples from XSum in Figure <a href="#S5.F3" title="Figure 3 ‣ Predictive Power of Different Heads ‣ 5 Discussions and Ablations ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a> to illustrate how <span id="S5.SS0.SSS0.Px4.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> guided decoding improves performance. Greedy decoding from LLaMA-2-7B-Chat results in a hallucination, i.e. <span id="S5.SS0.SSS0.Px4.p1.1.2" class="ltx_text ltx_font_italic">$100m (£64m)</span>, that does not exist in the input document. However, the <span id="S5.SS0.SSS0.Px4.p1.1.3" class="ltx_text ltx_font_italic">Lookback Lens</span> is able to assign low scores for the chunk candidates that have contextual hallucinations (as marked in red). Therefore, <span id="S5.SS0.SSS0.Px4.p1.1.4" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span> is able to help the model generate a summary that is factual to the given context.</p>
</div>
</section>
</section>
<section id="S6" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">6 </span>Related Work</h2>

<section id="S6.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Hallucinations in LLMs.</h4>

<div id="S6.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px1.p1.1" class="ltx_p"><cite class="ltx_cite ltx_citemacro_citet">Simhi et al. (<a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite> defined <em id="S6.SS0.SSS0.Px1.p1.1.1" class="ltx_emph ltx_font_italic">close-book hallucination</em> vs <em id="S6.SS0.SSS0.Px1.p1.1.2" class="ltx_emph ltx_font_italic">open-book hallucination</em> for settings of relying on parametric knowledge vs knowledge in context. We term <em id="S6.SS0.SSS0.Px1.p1.1.3" class="ltx_emph ltx_font_italic">open-book hallucination</em> as <em id="S6.SS0.SSS0.Px1.p1.1.4" class="ltx_emph ltx_font_italic">contextual hallucination</em> for better clarity. Previous studies in hallucinations primarily focus on close-book hallucinations <cite class="ltx_cite ltx_citemacro_cite">Chen et al. (<a href="#bib.bib4" title="" class="ltx_ref">2023</a>); Min et al. (<a href="#bib.bib29" title="" class="ltx_ref">2023</a>); Chern et al. (<a href="#bib.bib7" title="" class="ltx_ref">2023</a>)</cite> and
their detection <cite class="ltx_cite ltx_citemacro_citep">(Azaria and Mitchell, <a href="#bib.bib1" title="" class="ltx_ref">2023</a>; Simhi et al., <a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite> and mitigation <cite class="ltx_cite ltx_citemacro_cite">Li et al. (<a href="#bib.bib22" title="" class="ltx_ref">2024</a>); Chuang et al. (<a href="#bib.bib9" title="" class="ltx_ref">2024</a>); Chen et al. (<a href="#bib.bib5" title="" class="ltx_ref">2024a</a>); Zhang et al. (<a href="#bib.bib45" title="" class="ltx_ref">2024</a>)</cite>. Most of the studies focus on leveraging LLM’s internal representations, such as hidden states <cite class="ltx_cite ltx_citemacro_citep">(Burns et al., <a href="#bib.bib3" title="" class="ltx_ref">2023</a>; Azaria and Mitchell, <a href="#bib.bib1" title="" class="ltx_ref">2023</a>)</cite>, MLP outputs <cite class="ltx_cite ltx_citemacro_citep">(Zhang et al., <a href="#bib.bib45" title="" class="ltx_ref">2024</a>; Simhi et al., <a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite>, attention block outputs <cite class="ltx_cite ltx_citemacro_citep">(Zhang et al., <a href="#bib.bib45" title="" class="ltx_ref">2024</a>; Simhi et al., <a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite> and attention head outputs <cite class="ltx_cite ltx_citemacro_citep">(Li et al., <a href="#bib.bib22" title="" class="ltx_ref">2024</a>; Chen et al., <a href="#bib.bib6" title="" class="ltx_ref">2024b</a>; Simhi et al., <a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite>. Our work, however, focuses on contextual hallucinations, where models produce content inconsistent with the provided context <cite class="ltx_cite ltx_citemacro_cite">Maynez et al. (<a href="#bib.bib28" title="" class="ltx_ref">2020</a>); Fabbri et al. (<a href="#bib.bib12" title="" class="ltx_ref">2021</a>); Shi et al. (<a href="#bib.bib37" title="" class="ltx_ref">2023</a>)</cite>. Thus, different from prior studies, we focus on the attention maps instead of internal representations, as we believe that the attention maps patterns record how the LLM process the given contextual information.
Most of the prior studies treat detection and mitigation as two separate tasks, expect for <cite class="ltx_cite ltx_citemacro_citet">Simhi et al. (<a href="#bib.bib38" title="" class="ltx_ref">2024</a>); Chen et al. (<a href="#bib.bib5" title="" class="ltx_ref">2024a</a>)</cite>. Our work focuses not only on detection, but also tries to incorporate the detector into the decoding process to further mitigate the contextual hallucinations.
Recently, <cite class="ltx_cite ltx_citemacro_citet">Simhi et al. (<a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite> also explored detecting and mitigating both close-book and open-book hallucinations. However, their open-book hallucination setting is limited to DisentQA <cite class="ltx_cite ltx_citemacro_citep">(Neeman et al., <a href="#bib.bib31" title="" class="ltx_ref">2023</a>)</cite>, which creates knowledge conflicts between parametric knowledge and given context. In contrast, we focus on LLaMA-2’s naturally generated responses to capture general cases where LLMs fail to follow the context, not just due to knowledge conflicts.</p>
</div>
</section>
<section id="S6.SS0.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Classifier guided generation.</h4>

<div id="S6.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px2.p1.1" class="ltx_p">Classifier guided generation aims to control attributes like topic or sentiment in text generation. PPLM <cite class="ltx_cite ltx_citemacro_citep">(Dathathri et al., <a href="#bib.bib11" title="" class="ltx_ref">2019</a>)</cite> uses gradient ascent to adjust LM probabilities via attribute classifiers. FUDGE <cite class="ltx_cite ltx_citemacro_citep">(Yang and Klein, <a href="#bib.bib44" title="" class="ltx_ref">2021</a>)</cite> uses an attribute predictor on partial sequences to modify LM probabilities. Our method uniquely guides generation using classifiers on attention maps, setting it apart from prior approaches.</p>
</div>
</section>
<section id="S6.SS0.SSS0.Px3" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Self-attention and model behavior.</h4>

<div id="S6.SS0.SSS0.Px3.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px3.p1.1" class="ltx_p">The attention mechanism, initially introduced in RNN-based encoder-decoder for neural machine translation <cite class="ltx_cite ltx_citemacro_cite">Bahdanau et al. (<a href="#bib.bib2" title="" class="ltx_ref">2015</a>); Luong et al. (<a href="#bib.bib26" title="" class="ltx_ref">2015</a>)</cite>, was later adopted in the Transformer model’s self-attention module <cite class="ltx_cite ltx_citemacro_cite">Vaswani et al. (<a href="#bib.bib42" title="" class="ltx_ref">2017</a>)</cite>, enabling greater parallelization. Self-attention’s interpretability has led researchers to use it for understanding model behaviors <cite class="ltx_cite ltx_citemacro_cite">Clark et al. (<a href="#bib.bib10" title="" class="ltx_ref">2019</a>); Hao et al. (<a href="#bib.bib13" title="" class="ltx_ref">2021</a>); Vashishth et al. (<a href="#bib.bib41" title="" class="ltx_ref">2019</a>)</cite>. Our work demonstrates that attention maps in LLMs are effective for detecting contextual hallucinations, providing a lightweight and interpretable solution compared to complex hidden representation methods <cite class="ltx_cite ltx_citemacro_cite">Zhang et al. (<a href="#bib.bib45" title="" class="ltx_ref">2024</a>); Chen et al. (<a href="#bib.bib6" title="" class="ltx_ref">2024b</a>)</cite>.</p>
</div>
</section>
</section>
<section id="S7" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">7 </span>Conclusion</h2>

<div id="S7.p1" class="ltx_para">
<p id="S7.p1.1" class="ltx_p">We introduce the <span id="S7.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span>, a lightweight classifier designed to detect contextual hallucinations by utilizing the <span id="S7.p1.1.2" class="ltx_text ltx_font_italic">lookback ratio</span>, which is computed solely from attention weights. This classifier not only effectively identifies contextual hallucinations but also mitigates them through <span id="S7.p1.1.3" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span> from the LLM. Remarkably, the method is transferable across various tasks, and even across models after mapping their attention heads. This research opens up new possibilities for leveraging attention map information to combat hallucinations in large language models.</p>
</div>
</section>
<section id="Sx1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">Limitations</h2>

<div id="Sx1.p1" class="ltx_para">
<p id="Sx1.p1.1" class="ltx_p">Despite the effectiveness of the <span id="Sx1.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> and its decoding, there are several limitations to consider.</p>
<ul id="Sx1.I1" class="ltx_itemize">
<li id="Sx1.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="Sx1.I1.i1.p1" class="ltx_para">
<p id="Sx1.I1.i1.p1.1" class="ltx_p">First, the performance upper bound of <span id="Sx1.I1.i1.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span> is limited by the sampling capabilities of the LLM itself. If the LLM fails to sample the correct chunk among the eight candidates, the <span id="Sx1.I1.i1.p1.1.2" class="ltx_text ltx_font_italic">Lookback Lens</span> cannot correct the error.</p>
</div>
</li>
<li id="Sx1.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="Sx1.I1.i2.p1" class="ltx_para">
<p id="Sx1.I1.i2.p1.1" class="ltx_p">Second, although the <span id="Sx1.I1.i2.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> is a lightweight classifier with negligible inference time, the requirement to sample multiple candidates from the LLM increases the total inference time.
We argue that <span id="Sx1.I1.i2.p1.1.2" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span> is a preliminary approach that demonstrates the feasibility of integrating the <span id="Sx1.I1.i2.p1.1.3" class="ltx_text ltx_font_italic">Lookback Lens</span> into the decoding process, as well as a robustness test for the <span id="Sx1.I1.i2.p1.1.4" class="ltx_text ltx_font_italic">Lookback Lens</span> to handle various text generation scenarios.
However, other options, such as intervening in the attention map mechanism based on <span id="Sx1.I1.i2.p1.1.5" class="ltx_text ltx_font_italic">Lookback Lens</span> signals, could potentially achieve faster inference, and we leave this for future work.</p>
</div>
</li>
<li id="Sx1.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="Sx1.I1.i3.p1" class="ltx_para">
<p id="Sx1.I1.i3.p1.1" class="ltx_p">Lastly, the <span id="Sx1.I1.i3.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> relies on annotated examples of around 1k-2k to train the classifier. While other end-to-end methods <cite class="ltx_cite ltx_citemacro_citep">(Chuang et al., <a href="#bib.bib9" title="" class="ltx_ref">2024</a>)</cite> can mitigate close-book hallucinations without training data, they lack interpretability due to the absence of a detection step. Nevertheless, we believe that requiring 1,000 annotated examples is a feasible setting.</p>
</div>
</li>
</ul>
</div>
</section>
<section id="Sx2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">Ethics Statement</h2>

<div id="Sx2.p1" class="ltx_para">
<p id="Sx2.p1.1" class="ltx_p">In this research, we used publicly available datasets and we did not collect any personal information.
All datasets and models are used in accordance with their intended use and licenses.
Our method is designed to improve the factuality of large language models (LLMs), which can have a positive impact on various applications, such as question-answering systems, summarization systems, and other applications that rely on LLMs.
When deployed, however, our approach still carries the issues stemming from LLMs, which means that there is a risk that the LLM can produce biased, harmful, or offensive output.
Therefore, caution should be exercised before implementing similar approaches in real-world applications.</p>
</div>
<div class="ltx_pagination ltx_role_newpage"></div>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Azaria and Mitchell (2023)</span>
<span class="ltx_bibblock">
Amos Azaria and Tom Mitchell. 2023.

</span>
<span class="ltx_bibblock">The internal state of an llm knows when it’s lying.

</span>
<span class="ltx_bibblock">In <em id="bib.bib1.1.1" class="ltx_emph ltx_font_italic">Findings of the Association for Computational Linguistics: EMNLP 2023</em>, pages 967–976.

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bahdanau et al. (2015)</span>
<span class="ltx_bibblock">
Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. 2015.

</span>
<span class="ltx_bibblock">Neural machine translation by jointly learning to align and translate.

</span>
<span class="ltx_bibblock">In <em id="bib.bib2.1.1" class="ltx_emph ltx_font_italic">3rd International Conference on Learning Representations, ICLR 2015</em>.

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Burns et al. (2023)</span>
<span class="ltx_bibblock">
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2023.

</span>
<span class="ltx_bibblock">Discovering latent knowledge in language models without supervision.

</span>
<span class="ltx_bibblock">In <em id="bib.bib3.1.1" class="ltx_emph ltx_font_italic">The Eleventh International Conference on Learning Representations</em>.

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2023)</span>
<span class="ltx_bibblock">
Jifan Chen, Grace Kim, Aniruddh Sriram, Greg Durrett, and Eunsol Choi. 2023.

</span>
<span class="ltx_bibblock">Complex claim verification with evidence retrieved in the wild.

</span>
<span class="ltx_bibblock"><em id="bib.bib4.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2305.11859</em>.

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2024a)</span>
<span class="ltx_bibblock">
Shiqi Chen, Miao Xiong, Junteng Liu, Zhengxuan Wu, Teng Xiao, Siyang Gao, and Junxian He. 2024a.

</span>
<span class="ltx_bibblock">In-context sharpness as alerts: An inner representation perspective for hallucination mitigation.

</span>
<span class="ltx_bibblock"><em id="bib.bib5.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2403.01548</em>.

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2024b)</span>
<span class="ltx_bibblock">
Zhongzhi Chen, Xingwu Sun, Xianfeng Jiao, Fengzong Lian, Zhanhui Kang, Di Wang, and Chengzhong Xu. 2024b.

</span>
<span class="ltx_bibblock">Truth forest: Toward multi-scale truthfulness in large language models through intervention without tuning.

</span>
<span class="ltx_bibblock">In <em id="bib.bib6.1.1" class="ltx_emph ltx_font_italic">Proceedings of the AAAI Conference on Artificial Intelligence</em>, volume 38, pages 20967–20974.

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chern et al. (2023)</span>
<span class="ltx_bibblock">
I Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, Pengfei Liu, et al. 2023.

</span>
<span class="ltx_bibblock">Factool: Factuality detection in generative ai–a tool augmented framework for multi-task and multi-domain scenarios.

</span>
<span class="ltx_bibblock"><em id="bib.bib7.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2307.13528</em>.

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chiang and Lee (2023)</span>
<span class="ltx_bibblock">
Cheng-Han Chiang and Hung-Yi Lee. 2023.

</span>
<span class="ltx_bibblock">Can large language models be an alternative to human evaluations?

</span>
<span class="ltx_bibblock">In <em id="bib.bib8.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, pages 15607–15631.

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chuang et al. (2024)</span>
<span class="ltx_bibblock">
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R Glass, and Pengcheng He. 2024.

</span>
<span class="ltx_bibblock">Dola: Decoding by contrasting layers improves factuality in large language models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib9.1.1" class="ltx_emph ltx_font_italic">The Twelfth International Conference on Learning Representations</em>.

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Clark et al. (2019)</span>
<span class="ltx_bibblock">
Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. 2019.

</span>
<span class="ltx_bibblock">What does bert look at? an analysis of bert’s attention.

</span>
<span class="ltx_bibblock"><em id="bib.bib10.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1906.04341</em>.

</span>
</li>
<li id="bib.bib11" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dathathri et al. (2019)</span>
<span class="ltx_bibblock">
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019.

</span>
<span class="ltx_bibblock">Plug and play language models: A simple approach to controlled text generation.

</span>
<span class="ltx_bibblock">In <em id="bib.bib11.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>.

</span>
</li>
<li id="bib.bib12" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Fabbri et al. (2021)</span>
<span class="ltx_bibblock">
Alexander R Fabbri, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2021.

</span>
<span class="ltx_bibblock">Qafacteval: Improved qa-based factual consistency evaluation for summarization.

</span>
<span class="ltx_bibblock"><em id="bib.bib12.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2112.08542</em>.

</span>
</li>
<li id="bib.bib13" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hao et al. (2021)</span>
<span class="ltx_bibblock">
Yaru Hao, Li Dong, Furu Wei, and Ke Xu. 2021.

</span>
<span class="ltx_bibblock">Self-attention attribution: Interpreting information interactions inside transformer.

</span>
<span class="ltx_bibblock">In <em id="bib.bib13.1.1" class="ltx_emph ltx_font_italic">Proceedings of the AAAI Conference on Artificial Intelligence</em>, volume 35, pages 12963–12971.

</span>
</li>
<li id="bib.bib14" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">He et al. (2021)</span>
<span class="ltx_bibblock">
Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2111.09543" title="" class="ltx_ref ltx_href">Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib14.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2111.09543.

</span>
</li>
<li id="bib.bib15" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Honovich et al. (2022)</span>
<span class="ltx_bibblock">
Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022.

</span>
<span class="ltx_bibblock">True: Re-evaluating factual consistency evaluation.

</span>
<span class="ltx_bibblock">In <em id="bib.bib15.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies</em>, pages 3905–3920.

</span>
</li>
<li id="bib.bib16" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ji et al. (2023)</span>
<span class="ltx_bibblock">
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023.

</span>
<span class="ltx_bibblock">Survey of hallucination in natural language generation.

</span>
<span class="ltx_bibblock"><em id="bib.bib16.1.1" class="ltx_emph ltx_font_italic">ACM Computing Surveys</em>, 55(12):1–38.

</span>
</li>
<li id="bib.bib17" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kandpal et al. (2023)</span>
<span class="ltx_bibblock">
Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023.

</span>
<span class="ltx_bibblock">Large language models struggle to learn long-tail knowledge.

</span>
<span class="ltx_bibblock">In <em id="bib.bib17.1.1" class="ltx_emph ltx_font_italic">International Conference on Machine Learning</em>, pages 15696–15707. PMLR.

</span>
</li>
<li id="bib.bib18" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kwiatkowski et al. (2019)</span>
<span class="ltx_bibblock">
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019.

</span>
<span class="ltx_bibblock">Natural questions: a benchmark for question answering research.

</span>
<span class="ltx_bibblock"><em id="bib.bib18.1.1" class="ltx_emph ltx_font_italic">Transactions of the Association for Computational Linguistics</em>, 7:453–466.

</span>
</li>
<li id="bib.bib19" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Laban et al. (2022)</span>
<span class="ltx_bibblock">
Philippe Laban, Tobias Schnabel, Paul N Bennett, and Marti A Hearst. 2022.

</span>
<span class="ltx_bibblock">Summac: Re-visiting nli-based models for inconsistency detection in summarization.

</span>
<span class="ltx_bibblock"><em id="bib.bib19.1.1" class="ltx_emph ltx_font_italic">Transactions of the Association for Computational Linguistics</em>, 10:163–177.

</span>
</li>
<li id="bib.bib20" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Lewis et al. (2020)</span>
<span class="ltx_bibblock">
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020.

</span>
<span class="ltx_bibblock">Retrieval-augmented generation for knowledge-intensive nlp tasks.

</span>
<span class="ltx_bibblock"><em id="bib.bib20.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 33:9459–9474.

</span>
</li>
<li id="bib.bib21" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2023)</span>
<span class="ltx_bibblock">
Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023.

</span>
<span class="ltx_bibblock">Halueval: A large-scale hallucination evaluation benchmark for large language models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib21.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</em>, pages 6449–6464.

</span>
</li>
<li id="bib.bib22" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2024)</span>
<span class="ltx_bibblock">
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024.

</span>
<span class="ltx_bibblock">Inference-time intervention: Eliciting truthful answers from a language model.

</span>
<span class="ltx_bibblock"><em id="bib.bib22.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 36.

</span>
</li>
<li id="bib.bib23" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Lin (2004)</span>
<span class="ltx_bibblock">
Chin-Yew Lin. 2004.

</span>
<span class="ltx_bibblock">Rouge: A package for automatic evaluation of summaries.

</span>
<span class="ltx_bibblock">In <em id="bib.bib23.1.1" class="ltx_emph ltx_font_italic">Text summarization branches out</em>, pages 74–81.

</span>
</li>
<li id="bib.bib24" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Liu et al. (2024)</span>
<span class="ltx_bibblock">
Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024.

</span>
<span class="ltx_bibblock">Lost in the middle: How language models use long contexts.

</span>
<span class="ltx_bibblock"><em id="bib.bib24.1.1" class="ltx_emph ltx_font_italic">Transactions of the Association for Computational Linguistics</em>, 12:157–173.

</span>
</li>
<li id="bib.bib25" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Liu et al. (2023)</span>
<span class="ltx_bibblock">
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023.

</span>
<span class="ltx_bibblock">G-eval: Nlg evaluation using gpt-4 with better human alignment.

</span>
<span class="ltx_bibblock">In <em id="bib.bib25.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</em>, pages 2511–2522.

</span>
</li>
<li id="bib.bib26" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Luong et al. (2015)</span>
<span class="ltx_bibblock">
Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015.

</span>
<span class="ltx_bibblock">Effective approaches to attention-based neural machine translation.

</span>
<span class="ltx_bibblock">In <em id="bib.bib26.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing</em>, pages 1412–1421.

</span>
</li>
<li id="bib.bib27" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Mallen et al. (2023)</span>
<span class="ltx_bibblock">
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023.

</span>
<span class="ltx_bibblock">When not to trust language models: Investigating effectiveness of parametric and non-parametric memories.

</span>
<span class="ltx_bibblock">In <em id="bib.bib27.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, pages 9802–9822.

</span>
</li>
<li id="bib.bib28" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Maynez et al. (2020)</span>
<span class="ltx_bibblock">
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020.

</span>
<span class="ltx_bibblock">On faithfulness and factuality in abstractive summarization.

</span>
<span class="ltx_bibblock"><em id="bib.bib28.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2005.00661</em>.

</span>
</li>
<li id="bib.bib29" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Min et al. (2023)</span>
<span class="ltx_bibblock">
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023.

</span>
<span class="ltx_bibblock">Factscore: Fine-grained atomic evaluation of factual precision in long form text generation.

</span>
<span class="ltx_bibblock"><em id="bib.bib29.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2305.14251</em>.

</span>
</li>
<li id="bib.bib30" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Narayan et al. (2018)</span>
<span class="ltx_bibblock">
Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018.

</span>
<span class="ltx_bibblock">Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.

</span>
<span class="ltx_bibblock">In <em id="bib.bib30.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing</em>, pages 1797–1807.

</span>
</li>
<li id="bib.bib31" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Neeman et al. (2023)</span>
<span class="ltx_bibblock">
Ella Neeman, Roee Aharoni, Or Honovich, Leshem Choshen, Idan Szpektor, and Omri Abend. 2023.

</span>
<span class="ltx_bibblock">Disentqa: Disentangling parametric and contextual knowledge with counterfactual question answering.

</span>
<span class="ltx_bibblock">In <em id="bib.bib31.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, pages 10056–10070.

</span>
</li>
<li id="bib.bib32" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">OpenAI (2022)</span>
<span class="ltx_bibblock">
OpenAI. 2022.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://openai.com/blog/chatgpt" title="" class="ltx_ref ltx_href">Introducing chatgpt</a>.

</span>
</li>
<li id="bib.bib33" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">OpenAI (2023)</span>
<span class="ltx_bibblock">
OpenAI. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://cdn.openai.com/papers/gpt-4.pdf" title="" class="ltx_ref ltx_href">Gpt-4 technical report</a>.

</span>
</li>
<li id="bib.bib34" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">OpenAI (2024)</span>
<span class="ltx_bibblock">
OpenAI. 2024.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://openai.com/index/hello-gpt-4o/" title="" class="ltx_ref ltx_href">Hello gpt-4o</a>.

</span>
</li>
<li id="bib.bib35" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Schuster et al. (2021)</span>
<span class="ltx_bibblock">
Tal Schuster, Adam Fisch, and Regina Barzilay. 2021.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2021.naacl-main.52" title="" class="ltx_ref ltx_href">Get your vitamin C! robust fact verification with contrastive evidence</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib35.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies</em>, pages 624–643, Online. Association for Computational Linguistics.

</span>
</li>
<li id="bib.bib36" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">See et al. (2017)</span>
<span class="ltx_bibblock">
Abigail See, Peter J Liu, and Christopher D Manning. 2017.

</span>
<span class="ltx_bibblock">Get to the point: Summarization with pointer-generator networks.

</span>
<span class="ltx_bibblock">In <em id="bib.bib36.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, pages 1073–1083.

</span>
</li>
<li id="bib.bib37" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Shi et al. (2023)</span>
<span class="ltx_bibblock">
Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Scott Wen-tau Yih. 2023.

</span>
<span class="ltx_bibblock">Trusting your evidence: Hallucinate less with context-aware decoding.

</span>
<span class="ltx_bibblock"><em id="bib.bib37.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2305.14739</em>.

</span>
</li>
<li id="bib.bib38" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Simhi et al. (2024)</span>
<span class="ltx_bibblock">
Adi Simhi, Jonathan Herzig, Idan Szpektor, and Yonatan Belinkov. 2024.

</span>
<span class="ltx_bibblock">Constructing benchmarks and interventions for combating hallucinations in llms.

</span>
<span class="ltx_bibblock"><em id="bib.bib38.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2404.09971</em>.

</span>
</li>
<li id="bib.bib39" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Thorne et al. (2018)</span>
<span class="ltx_bibblock">
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018.

</span>
<span class="ltx_bibblock">FEVER: a large-scale dataset for fact extraction and VERification.

</span>
<span class="ltx_bibblock">In <em id="bib.bib39.1.1" class="ltx_emph ltx_font_italic">NAACL-HLT</em>.

</span>
</li>
<li id="bib.bib40" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Touvron et al. (2023)</span>
<span class="ltx_bibblock">
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023.

</span>
<span class="ltx_bibblock">Llama 2: Open foundation and fine-tuned chat models.

</span>
<span class="ltx_bibblock"><em id="bib.bib40.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2307.09288</em>.

</span>
</li>
<li id="bib.bib41" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Vashishth et al. (2019)</span>
<span class="ltx_bibblock">
Shikhar Vashishth, Shyam Upadhyay, Gaurav Singh Tomar, and Manaal Faruqui. 2019.

</span>
<span class="ltx_bibblock">Attention interpretability across nlp tasks.

</span>
<span class="ltx_bibblock"><em id="bib.bib41.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1909.11218</em>.

</span>
</li>
<li id="bib.bib42" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Vaswani et al. (2017)</span>
<span class="ltx_bibblock">
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.

</span>
<span class="ltx_bibblock">Attention is all you need.

</span>
<span class="ltx_bibblock"><em id="bib.bib42.1.1" class="ltx_emph ltx_font_italic">Advances in neural information processing systems</em>, 30.

</span>
</li>
<li id="bib.bib43" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Vectara (2023)</span>
<span class="ltx_bibblock">
Vectara. 2023.

</span>
<span class="ltx_bibblock">vectarahallucination_valuation_model.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://huggingface.co/vectara/hallucination_evaluation_model" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://huggingface.co/vectara/hallucination_evaluation_model</a>.

</span>
<span class="ltx_bibblock">Accessed: 2024-06-12.

</span>
</li>
<li id="bib.bib44" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yang and Klein (2021)</span>
<span class="ltx_bibblock">
Kevin Yang and Dan Klein. 2021.

</span>
<span class="ltx_bibblock">Fudge: Controlled text generation with future discriminators.

</span>
<span class="ltx_bibblock">In <em id="bib.bib44.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies</em>, pages 3511–3535.

</span>
</li>
<li id="bib.bib45" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhang et al. (2024)</span>
<span class="ltx_bibblock">
Shaolei Zhang, Tian Yu, and Yang Feng. 2024.

</span>
<span class="ltx_bibblock">Truthx: Alleviating hallucinations by editing large language models in truthful space.

</span>
<span class="ltx_bibblock"><em id="bib.bib45.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2402.17811</em>.

</span>
</li>
<li id="bib.bib46" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhang et al. (2019a)</span>
<span class="ltx_bibblock">
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019a.

</span>
<span class="ltx_bibblock">Bertscore: Evaluating text generation with bert.

</span>
<span class="ltx_bibblock">In <em id="bib.bib46.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>.

</span>
</li>
<li id="bib.bib47" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhang et al. (2019b)</span>
<span class="ltx_bibblock">
Yuan Zhang, Jason Baldridge, and Luheng He. 2019b.

</span>
<span class="ltx_bibblock">PAWS: Paraphrase Adversaries from Word Scrambling.

</span>
<span class="ltx_bibblock">In <em id="bib.bib47.1.1" class="ltx_emph ltx_font_italic">Proc. of NAACL</em>.

</span>
</li>
<li id="bib.bib48" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zheng et al. (2024)</span>
<span class="ltx_bibblock">
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2024.

</span>
<span class="ltx_bibblock">Judging llm-as-a-judge with mt-bench and chatbot arena.

</span>
<span class="ltx_bibblock"><em id="bib.bib48.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 36.

</span>
</li>
</ul>
</section>
<div class="ltx_pagination ltx_role_newpage"></div>
<section id="A1" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix A </span>Data Creation for <span id="A1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span>
</h2>

<div id="A1.p1" class="ltx_para">
<p id="A1.p1.1" class="ltx_p">Our experimental setup aims to evaluate the ability of <span id="A1.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> to detect hallucinations in large language models with attention maps. We consider the summarization task and question-answering (QA) task in data creation.</p>
</div>
<div id="A1.p2" class="ltx_para">
<p id="A1.p2.1" class="ltx_p">For the summarization task, we sampled 1,000 examples from the CNN/DM dataset <cite class="ltx_cite ltx_citemacro_citep">(See et al., <a href="#bib.bib36" title="" class="ltx_ref">2017</a>)</cite>.
For QA, we use 2,655 examples from the Natural Questions <cite class="ltx_cite ltx_citemacro_citep">(Kwiatkowski et al., <a href="#bib.bib18" title="" class="ltx_ref">2019</a>)</cite> from the setup of <cite class="ltx_cite ltx_citemacro_citet">Liu et al. (<a href="#bib.bib24" title="" class="ltx_ref">2024</a>)</cite> to mix the gold document with irrelevant documents. To keep our focus more on LLM hallucinations rather than being distracted by assessing LLMs’ long-context utilization ability, we limited context to three documents per question where the gold document containing the answer was placed in the middle, surrounded by two irrelevant documents.</p>
</div>
<div id="A1.p3" class="ltx_para">
<p id="A1.p3.1" class="ltx_p">We prompt LLaMA-2-7B-Chat <cite class="ltx_cite ltx_citemacro_citep">(Touvron et al., <a href="#bib.bib40" title="" class="ltx_ref">2023</a>)</cite> to generate correct responses by greedy decoding for both tasks to ensure that both hallucinated and non-hallucinated examples derive from the same source distribution. The max length of generation is set to 256 tokens, or until the EOS token is generated.</p>
</div>
<div id="A1.p4" class="ltx_para">
<p id="A1.p4.1" class="ltx_p">After the annotation was collected, we extract hallucinated and non-hallucinated spans, as well as the corresponding attention map lookback ratio, from the LLaMA-2-7B-Chat model, to train the <span id="A1.p4.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> classifiers.</p>
</div>
<div id="A1.p5" class="ltx_para">
<p id="A1.p5.1" class="ltx_p">In the predefined span setting, three types of spans are considered as non-hallucinated spans: 1) the text segment before the first hallucinated span in the response 2) the text segment after the last hallucinated span in the response 3) the response annotated as non-hallucinated. All the annotated hallucinated spans are used as negative data to train the <span id="A1.p5.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span>.</p>
</div>
<div id="A1.p6" class="ltx_para">
<p id="A1.p6.1" class="ltx_p">In the sliding window setting, we consider all the possible fixed sized chunk with size = 8. If a chunk is overlapping with any of the annotated hallucinated spans, then it is considered as hallucinated, otherwise it is non-hallucinated.</p>
</div>
<section id="A1.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Why not use existing data?</h4>

<div id="A1.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="A1.SS0.SSS0.Px1.p1.1" class="ltx_p">Initially, we considered using the HaluEval dataset <cite class="ltx_cite ltx_citemacro_citep">(Li et al., <a href="#bib.bib21" title="" class="ltx_ref">2023</a>)</cite>, which was created by prompting GPT-3.5 <cite class="ltx_cite ltx_citemacro_citep">(OpenAI, <a href="#bib.bib32" title="" class="ltx_ref">2022</a>)</cite> to generate “hallucinated examples” against human-annotated non-hallucinated responses, on summarization, QA, and dialogue tasks. However, we have concerns that their method introduces a bias by creating fundamentally different data distributions between hallucinated and non-hallucinated examples. This discrepancy could potentially lead the classifier to learn to distinguish the sources of responses rather than accurately detecting hallucinations.</p>
</div>
<div id="A1.SS0.SSS0.Px1.p2" class="ltx_para">
<p id="A1.SS0.SSS0.Px1.p2.1" class="ltx_p">Additionally, we argue that the LLM’s attention weight will be more meaningful if the text is generated by the same LLM itself, not from external sources and teacher forcing to obtain the attention weights.
To ensure an unbiased and controlled evaluation environment, we generated our own dataset on summarization and QA tasks.</p>
</div>
</section>
</section>
<section id="A2" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix B </span>Evaluation Prompt for GPT-4o</h2>

<div id="A2.p1" class="ltx_para">
<p id="A2.p1.1" class="ltx_p">We show the templates used to prompt GPT-4o (<span id="A2.p1.1.1" class="ltx_text ltx_font_typewriter">gpt-4o-2024-05-13</span>) in annotating the truthfulness of a response and the span-level hallucination segment prediction in Table <a href="#A2.T9" title="Table 9 ‣ Appendix B Evaluation Prompt for GPT-4o ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a> and Table <a href="#A2.T10" title="Table 10 ‣ Appendix B Evaluation Prompt for GPT-4o ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">10</span></a>, respectively for CNN/DM and Natural Questions.</p>
</div>
<div id="A2.p2" class="ltx_para">
<p id="A2.p2.1" class="ltx_p">This prompt is used for 1) collecting the data to train the <span id="A2.p2.1.1" class="ltx_text ltx_font_italic">Lookback Lens</span> in Table <a href="#S2.T1" title="Table 1 ‣ Data ‣ 2.2 Experimental Setup ‣ 2 Contextual Hallucinations Detection ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>, and 2) evaluating the XSum summarization task in Sections <a href="#S3" title="3 Contextual Hallucinations Mitigation ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>, <a href="#S4" title="4 Cross-model Transfer ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>, and <a href="#S5" title="5 Discussions and Ablations ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>.</p>
</div>
<figure id="A2.T9" class="ltx_table">
<table id="A2.T9.1" class="ltx_tabular ltx_centering ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="A2.T9.1.1.1" class="ltx_tr">
<td id="A2.T9.1.1.1.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_bb ltx_border_tt">
<span id="A2.T9.1.1.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="A2.T9.1.1.1.1.1.1" class="ltx_p" style="width:421.6pt;">
<span id="A2.T9.1.1.1.1.1.1.1" class="ltx_inline-block ltx_minipage ltx_align_top" style="width:433.6pt;">
<span id="A2.T9.1.1.1.1.1.1.1.1" class="ltx_p"><span id="A2.T9.1.1.1.1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">You will be provided with a document and a proposed summary. Your task is to determine if the proposed summary can be directly inferred from the document. If the summary contains any information not found in the document, it is considered false. Even if the summary is different from a ground truth summary, it might still be true, as long as it doesn’t contain false information.
<br class="ltx_break">For each proposed summary, explain why it is true or false based on the information from the document. Focus only on the original document’s content, disregarding any external context.
<br class="ltx_break">After your explanation, give your final conclusion as <span id="A2.T9.1.1.1.1.1.1.1.1.1.1" class="ltx_text" style="color:#FF0000;">Conclusion: True</span> if the proposed summary is completely accurate based on the document, or <span id="A2.T9.1.1.1.1.1.1.1.1.1.2" class="ltx_text" style="color:#FF0000;">Conclusion: False</span> if it contains any incorrect or unsupported information. If your conclusion is ’False’, identify the exact phrases or name entities from the summary that is incorrect by stating <span id="A2.T9.1.1.1.1.1.1.1.1.1.3" class="ltx_text" style="color:#FF0000;">Problematic Spans: [the inaccurate text spans from the summary, in Python list of strings format]</span>.
<br class="ltx_break">
<br class="ltx_break">#Document#: {document}
<br class="ltx_break">
<br class="ltx_break">#Ground Truth Summary#: {ground_truth_summary}
<br class="ltx_break">
<br class="ltx_break">#Proposed Summary#: {response}
<br class="ltx_break">
<br class="ltx_break">Write your explanation first, and then give your final conclusion as <span id="A2.T9.1.1.1.1.1.1.1.1.1.4" class="ltx_text" style="color:#FF0000;">Conclusion: True</span> if the proposed summary is completely accurate based on the document, or <span id="A2.T9.1.1.1.1.1.1.1.1.1.5" class="ltx_text" style="color:#FF0000;">Conclusion: False</span> if it contains any incorrect or unsupported information. Add <span id="A2.T9.1.1.1.1.1.1.1.1.1.6" class="ltx_text" style="color:#FF0000;">Problematic Spans: [the exact inaccurate text spans from the summary, in a list of strings]</span> if your conclusion is ’False’.</span></span>
</span></span>
</span>
</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 9: </span>Prompt template for GPT-4o in annotating the truthfulness and predicting span-level hallucinations on summarization tasks. Used for CNN/DM and XSum.</figcaption>
</figure>
<figure id="A2.T10" class="ltx_table">
<table id="A2.T10.1" class="ltx_tabular ltx_centering ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="A2.T10.1.1.1" class="ltx_tr">
<td id="A2.T10.1.1.1.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_bb ltx_border_tt">
<span id="A2.T10.1.1.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="A2.T10.1.1.1.1.1.1" class="ltx_p" style="width:421.6pt;">
<span id="A2.T10.1.1.1.1.1.1.1" class="ltx_inline-block ltx_minipage ltx_align_top" style="width:433.6pt;">
<span id="A2.T10.1.1.1.1.1.1.1.1" class="ltx_p"><span id="A2.T10.1.1.1.1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">You will be provided with a document and a proposed answer to a question. Your task is to determine if the proposed answer can be directly inferred from the document. If the answer contains any information not found in the document, it is considered false. Even if the answer is different from a ground truth answer, it might still be true, as long as it doesn’t contain false information.
<br class="ltx_break">For each proposed answer, explain why it is true or false based on the information from the document. Focus only on the original document’s content, disregarding any external context.
<br class="ltx_break">After your explanation, give your final conclusion as <span id="A2.T10.1.1.1.1.1.1.1.1.1.1" class="ltx_text" style="color:#FF0000;">Conclusion: True</span> if the proposed answer is completely accurate based on the document, or <span id="A2.T10.1.1.1.1.1.1.1.1.1.2" class="ltx_text" style="color:#FF0000;">Conclusion: False</span> if it contains any incorrect or unsupported information. If your conclusion is ’False’, identify the exact phrases or name entities from the answer that is incorrect by stating <span id="A2.T10.1.1.1.1.1.1.1.1.1.3" class="ltx_text" style="color:#FF0000;">Problematic Spans: [the inaccurate text spans from the answer, in Python list of strings format]</span>.
<br class="ltx_break">
<br class="ltx_break">#Document#: {document}
<br class="ltx_break">
<br class="ltx_break">#Ground Truth Answers (a list of valid answers)#: {ground_truth_answers}
<br class="ltx_break">
<br class="ltx_break">#Proposed Answer#: {response}
<br class="ltx_break">
<br class="ltx_break">Write your explanation first, and then give your final conclusion as <span id="A2.T10.1.1.1.1.1.1.1.1.1.4" class="ltx_text" style="color:#FF0000;">Conclusion: True</span> if the proposed answer is completely accurate based on the document, or <span id="A2.T10.1.1.1.1.1.1.1.1.1.5" class="ltx_text" style="color:#FF0000;">Conclusion: False</span> if it contains any incorrect or unsupported information. Add <span id="A2.T10.1.1.1.1.1.1.1.1.1.6" class="ltx_text" style="color:#FF0000;">Problematic Spans: [the exact inaccurate text spans from the answer, in a list of strings]</span> if your conclusion is ’False’.</span></span>
</span></span>
</span>
</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 10: </span>Prompt template for GPT-4o in annotating the truthfulness and predicting span-level hallucinations on question-answering tasks. Used for Natural Questions.</figcaption>
</figure>
</section>
<section id="A3" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix C </span>Pilot Study of Human Evaluation on GPT-4o Evaluation</h2>

<div id="A3.p1" class="ltx_para">
<p id="A3.p1.1" class="ltx_p">To assess the quality of GPT-4o’s evaluation in Appendix <a href="#A2" title="Appendix B Evaluation Prompt for GPT-4o ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">B</span></a>, we conducted a pilot study using 70 examples from XSum data, involving the authors and colleagues who participated voluntarily as evaluators. Evaluators are all native English speakers. The human evaluators are provided with the document, ground truth summary, summary from LLaMA-2-7B-Chat, and the judgment from GPT-4o. And the task is to give a binary judgment on whether the judgment from GPT-4o is correct. The screenshot of our interface is shown in Figure <a href="#A3.F4" title="Figure 4 ‣ Appendix C Pilot Study of Human Evaluation on GPT-4o Evaluation ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>. Every evaluator is informed and consents that the annotation result will only be used to report statistics in this paper. Our pilot study shows that 68 out of 70 (97%) GPT-4o outputs are correct, justifying the use of GPT-4o as an automatic evaluator.</p>
</div>
<figure id="A3.F4" class="ltx_figure"><img src="/html/2407.07071/assets/imgs/screenshot.png" id="A3.F4.1.g1" class="ltx_graphics ltx_centering ltx_img_portrait" width="598" height="1024" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 4: </span>Screenshot of human annotation interface.</figcaption>
</figure>
</section>
<section id="A4" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix D </span>Evaluation Prompt for MT-Bench</h2>

<div id="A4.p1" class="ltx_para">
<p id="A4.p1.1" class="ltx_p">We show the evaluation prompt for MT-Bench (hallucination) in Table <a href="#A4.T11" title="Table 11 ‣ Appendix D Evaluation Prompt for MT-Bench ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">11</span></a>. We follow standard practice for MT-Bench (original) evaluation<span id="footnote4" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">4</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">4</sup><span class="ltx_tag ltx_tag_note">4</span><a target="_blank" href="https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge</a>.</span></span></span> and show evaluation prompts in Table <a href="#A4.T12" title="Table 12 ‣ Appendix D Evaluation Prompt for MT-Bench ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">12</span></a>. We evaluate MT-bench (original) with their default GPT-4 model <span id="A4.p1.1.1" class="ltx_text ltx_font_typewriter">gpt-4-0613</span> and our proposed MT-Bench (hallucination) with the latest GPT-4o model (<span id="A4.p1.1.2" class="ltx_text ltx_font_typewriter">gpt-4o-2024-05-13</span>).</p>
</div>
<figure id="A4.T11" class="ltx_table">
<table id="A4.T11.1" class="ltx_tabular ltx_centering ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="A4.T11.1.1.1" class="ltx_tr">
<td id="A4.T11.1.1.1.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_bb ltx_border_tt">
<span id="A4.T11.1.1.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T11.1.1.1.1.1.1" class="ltx_p" style="width:421.6pt;">
<span id="A4.T11.1.1.1.1.1.1.1" class="ltx_inline-block ltx_minipage ltx_align_top" style="width:433.6pt;">
<span id="A4.T11.1.1.1.1.1.1.1.1" class="ltx_p"><span id="A4.T11.1.1.1.1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">Please act as an impartial judge and evaluate the <span id="A4.T11.1.1.1.1.1.1.1.1.1.1" class="ltx_text" style="color:#FF0000;">faithfulness and consistency</span> of the response provided by an AI assistant to the user question displayed below. Your evaluation should consider <span id="A4.T11.1.1.1.1.1.1.1.1.1.2" class="ltx_text" style="color:#FF0000;">whether the assistant’s answer to the second user question is faithful and consistent to the chat history. If the answer contains any misinformation not found or not supported by the chat history, it is considered a hallucination.</span> You evaluation should focus on the assistant’s answer to the second user question. Begin your evaluation by providing a short explanation. Be as objective as possible. After providing your explanation, you must rate the response on a scale of 1 to 10 by strictly following this format: ‘‘[[rating]]", for example: ‘‘Rating: [[5]]". 
<br class="ltx_break">
<br class="ltx_break">&lt;|The Start of Assistant A’s Conversation with User|&gt; 
<br class="ltx_break">
<br class="ltx_break">### User:
<br class="ltx_break">{question_1}
<br class="ltx_break">
<br class="ltx_break">### Assistant A:
<br class="ltx_break">{answer_1}
<br class="ltx_break">
<br class="ltx_break">### User:
<br class="ltx_break">{question_2}
<br class="ltx_break">
<br class="ltx_break">### Assistant A:
<br class="ltx_break">{answer_2} 
<br class="ltx_break">
<br class="ltx_break">&lt;|The End of Assistant A’s Conversation with User|&gt;</span></span>
</span></span>
</span>
</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 11: </span>GPT-4o evaluation prompt for MT-bench (hallucination).</figcaption>
</figure>
<figure id="A4.T12" class="ltx_table">
<table id="A4.T12.1" class="ltx_tabular ltx_centering ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="A4.T12.1.1.1" class="ltx_tr">
<td id="A4.T12.1.1.1.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_tt">
<span id="A4.T12.1.1.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T12.1.1.1.1.1.1" class="ltx_p" style="width:421.6pt;">
<span id="A4.T12.1.1.1.1.1.1.1" class="ltx_inline-block ltx_minipage ltx_align_top" style="width:433.6pt;">
<span id="A4.T12.1.1.1.1.1.1.1.1" class="ltx_p"><span id="A4.T12.1.1.1.1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">Please act as an impartial judge and evaluate the <span id="A4.T12.1.1.1.1.1.1.1.1.1.1" class="ltx_text" style="color:#FF0000;">quality</span> of the response provided by an AI assistant to the user question displayed below. Your evaluation should consider <span id="A4.T12.1.1.1.1.1.1.1.1.1.2" class="ltx_text" style="color:#FF0000;">factors such as the helpfulness, relevance, accuracy, depth, creativity, and level of detail of the response</span>. You evaluation should focus on the assistant’s answer to the second user question. Begin your evaluation by providing a short explanation. Be as objective as possible. After providing your explanation, you must rate the response on a scale of 1 to 10 by strictly following this format: "[[rating]]", for example: "Rating: [[5]]".
<br class="ltx_break">
<br class="ltx_break">&lt;|The Start of Assistant A’s Conversation with User|&gt;
<br class="ltx_break">
<br class="ltx_break">### User:
<br class="ltx_break">{question_1}
<br class="ltx_break">
<br class="ltx_break">### Assistant A:
<br class="ltx_break">{answer_1}
<br class="ltx_break">
<br class="ltx_break">### User:
<br class="ltx_break">{question_2}
<br class="ltx_break">
<br class="ltx_break">### Assistant A:
<br class="ltx_break">{answer_2}
<br class="ltx_break">
<br class="ltx_break">&lt;|The End of Assistant A’s Conversation with User|&gt;</span></span>
</span></span>
</span>
</td>
</tr>
<tr id="A4.T12.1.2.2" class="ltx_tr">
<td id="A4.T12.1.2.2.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_bb ltx_border_t">
<span id="A4.T12.1.2.2.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T12.1.2.2.1.1.1" class="ltx_p" style="width:421.6pt;">
<span id="A4.T12.1.2.2.1.1.1.1" class="ltx_inline-block ltx_minipage ltx_align_top" style="width:433.6pt;">
<span id="A4.T12.1.2.2.1.1.1.1.1" class="ltx_p"><span id="A4.T12.1.2.2.1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">Please act as an impartial judge and evaluate the <span id="A4.T12.1.2.2.1.1.1.1.1.1.1" class="ltx_text" style="color:#FF0000;">quality</span> of the response provided by an AI assistant to the user question. Your evaluation should consider <span id="A4.T12.1.2.2.1.1.1.1.1.1.2" class="ltx_text" style="color:#FF0000;">correctness and helpfulness. You will be given a reference answer and the assistant’s answer</span>. You evaluation should focus on the assistant’s answer to the second question. Begin your evaluation by comparing the assistant’s answer with the reference answer. Identify and correct any mistakes. Be as objective as possible. After providing your explanation, you must rate the response on a scale of 1 to 10 by strictly following this format: "[[rating]]", for example: "Rating: [[5]]".
<br class="ltx_break">
<br class="ltx_break">&lt;|The Start of Reference Answer|&gt;
<br class="ltx_break">
<br class="ltx_break">### User:
<br class="ltx_break">{question_1}
<br class="ltx_break">
<br class="ltx_break">### Reference answer:
<br class="ltx_break">{ref_answer_1}
<br class="ltx_break">
<br class="ltx_break">### User:
<br class="ltx_break">{question_2}
<br class="ltx_break">
<br class="ltx_break">### Reference answer:
<br class="ltx_break">{ref_answer_2}
<br class="ltx_break">
<br class="ltx_break">&lt;|The End of Reference Answer|&gt;
<br class="ltx_break">
<br class="ltx_break">
<br class="ltx_break">&lt;|The Start of Assistant A’s Conversation with User|&gt;
<br class="ltx_break">
<br class="ltx_break">### User:
<br class="ltx_break">{question_1}
<br class="ltx_break">
<br class="ltx_break">### Assistant A:
<br class="ltx_break">{answer_1}
<br class="ltx_break">
<br class="ltx_break">### User:
<br class="ltx_break">{question_2}
<br class="ltx_break">
<br class="ltx_break">### Assistant A:
<br class="ltx_break">{answer_2}
<br class="ltx_break">
<br class="ltx_break">&lt;|The End of Assistant A’s Conversation with User|&gt;
<br class="ltx_break"></span></span>
</span></span>
</span>
</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 12: </span>GPT-4 evaluation prompt for general questions (top) and math questions (bottom) on MT-bench (original).</figcaption>
</figure>
</section>
<section id="A5" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix E </span>Model Details</h2>

<section id="A5.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">State-of-the-art NLI Model</h4>

<div id="A5.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="A5.SS0.SSS0.Px1.p1.1" class="ltx_p">We give further detail on the pretrained SoTA NLI model <span id="footnote5" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">5</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">5</sup><span class="ltx_tag ltx_tag_note">5</span><a target="_blank" href="https://huggingface.co/vectara/hallucination_evaluation_model" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://huggingface.co/vectara/hallucination_evaluation_model</a></span></span></span> used as our topline hallucination detector. Specifically, the model is based on DeBERTa-V3-base <cite class="ltx_cite ltx_citemacro_cite">He et al. (<a href="#bib.bib14" title="" class="ltx_ref">2021</a>)</cite> and further finetuned on a range of NLI and summarization datasets with examples annotated with factual consistency, including FEVER <cite class="ltx_cite ltx_citemacro_citep">(Thorne et al., <a href="#bib.bib39" title="" class="ltx_ref">2018</a>)</cite>, Vitamin C <cite class="ltx_cite ltx_citemacro_citep">(Schuster et al., <a href="#bib.bib35" title="" class="ltx_ref">2021</a>)</cite> and PAWS <cite class="ltx_cite ltx_citemacro_citep">(Zhang et al., <a href="#bib.bib47" title="" class="ltx_ref">2019b</a>)</cite>. Roughly 731k data examples can be collected from the training set of the above three datasets.
The model is reported to have superior performance when evaluated on TRUE <cite class="ltx_cite ltx_citemacro_citep">(Honovich et al., <a href="#bib.bib15" title="" class="ltx_ref">2022</a>)</cite> SummaC Benchmark <cite class="ltx_cite ltx_citemacro_citep">(Laban et al., <a href="#bib.bib19" title="" class="ltx_ref">2022</a>)</cite> and AnyScale Ranking Test for Hallucinations <span id="footnote6" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">6</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">6</sup><span class="ltx_tag ltx_tag_note">6</span><a target="_blank" href="https://www.anyscale.com/blog/llama-2-is-about-as-factually-accurate-as-gpt-4-for-summaries-and-is-30x-cheaper" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://www.anyscale.com/blog/llama-2-is-about-as-factually-accurate-as-gpt-4-for-summaries-and-is-30x-cheaper</a></span></span></span>.</p>
</div>
</section>
<section id="A5.SS0.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Other Model Details and License</h4>

<div id="A5.SS0.SSS0.Px2.p1" class="ltx_para">
<ul id="A5.I1" class="ltx_itemize">
<li id="A5.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A5.I1.i1.p1" class="ltx_para">
<p id="A5.I1.i1.p1.2" class="ltx_p"><span id="A5.I1.i1.p1.2.1" class="ltx_text ltx_markedasmath ltx_font_typewriter">Llama-2-7B-Chat</span>: A 7B parameter model that is instruction fine-tuned. HuggingFace ID: <span id="A5.I1.i1.p1.2.2" class="ltx_text ltx_markedasmath ltx_font_typewriter">meta-llama/Llama-2-7b-chat-hf</span>.</p>
</div>
</li>
<li id="A5.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A5.I1.i2.p1" class="ltx_para">
<p id="A5.I1.i2.p1.2" class="ltx_p"><span id="A5.I1.i2.p1.2.1" class="ltx_text ltx_markedasmath ltx_font_typewriter">Llama-2-13B-Chat</span>: A 13B parameter model that is instruction fine-tuned. HuggingFace ID: <span id="A5.I1.i2.p1.2.2" class="ltx_text ltx_markedasmath ltx_font_typewriter">meta-llama/Llama-2-13b-chat-hf</span>.</p>
</div>
</li>
<li id="A5.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A5.I1.i3.p1" class="ltx_para">
<p id="A5.I1.i3.p1.3" class="ltx_p"><span id="A5.I1.i3.p1.3.1" class="ltx_text ltx_markedasmath ltx_font_typewriter">hallucination_evaluation_model</span>: Based on <span id="A5.I1.i3.p1.3.2" class="ltx_text ltx_markedasmath ltx_font_typewriter">microsoft/deberta-v3-base</span> which has 86M parameters. HuggingFace ID: <span id="A5.I1.i3.p1.3.3" class="ltx_text ltx_markedasmath ltx_font_typewriter">vectara/hallucination_evaluation_model</span>.</p>
</div>
</li>
<li id="A5.I1.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A5.I1.i4.p1" class="ltx_para">
<p id="A5.I1.i4.p1.2" class="ltx_p"><span id="A5.I1.i4.p1.2.1" class="ltx_text ltx_markedasmath ltx_font_typewriter">DeBERTa-V3-Base</span>: a 86M parameters encoder based model. HuggingFace ID: <span id="A5.I1.i4.p1.2.2" class="ltx_text ltx_markedasmath ltx_font_typewriter">microsoft/deberta-v3-base</span>.</p>
</div>
</li>
</ul>
</div>
<div id="A5.SS0.SSS0.Px2.p2" class="ltx_para">
<p id="A5.SS0.SSS0.Px2.p2.1" class="ltx_p">The above models have the following licenses.</p>
<ul id="A5.I2" class="ltx_itemize">
<li id="A5.I2.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A5.I2.i1.p1" class="ltx_para">
<p id="A5.I2.i1.p1.1" class="ltx_p"><span id="A5.I2.i1.p1.1.1" class="ltx_text ltx_markedasmath ltx_font_typewriter">Llama-2-7B-Chat</span> is under the Llama 2 Community License Agreement.</p>
</div>
</li>
<li id="A5.I2.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A5.I2.i2.p1" class="ltx_para">
<p id="A5.I2.i2.p1.1" class="ltx_p"><span id="A5.I2.i2.p1.1.1" class="ltx_text ltx_markedasmath ltx_font_typewriter">Llama-2-13B-Chat</span> is under the Llama 2 Community License Agreement.</p>
</div>
</li>
<li id="A5.I2.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A5.I2.i3.p1" class="ltx_para">
<p id="A5.I2.i3.p1.1" class="ltx_p"><span id="A5.I2.i3.p1.1.1" class="ltx_text ltx_markedasmath ltx_font_typewriter">vectara/hallucination_evaluation_model</span> is under the Apache 2.0 License.</p>
</div>
</li>
<li id="A5.I2.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A5.I2.i4.p1" class="ltx_para">
<p id="A5.I2.i4.p1.1" class="ltx_p"><span id="A5.I2.i4.p1.1.1" class="ltx_text ltx_markedasmath ltx_font_typewriter">DeBERTa-V3-Base</span> is under MIT License.</p>
</div>
</li>
</ul>
</div>
</section>
<section id="A5.SS0.SSS0.Px3" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Inference Details</h4>

<div id="A5.SS0.SSS0.Px3.p1" class="ltx_para">
<p id="A5.SS0.SSS0.Px3.p1.1" class="ltx_p">We run all the models on NVIDIA A6000 (48GB) and V100 (32GB) GPUs. We do not train the model, but only run the inference part. Each of the examples takes around 20-30 seconds for 7B model, 40-60 seconds for 13B model to generate responses using our <span id="A5.SS0.SSS0.Px3.p1.1.1" class="ltx_text ltx_font_italic">Lookback Lens Guided Decoding</span>. Please check Appendix <a href="#A7" title="Appendix G Dataset Details ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">G</span></a> to estimate the total running time on each of the datasets, as it depends on number of examples.</p>
</div>
<div id="A5.SS0.SSS0.Px3.p2" class="ltx_para">
<p id="A5.SS0.SSS0.Px3.p2.2" class="ltx_p">All the inferences are run with either greedy decoding or sampling using temperature 0.9 and top-<math id="A5.SS0.SSS0.Px3.p2.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="A5.SS0.SSS0.Px3.p2.1.m1.1a"><mi id="A5.SS0.SSS0.Px3.p2.1.m1.1.1" xref="A5.SS0.SSS0.Px3.p2.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="A5.SS0.SSS0.Px3.p2.1.m1.1b"><ci id="A5.SS0.SSS0.Px3.p2.1.m1.1.1.cmml" xref="A5.SS0.SSS0.Px3.p2.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="A5.SS0.SSS0.Px3.p2.1.m1.1c">p</annotation></semantics></math> sampling with <math id="A5.SS0.SSS0.Px3.p2.2.m2.1" class="ltx_Math" alttext="p=0.95" display="inline"><semantics id="A5.SS0.SSS0.Px3.p2.2.m2.1a"><mrow id="A5.SS0.SSS0.Px3.p2.2.m2.1.1" xref="A5.SS0.SSS0.Px3.p2.2.m2.1.1.cmml"><mi id="A5.SS0.SSS0.Px3.p2.2.m2.1.1.2" xref="A5.SS0.SSS0.Px3.p2.2.m2.1.1.2.cmml">p</mi><mo id="A5.SS0.SSS0.Px3.p2.2.m2.1.1.1" xref="A5.SS0.SSS0.Px3.p2.2.m2.1.1.1.cmml">=</mo><mn id="A5.SS0.SSS0.Px3.p2.2.m2.1.1.3" xref="A5.SS0.SSS0.Px3.p2.2.m2.1.1.3.cmml">0.95</mn></mrow><annotation-xml encoding="MathML-Content" id="A5.SS0.SSS0.Px3.p2.2.m2.1b"><apply id="A5.SS0.SSS0.Px3.p2.2.m2.1.1.cmml" xref="A5.SS0.SSS0.Px3.p2.2.m2.1.1"><eq id="A5.SS0.SSS0.Px3.p2.2.m2.1.1.1.cmml" xref="A5.SS0.SSS0.Px3.p2.2.m2.1.1.1"></eq><ci id="A5.SS0.SSS0.Px3.p2.2.m2.1.1.2.cmml" xref="A5.SS0.SSS0.Px3.p2.2.m2.1.1.2">𝑝</ci><cn type="float" id="A5.SS0.SSS0.Px3.p2.2.m2.1.1.3.cmml" xref="A5.SS0.SSS0.Px3.p2.2.m2.1.1.3">0.95</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A5.SS0.SSS0.Px3.p2.2.m2.1c">p=0.95</annotation></semantics></math>. The implementation is based on Huggingface Transformers packages.<span id="footnote7" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">7</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">7</sup><span class="ltx_tag ltx_tag_note">7</span><a target="_blank" href="https://github.com/huggingface/transformers" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/huggingface/transformers</a></span></span></span>
All the scores in the paper are from a single run due to the limited computation for the large models.</p>
</div>
</section>
<section id="A5.SS0.SSS0.Px4" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Classifier Training Details</h4>

<div id="A5.SS0.SSS0.Px4.p1" class="ltx_para">
<p id="A5.SS0.SSS0.Px4.p1.1" class="ltx_p">We use Scikit-Learn <span id="A5.SS0.SSS0.Px4.p1.1.1" class="ltx_text ltx_font_typewriter">sklearn.linear_model.LogisticRegression<span id="footnote8" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">8</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">8</sup><span class="ltx_tag ltx_tag_note"><span id="footnote8.1.1.1" class="ltx_text ltx_font_serif">8</span></span><span id="footnote8.5" class="ltx_text ltx_font_serif">
</span><a target="_blank" href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" title="" class="ltx_ref ltx_url">https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html</a></span></span></span></span> to train the classifiers of <span id="A5.SS0.SSS0.Px4.p1.1.2" class="ltx_text ltx_font_italic">Lookback Lens</span> on CPU machine. We use all the default hyperparameters, such as L2 penalty, etc, but we change the <span id="A5.SS0.SSS0.Px4.p1.1.3" class="ltx_text ltx_font_typewriter">max_iter</span> to 1000 to ensure it is converged.</p>
</div>
</section>
<section id="A5.SS0.SSS0.Px5" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Heads Mapping Details</h4>

<div id="A5.SS0.SSS0.Px5.p1" class="ltx_para">
<p id="A5.SS0.SSS0.Px5.p1.1" class="ltx_p">We use Scikit-Learn <span id="A5.SS0.SSS0.Px5.p1.1.1" class="ltx_text ltx_font_typewriter">sklearn.linear_model.LinearRegression<span id="footnote9" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">9</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">9</sup><span class="ltx_tag ltx_tag_note"><span id="footnote9.1.1.1" class="ltx_text ltx_font_serif">9</span></span><span id="footnote9.5" class="ltx_text ltx_font_serif">
</span><a target="_blank" href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html" title="" class="ltx_ref ltx_url">https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html</a></span></span></span></span> in Section <a href="#S4" title="4 Cross-model Transfer ‣ Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>, to fit a linear transformation from LLaMA-2-13B-Chat’s attention heads to LLaMA-2-7B-Chat’s attention heads. It is computed to solve the close-form Ordinary Least Squares optimization problem, without gradient descent. We use all the default hyperparameters and run it on our CPU machine.</p>
</div>
</section>
</section>
<section id="A6" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix F </span>Visualization</h2>

<figure id="A6.F5" class="ltx_figure"><img src="/html/2407.07071/assets/imgs/visualization.png" id="A6.F5.1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="598" height="437" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 5: </span>Top-10 positive/negative heads ranked from top to the bottom by the magnitude of their coefficients in the <span id="A6.F5.3.1" class="ltx_text ltx_font_italic">Lookback Lens</span> classifier.</figcaption>
</figure>
<div id="A6.p1" class="ltx_para">
<p id="A6.p1.1" class="ltx_p">We visualize the lookback ratio of the top-10 most positive/negative heads when LLaMA-2-7B-Chat decodes the answer for an NQ example.
The top-10 most positive/negative heads are selected with the most positive/negative coefficients from the classifier.
The green rectangle frames the part that contains the hallucinations, i.e. <span id="A6.p1.1.1" class="ltx_text ltx_font_italic">and in Germany in the 14th century</span>.
We can see that during the generation of the hallucinated span, the positive heads, especially for the top-1 heads (topmost), show a lower lookback ratio (in blue), while the negative heads show a slightly higher lookback ratio (in red). However, the behavior of <span id="A6.p1.1.2" class="ltx_text ltx_font_italic">Lookback Lens</span> still needs to be determined by the collective behavior of all heads and the weight and bias of the classifier.</p>
</div>
</section>
<section id="A7" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix G </span>Dataset Details</h2>

<div id="A7.p1" class="ltx_para">
<p id="A7.p1.1" class="ltx_p">The datasets we used in the paper have the following details:</p>
</div>
<div id="A7.p2" class="ltx_para">
<ul id="A7.I1" class="ltx_itemize">
<li id="A7.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A7.I1.i1.p1" class="ltx_para">
<p id="A7.I1.i1.p1.1" class="ltx_p">CNN/DM: sampled 1000 examples from the testing set. Apache-2.0 license. <a target="_blank" href="https://huggingface.co/datasets/abisee/cnn_dailymail" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://huggingface.co/datasets/abisee/cnn_dailymail</a></p>
</div>
</li>
<li id="A7.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A7.I1.i2.p1" class="ltx_para">
<p id="A7.I1.i2.p1.1" class="ltx_p">Natural Questions: Apache-2.0 license. Testing set: 2655 examples from <a target="_blank" href="https://github.com/nelson-liu/lost-in-the-middle" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/nelson-liu/lost-in-the-middle</a>. NQ-train: sampled 2499 examples from its training set, using the positive document provided by <a target="_blank" href="https://github.com/facebookresearch/DPR" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/facebookresearch/DPR</a></p>
</div>
</li>
<li id="A7.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A7.I1.i3.p1" class="ltx_para">
<p id="A7.I1.i3.p1.1" class="ltx_p">XSum: 1000 examples sampled from the testing set. MIT license. <a target="_blank" href="https://github.com/EdinburghNLP/XSum" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/EdinburghNLP/XSum</a></p>
</div>
</li>
<li id="A7.I1.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A7.I1.i4.p1" class="ltx_para">
<p id="A7.I1.i4.p1.1" class="ltx_p">MT-bench: 80 examples. Apache-2.0 license. <a target="_blank" href="https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge</a></p>
</div>
</li>
</ul>
</div>
<div class="ltx_pagination ltx_role_newpage"></div>
</section>
</article>
</div>
<div class="ar5iv-footer"><a href="/html/2407.07070" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2407.07071" class="ar5iv-text-button ar5iv-severity-warning">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2407.07071">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2407.07071" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2407.07072" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Mon Aug  5 16:10:45 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2407.07071
When asked to summarize articles or answer questions given a passage, large language models (LLMs)  can hallucinate details and respond with unsubstantiated answers that are inaccurate with respect to the input context.
This paper describes a simple approach for detecting suchcontextual hallucinations. We hypothesize that contextual hallucinations are related to the extent to which an LLM attends to information in the provided context versus its own generations. Based on this intuition, we propose a simple hallucination detection model whose input features are given by the ratio of attention weights on the context versus newly generated tokens (for each attention head) . We find that a linear classifier based on theselookback ratiofeatures is as effective as a richer detector that utilizes the entire hidden states of an LLM or a text-based entailment model.
The lookback ratio-based detector—Lookback Lens—is found to transfer across tasks and even models, allowing a detector that is trained on a 7B model to be applied (without retraining)  to a larger 13B model.
We further apply this detector to mitigate contextual hallucinations, and find that a simple classifier-guided decoding approach is able to reduce the amount of hallucination, for example by 9.6% in the XSum summarization task.111Source code:github.com/voidism/Lookback-Lens
abstract
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models UsingOnlyAttention Maps
Abstract
2407.07071
1IntroductionDespite the utility and impressive capabilities of large language models (LLMs) , their tendency to generate hallucinations, i.e., content that deviates from facts or contextually relevant information, presents a significant challenge in their deployment.
In this work, we focus on the scenarios where the model is provided with the correct facts within the input context but still fails to generate accurate outputs, a phenomenon we termcontextual hallucination. Despite the simplicity of this setup, LLMs struggle with contextual hallucinations, frequently producing errors in tasks such as summarization and document-based question answering (e.g., Table1) , which can cause serious issues in applications such as retrieval-augmented generation (RAG) , even when correct documents are retrieved.Figure 1:An illustration of theLookback Lens. We extract attention weights and calculate the lookback ratios for all layers and all heads. We train a linear classifier on the concatenated features to predict truthfulness of the generation.Most prior studies that propose methods to combat hallucination focus on the scenariowithoutany input context, where the hallucinations arise from the LLMs’ parametric knowledge. These works detect and mitigate hallucinations by generally using the LLM’s representations, such as hidden states, MLP outputs, attention block outputsand attention head outputs. In contrast, the provided contextual information plays a key role in detecting contextual hallucinations. Insofar as attention (more so than other model internals)  provides a human-meaningful measure of how much weight is given to the context during generation, this motivates the use of signals from the attention maps for hallucination detection and mitigation.To leverage signals from attention maps, we start by hypothesizing that contextual hallucinations are related to the extent to which an LLM attends to the provided contextual information.
Concretely, we propose a simple feature calledlookback ratio, which is computed as the ratio of attention weights on the given context versus the newly generated tokens. At each time step, we calculate this lookback ratio for each attention head, and train a linear classifier, which we call theLookback Lens, to detect contextual hallucinations based on the lookback ratio features, as illustrated in Figure1.
TheLookback Lensperforms on par with, and sometimes even surpasses, more complex feature-based detectors that utilize hidden states from LLMs or text-based entailment models trained on extensively annotated datasets.
We can further integrate this detector during decoding to derive aLookback Lens Guided Decodingstrategy which can reduce contextual hallucinations by 9.6% from LLaMA-2-7B-Chat in the XSum summarization task.
Furthermore, our use of “higher level” attention map features makes it possible to transfer the detector across modelswithoutretraining, allowing a LLaMA2-13B-Chat model to use the same detector that has been trained on LLaMA-2-7B-Chat, and still reduce hallucinations by 3.2% in XSum. These results collectively highlight the potential of combating contextual hallucination by leveraging the information from attention maps.
introduction
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models UsingOnlyAttention Maps
1Introduction
2407.07071
2Contextual Hallucinations Detection2.1Lookback LensTodetectcontextual hallucinations in LLMs, we introduce a lookback ratio, a measure based on the attention distribution of a transformer model. Given a transformer withlayers, each withheads, the model processes an input sequence of context tokensof lengthfollowed by a set of newly generated tokensto generate the next token.
For time step, and for each head, we calculate the ratio of attention weights focused on the context tokens versus the newly generated tokens.
Formally, for each headin layer, we define:whereandare softmax-ed attention weights assigned to context tokensand new tokensrespectively.
The lookback ratiofor headin layerat time stepis then calculated as:To utilize these lookback ratios as input features in detecting hallucinations, we concatenate the lookback ratios across all heads and layers into a feature vector for the time step:Given a text span of interest, we average the corresponding lookback ratio vectorsinto a single vector.
We then employ a logistic regression classifierto predict if the span is factual (1)  or hallucinated (0)  based on the averaged lookback ratio vector.wheredenotes the sigmoid function,is the weight vector, andis the bias term of the classifier.Defining SpanTheLookback Lenspredicts the probability of hallucinations over spans. We consider two ways to obtain spans for a given sequence:predefined spansorsliding window.1)  Predefined Spans:When the hallucinated and non-hallucinated span annotations are available, we directly train the classifier to differentiate between them. This is a clean setting where all spans are either hallucinated or non-hallucinated.2)  Sliding Window:In practice, we do not have any predefined spans during decoding, thus we need a sliding window setup that iterates over all possible spans.
Specifically, we process the sentences into fixed-sized chunks and train the classifier to predict a label of 0 if any hallucinated content exists within a chunk, and 1 otherwise. Here, the annotated data is only used for creating labels, not for the span segmentation. This is more realistic for classifier-guided decoding, but it presents greater challenges because a chunk can contain both hallucinated and non-hallucinated content.2.2Experimental SetupDataTraining theLookback Lensrequires labels for hallucinated and non-hallucinated examples. To obtain these examples, we first prompt LLaMA-2-7B-Chatto greedy decode responses for 1,000 summarization examples from the CNN/DM datasetand 2,655 QA examples from the Natural Questionsfollowing the setup of. More details are shown in AppendixA.
Although being prompted to generate correct responses, the decoded responses will contain both hallucinated and non-hallucinated information as the LLaMA model is still not perfect.
Then, we employed GPT-4oto verify the truthfulness of these responses and provide span-level annotations on hallucinated segments (detailed prompts in AppendixB) . Additionally, we performed a pilot study of human annotation on a subset of 70 examples of the summarization task (details in AppendixC) , confirming a 97% consistency rate between GPT-4o annotations and human judgments, and validating the reliability of the automated annotations.
We show LLaMA-2-7B-Chat’s results on both tasks, as evaluated by GPT-4o, in Table1.
The results show that the generated summaries from LLaMA-2-7B-Chat still exhibit hallucinations about half of the time, highlighting the challenge of summarization tasks.DatasetExamplesCorrectCNN/DM100049.6%NQ265567.8%Table 1:Dataset statistics and GPT-4o evaluation results on responses greedy decoded by LLaMA-2-7B-chat.BaselinesWe compare our detection method against several baselines:1)  Text-based entailment classifier:We fine-tune the DeBERTa-v3-basemodel on the same dataset of CNN/DM and NQ as a natural language entailment (NLI)  task. Additionally, we include the results from a state-of-the-art entailment modeltrained on a huge amount of annotated NLI data (see details in AppendixE.2)  Hidden states-based classifier:We train classifiers using the same setting as theLookback Lensbut used input features from the hidden states of LLaMA-2-7B-Chat from its 24th, 28th, and 32nd layers instead of the lookback ratio. This baseline resembles a broad range of existing methods in the literature.Predefined SpanSliding Window = 8MethodSourceTargetSourceTargetSourceTargetTrainTestTransferTrainTestTransferText based NLISoTA NLI–Sum.––76.6––57.1SoTA NLI–QA––58.6––61.8NLI (our impl.) QASum.––55.1––53.0NLI (our impl.) Sum.QA––71.0––64.9Hidden states based32nd LayerQASum.100.089.679.499.097.156.132nd LayerSum.QA100.082.581.897.094.859.428th LayerQASum.100.091.483.699.297.357.728th LayerSum.QA100.083.384.797.295.258.824th LayerQASum.100.092.081.399.297.458.324th LayerSum.QA100.083.183.099.297.458.3Attention maps based (Ours) Lookback LensQASum.98.391.285.388.387.166.1Lookback LensSum.QA97.788.882.086.285.366.0Table 2:AUROC of the classification tasks using predefined span segmentation and sliding window (size = 8)  on NQ (QA)  and CNN/DM (Sum.) . The source task scores (Train/Test)  are averaged over two-fold validation.2.3ResultsOur results are presented in Table2. We consider both predefined span segmentation and sliding window with a window size of 8.
We include the two-fold validation setting on the source task and the out-of-domain transfer setting on the target task, with the tasks either question answering (QA)  or summarization (Sum.) . We find that theLookback Lensachieves slightly better performance than the hidden states-based classifier and significantly outperforms the NLI models (SoTA and our impl.) . The advantage of theLookback Lensover the hidden states-based classifier is more significant in the sliding window settings, as shown in the right-hand side of Table2.Additionally, we observe that the hidden states-based classifier tends to overfit the training sets during the two-fold validation, and present a substantial performance drop when transferred to out-of-domain tasks. In contrast,Lookback Lens, while not always fitting the training set perfectly, consistently exhibits better performance when applied to out-of-domain tasks. This contrast highlights the effectiveness and generalizability of the lookback ratio features we extract from the attention maps.
body
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models UsingOnlyAttention Maps
2Contextual Hallucinations Detection
2407.07071
3Contextual Hallucinations MitigationFigure 2:Lookback Lens Guided Decoding: sample multiple chunk candidates, compute lookback ratios from attention maps to be scored byLookback Lens, and select the best candidate that is less likely to be hallucinations.3.1Lookback Lens Guided DecodingTo mitigate the impact of contextual hallucinations identified by theLookback Lens, we introduce a classifier-guided decoding strategy to guide the generation toward more contextually accurate outputs.
This approach serves as a robustness test of theLookback Lens’ ability to handle various text generation scenarios.
While prior studies on controllable text generation adjust the output probabilities using classifiers based on the output tokens, our method fundamentally differs by not using the tokens themselves but rather their attention maps during generation.We proposeLookback Lens Guided Decoding, which incorporatesLookback Lens()  into the decoding process.
Since all tokens in the vocabulary share the same attention pattern during one decoding step,cannot directly influence one-step token choice.
Instead,can evaluate multiple-token chunks, as each chunk causes different attention patterns in multiple decoding steps.Given the context and partially generated text, we independently sample a set ofcandidate chunksat the same decoding step.
For each chunk, the associated lookback ratios are averaged to form a feature vector. As shown in Figure2, we select the best candidatepredicted byand append to the generation,We repeat this process until it generates the EOS token or reaches the maximum length.3.2Experimental SetupWe evaluateLookback Lens Guided Decodingon three tasks that involve generating texts conditioned on given contexts, including summarization with XSum, QA with NQ, and multi-turn conversations with MT-bench.For testing the generalization ability of theLookback Lens, we only train it with the CNN/DM summarization dataset from the detection task in Section2.2. Thus, only the XSum dataset will be the same-task transfer setting, while NQ and MT-bench will be cross-task transfer setting.XSumTo test theLookback Lens’s effectiveness at transferring across data distributions for the same task (summarization) , we use 1,000 examples sampled from the testing set of XSum. Prior studiesindicate that traditional evaluation metrics such as ROUGEor BERTScorecorrelated poorly with human evaluation on faithfulness and factuality. Recent studiesalso show a strong correlation between GPT-4evaluation and human evaluation. Thus, we report the averaged accuracy from the binary judgments of GPT-4o, with the prompts in AppendixB. We also conduct a pilot study for human evaluation on GPT-4o’s judgment in AppendixC, finding that 97% of the GPT-4o judgments are consistent with human judgment.Natural QuestionsWe use the NQ data from the setup ofwe describe in AppendixGand evaluate the best span exact match following.MT-BenchWe consider a multi-turn conversations setup where the model needs to follow previous chat history. We use MT-bench, a multi-turn instruction-following benchmark covering 8 categories. We focus exclusively on generating responses for the second turn and use GPT-3.5’s responses as the default for the first turn. We use GPT-4 to score the model’s answers on a scale of 1 to 10 based on various factors, includinghelpfulness, relevance, accuracy, depth, creativity, and level of detail of the response.Additionally, since we are particularly interested in mitigating contextual hallucinations, we further exclude math questions and evaluate the remaining 50 general questions. We specifically instruct GPT-4o to focus on whether the responses are faithful to the chat history (see prompt in AppendixB) . We refer to this setup as MT-Bench (hallu.) .BaselinesTo evaluate the performance of our proposed method, we compared it against the following baselines:1)  Greedy Decoding:generating responses using the LLaMA-2-7B-Chat modelthrough greedy decoding.2)  Other Classifier-Guided Decoding:using exactly the same setting but with different classifiers introduced in Section2.2, including
text-based entailment classifiers and hidden states-based classifiers.3.3Main ResultsMethodXSumNQMT-BenchHallu.Ori.Greedy Decoding49.071.26.085.10Text-based classifier guided decodingSoTA NLI†59.074.26.125.03NLI (our impl.) 44.172.55.724.99Hidden states based classifier guided decoding32nd layer48.373.95.494.9128th layer48.973.05.715.0624th layer47.573.95.655.16Lookback Lens guided decodingOurs58.674.26.275.10Table 3:Decoding results using 8 candidates per chunk in a chunk size of 8. We compare our methods with greedy decoding and classifier-guided decoding using the NLI models, and hidden state representations of different layers.†The SoTA NLI is trained on 731k examples so it may not be directly comparable.We show our results using 8 candidates per chunk in a chunk size of 8 in Table3, and the ablation with different chunk sizes is shown in Table6.Lookback Lens Guided Decodingcan improve the performance on both in-domain task (XSum, by 9.6%)  and out-of-domain tasks (NQ, by 3%) . The original greedy decoding results on XSum achieved 49.0% correct
which means 510 examples were hallucinated. Our decoding method significantly reduced the number of hallucinated examples from 510 to 414, resulting in an 18.8% reduction in the hallucinated examples. This result is on par with using SoTA NLI to guide the decoding, where SoTA NLI is trained on roughly 731k annotated summarization examples, which is 700larger compared to our 1k training set. (See AppendixE.)  In contrast, decoding guided by hidden states-based or the NLI (our implementation)  classifiers, both trained on the same data of our method, can only slightly improve the performance on NQ, but not for XSum, probably due to the issue of distribution shift, highlighting the advantages ofLookback Lensin generalization ability.For MT-bench, we evaluate both settings: the original setting (ori.)  and the setting that is specifically for judging contextual hallucinations (hallu.) .
We do not expect our method can improve on the original setting, because it evaluates many factors such as helpfulness, relevance, etc. But we expect to see an improvement on the hallucination setting.
The results shown in Table3suggest that our decoding method can boost the performance on the hallucination setting while maintaining the same performance in the original setting, which shows that our decoding method is effective in reducing hallucinations without compromising the overall generation quality.
body
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models UsingOnlyAttention Maps
3Contextual Hallucinations Mitigation
2407.07071
4Cross-model TransferOne benefit of using the lookback ratio to capture higher-level model patterns for hallucination detection is its potential to better transfer across models. A classifier trained with one model’s lookback ratio could potentially be applied to another modelwithoutretraining, provided correlation between the target model’s attention pattern and that of the original model.
Here, we show that we can transfer aLookback Lenstrained on attention maps from LLaMA-2-7B-Chat to LLaMA-2-13B-Chat without any retraining.SourceTargetPredefinedSlidingSpanWindowLookback Lens: Train 13BTest 13BQASum.84.060.4Sum.QA84.360.8QA-trainQA93.363.7Lookback Lens: Train 7BTest 13BQASum.73.558.8Sum.QA78.260.5QA-trainQA80.662.4Table 4:Cross model transfer results on detection tasks.Since the total numbers of attention heads are different in 7B and 13B models, and there is no obvious one-to-one mapping between the heads, we use a linear regression model to map the heads from the 13B model to the heads in 7B model. Concretely, we have 1024 heads in 7B and 1600 heads in 13B. We extract the averaged lookback ratio per head for all thetraining examples, resulting in amatrix and amatrix.222To ensure that two models are generating the same content when extracting lookback ratio, we decode from 7B and run the 13B model on the 7B outputs.We then fit a linear regression model to map the heads to reconstruct the 7B heads from 13B heads. After applying the linear transformation to the lookback ratio from 13B, the transformed heads can be directly used by 7B’s classifiers. See details in AppendixE.The detection results are shown in Table4. We first show the same-model (13B13B)  + cross-task transfer result, and the cross-model (7B13B)  + cross-task transfer result. Although cross-model transfer yields slightly worse results compared to same-model transfer, the AUROC scores are still non-trivially high. Consider that doing cross-model + cross-task transfer at the same time may be tough toLookback Lens, we also include one more setting that does training on 2.5K examples of the NQ training set333The NQ-train 2.5K data is annotated in the same method to annotate NQ testing set, as described in Section2.2.and then transfer to the NQ testing set. We see the cross-model same-task transfer results are even closer to the same-model transfer results.Given promising results on detection tasks, we apply cross-model transfer toLookback Lens Guided Decoding. We conduct the same-task transfer setting: NQ-train (7B)  to NQ (13B) , and CNN/DM (7B)  to XSum (13B) .
In Table5, we observe a performance improvement similar to same-model transfer using 13B itself, or using the SoTA NLI model applied on the 13B decoding.
However, on cross-task + cross-model transfer settings: CNN/DM (7B)  to NQ (13B) , we do not observe significant improvements where we attribute to the larger distribution shift. We leave this challenging setting for future work.MethodXSumNQGreedy52.974.0Text-based classifier guided decodingSoTA NLI†59.674.4MethodCNN/DMNQ-trainCNN/DMXSumNQNQLookback Lens guided decoding13B13B57.975.674.87B13B56.176.473.7Table 5:Cross model transfer from LLaMA-2-7B-chat to LLaMA-2-13B-chat using greedy decoding and classifier guided sampling methods with chunk size 8.
body
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models UsingOnlyAttention Maps
4Cross-model Transfer
2407.07071
5Discussions and AblationsIn this section, we further conduct various experiments and ablation studies on theLookback Lensand its corresponding classifier guided decoding.Effect of Chunk SizeIn Section3.3(Table3) , we experiment with chunk size = 8. Here, we study the effect of varying chunk sizes, from 4, 8, to 16. We see that there is a slight trend thatLookback Lensguided decoding prefers shorter chunk size for NQ and longer chunk size for XSum. However, in general the improvements are consistent across different chunk sizes, thus reducing the need to optimize for chunk sizes.MethodNQXSumChunk size=48164816Greedy71.249.0Text-based classifier guided decodingSoTA NLI†73.774.274.457.359.062.1Hidden states based classifier guided decoding32nd layer72.673.972.748.948.348.328th layer72.973.074.147.248.947.124th layer75.073.972.547.647.551.2Lookback Lens guided decodingOurs75.474.274.353.258.657.7Table 6:Performance comparison on various datasets using different methods and chunk sizes.Predictive Power of Different HeadsMethodPredefined SpanQASum.Sum.QAAll heads85.382.0Top-heads onlywith10501001050100Largest mag.71.282.382.879.280.381.1Most positive65.174.975.466.370.374.4Most negative59.567.574.466.470.273.0Table 7:Cross-task transfer AUROC using top-attention heads selected according to: coefficients with the largest magnitude (largest mag.) , most positive, and most negative. We consider= 10, 50, and 100.In the aforementioned experiments, we utilize all attention heads to train theLookback Lens. We are thus interested in
how the predictive power is distributed among different heads in making predictions. That is, how much performance can we recover if we only utilize a subset of heads?
To answer this, we use the coefficients in the linear classifier of theLookback Lens(in Section2)  to estimate the importance of each head in detecting hallucinations.In Table7, we show the results on detection tasks achieved by different detectors trained using only a subset of top-heads with the largest magnitude of coefficients in the originalLookback Lenstrained will all heads.
The results show that the predictive power is not concentrated only on a subset of heads.
Using only top-10 heads is worse than using all heads, and increasingconsistently improves performance and top-100 heads largely recover the model’s performance using all heads.More interestingly, we also include the results that only select the top-heads among the heads with most positive/negative coefficients, which are positive/negatively correlated to factuality.
On the heads with positive coefficients, higher lookback ratio (i.e., when the heads attend at the context more)  indicates higher factuality and less hallucination; conversely, heads with negative coefficients suggest a lower lookback ratio (i.e., attending to generated tokens more)  is more likely to be truthful.
Table7shows that none of positive or negative heads alone can be on par with using the top-largest magnitude heads.
This result implies that both positive and negative heads are critical for a model to generate factual responses. We conjecture that the positive heads may specialize at context grounding, and thus higher lookback ratio on these heads leads to more factual response. On the other hand, the negative heads may be critical at ensuring consistency in its own generation, and thus should attend to the generated tokens more.
We leave further investigation on this interesting balance for future work. Meanwhile, we visualize the lookback ratio of positive/negative heads in AppendixF.Figure 3:Qualitative example on XSum using the LLaMA-2-7B-Chat model with greedy decoding andLookback Lens Guided Decoding. The numbers in the parenthesis show the predicted scores from theLookback Lens.LayersPredefined SpanQASum.Sum.QALayer 1-469.664.0Layer 5-875.670.1Layer 9-1275.468.3Layer 13-1681.278.2Layer 17-2080.878.2Layer 21-2464.473.1Layer 25-2866.074.4Layer 29-3266.471.4Layer 1-3285.382.0Table 8:Cross-task transfer AUROC among layers.Reducing Number of LayersWe experiment with using only a subset of layers forLookback Lens, as shown in Table8. We can see that the predictive power is not concentrated in any subset of layers, as none of them can recover the performance of the full model that uses all layers. However, we observe that the middle layers (13-16, 17-20)  are slightly more useful than other layers.Qualitative StudyWe show qualitative examples from XSum in Figure3to illustrate howLookback Lensguided decoding improves performance. Greedy decoding from LLaMA-2-7B-Chat results in a hallucination, i.e.$100m (£64m) , that does not exist in the input document. However, theLookback Lensis able to assign low scores for the chunk candidates that have contextual hallucinations (as marked in red) . Therefore,Lookback Lens Guided Decodingis able to help the model generate a summary that is factual to the given context.
body
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models UsingOnlyAttention Maps
5Discussions and Ablations
2407.07071
6Related WorkHallucinations in LLMs.definedclose-book hallucinationvsopen-book hallucinationfor settings of relying on parametric knowledge vs knowledge in context. We termopen-book hallucinationascontextual hallucinationfor better clarity. Previous studies in hallucinations primarily focus on close-book hallucinationsand
their detectionand mitigation. Most of the studies focus on leveraging LLM’s internal representations, such as hidden states, MLP outputs, attention block outputsand attention head outputs. Our work, however, focuses on contextual hallucinations, where models produce content inconsistent with the provided context. Thus, different from prior studies, we focus on the attention maps instead of internal representations, as we believe that the attention maps patterns record how the LLM process the given contextual information.
Most of the prior studies treat detection and mitigation as two separate tasks, expect for. Our work focuses not only on detection, but also tries to incorporate the detector into the decoding process to further mitigate the contextual hallucinations.
Recently,also explored detecting and mitigating both close-book and open-book hallucinations. However, their open-book hallucination setting is limited to DisentQA, which creates knowledge conflicts between parametric knowledge and given context. In contrast, we focus on LLaMA-2’s naturally generated responses to capture general cases where LLMs fail to follow the context, not just due to knowledge conflicts.Classifier guided generation.Classifier guided generation aims to control attributes like topic or sentiment in text generation. PPLMuses gradient ascent to adjust LM probabilities via attribute classifiers. FUDGEuses an attribute predictor on partial sequences to modify LM probabilities. Our method uniquely guides generation using classifiers on attention maps, setting it apart from prior approaches.Self-attention and model behavior.The attention mechanism, initially introduced in RNN-based encoder-decoder for neural machine translation, was later adopted in the Transformer model’s self-attention module, enabling greater parallelization. Self-attention’s interpretability has led researchers to use it for understanding model behaviors. Our work demonstrates that attention maps in LLMs are effective for detecting contextual hallucinations, providing a lightweight and interpretable solution compared to complex hidden representation methods.
body
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models UsingOnlyAttention Maps
6Related Work
2407.07071
7ConclusionWe introduce theLookback Lens, a lightweight classifier designed to detect contextual hallucinations by utilizing thelookback ratio, which is computed solely from attention weights. This classifier not only effectively identifies contextual hallucinations but also mitigates them throughLookback Lens Guided Decodingfrom the LLM. Remarkably, the method is transferable across various tasks, and even across models after mapping their attention heads. This research opens up new possibilities for leveraging attention map information to combat hallucinations in large language models.
body
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models UsingOnlyAttention Maps
7Conclusion
2407.07071
LimitationsDespite the effectiveness of theLookback Lensand its decoding, there are several limitations to consider.•First, the performance upper bound ofLookback Lens Guided Decodingis limited by the sampling capabilities of the LLM itself. If the LLM fails to sample the correct chunk among the eight candidates, theLookback Lenscannot correct the error.•Second, although theLookback Lensis a lightweight classifier with negligible inference time, the requirement to sample multiple candidates from the LLM increases the total inference time.
We argue thatLookback Lens Guided Decodingis a preliminary approach that demonstrates the feasibility of integrating theLookback Lensinto the decoding process, as well as a robustness test for theLookback Lensto handle various text generation scenarios.
However, other options, such as intervening in the attention map mechanism based onLookback Lenssignals, could potentially achieve faster inference, and we leave this for future work.•Lastly, theLookback Lensrelies on annotated examples of around 1k-2k to train the classifier. While other end-to-end methodscan mitigate close-book hallucinations without training data, they lack interpretability due to the absence of a detection step. Nevertheless, we believe that requiring 1,000 annotated examples is a feasible setting.
body
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models UsingOnlyAttention Maps
Limitations
2407.07071
Ethics StatementIn this research, we used publicly available datasets and we did not collect any personal information.
All datasets and models are used in accordance with their intended use and licenses.
Our method is designed to improve the factuality of large language models (LLMs) , which can have a positive impact on various applications, such as question-answering systems, summarization systems, and other applications that rely on LLMs.
When deployed, however, our approach still carries the issues stemming from LLMs, which means that there is a risk that the LLM can produce biased, harmful, or offensive output.
Therefore, caution should be exercised before implementing similar approaches in real-world applications.
conclusion
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models UsingOnlyAttention Maps
Ethics Statement