<!DOCTYPE html><html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2402.11782] What Evidence Do Language Models Find Convincing?</title><meta property="og:description" content="Retrieval-augmented language models are being increasingly tasked with subjective, contentious, and conflicting queries such as “is aspartame linked to cancer”.
To resolve these ambiguous queries, one must search throu…">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="What Evidence Do Language Models Find Convincing?">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="What Evidence Do Language Models Find Convincing?">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2402.11782">

<!--Generated on Tue Mar  5 16:41:15 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document">
<h1 class="ltx_title ltx_title_document">What Evidence Do Language Models Find Convincing?</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname"><span id="id1.1.id1" class="ltx_text"></span> <span id="id2.2.id2" class="ltx_text">
<span id="id2.2.id2.1" class="ltx_tabular ltx_align_middle">
<span id="id2.2.id2.1.1" class="ltx_tr">
<span id="id2.2.id2.1.1.1" class="ltx_td ltx_nopad_r ltx_align_center">Alexander Wan, Eric Wallace, Dan Klein</span></span>
</span></span> <span id="id3.3.id3" class="ltx_text"></span> 
<br class="ltx_break">UC Berkeley 
<br class="ltx_break"><span id="id4.4.id4" class="ltx_text ltx_font_typewriter">{alexwan, ericwallace, klein}@berkeley.edu</span> 
<br class="ltx_break">
</span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id5.id1" class="ltx_p">Retrieval-augmented language models are being increasingly tasked with subjective, contentious, and conflicting queries such as “is aspartame linked to cancer”.
To resolve these ambiguous queries, one must search through a large range of websites and consider <em id="id5.id1.1" class="ltx_emph ltx_font_italic">which, if any, of this evidence do I find convincing?</em>
In this work, we study how LLMs answer this question.
In particular, we construct <span id="id5.id1.2" class="ltx_text ltx_font_smallcaps">ConflictingQA</span>, a dataset that pairs controversial queries with a series of real-world evidence documents that contain different facts (e.g., quantitative results), argument styles (e.g., appeals to authority), and answers (<span id="id5.id1.3" class="ltx_text ltx_font_typewriter">Yes</span> or <span id="id5.id1.4" class="ltx_text ltx_font_typewriter">No</span>). We use this dataset to perform sensitivity and counterfactual analyses to explore which text features most affect LLM predictions. Overall, we find that current models rely heavily on the <span id="id5.id1.5" class="ltx_text ltx_font_italic">relevance</span> of a website to the query, while largely ignoring <span id="id5.id1.6" class="ltx_text ltx_font_italic">stylistic</span> features that humans find important such as whether a text contains scientific references or is written with a neutral tone.
Taken together, these results highlight the importance of RAG corpus quality (e.g., the need to filter misinformation), and possibly even a shift in how LLMs are trained to better align with human judgements.</p>
</div>
<div id="p1" class="ltx_para ltx_noindent">
<div id="p1.1" class="ltx_block ltx_align_bottom">
<p id="p1.1.1" class="ltx_p"><span id="p1.1.1.1" class="ltx_text ltx_font_bold">What Evidence Do Language Models Find Convincing?</span></p>
<br class="ltx_break ltx_centering">
<p id="p1.1.2" class="ltx_p ltx_align_center" style="width:433.6pt;"><span id="p1.1.2.1" class="ltx_text ltx_inline-block" style="width:0.0pt;">
<span id="p1.1.2.1.1" class="ltx_tabular ltx_align_top">
<span id="p1.1.2.1.1.1" class="ltx_tr">
<span id="p1.1.2.1.1.1.1" class="ltx_td ltx_align_center"><span id="p1.1.2.1.1.1.1.1" class="ltx_text"></span><span id="p1.1.2.1.1.1.1.2" class="ltx_text ltx_font_bold"> <span id="p1.1.2.1.1.1.1.2.1" class="ltx_text">
<span id="p1.1.2.1.1.1.1.2.1.1" class="ltx_tabular ltx_align_middle">
<span id="p1.1.2.1.1.1.1.2.1.1.1" class="ltx_tr">
<span id="p1.1.2.1.1.1.1.2.1.1.1.1" class="ltx_td ltx_nopad_r ltx_align_center">Alexander Wan, Eric Wallace, Dan Klein</span></span>
</span></span><span id="p1.1.2.1.1.1.1.2.2" class="ltx_text"></span></span></span></span>
<span id="p1.1.2.1.1.2" class="ltx_tr">
<span id="p1.1.2.1.1.2.1" class="ltx_td ltx_align_center">UC Berkeley</span></span>
<span id="p1.1.2.1.1.3" class="ltx_tr">
<span id="p1.1.2.1.1.3.1" class="ltx_td ltx_align_center"><span id="p1.1.2.1.1.3.1.1" class="ltx_text ltx_font_typewriter">{alexwan, ericwallace, klein}@berkeley.edu</span></span></span>
</span></span></p>
<br class="ltx_break ltx_centering">
</div>
</div>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Introduction</h2>

<figure id="S1.F1" class="ltx_figure"><img src="/html/2402.11782/assets/x1.png" id="S1.F1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="332" height="239" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 1: </span>In <span id="S1.F1.7.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span>, we create contentious questions such as “<span id="S1.F1.8.2" class="ltx_text ltx_font_italic">is aspartame linked to cancer</span>”. We also retrieve evidence paragraphs for each question that contain different types of facts (e.g., quantitative results), argument styles (e.g., appeals to authority), and answers (<span id="S1.F1.9.3" class="ltx_text ltx_font_typewriter">Yes</span> or <span id="S1.F1.10.4" class="ltx_text ltx_font_typewriter">No</span>). For example, in the figure above we show two evidence paragraphs with their key arguments highlighted. Using <span id="S1.F1.11.5" class="ltx_text ltx_font_smallcaps">ConflictingQA</span>, we study <span id="S1.F1.12.6" class="ltx_text ltx_font_italic">why</span> LLMs trust certain types of evidence paragraphs and argument styles over others.</figcaption>
</figure>
<div id="S1.p1" class="ltx_para">
<p id="S1.p1.1" class="ltx_p">LLMs are widely deployed in settings that require understanding context—from retrieval-augmented systems to web agents, models condition on sources that range from internet paragraphs <cite class="ltx_cite ltx_citemacro_citep">(Karpukhin et al., <a href="#bib.bib24" title="" class="ltx_ref">2020</a>)</cite> to Python interpreters <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>)</cite>. At the same time, today’s models are also given tasks that are increasingly open-ended and controversial, such as “tell me if aspartame causes cancer”. To answer such questions, LLMs will read real-world paragraphs that are contradictory, noisy, and rife with misinformation <cite class="ltx_cite ltx_citemacro_citep">(Bush and Zaheer, <a href="#bib.bib6" title="" class="ltx_ref">2019</a>)</cite>.</p>
</div>
<div id="S1.p2" class="ltx_para">
<p id="S1.p2.1" class="ltx_p">Humans have techniques to sift through such large quantities of complex and contradictory evidence by answering the question: <em id="S1.p2.1.1" class="ltx_emph ltx_font_italic">which, if any, of this evidence do I find convincing?</em> To do so, humans combine multiple strategies, including fact checking and evaluating a source’s credibility <cite class="ltx_cite ltx_citemacro_citep">(Fogg et al., <a href="#bib.bib13" title="" class="ltx_ref">2003</a>)</cite>, harnessing prior knowledge and beliefs <cite class="ltx_cite ltx_citemacro_citep">(Kakol et al., <a href="#bib.bib23" title="" class="ltx_ref">2017</a>)</cite>, and critically evaluating logical arguments <cite class="ltx_cite ltx_citemacro_citep">(Metzger et al., <a href="#bib.bib27" title="" class="ltx_ref">2010</a>)</cite>.</p>
</div>
<div id="S1.p3" class="ltx_para">
<p id="S1.p3.1" class="ltx_p">In this work, we explore how LLMs resolve similar ambiguities when faced with conflicting open-ended questions. To study this, we create <span id="S1.p3.1.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span>, a dataset consisting of questions and real web documents that lead to conflicting answers. For instance, in Figure <a href="#S1.F1" title="Figure 1 ‣ 1 Introduction ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> we show an example where we pair the question “is aspartame linked to cancer” with a series of conflicting evidence documents collected from Google search. In our experiments, we evaluate the <span id="S1.p3.1.2" class="ltx_text ltx_font_italic">convincingness</span> of each evidence document by computing the rate at which a model’s predictions align with that document’s viewpoint (i.e., its win-rate).</p>
</div>
<div id="S1.p4" class="ltx_para">
<p id="S1.p4.1" class="ltx_p">We perform sensitivity and counterfactual analyses to find in-the-wild features that correlate with document convincingness. We consider a mix of features that describe (1) stylistic properties of a document and (2) relevance of a document to the question. Many of these were inspired by results from studies of human credibility, for example, we consider whether adding scientific references makes text appear more convincing.</p>
</div>
<div id="S1.p5" class="ltx_para">
<p id="S1.p5.1" class="ltx_p">Overall, we find that stylistic features play a considerably less impactful role in determining the convincingness of text than measures of relevance. Notably, we show that a simple perturbation targeting a website’s relevance—prefixing the page with “The following text is about the question: <span id="S1.p5.1.1" class="ltx_text ltx_font_typewriter">[question]</span>”—is enough to substantially improves its win-rate. On the other hand, stylistic features like the informational content, whether a page contains references, or its confidence, tend to only have a neutral to negative effect on win-rate. These results show that LLM perceptions of convincingness, when grounded in real-world QA tasks, do not align with humans. Taken together, these results suggest there should be an increasing focus on the <span id="S1.p5.1.2" class="ltx_text ltx_font_italic">quality</span> of retrieved evidence and a shift in how LLMs are trained to align with human preferences. We release our code at <a target="_blank" href="https://github.com/AlexWan0/rag-convincingness" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/AlexWan0/rag-convincingness</a>.</p>
</div>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Background and Motivations</h2>

<figure id="S2.F2" class="ltx_figure"><img src="/html/2402.11782/assets/x2.png" id="S2.F2.1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="266" height="160" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 2: </span><span id="S2.F2.3.1" class="ltx_text ltx_font_italic">Models over-rely on document relevance</span>. We study how the convincingness of a particular evidence paragraph (measured through win-rate) changes when we modify it. We compare the effect of these changes to a baseline perturbation where we append “Thanks for reading!” to the end (indicated by the dotted line). We find that many stylistic changes—inspired by factors that influence humans—have a neutral or even negative effect on models. On the other hand, perturbations that increase the texts relevance but minimally change its style have a substantial positive effect on models. Descriptions for each perturbation can be found in Appendix <a href="#A5" title="Appendix E Counterfactual Perturbations ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">E</span></a>.</figcaption>
</figure>
<div id="S2.p1" class="ltx_para">
<p id="S2.p1.1" class="ltx_p">Standard LLMs can be used to solve tasks that do not require context, e.g., writing basic Python code or answering simple trivia questions <cite class="ltx_cite ltx_citemacro_citep">(Brown et al., <a href="#bib.bib5" title="" class="ltx_ref">2020</a>; Raffel et al., <a href="#bib.bib33" title="" class="ltx_ref">2020</a>; Touvron et al., <a href="#bib.bib43" title="" class="ltx_ref">2023a</a>)</cite>. To give these models more knowledge, agency, and capabilities, recent efforts have augmented LLMs with retrieval <cite class="ltx_cite ltx_citemacro_citep">(Guu et al., <a href="#bib.bib18" title="" class="ltx_ref">2020</a>; Karpukhin et al., <a href="#bib.bib24" title="" class="ltx_ref">2020</a>)</cite>, domain-specific tools <cite class="ltx_cite ltx_citemacro_citep">(Schick et al., <a href="#bib.bib38" title="" class="ltx_ref">2023</a>; Gao et al., <a href="#bib.bib14" title="" class="ltx_ref">2023</a>; Mialon et al., <a href="#bib.bib28" title="" class="ltx_ref">2023</a>)</cite>, or even generic web access <cite class="ltx_cite ltx_citemacro_citep">(Nakano et al., <a href="#bib.bib30" title="" class="ltx_ref">2021</a>; Adept, <a href="#bib.bib2" title="" class="ltx_ref">2022</a>; Richards, <a href="#bib.bib36" title="" class="ltx_ref">2023</a>)</cite>. These enhancements allow LLMs to answer more challenging open-domain questions (e.g., “is aspartame linked to cancer?”) or accomplish open-ended tasks (e.g., “buy me a size 9 pair of blue running shoes”).</p>
</div>
<div id="S2.p2" class="ltx_para ltx_noindent">
<p id="S2.p2.1" class="ltx_p"><span id="S2.p2.1.1" class="ltx_text ltx_font_bold">Handling conflicting evidence.</span>
 A key question is how retrieval-augmented LLMs handle scenarios where their context is conflicting, ambiguous, or uncertain.
There has been a large body of work that studies how <span id="S2.p2.1.2" class="ltx_text ltx_font_italic">humans</span> handle such conflicting evidence using HCI studies <cite class="ltx_cite ltx_citemacro_cite">Fogg et al. (<a href="#bib.bib13" title="" class="ltx_ref">2003</a>); Kakol et al. (<a href="#bib.bib22" title="" class="ltx_ref">2013</a>); Flanagin and Metzger (<a href="#bib.bib12" title="" class="ltx_ref">2000</a>); Metzger et al. (<a href="#bib.bib27" title="" class="ltx_ref">2010</a>); Kakol et al. (<a href="#bib.bib23" title="" class="ltx_ref">2017</a>)</cite> or by trying to predict human argument preferences <cite class="ltx_cite ltx_citemacro_cite">Gleize et al. (<a href="#bib.bib15" title="" class="ltx_ref">2019</a>); Toledo et al. (<a href="#bib.bib42" title="" class="ltx_ref">2019</a>); Gretz et al. (<a href="#bib.bib16" title="" class="ltx_ref">2019</a>)</cite>, but little work has been done on evaluating how AI models handle such conflicts.</p>
</div>
<div id="S2.p3" class="ltx_para">
<p id="S2.p3.1" class="ltx_p">The existing work in AI has focused on conflicts between facts learned during pre-training and the evidence given during inference, finding that models are largely receptive to retrieved samples <cite class="ltx_cite ltx_citemacro_citep">(Longpre et al., <a href="#bib.bib26" title="" class="ltx_ref">2021</a>; Xie et al., <a href="#bib.bib46" title="" class="ltx_ref">2023</a>; Chen et al., <a href="#bib.bib8" title="" class="ltx_ref">2022</a>)</cite>. However, these works focus on restricted settings such as QA over Wikipedia, where there are relatively uncontroversial factoid questions that have trusted evidence paragraphs.
Moreover, they do not focus on <span id="S2.p3.1.1" class="ltx_text ltx_font_italic">what types</span> of evidence models prefer. Our goal is to design a more realistic question answering benchmark to better analyze features about the evidence itself.</p>
</div>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>The <span id="S3.1.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span> Dataset</h2>

<figure id="S3.T1" class="ltx_table">
<table id="S3.T1.1" class="ltx_tabular ltx_centering ltx_align_middle">
<tr id="S3.T1.1.1" class="ltx_tr">
<td id="S3.T1.1.1.1" class="ltx_td ltx_align_left ltx_border_tt">
<span id="S3.T1.1.1.1.1" class="ltx_inline-block">
<span id="S3.T1.1.1.1.1.1" class="ltx_p"><span id="S3.T1.1.1.1.1.1.1" class="ltx_text ltx_font_bold">Category</span></span>
</span>
</td>
<td id="S3.T1.1.1.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_tt">
<span id="S3.T1.1.1.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.1.2.1.1" class="ltx_p" style="width:227.6pt;"><span id="S3.T1.1.1.2.1.1.1" class="ltx_text"></span><span id="S3.T1.1.1.2.1.1.2" class="ltx_text ltx_font_bold"> <span id="S3.T1.1.1.2.1.1.2.1" class="ltx_text">
<span id="S3.T1.1.1.2.1.1.2.1.1" class="ltx_tabular ltx_align_middle">
<span id="S3.T1.1.1.2.1.1.2.1.1.1" class="ltx_tr">
<span id="S3.T1.1.1.2.1.1.2.1.1.1.1" class="ltx_td ltx_nopad_r ltx_align_center">Example Question</span></span>
</span></span><span id="S3.T1.1.1.2.1.1.2.2" class="ltx_text"></span></span></span>
</span>
</td>
<td id="S3.T1.1.1.3" class="ltx_td ltx_align_center ltx_border_tt">
<span id="S3.T1.1.1.3.1" class="ltx_inline-block">
<span id="S3.T1.1.1.3.1.1" class="ltx_p"><span id="S3.T1.1.1.3.1.1.1" class="ltx_text ltx_font_bold">Num Evidence Docs</span></span>
</span>
</td>
</tr>
<tr id="S3.T1.1.2" class="ltx_tr">
<td id="S3.T1.1.2.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.2.1.1" class="ltx_text" style="background-color:#FFC0C0;">Pharmacology</span></td>
<td id="S3.T1.1.2.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S3.T1.1.2.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.2.2.1.1" class="ltx_p" style="width:227.6pt;">Are antidepressants more effective than placebo?</span>
</span>
</td>
<td id="S3.T1.1.2.3" class="ltx_td ltx_align_center ltx_border_t">10</td>
</tr>
<tr id="S3.T1.1.3" class="ltx_tr">
<td id="S3.T1.1.3.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.3.1.1" class="ltx_text" style="background-color:#FFDAC0;">Online Learning</span></td>
<td id="S3.T1.1.3.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S3.T1.1.3.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.3.2.1.1" class="ltx_p" style="width:227.6pt;">Are online degrees valued less by employers?</span>
</span>
</td>
<td id="S3.T1.1.3.3" class="ltx_td ltx_align_center ltx_border_t">10</td>
</tr>
<tr id="S3.T1.1.4" class="ltx_tr">
<td id="S3.T1.1.4.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.4.1.1" class="ltx_text" style="background-color:#FFFDC0;">Biodiversity</span></td>
<td id="S3.T1.1.4.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S3.T1.1.4.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.4.2.1.1" class="ltx_p" style="width:227.6pt;">Are bees the most important pollinators?</span>
</span>
</td>
<td id="S3.T1.1.4.3" class="ltx_td ltx_align_center ltx_border_t">10</td>
</tr>
<tr id="S3.T1.1.5" class="ltx_tr">
<td id="S3.T1.1.5.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.5.1.1" class="ltx_text" style="background-color:#D0FFC0;">Web Design</span></td>
<td id="S3.T1.1.5.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S3.T1.1.5.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.5.2.1.1" class="ltx_p" style="width:227.6pt;">Does longer website content rank better on Google?</span>
</span>
</td>
<td id="S3.T1.1.5.3" class="ltx_td ltx_align_center ltx_border_t">13</td>
</tr>
<tr id="S3.T1.1.6" class="ltx_tr">
<td id="S3.T1.1.6.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.6.1.1" class="ltx_text" style="background-color:#C0FFFD;">Sustainability</span></td>
<td id="S3.T1.1.6.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S3.T1.1.6.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.6.2.1.1" class="ltx_p" style="width:227.6pt;">Are electric cars really green?</span>
</span>
</td>
<td id="S3.T1.1.6.3" class="ltx_td ltx_align_center ltx_border_t">9</td>
</tr>
<tr id="S3.T1.1.7" class="ltx_tr">
<td id="S3.T1.1.7.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.7.1.1" class="ltx_text" style="background-color:#C0D0FF;">Philosophy</span></td>
<td id="S3.T1.1.7.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S3.T1.1.7.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.7.2.1.1" class="ltx_p" style="width:227.6pt;">Are humans fundamentally good or evil?</span>
</span>
</td>
<td id="S3.T1.1.7.3" class="ltx_td ltx_align_center ltx_border_t">7</td>
</tr>
<tr id="S3.T1.1.8" class="ltx_tr">
<td id="S3.T1.1.8.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.8.1.1" class="ltx_text" style="background-color:#C0C0FF;">Nuclear Energy</span></td>
<td id="S3.T1.1.8.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S3.T1.1.8.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.8.2.1.1" class="ltx_p" style="width:227.6pt;">Can nuclear power solve climate change?</span>
</span>
</td>
<td id="S3.T1.1.8.3" class="ltx_td ltx_align_center ltx_border_t">7</td>
</tr>
<tr id="S3.T1.1.9" class="ltx_tr">
<td id="S3.T1.1.9.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.9.1.1" class="ltx_text" style="background-color:#DAC0FF;">Work-Life Balance</span></td>
<td id="S3.T1.1.9.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S3.T1.1.9.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.9.2.1.1" class="ltx_p" style="width:227.6pt;">Is unlimited vacation time beneficial for employees?</span>
</span>
</td>
<td id="S3.T1.1.9.3" class="ltx_td ltx_align_center ltx_border_t">10</td>
</tr>
<tr id="S3.T1.1.10" class="ltx_tr">
<td id="S3.T1.1.10.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.10.1.1" class="ltx_text" style="background-color:#FFC0FF;">Somnology</span></td>
<td id="S3.T1.1.10.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S3.T1.1.10.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.10.2.1.1" class="ltx_p" style="width:227.6pt;">Do older people need less sleep?</span>
</span>
</td>
<td id="S3.T1.1.10.3" class="ltx_td ltx_align_center ltx_border_t">8</td>
</tr>
<tr id="S3.T1.1.11" class="ltx_tr">
<td id="S3.T1.1.11.1" class="ltx_td ltx_align_left ltx_border_bb ltx_border_t"><span id="S3.T1.1.11.1.1" class="ltx_text" style="background-color:#FFC0DA;">Biomechanics</span></td>
<td id="S3.T1.1.11.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_bb ltx_border_t">
<span id="S3.T1.1.11.2.1" class="ltx_inline-block ltx_align_top">
<span id="S3.T1.1.11.2.1.1" class="ltx_p" style="width:227.6pt;">Do compression garments improve athletic performance?</span>
</span>
</td>
<td id="S3.T1.1.11.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">13</td>
</tr>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 1: </span>In <span id="S3.T1.5.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span>, we create controversial questions for 136 different categories (see Table <a href="#A1.T5" title="Table 5 ‣ Appendix A Additional Details on ConflictingQA ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a> for the complete list). Above, we show an example question for ten different categories, as well as the number of evidence paragraphs for each one. The evidence paragraphs contain a mix of <span id="S3.T1.6.2" class="ltx_text ltx_font_typewriter">Yes</span> and <span id="S3.T1.7.3" class="ltx_text ltx_font_typewriter">No</span> answers.</figcaption>
</figure>
<div id="S3.p1" class="ltx_para">
<p id="S3.p1.1" class="ltx_p">Here, we describe the construction of <span id="S3.p1.1.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span>, our dataset that evaluates what types of evidence are convincing for LLMs.
We design <span id="S3.p1.1.2" class="ltx_text ltx_font_smallcaps">ConflictingQA</span> to emulate the common setup for deploying retrieval-augmented LLMs: we retrieve the most relevant documents for a particular user query and place them in the LLM’s context window <cite class="ltx_cite ltx_citemacro_citep">(Chen et al., <a href="#bib.bib7" title="" class="ltx_ref">2017</a>; Shi et al., <a href="#bib.bib40" title="" class="ltx_ref">2023</a>; Ram et al., <a href="#bib.bib34" title="" class="ltx_ref">2023</a>)</cite>.
To build our dataset, we tackle three challenges: collecting contentious questions, identifying relevant and diverse evidence paragraphs, and grouping evidence paragraphs together to create conflicting examples.</p>
</div>
<div id="S3.p2" class="ltx_para ltx_noindent">
<p id="S3.p2.1" class="ltx_p"><span id="S3.p2.1.1" class="ltx_text ltx_font_bold">Collecting contentious questions.</span>
 We first create a series of realistic open-ended questions for which there exists conflicting evidence online. Critically, unlike past work on ambiguity in QA <cite class="ltx_cite ltx_citemacro_citep">(Min et al., <a href="#bib.bib29" title="" class="ltx_ref">2020</a>; Zhang and Choi, <a href="#bib.bib49" title="" class="ltx_ref">2021</a>; Sun et al., <a href="#bib.bib41" title="" class="ltx_ref">2023</a>)</cite>, we want to collect <span id="S3.p2.1.2" class="ltx_text ltx_font_italic">unambiguous</span> questions that still have answer conflicts. For example, in Figure <a href="#S1.F1" title="Figure 1 ‣ 1 Introduction ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>, we show a question “are artificial sweeteners linked to cancer?”, which is a widely-debated query in which there exist websites that support both answers. We design the questions to elicit binaries responses of <span id="S3.p2.1.3" class="ltx_text ltx_font_typewriter">Yes</span> or <span id="S3.p2.1.4" class="ltx_text ltx_font_typewriter">No</span> to simplify evaluation.</p>
</div>
<div id="S3.p3" class="ltx_para">
<p id="S3.p3.1" class="ltx_p">We create questions using GPT-4. To ensure that the model generates a diverse set of questions we take inspiration from previous work in synthetic dataset generation <cite class="ltx_cite ltx_citemacro_citep">(Gunasekar et al., <a href="#bib.bib17" title="" class="ltx_ref">2023</a>; Eldan and Li, <a href="#bib.bib11" title="" class="ltx_ref">2023</a>)</cite> and stratify the generations by topic: we first generate question categories (e.g., climate change, robotics, oncology) then generate sets of questions conditioned on each category (full prompt provided in Table <a href="#A1.T6" title="Table 6 ‣ Appendix A Additional Details on ConflictingQA ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">6</span></a> in Appendix <a href="#A1" title="Appendix A Additional Details on ConflictingQA ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">A</span></a>). We qualitatively find that the questions are diverse and challenging; we show ten examples of them in Table <a href="#S3.T1" title="Table 1 ‣ 3 The ConflictingQA Dataset ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>. We additionally manually remove duplicate questions in the dataset.</p>
</div>
<div id="S3.p4" class="ltx_para ltx_noindent">
<p id="S3.p4.2" class="ltx_p"><span id="S3.p4.2.1" class="ltx_text ltx_font_bold">Collecting evidence paragraphs.</span>
 Given these questions, we want to find evidence paragraphs that support both the answers of <span id="S3.p4.2.2" class="ltx_text ltx_font_typewriter">Yes</span> and <span id="S3.p4.2.3" class="ltx_text ltx_font_typewriter">No</span>. We also want these paragraphs to (1) contain a diverse range of argument styles, factual information, etc., and (2) be realistic inputs to an LLM. To handle this, we emulate running an real-world retrieval-augmented LLM system that uses the Google Search API as its retrieval engine. Concretely, we take the user’s query, reformulate it, and take the top-<math id="S3.p4.1.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S3.p4.1.m1.1a"><mi id="S3.p4.1.m1.1.1" xref="S3.p4.1.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S3.p4.1.m1.1b"><ci id="S3.p4.1.m1.1.1.cmml" xref="S3.p4.1.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p4.1.m1.1c">k</annotation></semantics></math> results from Google search for the answer <span id="S3.p4.2.4" class="ltx_text ltx_font_typewriter">Yes</span> and the top-<math id="S3.p4.2.m2.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S3.p4.2.m2.1a"><mi id="S3.p4.2.m2.1.1" xref="S3.p4.2.m2.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S3.p4.2.m2.1b"><ci id="S3.p4.2.m2.1.1.cmml" xref="S3.p4.2.m2.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p4.2.m2.1c">k</annotation></semantics></math> for the answer <span id="S3.p4.2.5" class="ltx_text ltx_font_typewriter">No</span>.</p>
</div>
<figure id="S3.T3" class="ltx_table">
<div class="ltx_flex_figure ltx_flex_table">
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="S3.T3.1" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_center ltx_align_top" style="width:229.8pt;">
<table id="S3.T3.1.1" class="ltx_tabular ltx_align_middle">
<tr id="S3.T3.1.1.2" class="ltx_tr">
<td id="S3.T3.1.1.2.1" class="ltx_td ltx_align_left ltx_border_tt">Number of questions</td>
<td id="S3.T3.1.1.2.2" class="ltx_td ltx_align_right ltx_border_tt">238</td>
</tr>
<tr id="S3.T3.1.1.3" class="ltx_tr">
<td id="S3.T3.1.1.3.1" class="ltx_td ltx_align_left">Number of question categories</td>
<td id="S3.T3.1.1.3.2" class="ltx_td ltx_align_right">144</td>
</tr>
<tr id="S3.T3.1.1.4" class="ltx_tr">
<td id="S3.T3.1.1.4.1" class="ltx_td ltx_align_left ltx_border_t">Number of retrieved paragraphs</td>
<td id="S3.T3.1.1.4.2" class="ltx_td ltx_align_right ltx_border_t">2,208</td>
</tr>
<tr id="S3.T3.1.1.5" class="ltx_tr">
<td id="S3.T3.1.1.5.1" class="ltx_td ltx_align_left">Average paragraph length (words)</td>
<td id="S3.T3.1.1.5.2" class="ltx_td ltx_align_right">365.01</td>
</tr>
<tr id="S3.T3.1.1.1" class="ltx_tr">
<td id="S3.T3.1.1.1.1" class="ltx_td ltx_align_left ltx_border_t">Number of paragraphs with <math id="S3.T3.1.1.1.1.m1.1" class="ltx_Math" alttext="\geq 5" display="inline"><semantics id="S3.T3.1.1.1.1.m1.1a"><mrow id="S3.T3.1.1.1.1.m1.1.1" xref="S3.T3.1.1.1.1.m1.1.1.cmml"><mi id="S3.T3.1.1.1.1.m1.1.1.2" xref="S3.T3.1.1.1.1.m1.1.1.2.cmml"></mi><mo id="S3.T3.1.1.1.1.m1.1.1.1" xref="S3.T3.1.1.1.1.m1.1.1.1.cmml">≥</mo><mn id="S3.T3.1.1.1.1.m1.1.1.3" xref="S3.T3.1.1.1.1.m1.1.1.3.cmml">5</mn></mrow><annotation-xml encoding="MathML-Content" id="S3.T3.1.1.1.1.m1.1b"><apply id="S3.T3.1.1.1.1.m1.1.1.cmml" xref="S3.T3.1.1.1.1.m1.1.1"><geq id="S3.T3.1.1.1.1.m1.1.1.1.cmml" xref="S3.T3.1.1.1.1.m1.1.1.1"></geq><csymbol cd="latexml" id="S3.T3.1.1.1.1.m1.1.1.2.cmml" xref="S3.T3.1.1.1.1.m1.1.1.2">absent</csymbol><cn type="integer" id="S3.T3.1.1.1.1.m1.1.1.3.cmml" xref="S3.T3.1.1.1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.T3.1.1.1.1.m1.1c">\geq 5</annotation></semantics></math> comparisons</td>
<td id="S3.T3.1.1.1.2" class="ltx_td ltx_align_right ltx_border_t">912</td>
</tr>
<tr id="S3.T3.1.1.6" class="ltx_tr">
<td id="S3.T3.1.1.6.1" class="ltx_td ltx_align_left ltx_border_bb">Average number of comparisons per paragraph</td>
<td id="S3.T3.1.1.6.2" class="ltx_td ltx_align_right ltx_border_bb">6.54</td>
</tr>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Table 2: </span>Basic statistics for <span id="S3.T3.1.3.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span> when evaluating LLaMA-2 Chat. We start by collecting a set of controversial questions for different categories (top). For each question, we retrieve a series of paragraphs from a variety of domains (middle). To determine the convincingness of a paragraph, we compare it against at least five different paragraphs that have the opposite stance/viewpoint (bottom).</figcaption>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="S3.T3.fig1" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_center ltx_align_top" style="width:190.8pt;">
<table id="S3.T3.fig1.1" class="ltx_tabular ltx_align_middle">
<tr id="S3.T3.fig1.1.1" class="ltx_tr">
<td id="S3.T3.fig1.1.1.1" class="ltx_td ltx_align_left ltx_border_tt">
<span id="S3.T3.fig1.1.1.1.1" class="ltx_inline-block">
<span id="S3.T3.fig1.1.1.1.1.1" class="ltx_p"><span id="S3.T3.fig1.1.1.1.1.1.1" class="ltx_text ltx_font_bold">Domain</span></span>
</span>
</td>
<td id="S3.T3.fig1.1.1.2" class="ltx_td ltx_align_right ltx_border_tt">
<span id="S3.T3.fig1.1.1.2.1" class="ltx_inline-block">
<span id="S3.T3.fig1.1.1.2.1.1" class="ltx_p"><span id="S3.T3.fig1.1.1.2.1.1.1" class="ltx_text ltx_font_bold">Count</span></span>
</span>
</td>
</tr>
<tr id="S3.T3.fig1.1.2" class="ltx_tr">
<td id="S3.T3.fig1.1.2.1" class="ltx_td ltx_align_left ltx_border_t">.com</td>
<td id="S3.T3.fig1.1.2.2" class="ltx_td ltx_align_right ltx_border_t">527</td>
</tr>
<tr id="S3.T3.fig1.1.3" class="ltx_tr">
<td id="S3.T3.fig1.1.3.1" class="ltx_td ltx_align_left">.org</td>
<td id="S3.T3.fig1.1.3.2" class="ltx_td ltx_align_right">175</td>
</tr>
<tr id="S3.T3.fig1.1.4" class="ltx_tr">
<td id="S3.T3.fig1.1.4.1" class="ltx_td ltx_align_left">.gov</td>
<td id="S3.T3.fig1.1.4.2" class="ltx_td ltx_align_right">59</td>
</tr>
<tr id="S3.T3.fig1.1.5" class="ltx_tr">
<td id="S3.T3.fig1.1.5.1" class="ltx_td ltx_align_left">.edu</td>
<td id="S3.T3.fig1.1.5.2" class="ltx_td ltx_align_right">57</td>
</tr>
<tr id="S3.T3.fig1.1.6" class="ltx_tr">
<td id="S3.T3.fig1.1.6.1" class="ltx_td ltx_align_left">.net</td>
<td id="S3.T3.fig1.1.6.2" class="ltx_td ltx_align_right">12</td>
</tr>
<tr id="S3.T3.fig1.1.7" class="ltx_tr">
<td id="S3.T3.fig1.1.7.1" class="ltx_td ltx_align_left ltx_border_bb ltx_border_t"># unique</td>
<td id="S3.T3.fig1.1.7.2" class="ltx_td ltx_align_right ltx_border_bb ltx_border_t">39</td>
</tr>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Table 3: </span>The top five most common top-level domains found in <span id="S3.T3.fig1.6.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span> for evaluating LLaMA-2 Chat. The dataset consists of a diverse range of sources, including organizations (<span id="S3.T3.fig1.7.2" class="ltx_text ltx_font_typewriter">.org</span>), schools (<span id="S3.T3.fig1.8.3" class="ltx_text ltx_font_typewriter">.edu</span>), and governments (<span id="S3.T3.fig1.9.4" class="ltx_text ltx_font_typewriter">.gov</span>).</figcaption>
</figure>
</div>
</div>
</figure>
<div id="S3.p5" class="ltx_para">
<p id="S3.p5.1" class="ltx_p">We first turn each question into affirmative and negative statements, e.g., the question “is asparatame safe?” is converted to “asparatame is safe” and “asparatame is harmful” using GPT-4. We also put double quotes (to indicate to Google Search that we have exact-match keywords) around any tokens that do not change after rephrasing the question into either statements (e.g., “aspartame”). For both the affirmative and negative statements, we search the queries using the Google Search API and retrieve top-<math id="S3.p5.1.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S3.p5.1.m1.1a"><mi id="S3.p5.1.m1.1.1" xref="S3.p5.1.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S3.p5.1.m1.1b"><ci id="S3.p5.1.m1.1.1.cmml" xref="S3.p5.1.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p5.1.m1.1c">k</annotation></semantics></math> documents.<span id="footnote1" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_tag ltx_tag_note">1</span>We set <math id="footnote1.m1.1" class="ltx_Math" alttext="k=20" display="inline"><semantics id="footnote1.m1.1b"><mrow id="footnote1.m1.1.1" xref="footnote1.m1.1.1.cmml"><mi id="footnote1.m1.1.1.2" xref="footnote1.m1.1.1.2.cmml">k</mi><mo id="footnote1.m1.1.1.1" xref="footnote1.m1.1.1.1.cmml">=</mo><mn id="footnote1.m1.1.1.3" xref="footnote1.m1.1.1.3.cmml">20</mn></mrow><annotation-xml encoding="MathML-Content" id="footnote1.m1.1c"><apply id="footnote1.m1.1.1.cmml" xref="footnote1.m1.1.1"><eq id="footnote1.m1.1.1.1.cmml" xref="footnote1.m1.1.1.1"></eq><ci id="footnote1.m1.1.1.2.cmml" xref="footnote1.m1.1.1.2">𝑘</ci><cn type="integer" id="footnote1.m1.1.1.3.cmml" xref="footnote1.m1.1.1.3">20</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote1.m1.1d">k=20</annotation></semantics></math> because qualitatively the relevancy of the results dropped off significantly after this point.</span></span></span> As is common in many retrieval-augmented models <cite class="ltx_cite ltx_citemacro_citep">(Nakano et al., <a href="#bib.bib30" title="" class="ltx_ref">2021</a>)</cite>, we do not consider any visual features of the web page. Instead, we extract the raw text from each document using jusText.<span id="footnote2" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">2</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">2</sup><span class="ltx_tag ltx_tag_note">2</span>Package available at <a target="_blank" href="https://github.com/miso-belica/jusText" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/miso-belica/jusText</a>. Although humans use visual features when considering the credibility and trustworthiness of a source <cite class="ltx_cite ltx_citemacro_cite">Kakol et al. (<a href="#bib.bib23" title="" class="ltx_ref">2017</a>); Fogg et al. (<a href="#bib.bib13" title="" class="ltx_ref">2003</a>)</cite>, we do not consider these features as most state-of-the-art LLMs do not use visual inputs.</span></span></span> Additionally, we do not explicitly include metadata like source URL, publication date, or page headings.</p>
</div>
<div id="S3.p6" class="ltx_para">
<p id="S3.p6.1" class="ltx_p">When searching queries such as “aspartame is safe”, we still retrieve documents that argue that aspartame is unsafe. To label the documents actual stance, we use an ensemble of <span id="S3.p6.1.1" class="ltx_text ltx_font_typewriter">claude-instant-v1</span> and <span id="S3.p6.1.2" class="ltx_text ltx_font_typewriter">GPT-4-1106-preview</span> and keep only the samples where the two models agree (see Table <a href="#A1.T7" title="Table 7 ‣ Appendix A Additional Details on ConflictingQA ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a> in Appendix <a href="#A1" title="Appendix A Additional Details on ConflictingQA ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">A</span></a> for the prompts).<span id="footnote3" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">3</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">3</sup><span class="ltx_tag ltx_tag_note">3</span>After identifying the stance, we also feed the paragraphs into the downstream LLM that we are testing and make sure that its answer aligns with the paragraphs predicted stances. This further filters and balances the data, accounting for mistakes in the downstream model. See Appendix <a href="#A2" title="Appendix B Model-based Data Filtering ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">B</span></a> for details.</span></span></span>
Furthermore, we allow the LLM to say that a document is irrelevant to the query; if so, we also filter it from the input.</p>
</div>
<div id="S3.p7" class="ltx_para">
<p id="S3.p7.1" class="ltx_p">Finally, we want to isolate <span id="S3.p7.1.1" class="ltx_text ltx_font_italic">paragraphs</span> from these larger documents to feed into the LLMs (as is common in RAG systems). To do this, we extract the most relevant 512 token window of text inside the document. We run the TAS-B model <cite class="ltx_cite ltx_citemacro_cite">Hofstätter et al. (<a href="#bib.bib19" title="" class="ltx_ref">2021</a>)</cite> across windows of 512 tokens with a 256 token stride, compute the dot product between the model’s embedding of that window and the model’s embedding of the question, and take the highest scoring window. We filter out any documents whose highest-scoring window has a dot product below 95.</p>
</div>
<div id="S3.p8" class="ltx_para ltx_noindent">
<p id="S3.p8.1" class="ltx_p"><span id="S3.p8.1.1" class="ltx_text ltx_font_bold">Creating conflicting examples.</span>
 The end result of our data collection process is (1) a set of controversial questions that (2) have evidence paragraphs which contradict one another. This data can be used in a variety of ways to “stress test” RAG systems in order to understand how they behave under conflicting scenarios. One example of this is shown in Figure <a href="#S1.F1" title="Figure 1 ‣ 1 Introduction ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>, and the subsequent section will explore numerous possible uses of <span id="S3.p8.1.2" class="ltx_text ltx_font_smallcaps">ConflictingQA</span>.
Table <a href="#S3.T3" title="Table 3 ‣ 3 The ConflictingQA Dataset ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a> and Table <a href="#S3.T3" title="Table 3 ‣ 3 The ConflictingQA Dataset ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a> present basic statistics for our final data, accounting for specific filtering done for LLaMA-2 Chat.</p>
</div>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>Experimental Results</h2>

<div id="S4.p1" class="ltx_para">
<p id="S4.p1.1" class="ltx_p">In this section, we use <span id="S4.p1.1.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span> to evaluate what types of evidence models find convincing.</p>
</div>
<section id="S4.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.1 </span>Convincingness as Paragraph Win Rate</h3>

<div id="S4.SS1.p1" class="ltx_para">
<p id="S4.SS1.p1.1" class="ltx_p">We mainly focus on using <span id="S4.SS1.p1.1.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span> in a setup where we ask an LLM a question while providing two conflicting evidence paragraphs (one that supports <span id="S4.SS1.p1.1.2" class="ltx_text ltx_font_typewriter">Yes</span> and one that supports <span id="S4.SS1.p1.1.3" class="ltx_text ltx_font_typewriter">No</span>). Then, we measure which paragraph the model’s answer aligns with. By repeating this for all pairs of paragraphs, we can define the convincingness of a particular paragraph as its <span id="S4.SS1.p1.1.4" class="ltx_text ltx_font_italic">win-rate</span>, i.e., what percent of the time a model picks the answer in that paragraph over the other paragraphs.</p>
</div>
<div id="S4.SS1.p2" class="ltx_para">
<p id="S4.SS1.p2.9" class="ltx_p">Concretely, let <math id="S4.SS1.p2.1.m1.2" class="ltx_Math" alttext="\mathcal{P}_{q,s}" display="inline"><semantics id="S4.SS1.p2.1.m1.2a"><msub id="S4.SS1.p2.1.m1.2.3" xref="S4.SS1.p2.1.m1.2.3.cmml"><mi class="ltx_font_mathcaligraphic" id="S4.SS1.p2.1.m1.2.3.2" xref="S4.SS1.p2.1.m1.2.3.2.cmml">𝒫</mi><mrow id="S4.SS1.p2.1.m1.2.2.2.4" xref="S4.SS1.p2.1.m1.2.2.2.3.cmml"><mi id="S4.SS1.p2.1.m1.1.1.1.1" xref="S4.SS1.p2.1.m1.1.1.1.1.cmml">q</mi><mo id="S4.SS1.p2.1.m1.2.2.2.4.1" xref="S4.SS1.p2.1.m1.2.2.2.3.cmml">,</mo><mi id="S4.SS1.p2.1.m1.2.2.2.2" xref="S4.SS1.p2.1.m1.2.2.2.2.cmml">s</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.1.m1.2b"><apply id="S4.SS1.p2.1.m1.2.3.cmml" xref="S4.SS1.p2.1.m1.2.3"><csymbol cd="ambiguous" id="S4.SS1.p2.1.m1.2.3.1.cmml" xref="S4.SS1.p2.1.m1.2.3">subscript</csymbol><ci id="S4.SS1.p2.1.m1.2.3.2.cmml" xref="S4.SS1.p2.1.m1.2.3.2">𝒫</ci><list id="S4.SS1.p2.1.m1.2.2.2.3.cmml" xref="S4.SS1.p2.1.m1.2.2.2.4"><ci id="S4.SS1.p2.1.m1.1.1.1.1.cmml" xref="S4.SS1.p2.1.m1.1.1.1.1">𝑞</ci><ci id="S4.SS1.p2.1.m1.2.2.2.2.cmml" xref="S4.SS1.p2.1.m1.2.2.2.2">𝑠</ci></list></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.1.m1.2c">\mathcal{P}_{q,s}</annotation></semantics></math> be the set of top-<math id="S4.SS1.p2.2.m2.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S4.SS1.p2.2.m2.1a"><mi id="S4.SS1.p2.2.m2.1.1" xref="S4.SS1.p2.2.m2.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.2.m2.1b"><ci id="S4.SS1.p2.2.m2.1.1.cmml" xref="S4.SS1.p2.2.m2.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.2.m2.1c">k</annotation></semantics></math> paragraphs corresponding to a controversial question <math id="S4.SS1.p2.3.m3.1" class="ltx_Math" alttext="q" display="inline"><semantics id="S4.SS1.p2.3.m3.1a"><mi id="S4.SS1.p2.3.m3.1.1" xref="S4.SS1.p2.3.m3.1.1.cmml">q</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.3.m3.1b"><ci id="S4.SS1.p2.3.m3.1.1.cmml" xref="S4.SS1.p2.3.m3.1.1">𝑞</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.3.m3.1c">q</annotation></semantics></math> with stance <math id="S4.SS1.p2.4.m4.2" class="ltx_Math" alttext="s\in\{\text{{yes}},\text{{no}}\}" display="inline"><semantics id="S4.SS1.p2.4.m4.2a"><mrow id="S4.SS1.p2.4.m4.2.3" xref="S4.SS1.p2.4.m4.2.3.cmml"><mi id="S4.SS1.p2.4.m4.2.3.2" xref="S4.SS1.p2.4.m4.2.3.2.cmml">s</mi><mo id="S4.SS1.p2.4.m4.2.3.1" xref="S4.SS1.p2.4.m4.2.3.1.cmml">∈</mo><mrow id="S4.SS1.p2.4.m4.2.3.3.2" xref="S4.SS1.p2.4.m4.2.3.3.1.cmml"><mo stretchy="false" id="S4.SS1.p2.4.m4.2.3.3.2.1" xref="S4.SS1.p2.4.m4.2.3.3.1.cmml">{</mo><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.4.m4.1.1" xref="S4.SS1.p2.4.m4.1.1a.cmml">yes</mtext><mo id="S4.SS1.p2.4.m4.2.3.3.2.2" xref="S4.SS1.p2.4.m4.2.3.3.1.cmml">,</mo><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.4.m4.2.2" xref="S4.SS1.p2.4.m4.2.2a.cmml">no</mtext><mo stretchy="false" id="S4.SS1.p2.4.m4.2.3.3.2.3" xref="S4.SS1.p2.4.m4.2.3.3.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.4.m4.2b"><apply id="S4.SS1.p2.4.m4.2.3.cmml" xref="S4.SS1.p2.4.m4.2.3"><in id="S4.SS1.p2.4.m4.2.3.1.cmml" xref="S4.SS1.p2.4.m4.2.3.1"></in><ci id="S4.SS1.p2.4.m4.2.3.2.cmml" xref="S4.SS1.p2.4.m4.2.3.2">𝑠</ci><set id="S4.SS1.p2.4.m4.2.3.3.1.cmml" xref="S4.SS1.p2.4.m4.2.3.3.2"><ci id="S4.SS1.p2.4.m4.1.1a.cmml" xref="S4.SS1.p2.4.m4.1.1"><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.4.m4.1.1.cmml" xref="S4.SS1.p2.4.m4.1.1">yes</mtext></ci><ci id="S4.SS1.p2.4.m4.2.2a.cmml" xref="S4.SS1.p2.4.m4.2.2"><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.4.m4.2.2.cmml" xref="S4.SS1.p2.4.m4.2.2">no</mtext></ci></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.4.m4.2c">s\in\{\text{{yes}},\text{{no}}\}</annotation></semantics></math>. We take an LLM <math id="S4.SS1.p2.5.m5.1" class="ltx_Math" alttext="f" display="inline"><semantics id="S4.SS1.p2.5.m5.1a"><mi id="S4.SS1.p2.5.m5.1.1" xref="S4.SS1.p2.5.m5.1.1.cmml">f</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.5.m5.1b"><ci id="S4.SS1.p2.5.m5.1.1.cmml" xref="S4.SS1.p2.5.m5.1.1">𝑓</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.5.m5.1c">f</annotation></semantics></math> (e.g., LLaMA-2 Chat) and ask it for a binary prediction for the question <math id="S4.SS1.p2.6.m6.1" class="ltx_Math" alttext="q" display="inline"><semantics id="S4.SS1.p2.6.m6.1a"><mi id="S4.SS1.p2.6.m6.1.1" xref="S4.SS1.p2.6.m6.1.1.cmml">q</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.6.m6.1b"><ci id="S4.SS1.p2.6.m6.1.1.cmml" xref="S4.SS1.p2.6.m6.1.1">𝑞</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.6.m6.1c">q</annotation></semantics></math>, based on two paragraphs selected from the larger set, <math id="S4.SS1.p2.7.m7.2" class="ltx_Math" alttext="p_{\text{{yes}}}\in\mathcal{P}_{q,\text{{yes}}}" display="inline"><semantics id="S4.SS1.p2.7.m7.2a"><mrow id="S4.SS1.p2.7.m7.2.3" xref="S4.SS1.p2.7.m7.2.3.cmml"><msub id="S4.SS1.p2.7.m7.2.3.2" xref="S4.SS1.p2.7.m7.2.3.2.cmml"><mi id="S4.SS1.p2.7.m7.2.3.2.2" xref="S4.SS1.p2.7.m7.2.3.2.2.cmml">p</mi><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.7.m7.2.3.2.3" xref="S4.SS1.p2.7.m7.2.3.2.3a.cmml">yes</mtext></msub><mo id="S4.SS1.p2.7.m7.2.3.1" xref="S4.SS1.p2.7.m7.2.3.1.cmml">∈</mo><msub id="S4.SS1.p2.7.m7.2.3.3" xref="S4.SS1.p2.7.m7.2.3.3.cmml"><mi class="ltx_font_mathcaligraphic" id="S4.SS1.p2.7.m7.2.3.3.2" xref="S4.SS1.p2.7.m7.2.3.3.2.cmml">𝒫</mi><mrow id="S4.SS1.p2.7.m7.2.2.2.4" xref="S4.SS1.p2.7.m7.2.2.2.3.cmml"><mi id="S4.SS1.p2.7.m7.1.1.1.1" xref="S4.SS1.p2.7.m7.1.1.1.1.cmml">q</mi><mo id="S4.SS1.p2.7.m7.2.2.2.4.1" xref="S4.SS1.p2.7.m7.2.2.2.3.cmml">,</mo><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.7.m7.2.2.2.2" xref="S4.SS1.p2.7.m7.2.2.2.2a.cmml">yes</mtext></mrow></msub></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.7.m7.2b"><apply id="S4.SS1.p2.7.m7.2.3.cmml" xref="S4.SS1.p2.7.m7.2.3"><in id="S4.SS1.p2.7.m7.2.3.1.cmml" xref="S4.SS1.p2.7.m7.2.3.1"></in><apply id="S4.SS1.p2.7.m7.2.3.2.cmml" xref="S4.SS1.p2.7.m7.2.3.2"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.2.3.2.1.cmml" xref="S4.SS1.p2.7.m7.2.3.2">subscript</csymbol><ci id="S4.SS1.p2.7.m7.2.3.2.2.cmml" xref="S4.SS1.p2.7.m7.2.3.2.2">𝑝</ci><ci id="S4.SS1.p2.7.m7.2.3.2.3a.cmml" xref="S4.SS1.p2.7.m7.2.3.2.3"><mtext class="ltx_mathvariant_monospace" mathsize="70%" id="S4.SS1.p2.7.m7.2.3.2.3.cmml" xref="S4.SS1.p2.7.m7.2.3.2.3">yes</mtext></ci></apply><apply id="S4.SS1.p2.7.m7.2.3.3.cmml" xref="S4.SS1.p2.7.m7.2.3.3"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.2.3.3.1.cmml" xref="S4.SS1.p2.7.m7.2.3.3">subscript</csymbol><ci id="S4.SS1.p2.7.m7.2.3.3.2.cmml" xref="S4.SS1.p2.7.m7.2.3.3.2">𝒫</ci><list id="S4.SS1.p2.7.m7.2.2.2.3.cmml" xref="S4.SS1.p2.7.m7.2.2.2.4"><ci id="S4.SS1.p2.7.m7.1.1.1.1.cmml" xref="S4.SS1.p2.7.m7.1.1.1.1">𝑞</ci><ci id="S4.SS1.p2.7.m7.2.2.2.2a.cmml" xref="S4.SS1.p2.7.m7.2.2.2.2"><mtext class="ltx_mathvariant_monospace" mathsize="70%" id="S4.SS1.p2.7.m7.2.2.2.2.cmml" xref="S4.SS1.p2.7.m7.2.2.2.2">yes</mtext></ci></list></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.7.m7.2c">p_{\text{{yes}}}\in\mathcal{P}_{q,\text{{yes}}}</annotation></semantics></math> and <math id="S4.SS1.p2.8.m8.2" class="ltx_Math" alttext="p_{\text{{no}}}\in\mathcal{P}_{q,\text{{yes}}}" display="inline"><semantics id="S4.SS1.p2.8.m8.2a"><mrow id="S4.SS1.p2.8.m8.2.3" xref="S4.SS1.p2.8.m8.2.3.cmml"><msub id="S4.SS1.p2.8.m8.2.3.2" xref="S4.SS1.p2.8.m8.2.3.2.cmml"><mi id="S4.SS1.p2.8.m8.2.3.2.2" xref="S4.SS1.p2.8.m8.2.3.2.2.cmml">p</mi><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.8.m8.2.3.2.3" xref="S4.SS1.p2.8.m8.2.3.2.3a.cmml">no</mtext></msub><mo id="S4.SS1.p2.8.m8.2.3.1" xref="S4.SS1.p2.8.m8.2.3.1.cmml">∈</mo><msub id="S4.SS1.p2.8.m8.2.3.3" xref="S4.SS1.p2.8.m8.2.3.3.cmml"><mi class="ltx_font_mathcaligraphic" id="S4.SS1.p2.8.m8.2.3.3.2" xref="S4.SS1.p2.8.m8.2.3.3.2.cmml">𝒫</mi><mrow id="S4.SS1.p2.8.m8.2.2.2.4" xref="S4.SS1.p2.8.m8.2.2.2.3.cmml"><mi id="S4.SS1.p2.8.m8.1.1.1.1" xref="S4.SS1.p2.8.m8.1.1.1.1.cmml">q</mi><mo id="S4.SS1.p2.8.m8.2.2.2.4.1" xref="S4.SS1.p2.8.m8.2.2.2.3.cmml">,</mo><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.8.m8.2.2.2.2" xref="S4.SS1.p2.8.m8.2.2.2.2a.cmml">yes</mtext></mrow></msub></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.8.m8.2b"><apply id="S4.SS1.p2.8.m8.2.3.cmml" xref="S4.SS1.p2.8.m8.2.3"><in id="S4.SS1.p2.8.m8.2.3.1.cmml" xref="S4.SS1.p2.8.m8.2.3.1"></in><apply id="S4.SS1.p2.8.m8.2.3.2.cmml" xref="S4.SS1.p2.8.m8.2.3.2"><csymbol cd="ambiguous" id="S4.SS1.p2.8.m8.2.3.2.1.cmml" xref="S4.SS1.p2.8.m8.2.3.2">subscript</csymbol><ci id="S4.SS1.p2.8.m8.2.3.2.2.cmml" xref="S4.SS1.p2.8.m8.2.3.2.2">𝑝</ci><ci id="S4.SS1.p2.8.m8.2.3.2.3a.cmml" xref="S4.SS1.p2.8.m8.2.3.2.3"><mtext class="ltx_mathvariant_monospace" mathsize="70%" id="S4.SS1.p2.8.m8.2.3.2.3.cmml" xref="S4.SS1.p2.8.m8.2.3.2.3">no</mtext></ci></apply><apply id="S4.SS1.p2.8.m8.2.3.3.cmml" xref="S4.SS1.p2.8.m8.2.3.3"><csymbol cd="ambiguous" id="S4.SS1.p2.8.m8.2.3.3.1.cmml" xref="S4.SS1.p2.8.m8.2.3.3">subscript</csymbol><ci id="S4.SS1.p2.8.m8.2.3.3.2.cmml" xref="S4.SS1.p2.8.m8.2.3.3.2">𝒫</ci><list id="S4.SS1.p2.8.m8.2.2.2.3.cmml" xref="S4.SS1.p2.8.m8.2.2.2.4"><ci id="S4.SS1.p2.8.m8.1.1.1.1.cmml" xref="S4.SS1.p2.8.m8.1.1.1.1">𝑞</ci><ci id="S4.SS1.p2.8.m8.2.2.2.2a.cmml" xref="S4.SS1.p2.8.m8.2.2.2.2"><mtext class="ltx_mathvariant_monospace" mathsize="70%" id="S4.SS1.p2.8.m8.2.2.2.2.cmml" xref="S4.SS1.p2.8.m8.2.2.2.2">yes</mtext></ci></list></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.8.m8.2c">p_{\text{{no}}}\in\mathcal{P}_{q,\text{{yes}}}</annotation></semantics></math>. The model makes a prediction: <math id="S4.SS1.p2.9.m9.5" class="ltx_Math" alttext="f(p_{\text{{yes}}},p_{\text{{no}}},q)\in\{\text{{yes}},\text{{no}}\}" display="inline"><semantics id="S4.SS1.p2.9.m9.5a"><mrow id="S4.SS1.p2.9.m9.5.5" xref="S4.SS1.p2.9.m9.5.5.cmml"><mrow id="S4.SS1.p2.9.m9.5.5.2" xref="S4.SS1.p2.9.m9.5.5.2.cmml"><mi id="S4.SS1.p2.9.m9.5.5.2.4" xref="S4.SS1.p2.9.m9.5.5.2.4.cmml">f</mi><mo lspace="0em" rspace="0em" id="S4.SS1.p2.9.m9.5.5.2.3" xref="S4.SS1.p2.9.m9.5.5.2.3.cmml">​</mo><mrow id="S4.SS1.p2.9.m9.5.5.2.2.2" xref="S4.SS1.p2.9.m9.5.5.2.2.3.cmml"><mo stretchy="false" id="S4.SS1.p2.9.m9.5.5.2.2.2.3" xref="S4.SS1.p2.9.m9.5.5.2.2.3.cmml">(</mo><msub id="S4.SS1.p2.9.m9.4.4.1.1.1.1" xref="S4.SS1.p2.9.m9.4.4.1.1.1.1.cmml"><mi id="S4.SS1.p2.9.m9.4.4.1.1.1.1.2" xref="S4.SS1.p2.9.m9.4.4.1.1.1.1.2.cmml">p</mi><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.9.m9.4.4.1.1.1.1.3" xref="S4.SS1.p2.9.m9.4.4.1.1.1.1.3a.cmml">yes</mtext></msub><mo id="S4.SS1.p2.9.m9.5.5.2.2.2.4" xref="S4.SS1.p2.9.m9.5.5.2.2.3.cmml">,</mo><msub id="S4.SS1.p2.9.m9.5.5.2.2.2.2" xref="S4.SS1.p2.9.m9.5.5.2.2.2.2.cmml"><mi id="S4.SS1.p2.9.m9.5.5.2.2.2.2.2" xref="S4.SS1.p2.9.m9.5.5.2.2.2.2.2.cmml">p</mi><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.9.m9.5.5.2.2.2.2.3" xref="S4.SS1.p2.9.m9.5.5.2.2.2.2.3a.cmml">no</mtext></msub><mo id="S4.SS1.p2.9.m9.5.5.2.2.2.5" xref="S4.SS1.p2.9.m9.5.5.2.2.3.cmml">,</mo><mi id="S4.SS1.p2.9.m9.1.1" xref="S4.SS1.p2.9.m9.1.1.cmml">q</mi><mo stretchy="false" id="S4.SS1.p2.9.m9.5.5.2.2.2.6" xref="S4.SS1.p2.9.m9.5.5.2.2.3.cmml">)</mo></mrow></mrow><mo id="S4.SS1.p2.9.m9.5.5.3" xref="S4.SS1.p2.9.m9.5.5.3.cmml">∈</mo><mrow id="S4.SS1.p2.9.m9.5.5.4.2" xref="S4.SS1.p2.9.m9.5.5.4.1.cmml"><mo stretchy="false" id="S4.SS1.p2.9.m9.5.5.4.2.1" xref="S4.SS1.p2.9.m9.5.5.4.1.cmml">{</mo><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.9.m9.2.2" xref="S4.SS1.p2.9.m9.2.2a.cmml">yes</mtext><mo id="S4.SS1.p2.9.m9.5.5.4.2.2" xref="S4.SS1.p2.9.m9.5.5.4.1.cmml">,</mo><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.9.m9.3.3" xref="S4.SS1.p2.9.m9.3.3a.cmml">no</mtext><mo stretchy="false" id="S4.SS1.p2.9.m9.5.5.4.2.3" xref="S4.SS1.p2.9.m9.5.5.4.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.9.m9.5b"><apply id="S4.SS1.p2.9.m9.5.5.cmml" xref="S4.SS1.p2.9.m9.5.5"><in id="S4.SS1.p2.9.m9.5.5.3.cmml" xref="S4.SS1.p2.9.m9.5.5.3"></in><apply id="S4.SS1.p2.9.m9.5.5.2.cmml" xref="S4.SS1.p2.9.m9.5.5.2"><times id="S4.SS1.p2.9.m9.5.5.2.3.cmml" xref="S4.SS1.p2.9.m9.5.5.2.3"></times><ci id="S4.SS1.p2.9.m9.5.5.2.4.cmml" xref="S4.SS1.p2.9.m9.5.5.2.4">𝑓</ci><vector id="S4.SS1.p2.9.m9.5.5.2.2.3.cmml" xref="S4.SS1.p2.9.m9.5.5.2.2.2"><apply id="S4.SS1.p2.9.m9.4.4.1.1.1.1.cmml" xref="S4.SS1.p2.9.m9.4.4.1.1.1.1"><csymbol cd="ambiguous" id="S4.SS1.p2.9.m9.4.4.1.1.1.1.1.cmml" xref="S4.SS1.p2.9.m9.4.4.1.1.1.1">subscript</csymbol><ci id="S4.SS1.p2.9.m9.4.4.1.1.1.1.2.cmml" xref="S4.SS1.p2.9.m9.4.4.1.1.1.1.2">𝑝</ci><ci id="S4.SS1.p2.9.m9.4.4.1.1.1.1.3a.cmml" xref="S4.SS1.p2.9.m9.4.4.1.1.1.1.3"><mtext class="ltx_mathvariant_monospace" mathsize="70%" id="S4.SS1.p2.9.m9.4.4.1.1.1.1.3.cmml" xref="S4.SS1.p2.9.m9.4.4.1.1.1.1.3">yes</mtext></ci></apply><apply id="S4.SS1.p2.9.m9.5.5.2.2.2.2.cmml" xref="S4.SS1.p2.9.m9.5.5.2.2.2.2"><csymbol cd="ambiguous" id="S4.SS1.p2.9.m9.5.5.2.2.2.2.1.cmml" xref="S4.SS1.p2.9.m9.5.5.2.2.2.2">subscript</csymbol><ci id="S4.SS1.p2.9.m9.5.5.2.2.2.2.2.cmml" xref="S4.SS1.p2.9.m9.5.5.2.2.2.2.2">𝑝</ci><ci id="S4.SS1.p2.9.m9.5.5.2.2.2.2.3a.cmml" xref="S4.SS1.p2.9.m9.5.5.2.2.2.2.3"><mtext class="ltx_mathvariant_monospace" mathsize="70%" id="S4.SS1.p2.9.m9.5.5.2.2.2.2.3.cmml" xref="S4.SS1.p2.9.m9.5.5.2.2.2.2.3">no</mtext></ci></apply><ci id="S4.SS1.p2.9.m9.1.1.cmml" xref="S4.SS1.p2.9.m9.1.1">𝑞</ci></vector></apply><set id="S4.SS1.p2.9.m9.5.5.4.1.cmml" xref="S4.SS1.p2.9.m9.5.5.4.2"><ci id="S4.SS1.p2.9.m9.2.2a.cmml" xref="S4.SS1.p2.9.m9.2.2"><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.9.m9.2.2.cmml" xref="S4.SS1.p2.9.m9.2.2">yes</mtext></ci><ci id="S4.SS1.p2.9.m9.3.3a.cmml" xref="S4.SS1.p2.9.m9.3.3"><mtext class="ltx_mathvariant_monospace" id="S4.SS1.p2.9.m9.3.3.cmml" xref="S4.SS1.p2.9.m9.3.3">no</mtext></ci></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.9.m9.5c">f(p_{\text{{yes}}},p_{\text{{no}}},q)\in\{\text{{yes}},\text{{no}}\}</annotation></semantics></math>.</p>
</div>
<div id="S4.SS1.p3" class="ltx_para">
<p id="S4.SS1.p3.1" class="ltx_p">For each paragraph, we define its win-rate as the empirical probability of the model’s prediction aligning with its stance when paired with a set of conflicting paragraphs, i.e.,</p>
<table id="A5.EGx1" class="ltx_equationgroup ltx_eqn_align ltx_eqn_table">

<tbody id="S4.Ex1"><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="S4.Ex1.m1.8" class="ltx_Math" alttext="\displaystyle\text{WR}(p_{\text{{yes}}},q)=\operatorname{\mathbb{E}}_{p\sim\mathcal{P}_{q,\text{{no}}}}[\mathds{1}[f(p_{\text{{yes}}},p,q)=\text{{yes}}]]" display="inline"><semantics id="S4.Ex1.m1.8a"><mrow id="S4.Ex1.m1.8.8" xref="S4.Ex1.m1.8.8.cmml"><mrow id="S4.Ex1.m1.6.6.1" xref="S4.Ex1.m1.6.6.1.cmml"><mtext id="S4.Ex1.m1.6.6.1.3" xref="S4.Ex1.m1.6.6.1.3a.cmml">WR</mtext><mo lspace="0em" rspace="0em" id="S4.Ex1.m1.6.6.1.2" xref="S4.Ex1.m1.6.6.1.2.cmml">​</mo><mrow id="S4.Ex1.m1.6.6.1.1.1" xref="S4.Ex1.m1.6.6.1.1.2.cmml"><mo stretchy="false" id="S4.Ex1.m1.6.6.1.1.1.2" xref="S4.Ex1.m1.6.6.1.1.2.cmml">(</mo><msub id="S4.Ex1.m1.6.6.1.1.1.1" xref="S4.Ex1.m1.6.6.1.1.1.1.cmml"><mi id="S4.Ex1.m1.6.6.1.1.1.1.2" xref="S4.Ex1.m1.6.6.1.1.1.1.2.cmml">p</mi><mtext class="ltx_mathvariant_monospace" id="S4.Ex1.m1.6.6.1.1.1.1.3" xref="S4.Ex1.m1.6.6.1.1.1.1.3a.cmml">yes</mtext></msub><mo id="S4.Ex1.m1.6.6.1.1.1.3" xref="S4.Ex1.m1.6.6.1.1.2.cmml">,</mo><mi id="S4.Ex1.m1.3.3" xref="S4.Ex1.m1.3.3.cmml">q</mi><mo stretchy="false" id="S4.Ex1.m1.6.6.1.1.1.4" xref="S4.Ex1.m1.6.6.1.1.2.cmml">)</mo></mrow></mrow><mo id="S4.Ex1.m1.8.8.4" xref="S4.Ex1.m1.8.8.4.cmml">=</mo><mrow id="S4.Ex1.m1.8.8.3.2" xref="S4.Ex1.m1.8.8.3.3.cmml"><msub id="S4.Ex1.m1.7.7.2.1.1" xref="S4.Ex1.m1.7.7.2.1.1.cmml"><mi id="S4.Ex1.m1.7.7.2.1.1.2" xref="S4.Ex1.m1.7.7.2.1.1.2.cmml">𝔼</mi><mrow id="S4.Ex1.m1.2.2.2" xref="S4.Ex1.m1.2.2.2.cmml"><mi id="S4.Ex1.m1.2.2.2.4" xref="S4.Ex1.m1.2.2.2.4.cmml">p</mi><mo id="S4.Ex1.m1.2.2.2.3" xref="S4.Ex1.m1.2.2.2.3.cmml">∼</mo><msub id="S4.Ex1.m1.2.2.2.5" xref="S4.Ex1.m1.2.2.2.5.cmml"><mi class="ltx_font_mathcaligraphic" id="S4.Ex1.m1.2.2.2.5.2" xref="S4.Ex1.m1.2.2.2.5.2.cmml">𝒫</mi><mrow id="S4.Ex1.m1.2.2.2.2.2.4" xref="S4.Ex1.m1.2.2.2.2.2.3.cmml"><mi id="S4.Ex1.m1.1.1.1.1.1.1" xref="S4.Ex1.m1.1.1.1.1.1.1.cmml">q</mi><mo id="S4.Ex1.m1.2.2.2.2.2.4.1" xref="S4.Ex1.m1.2.2.2.2.2.3.cmml">,</mo><mtext class="ltx_mathvariant_monospace" id="S4.Ex1.m1.2.2.2.2.2.2" xref="S4.Ex1.m1.2.2.2.2.2.2a.cmml">no</mtext></mrow></msub></mrow></msub><mo id="S4.Ex1.m1.8.8.3.2a" xref="S4.Ex1.m1.8.8.3.3.cmml">⁡</mo><mrow id="S4.Ex1.m1.8.8.3.2.2" xref="S4.Ex1.m1.8.8.3.3.cmml"><mo stretchy="false" id="S4.Ex1.m1.8.8.3.2.2.2" xref="S4.Ex1.m1.8.8.3.3.cmml">[</mo><mrow id="S4.Ex1.m1.8.8.3.2.2.1" xref="S4.Ex1.m1.8.8.3.2.2.1.cmml"><mn id="S4.Ex1.m1.8.8.3.2.2.1.3" xref="S4.Ex1.m1.8.8.3.2.2.1.3.cmml">𝟙</mn><mo lspace="0em" rspace="0em" id="S4.Ex1.m1.8.8.3.2.2.1.2" xref="S4.Ex1.m1.8.8.3.2.2.1.2.cmml">​</mo><mrow id="S4.Ex1.m1.8.8.3.2.2.1.1.1" xref="S4.Ex1.m1.8.8.3.2.2.1.1.2.cmml"><mo stretchy="false" id="S4.Ex1.m1.8.8.3.2.2.1.1.1.2" xref="S4.Ex1.m1.8.8.3.2.2.1.1.2.1.cmml">[</mo><mrow id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.cmml"><mrow id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.cmml"><mi id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.3" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.3.cmml">f</mi><mo lspace="0em" rspace="0em" id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.2" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.2.cmml">​</mo><mrow id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.2.cmml"><mo stretchy="false" id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.2" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.2.cmml">(</mo><msub id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.cmml"><mi id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.2" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.2.cmml">p</mi><mtext class="ltx_mathvariant_monospace" id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.3" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.3a.cmml">yes</mtext></msub><mo id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.3" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.2.cmml">,</mo><mi id="S4.Ex1.m1.4.4" xref="S4.Ex1.m1.4.4.cmml">p</mi><mo id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.4" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.2.cmml">,</mo><mi id="S4.Ex1.m1.5.5" xref="S4.Ex1.m1.5.5.cmml">q</mi><mo stretchy="false" id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.5" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.2.cmml">)</mo></mrow></mrow><mo id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.2" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.2.cmml">=</mo><mtext class="ltx_mathvariant_monospace" id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.3" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.3a.cmml">yes</mtext></mrow><mo stretchy="false" id="S4.Ex1.m1.8.8.3.2.2.1.1.1.3" xref="S4.Ex1.m1.8.8.3.2.2.1.1.2.1.cmml">]</mo></mrow></mrow><mo stretchy="false" id="S4.Ex1.m1.8.8.3.2.2.3" xref="S4.Ex1.m1.8.8.3.3.cmml">]</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.Ex1.m1.8b"><apply id="S4.Ex1.m1.8.8.cmml" xref="S4.Ex1.m1.8.8"><eq id="S4.Ex1.m1.8.8.4.cmml" xref="S4.Ex1.m1.8.8.4"></eq><apply id="S4.Ex1.m1.6.6.1.cmml" xref="S4.Ex1.m1.6.6.1"><times id="S4.Ex1.m1.6.6.1.2.cmml" xref="S4.Ex1.m1.6.6.1.2"></times><ci id="S4.Ex1.m1.6.6.1.3a.cmml" xref="S4.Ex1.m1.6.6.1.3"><mtext id="S4.Ex1.m1.6.6.1.3.cmml" xref="S4.Ex1.m1.6.6.1.3">WR</mtext></ci><interval closure="open" id="S4.Ex1.m1.6.6.1.1.2.cmml" xref="S4.Ex1.m1.6.6.1.1.1"><apply id="S4.Ex1.m1.6.6.1.1.1.1.cmml" xref="S4.Ex1.m1.6.6.1.1.1.1"><csymbol cd="ambiguous" id="S4.Ex1.m1.6.6.1.1.1.1.1.cmml" xref="S4.Ex1.m1.6.6.1.1.1.1">subscript</csymbol><ci id="S4.Ex1.m1.6.6.1.1.1.1.2.cmml" xref="S4.Ex1.m1.6.6.1.1.1.1.2">𝑝</ci><ci id="S4.Ex1.m1.6.6.1.1.1.1.3a.cmml" xref="S4.Ex1.m1.6.6.1.1.1.1.3"><mtext class="ltx_mathvariant_monospace" mathsize="70%" id="S4.Ex1.m1.6.6.1.1.1.1.3.cmml" xref="S4.Ex1.m1.6.6.1.1.1.1.3">yes</mtext></ci></apply><ci id="S4.Ex1.m1.3.3.cmml" xref="S4.Ex1.m1.3.3">𝑞</ci></interval></apply><apply id="S4.Ex1.m1.8.8.3.3.cmml" xref="S4.Ex1.m1.8.8.3.2"><apply id="S4.Ex1.m1.7.7.2.1.1.cmml" xref="S4.Ex1.m1.7.7.2.1.1"><csymbol cd="ambiguous" id="S4.Ex1.m1.7.7.2.1.1.1.cmml" xref="S4.Ex1.m1.7.7.2.1.1">subscript</csymbol><ci id="S4.Ex1.m1.7.7.2.1.1.2.cmml" xref="S4.Ex1.m1.7.7.2.1.1.2">𝔼</ci><apply id="S4.Ex1.m1.2.2.2.cmml" xref="S4.Ex1.m1.2.2.2"><csymbol cd="latexml" id="S4.Ex1.m1.2.2.2.3.cmml" xref="S4.Ex1.m1.2.2.2.3">similar-to</csymbol><ci id="S4.Ex1.m1.2.2.2.4.cmml" xref="S4.Ex1.m1.2.2.2.4">𝑝</ci><apply id="S4.Ex1.m1.2.2.2.5.cmml" xref="S4.Ex1.m1.2.2.2.5"><csymbol cd="ambiguous" id="S4.Ex1.m1.2.2.2.5.1.cmml" xref="S4.Ex1.m1.2.2.2.5">subscript</csymbol><ci id="S4.Ex1.m1.2.2.2.5.2.cmml" xref="S4.Ex1.m1.2.2.2.5.2">𝒫</ci><list id="S4.Ex1.m1.2.2.2.2.2.3.cmml" xref="S4.Ex1.m1.2.2.2.2.2.4"><ci id="S4.Ex1.m1.1.1.1.1.1.1.cmml" xref="S4.Ex1.m1.1.1.1.1.1.1">𝑞</ci><ci id="S4.Ex1.m1.2.2.2.2.2.2a.cmml" xref="S4.Ex1.m1.2.2.2.2.2.2"><mtext class="ltx_mathvariant_monospace" mathsize="50%" id="S4.Ex1.m1.2.2.2.2.2.2.cmml" xref="S4.Ex1.m1.2.2.2.2.2.2">no</mtext></ci></list></apply></apply></apply><apply id="S4.Ex1.m1.8.8.3.2.2.1.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1"><times id="S4.Ex1.m1.8.8.3.2.2.1.2.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.2"></times><cn type="integer" id="S4.Ex1.m1.8.8.3.2.2.1.3.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.3">1</cn><apply id="S4.Ex1.m1.8.8.3.2.2.1.1.2.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1"><csymbol cd="latexml" id="S4.Ex1.m1.8.8.3.2.2.1.1.2.1.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.2">delimited-[]</csymbol><apply id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1"><eq id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.2.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.2"></eq><apply id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1"><times id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.2.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.2"></times><ci id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.3.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.3">𝑓</ci><vector id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.2.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1"><apply id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.1.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.2.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.2">𝑝</ci><ci id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.3a.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.3"><mtext class="ltx_mathvariant_monospace" mathsize="70%" id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.3.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.1.1.1.1.3">yes</mtext></ci></apply><ci id="S4.Ex1.m1.4.4.cmml" xref="S4.Ex1.m1.4.4">𝑝</ci><ci id="S4.Ex1.m1.5.5.cmml" xref="S4.Ex1.m1.5.5">𝑞</ci></vector></apply><ci id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.3a.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.3"><mtext class="ltx_mathvariant_monospace" id="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.3.cmml" xref="S4.Ex1.m1.8.8.3.2.2.1.1.1.1.3">yes</mtext></ci></apply></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.Ex1.m1.8c">\displaystyle\text{WR}(p_{\text{{yes}}},q)=\operatorname{\mathbb{E}}_{p\sim\mathcal{P}_{q,\text{{no}}}}[\mathds{1}[f(p_{\text{{yes}}},p,q)=\text{{yes}}]]</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S4.SS1.p3.2" class="ltx_p">Finally, as the ordering of the retrieved evidence is known to bias model predictions <cite class="ltx_cite ltx_citemacro_citep">(Xie et al., <a href="#bib.bib46" title="" class="ltx_ref">2023</a>)</cite>, we calculate win-rate based on both orderings of the retrieved paragraphs. We additionally filter our dataset to ensure that each win-rate calculation consists of comparisons with at least five unique paragraphs.</p>
</div>
<div id="S4.SS1.p4" class="ltx_para ltx_noindent">
<p id="S4.SS1.p4.1" class="ltx_p"><span id="S4.SS1.p4.1.1" class="ltx_text ltx_font_bold">Models Cannot Predict Convincingness</span>
 We designed the above experimental setting to emulate how production RAG models work. However, we could have instead just directly asked the LLM, “<span id="S4.SS1.p4.1.2" class="ltx_text ltx_font_italic">do you find paragraph X to be persuasive?</span>”. This is how humans are typically asked to judge the convincingness of a piece of evidence <cite class="ltx_cite ltx_citemacro_citep">(Kakol et al., <a href="#bib.bib23" title="" class="ltx_ref">2017</a>; Jo et al., <a href="#bib.bib21" title="" class="ltx_ref">2019</a>; Kakol et al., <a href="#bib.bib22" title="" class="ltx_ref">2013</a>)</cite>. However, we find that LLMs are largely incapable of expressing the convincingness of a paragraph in words, e.g., there is little correlation in which paragraphs are marked as convincing in the two settings (Figure <a href="#S4.F3" title="Figure 3 ‣ 4.2 Implementation Details ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>).<span id="footnote4" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">4</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">4</sup><span class="ltx_tag ltx_tag_note">4</span>Our methodology for this setting is described in more detail in Appendix <a href="#A4" title="Appendix D Expressing Convincingness in Isolation ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">D</span></a>.</span></span></span> We thus focus on the more practically-grounded setting going forward.</p>
</div>
</section>
<section id="S4.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.2 </span>Implementation Details</h3>

<div id="S4.SS2.p1" class="ltx_para">
<p id="S4.SS2.p1.1" class="ltx_p">We evaluate a mix of open-source (LLaMA-2 Chat <cite class="ltx_cite ltx_citemacro_citep">(Touvron et al., <a href="#bib.bib44" title="" class="ltx_ref">2023b</a>)</cite>, Vicuna v1.5 <cite class="ltx_cite ltx_citemacro_citep">(Chiang et al., <a href="#bib.bib9" title="" class="ltx_ref">2023</a>)</cite>, and WizardLM v1.2 <cite class="ltx_cite ltx_citemacro_citep">(Xu et al., <a href="#bib.bib47" title="" class="ltx_ref">2023</a>)</cite>) and closed source (GPT-4, Anthropic Claude v1 Instant) models. Importantly, we specify “Use only the information in the above text to answer the question” as we are looking to see how models judge stylistic differences in evidence, rather than their prior stances on the question.</p>
</div>
<div id="S4.SS2.p2" class="ltx_para">
<p id="S4.SS2.p2.1" class="ltx_p">We extract binary <span id="S4.SS2.p2.1.1" class="ltx_text ltx_font_typewriter">Yes</span>/<span id="S4.SS2.p2.1.2" class="ltx_text ltx_font_typewriter">No</span> predictions from the model for each question. For open-source models, we compare the log-probabilities of the next-token. For the closed-source models, we prompt them to output only <span id="S4.SS2.p2.1.3" class="ltx_text ltx_font_typewriter">Yes</span> or <span id="S4.SS2.p2.1.4" class="ltx_text ltx_font_typewriter">No</span>. See Table <a href="#A1.T8" title="Table 8 ‣ Appendix A Additional Details on ConflictingQA ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a> for the prompt used for question answering.</p>
</div>
<figure id="S4.F3" class="ltx_figure"><img src="/html/2402.11782/assets/x3.png" id="S4.F3.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="159" height="120" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 3: </span>Humans can read a paragraph in isolation and evaluate how convincing it is. For LLMs, when they are given a paragraph in isolation, they are <span id="S4.F3.2.1" class="ltx_text ltx_font_italic">unable to express its convincingness in words</span>. Concretely, we plot the win rate of paragraphs versus what a model outputs when it is asked to judge the convincingness on a 1–5 Likert scale. The error bars show a 95% CI.</figcaption>
</figure>
<figure id="S4.F4" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="S4.F4.sf1" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x4.png" id="S4.F4.sf1.g1" class="ltx_graphics ltx_img_landscape" width="105" height="77" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">(a) </span>Flesh Kincaid Readability</figcaption>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="S4.F4.sf2" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x5.png" id="S4.F4.sf2.g1" class="ltx_graphics ltx_img_landscape" width="105" height="78" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">(b) </span>Num Unique Tokens</figcaption>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="S4.F4.sf3" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x6.png" id="S4.F4.sf3.g1" class="ltx_graphics ltx_img_landscape" width="105" height="76" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">(c) </span>Sentiment (FLAN confidence)</figcaption>
</figure>
</div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="S4.F4.sf4" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x7.png" id="S4.F4.sf4.g1" class="ltx_graphics ltx_img_landscape" width="105" height="77" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">(d) </span>Perplexity (GPT-2 Medium)</figcaption>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="S4.F4.sf5" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="" id="S4.F4.sf5.g1" class="ltx_graphics ltx_missing ltx_missing_image" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">(e) </span><math id="S4.F4.sf5.2.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S4.F4.sf5.2.m1.1b"><mi id="S4.F4.sf5.2.m1.1.1" xref="S4.F4.sf5.2.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S4.F4.sf5.2.m1.1c"><ci id="S4.F4.sf5.2.m1.1.1.cmml" xref="S4.F4.sf5.2.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F4.sf5.2.m1.1d">n</annotation></semantics></math>-gram Overlap w/ Question</figcaption>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="S4.F4.sf6" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x9.png" id="S4.F4.sf6.g1" class="ltx_graphics ltx_img_landscape" width="105" height="78" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">(f) </span>Question-Para Embedding Sim.</figcaption>
</figure>
</div>
</div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 4: </span>
<span id="S4.F4.2.1" class="ltx_text ltx_font_italic">Why do models prefer certain paragraphs over others?</span> We test correlations between different features and paragraph win-rates. Here, we show LLaMA-2 Chat 13B (see all other models in Appendix <a href="#A3" title="Appendix C Additional Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">C</span></a>), where the model tends to prefer samples with low-perplexity <a href="#S4.F4.sf4" title="In Figure 4 ‣ 4.2 Implementation Details ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref ltx_font_bold"><span class="ltx_text ltx_ref_tag">4(d)</span></a>. In addition, paragraphs with high relevancy scores—particularly high question-paragraph embedding similarity are also highly convincing <a href="#S4.F4.sf6" title="In Figure 4 ‣ 4.2 Implementation Details ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref ltx_font_bold"><span class="ltx_text ltx_ref_tag">4(f)</span></a>. See Figure <a href="#S2.F2" title="Figure 2 ‣ 2 Background and Motivations ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> for additional analysis. The error bars show the 95% CI (n = 242).</figcaption>
</figure>
</section>
<section id="S4.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.3 </span>What Correlates With Convincingness?</h3>

<div id="S4.SS3.p1" class="ltx_para">
<p id="S4.SS3.p1.1" class="ltx_p">After collecting the win rates for each paragraph, we look to explain <span id="S4.SS3.p1.1.1" class="ltx_text ltx_font_italic">why</span> models pick some paragraphs over others. We first compute several automatic metrics and correlate them with the win-rate:</p>
<ul id="S4.I1" class="ltx_itemize">
<li id="S4.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S4.I1.i1.p1" class="ltx_para">
<p id="S4.I1.i1.p1.1" class="ltx_p"><span id="S4.I1.i1.p1.1.1" class="ltx_text ltx_font_bold">Readability</span>: We use the Flesch-Kincaid readability test <cite class="ltx_cite ltx_citemacro_citep">(Kincaid et al., <a href="#bib.bib25" title="" class="ltx_ref">1975</a>)</cite>. This metric considers readability as a function of the average number of words per sentence and average number of syllables per word.</p>
</div>
</li>
<li id="S4.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S4.I1.i2.p1" class="ltx_para">
<p id="S4.I1.i2.p1.1" class="ltx_p"><span id="S4.I1.i2.p1.1.1" class="ltx_text ltx_font_bold">Number of unique tokens</span>: We measure the number of unique lemmas in the text.<span id="footnote5" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">5</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">5</sup><span class="ltx_tag ltx_tag_note">5</span>We use the <span id="footnote5.1" class="ltx_text ltx_font_typewriter">WordNetLemmatizer</span> from the <span id="footnote5.2" class="ltx_text ltx_font_typewriter">nltk</span> library.</span></span></span></p>
</div>
</li>
<li id="S4.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S4.I1.i3.p1" class="ltx_para">
<p id="S4.I1.i3.p1.1" class="ltx_p"><span id="S4.I1.i3.p1.1.1" class="ltx_text ltx_font_bold">Binary sentiment</span>: We measure the probability of positive sentiment using the FLAN-large model <cite class="ltx_cite ltx_citemacro_citep">(Wei et al., <a href="#bib.bib45" title="" class="ltx_ref">2022</a>)</cite>.</p>
</div>
</li>
<li id="S4.I1.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S4.I1.i4.p1" class="ltx_para">
<p id="S4.I1.i4.p1.1" class="ltx_p"><span id="S4.I1.i4.p1.1.1" class="ltx_text ltx_font_bold">Perplexity</span>: We measure this using the GPT-2 medium model <cite class="ltx_cite ltx_citemacro_cite">Radford et al. (<a href="#bib.bib32" title="" class="ltx_ref">2019</a>)</cite>.</p>
</div>
</li>
<li id="S4.I1.i5" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S4.I1.i5.p1" class="ltx_para">
<p id="S4.I1.i5.p1.1" class="ltx_p"><span id="S4.I1.i5.p1.1.1" class="ltx_text ltx_font_bold">n-gram overlap</span>: We measure the maximum length n-gram that is common to the question and paragraph.</p>
</div>
</li>
<li id="S4.I1.i6" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S4.I1.i6.p1" class="ltx_para">
<p id="S4.I1.i6.p1.1" class="ltx_p"><span id="S4.I1.i6.p1.1.1" class="ltx_text ltx_font_bold">Question embedding similarity</span>: We use TAS-B to measure the relevance of the question to the paragraph (as described in Section <a href="#S3" title="3 The ConflictingQA Dataset ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>).</p>
</div>
</li>
</ul>
</div>
<div id="S4.SS3.p2" class="ltx_para ltx_noindent">
<p id="S4.SS3.p2.1" class="ltx_p"><span id="S4.SS3.p2.1.1" class="ltx_text ltx_font_bold">Results</span>
 <span id="S4.SS3.p2.1.2" class="ltx_text ltx_font_italic">Stylistic features are poor predictors of paragraph convincingness</span>. Figure <a href="#S4.F4" title="Figure 4 ‣ 4.2 Implementation Details ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a> shows the results for the LLaMA-2 Chat model and Figures <a href="#A3.F5" title="Figure 5 ‣ Appendix C Additional Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>–<a href="#A3.F8" title="Figure 8 ‣ Appendix C Additional Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a> shows the results for other models. For example, across all models the Flesch-Kincaid score and number of unique tokens does not correlate with convincingness. Similarly, paragraphs with a more positive sentiment and lower perplexity tend to have some small impact on convincingness, with varying strengths from model to model.</p>
</div>
<div id="S4.SS3.p3" class="ltx_para">
<p id="S4.SS3.p3.1" class="ltx_p">On the other hand, question-paragraph embedding similarity correlates strongly with win-rate across all models except for GPT-4. Similarly, a positive (but weaker) correlational exists between n-gram overlap and win-rate.</p>
</div>
</section>
<section id="S4.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.4 </span>Counterfactual Analysis</h3>

<div id="S4.SS4.p1" class="ltx_para">
<p id="S4.SS4.p1.1" class="ltx_p">Rather than a correlational study, we also test how win-rates change in a counterfactual setting where we directly edit paragraphs using an LLM. We make perturbations using <span id="S4.SS4.p1.1.1" class="ltx_text ltx_font_typewriter">claude-v1-instant</span>, examples of which are shown in Figure <a href="#A5.F9" title="Figure 9 ‣ Appendix E Counterfactual Perturbations ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a>.</p>
</div>
<div id="S4.SS4.p2" class="ltx_para ltx_noindent">
<p id="S4.SS4.p2.1" class="ltx_p"><span id="S4.SS4.p2.1.1" class="ltx_text ltx_font_bold">Stylistic changes</span>
 We first consider changes inspired by factors that humans find important for the credibility of a text. For example, adding more information, adding scientific references, or making the text sound more objective. Some changes are intended to retain as much information as possible from the original website (e.g., <span id="S4.SS4.p2.1.2" class="ltx_text ltx_font_typewriter">Add Science Reference</span>, <span id="S4.SS4.p2.1.3" class="ltx_text ltx_font_typewriter">Add More Info</span>). Others involve significantly changing the entire paragraph (e.g., <span id="S4.SS4.p2.1.4" class="ltx_text ltx_font_typewriter">Rewrite Objective</span>, <span id="S4.SS4.p2.1.5" class="ltx_text ltx_font_typewriter">Rewrite Tech. Language</span>). All of the perturbations are described further in Appendix <a href="#A5" title="Appendix E Counterfactual Perturbations ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">E</span></a>.</p>
</div>
<div id="S4.SS4.p3" class="ltx_para ltx_noindent">
<p id="S4.SS4.p3.1" class="ltx_p"><span id="S4.SS4.p3.1.1" class="ltx_text ltx_font_bold">Relevancy changes</span>
 Based on the results in Section <a href="#S4.SS3" title="4.3 What Correlates With Convincingness? ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.3</span></a>, we also consider several changes that make the text more relevant to the question. This includes rewriting the text (<span id="S4.SS4.p3.1.2" class="ltx_text ltx_font_typewriter">Rewrite Relevance</span>), adding keywords (<span id="S4.SS4.p3.1.3" class="ltx_text ltx_font_typewriter">Keyword Stuffing</span>), and prefixing the paragraph with “The following text is about the question: <span id="S4.SS4.p3.1.4" class="ltx_text ltx_font_typewriter">[question]</span>.” (<span id="S4.SS4.p3.1.5" class="ltx_text ltx_font_typewriter">Question Prefix</span>). Finally, we consider a perturbation inspired by the “AddSent” perturbation in <cite class="ltx_cite ltx_citemacro_cite">Jia and Liang (<a href="#bib.bib20" title="" class="ltx_ref">2017</a>)</cite> where we use <span id="S4.SS4.p3.1.6" class="ltx_text ltx_font_typewriter">claude-v1-instant</span> to add a single sentence to make stance of a text obvious (<span id="S4.SS4.p3.1.7" class="ltx_text ltx_font_typewriter">Add Single Sentence</span>). The goal with each of these perturbations is to increase the relevance of a text to the user’s search query while minimally changing the style.</p>
</div>
<div id="S4.SS4.p4" class="ltx_para">
<p id="S4.SS4.p4.1" class="ltx_p">We also compare these perturbations against a “control” perturbation where text is suffixed with “Thanks for reading!” This perturbation minimally influences both style and relevance. For simplicity we only perturb the paragraphs with the <span id="S4.SS4.p4.1.1" class="ltx_text ltx_font_typewriter">Yes</span> stance.</p>
</div>
<div id="S4.SS4.p5" class="ltx_para ltx_noindent">
<p id="S4.SS4.p5.1" class="ltx_p"><span id="S4.SS4.p5.1.1" class="ltx_text ltx_font_bold">Counterfactual results</span>
 The results for the counterfactual experiments are shown in Figure <a href="#S2.F2" title="Figure 2 ‣ 2 Background and Motivations ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>: compared to the effect of the control perturbation, stylistic features tend to have a neutral to negative effect while relevancy-based features significantly improve win-rate. Note that many of these perturbations change a smaller amount of tokens than stylistic features—leaving the <span id="S4.SS4.p5.1.2" class="ltx_text ltx_font_italic">content</span> of the website largely unchanged (e.g., <span id="S4.SS4.p5.1.3" class="ltx_text ltx_font_typewriter">Add Single Sentence</span>, <span id="S4.SS4.p5.1.4" class="ltx_text ltx_font_typewriter">Question Prefix</span>)—but are still able to improve the convincingness of websites.</p>
</div>
<div id="S4.SS4.p6" class="ltx_para">
<p id="S4.SS4.p6.1" class="ltx_p">Overall, we find that, as compared to typical finding from human experiments <cite class="ltx_cite ltx_citemacro_cite">Fogg et al. (<a href="#bib.bib13" title="" class="ltx_ref">2003</a>); Kakol et al. (<a href="#bib.bib22" title="" class="ltx_ref">2013</a>); Metzger et al. (<a href="#bib.bib27" title="" class="ltx_ref">2010</a>)</cite>, <span id="S4.SS4.p6.1.1" class="ltx_text ltx_font_italic">LLMs tend to overindex on relevancy</span>. They consider features such as the informational content or style of argumentation to be largely unimportant for deciding on an answer to a question. Instead, making simplistic changes like increasing the amount of <math id="S4.SS4.p6.1.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S4.SS4.p6.1.m1.1a"><mi id="S4.SS4.p6.1.m1.1.1" xref="S4.SS4.p6.1.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S4.SS4.p6.1.m1.1b"><ci id="S4.SS4.p6.1.m1.1.1.cmml" xref="S4.SS4.p6.1.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS4.p6.1.m1.1c">n</annotation></semantics></math>-gram overlap between the question and the paragraph can substantially improve its convincingness.</p>
</div>
<figure id="S4.T4" class="ltx_table">
<table id="S4.T4.1" class="ltx_tabular ltx_centering ltx_align_middle">
<tr id="S4.T4.1.1" class="ltx_tr">
<td id="S4.T4.1.1.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_tt">
<span id="S4.T4.1.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.1.1.1.1" class="ltx_p" style="width:166.2pt;">
<span id="S4.T4.1.1.1.1.1.1" class="ltx_inline-block">
<span id="S4.T4.1.1.1.1.1.1.1" class="ltx_p"><span id="S4.T4.1.1.1.1.1.1.1.1" class="ltx_text ltx_font_bold">Question</span></span>
</span></span>
</span>
</td>
<td id="S4.T4.1.1.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_tt">
<span id="S4.T4.1.1.2.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.1.2.1.1" class="ltx_p" style="width:122.9pt;">
<span id="S4.T4.1.1.2.1.1.1" class="ltx_inline-block">
<span id="S4.T4.1.1.2.1.1.1.1" class="ltx_p"><span id="S4.T4.1.1.2.1.1.1.1.1" class="ltx_text ltx_font_bold">Affirmative</span></span>
</span></span>
</span>
</td>
<td id="S4.T4.1.1.3" class="ltx_td ltx_align_justify ltx_align_top ltx_border_tt">
<span id="S4.T4.1.1.3.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.1.3.1.1" class="ltx_p" style="width:126.5pt;">
<span id="S4.T4.1.1.3.1.1.1" class="ltx_inline-block">
<span id="S4.T4.1.1.3.1.1.1.1" class="ltx_p"><span id="S4.T4.1.1.3.1.1.1.1.1" class="ltx_text ltx_font_bold">Negative</span></span>
</span></span>
</span>
</td>
</tr>
<tr id="S4.T4.1.2" class="ltx_tr">
<td id="S4.T4.1.2.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S4.T4.1.2.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.2.1.1.1" class="ltx_p" style="width:166.2pt;">Are Coral snakes found in Africa?</span>
</span>
</td>
<td id="S4.T4.1.2.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S4.T4.1.2.2.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.2.2.1.1" class="ltx_p" style="width:122.9pt;">Old-world coral snakes are found in Africa, the Middle East, India, and parts of Southeast Asia. New World coral snakes can be found in North America, Central America, and South America.</span>
</span>
</td>
<td id="S4.T4.1.2.3" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S4.T4.1.2.3.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.2.3.1.1" class="ltx_p" style="width:126.5pt;">Coral snakes are found in scattered localities in the southern coastal plains from North Carolina to Louisiana, including all of Florida.</span>
</span>
</td>
</tr>
<tr id="S4.T4.1.3" class="ltx_tr">
<td id="S4.T4.1.3.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S4.T4.1.3.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.3.1.1.1" class="ltx_p" style="width:166.2pt;">Are Florida Panthers on the brink of extinction?</span>
</span>
</td>
<td id="S4.T4.1.3.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S4.T4.1.3.2.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.3.2.1.1" class="ltx_p" style="width:122.9pt;">As Florida’s panther numbers plummeted, the state’s human population nearly doubled over the past 30 years. Recent development patterns pose threats to panthers.</span>
</span>
</td>
<td id="S4.T4.1.3.3" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="S4.T4.1.3.3.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.3.3.1.1" class="ltx_p" style="width:126.5pt;">Now, though, their population is on the upswing … Both the numbers and the genetic diversity of Florida panthers improved.</span>
</span>
</td>
</tr>
<tr id="S4.T4.1.4" class="ltx_tr">
<td id="S4.T4.1.4.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_bb ltx_border_t">
<span id="S4.T4.1.4.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.4.1.1.1" class="ltx_p" style="width:166.2pt;">Are artificial sweeteners safe for diabetics?</span>
</span>
</td>
<td id="S4.T4.1.4.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_bb ltx_border_t">
<span id="S4.T4.1.4.2.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.4.2.1.1" class="ltx_p" style="width:122.9pt;">A new study published in February revealed that consuming large amounts of the artificial sweetener erythritol can lead to an increased risk of heart attacks and strokes.</span>
</span>
</td>
<td id="S4.T4.1.4.3" class="ltx_td ltx_align_justify ltx_align_top ltx_border_bb ltx_border_t">
<span id="S4.T4.1.4.3.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T4.1.4.3.1.1" class="ltx_p" style="width:126.5pt;">Furthermore, xylitol does not need insulin to be metabolized, so it can be safely consumed by diabetics.</span>
</span>
</td>
</tr>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 4: </span>We show examples of knowledge conflicts in real retrieved evidence. For example, questions may be underspecified (e.g., “old-world” vs “new-world” coral snakes). In other cases, the answer is dependent on the publication date (e.g., <span id="S4.T4.3.1" class="ltx_text ltx_font_italic">currently</span> on the brink vs recent upswing). Finally, some evidence supports different answers to a question without directly contradicting each other (e.g., the safety of two different artificial sweeteners).</figcaption>
</figure>
</section>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Discussion &amp; Related Work</h2>

<div id="S5.p1" class="ltx_para ltx_noindent">
<p id="S5.p1.1" class="ltx_p"><span id="S5.p1.1.1" class="ltx_text ltx_font_bold">How should systems handle ambiguity?</span>
 One reasonable suggestion is that agents should not make their own autonomous decisions when faced with ambiguous or conflicting evidence. For example, they may summarize <span id="S5.p1.1.2" class="ltx_text ltx_font_italic">both</span> sides of the aspartame argument, or they may ask the user to clarify their preferences. There is naturally a trade-off between autonomy and clarity. Past work has explored one side of this trade-off, for example by abstaining from answering in cases of ambiguity <cite class="ltx_cite ltx_citemacro_cite">Chen et al. (<a href="#bib.bib8" title="" class="ltx_ref">2022</a>)</cite>, by trying to provide multiple perspectives on the answer <cite class="ltx_cite ltx_citemacro_cite">Min et al. (<a href="#bib.bib29" title="" class="ltx_ref">2020</a>)</cite>, or by asking clarification questions <cite class="ltx_cite ltx_citemacro_citep">(Rao and Daumé III, <a href="#bib.bib35" title="" class="ltx_ref">2018</a>; Zamani et al., <a href="#bib.bib48" title="" class="ltx_ref">2020</a>)</cite>. Our work explores the other side of the trade-off: we analyze the behavior of models when they are expected to resolve ambiguity with more autonomy.</p>
</div>
<div id="S5.p2" class="ltx_para">
<p id="S5.p2.1" class="ltx_p">Additionally, our dataset serves as a benchmark for exploring these questions as it reflects real-world ambiguities in question-answering. For example, in Table <a href="#S4.T4" title="Table 4 ‣ 4.4 Counterfactual Analysis ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>) to best answer “Are Coral snakes found in Africa?”, additional clarification questions would be needed from the user.</p>
</div>
<section id="S5.SS0.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Optimizing misinformation and SEO.</h5>

<div id="S5.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="S5.SS0.SSS0.Px1.p1.1" class="ltx_p">In principle, our insights could also be used to <span id="S5.SS0.SSS0.Px1.p1.1.1" class="ltx_text ltx_font_italic">optimize</span> paragraphs to increase the chance that a QA model is convinced by it. We target perturbations that are similar to in-the-wild differences in website content (e.g., scientific references, informational content, etc.) but past work has more directly created adversarial examples <cite class="ltx_cite ltx_citemacro_citep">(Du et al., <a href="#bib.bib10" title="" class="ltx_ref">2022</a>; Abdelnabi and Fritz, <a href="#bib.bib1" title="" class="ltx_ref">2023</a>; Pan et al., <a href="#bib.bib31" title="" class="ltx_ref">2023</a>; Aggarwal et al., <a href="#bib.bib3" title="" class="ltx_ref">2023</a>)</cite>. Our counterfactual perturbations could also be used in a search engine optimization (SEO) fashion to increase how often a certain product or company is mentioned in a RAG LLM’s answer <cite class="ltx_cite ltx_citemacro_citep">(Sharma et al., <a href="#bib.bib39" title="" class="ltx_ref">2019</a>)</cite>. Indeed, concurrent work has explored ideas such as this <cite class="ltx_cite ltx_citemacro_citep">(Aggarwal et al., <a href="#bib.bib3" title="" class="ltx_ref">2023</a>)</cite>, where they aim to optimize “impressions” in long-form answers by maximizing the number of tokens from a particular paragraph that appear in an output. We instead study how model <span id="S5.SS0.SSS0.Px1.p1.1.2" class="ltx_text ltx_font_italic">answers</span> can be manipulated.</p>
</div>
<div id="S5.SS0.SSS0.Px1.p2" class="ltx_para ltx_noindent">
<p id="S5.SS0.SSS0.Px1.p2.1" class="ltx_p"><span id="S5.SS0.SSS0.Px1.p2.1.1" class="ltx_text ltx_font_bold">Improving model judgements.</span>
 Our work highlights the gap between human and model judgements of text credibility. This solution to this, however, is not clear cut. For one, it is not clear the level of discretion models should have when making predictions. Human judgements of website credibility differ from person to person <cite class="ltx_cite ltx_citemacro_citep">(Kakol et al., <a href="#bib.bib22" title="" class="ltx_ref">2013</a>)</cite>, and users may not be comfortable with the idea that models are “choosing” for them what source to trust. One approach is to incorporate extraneous information about source trustworthiness. For example, <cite class="ltx_cite ltx_citemacro_citet">Bashlovkina et al. (<a href="#bib.bib4" title="" class="ltx_ref">2023</a>)</cite> propose aligning model predictions with that of known trustworthy sources via prompting. Another solution may be to limit retrieval to a set of trustworthy sources.</p>
</div>
</section>
</section>
<section id="S6" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">6 </span>Conclusion</h2>

<div id="S6.p1" class="ltx_para">
<p id="S6.p1.1" class="ltx_p">We study how RAG model judge convincingness by collecting a diverse set of controversial questions and website text (<span id="S6.p1.1.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span>), and designing a realistic evaluation framework based on how these models are used in practice. Our results show that today’s LLMs tend to overrely on relevancy and ignore many stylistic features of text that humans often deem important. Future work should explore how integrating other forms of information (e.g., metadata, visual content) can influence these behaviors. In addition, given the possible flood of LLM-generated content on the internet, it is important to consider how these synthetic texts may influence LLM judgements of convincingness.</p>
</div>
</section>
<section id="Sx1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">Limitations</h2>

<div id="Sx1.p1" class="ltx_para">
<p id="Sx1.p1.1" class="ltx_p">While <span id="Sx1.p1.1.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span> is diverse and simulates real-world uses of RAG models, it may not fully capture the complexity of how LLMs are used in practice. In particular, we may not evaluate all types of controversial questions and website text, and we focus on a setting with two paragraphs as input. We also only consider a binary Yes or No answer to contentious questions whereas LLM outputs in practice may be more nuanced. Moreover, we focus primarily on text-based content, and future work should consider the impact of metadata, visual content, and other forms of information that could influence LLM judgements of convincingness. Finally, we acknowledge that our study does not address the broader ethical and societal implications of LLMs both reading and generating most of the content on the web. Future research can help to explore some of these questions in further depth.</p>
</div>
<section id="Sx1.SS0.SSSx1" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">Acknowledgments</h4>

<div id="Sx1.SS0.SSSx1.p1" class="ltx_para">
<p id="Sx1.SS0.SSSx1.p1.1" class="ltx_p">We thank Sewon Min and the members of Berkeley NLP for useful feedback on this project. We thank Ruiqi Zhong for helping us run the D5 codebase. Eric Wallace is supported by the Apple Scholars in AI/ML Fellowship.
Part of this research was supported with Cloud TPUs from Google’s TPU Research Cloud (TRC).</p>
</div>
</section>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Abdelnabi and Fritz (2023)</span>
<span class="ltx_bibblock">
Sahar Abdelnabi and Mario Fritz. 2023.

</span>
<span class="ltx_bibblock">Fact-Saboteurs: A taxonomy of evidence manipulation attacks
against fact-verification systems.

</span>
<span class="ltx_bibblock">In <em id="bib.bib1.1.1" class="ltx_emph ltx_font_italic">USENIX</em>.

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Adept (2022)</span>
<span class="ltx_bibblock">
Adept. 2022.

</span>
<span class="ltx_bibblock">ACT-1: Transformer for actions.

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Aggarwal et al. (2023)</span>
<span class="ltx_bibblock">
Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik R
Narasimhan, and Ameet Deshpande. 2023.

</span>
<span class="ltx_bibblock">GEO: Generative engine optimization.

</span>
<span class="ltx_bibblock"><em id="bib.bib3.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2311.09735</em>.

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bashlovkina et al. (2023)</span>
<span class="ltx_bibblock">
Vasilisa Bashlovkina, Zhaobin Kuang, Riley Matthews, Edward Clifford, Yennie
Jun, William W. Cohen, and Simon Baumgartner. 2023.

</span>
<span class="ltx_bibblock">Trusted source alignment in large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib4.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2311.06697</em>.

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Brown et al. (2020)</span>
<span class="ltx_bibblock">
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan,
Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, et al. 2020.

</span>
<span class="ltx_bibblock">Language models are few-shot learners.

</span>
<span class="ltx_bibblock">In <em id="bib.bib5.1.1" class="ltx_emph ltx_font_italic">NeurIPS</em>.

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bush and Zaheer (2019)</span>
<span class="ltx_bibblock">
Daniel Bush and Alex Zaheer. 2019.

</span>
<span class="ltx_bibblock">Bing’s top search results contain an alarming amount of
disinformation.

</span>
<span class="ltx_bibblock"><em id="bib.bib6.1.1" class="ltx_emph ltx_font_italic">Internet Observatory News</em>.

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2017)</span>
<span class="ltx_bibblock">
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017.

</span>
<span class="ltx_bibblock">Reading Wikipedia to answer open-domain questions.

</span>
<span class="ltx_bibblock">In <em id="bib.bib7.1.1" class="ltx_emph ltx_font_italic">ACL</em>.

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2022)</span>
<span class="ltx_bibblock">
Hung-Ting Chen, Michael JQ Zhang, and Eunsol Choi. 2022.

</span>
<span class="ltx_bibblock">Rich knowledge sources bring complex knowledge conflicts:
Recalibrating models to reflect conflicting evidence.

</span>
<span class="ltx_bibblock">In <em id="bib.bib8.1.1" class="ltx_emph ltx_font_italic">EMNLP</em>.

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chiang et al. (2023)</span>
<span class="ltx_bibblock">
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin
Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and
Eric P. Xing. 2023.

</span>
<span class="ltx_bibblock">Vicuna: An open-source chatbot impressing GPT-4 with 90%*
ChatGPT quality.

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Du et al. (2022)</span>
<span class="ltx_bibblock">
Yibing Du, Antoine Bosselut, and Christopher D Manning. 2022.

</span>
<span class="ltx_bibblock">Synthetic disinformation attacks on automated fact verification
systems.

</span>
<span class="ltx_bibblock">In <em id="bib.bib10.1.1" class="ltx_emph ltx_font_italic">AAAI</em>.

</span>
</li>
<li id="bib.bib11" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Eldan and Li (2023)</span>
<span class="ltx_bibblock">
Ronen Eldan and Yuanzhi Li. 2023.

</span>
<span class="ltx_bibblock">Tinystories: How small can language models be and still speak
coherent english?

</span>
</li>
<li id="bib.bib12" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Flanagin and Metzger (2000)</span>
<span class="ltx_bibblock">
Andrew J. Flanagin and Miriam J. Metzger. 2000.

</span>
<span class="ltx_bibblock">Perceptions of internet information credibility.

</span>
<span class="ltx_bibblock"><em id="bib.bib12.1.1" class="ltx_emph ltx_font_italic">Journalism &amp; Mass Communication Quarterly</em>.

</span>
</li>
<li id="bib.bib13" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Fogg et al. (2003)</span>
<span class="ltx_bibblock">
B. J. Fogg, Cathy Soohoo, David R. Danielson, Leslie Marable, Julianne
Stanford, and Ellen R. Tauber. 2003.

</span>
<span class="ltx_bibblock">How do users evaluate the credibility of web sites? A study with
over 2,500 participants.

</span>
<span class="ltx_bibblock">In <em id="bib.bib13.1.1" class="ltx_emph ltx_font_italic">Designing for User Experiences</em>.

</span>
</li>
<li id="bib.bib14" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gao et al. (2023)</span>
<span class="ltx_bibblock">
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie
Callan, and Graham Neubig. 2023.

</span>
<span class="ltx_bibblock">PAL: Program-aided language models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib14.1.1" class="ltx_emph ltx_font_italic">ICML</em>.

</span>
</li>
<li id="bib.bib15" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gleize et al. (2019)</span>
<span class="ltx_bibblock">
Martin Gleize, Eyal Shnarch, Leshem Choshen, Lena Dankin, Guy Moshkowich, Ranit
Aharonov, and Noam Slonim. 2019.

</span>
<span class="ltx_bibblock">Are you convinced? Choosing the more convincing evidence with a
siamese network.

</span>
<span class="ltx_bibblock">In <em id="bib.bib15.1.1" class="ltx_emph ltx_font_italic">ACL</em>.

</span>
</li>
<li id="bib.bib16" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gretz et al. (2019)</span>
<span class="ltx_bibblock">
Shai Gretz, Roni Friedman, Edo Cohen-Karlik, Assaf Toledo, Dan Lahav, Ranit
Aharonov, and Noam Slonim. 2019.

</span>
<span class="ltx_bibblock">A large-scale dataset for argument quality ranking: Construction and
analysis.

</span>
</li>
<li id="bib.bib17" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gunasekar et al. (2023)</span>
<span class="ltx_bibblock">
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del
Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa,
Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang,
Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi
Li. 2023.

</span>
<span class="ltx_bibblock">Textbooks are all you need.

</span>
</li>
<li id="bib.bib18" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Guu et al. (2020)</span>
<span class="ltx_bibblock">
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020.

</span>
<span class="ltx_bibblock">Retrieval augmented language model pre-training.

</span>
<span class="ltx_bibblock">In <em id="bib.bib18.1.1" class="ltx_emph ltx_font_italic">ICML</em>.

</span>
</li>
<li id="bib.bib19" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hofstätter et al. (2021)</span>
<span class="ltx_bibblock">
Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and
Allan Hanbury. 2021.

</span>
<span class="ltx_bibblock">Efficiently teaching an effective dense retriever with balanced topic
aware sampling.

</span>
<span class="ltx_bibblock">In <em id="bib.bib19.1.1" class="ltx_emph ltx_font_italic">SIGIR</em>.

</span>
</li>
<li id="bib.bib20" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Jia and Liang (2017)</span>
<span class="ltx_bibblock">
Robin Jia and Percy Liang. 2017.

</span>
<span class="ltx_bibblock">Adversarial examples for evaluating reading comprehension systems.

</span>
<span class="ltx_bibblock">In <em id="bib.bib20.1.1" class="ltx_emph ltx_font_italic">EMNLP</em>.

</span>
</li>
<li id="bib.bib21" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Jo et al. (2019)</span>
<span class="ltx_bibblock">
Yonggeol Jo, Minwoo Kim, and Kyungsik Han. 2019.

</span>
<span class="ltx_bibblock">How do humans assess the credibility on web blogs: Qualifying and
verifying human factors with machine learning.

</span>
<span class="ltx_bibblock"><em id="bib.bib21.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2019 CHI Conference on Human Factors in
Computing Systems</em>.

</span>
</li>
<li id="bib.bib22" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kakol et al. (2013)</span>
<span class="ltx_bibblock">
Michal Kakol, Michał Jankowski-Lorek, Katarzyna Abramczuk, Adam Wierzbicki,
and Michele Catasta. 2013.

</span>
<span class="ltx_bibblock">On the subjectivity and bias of web content credibility evaluations.

</span>
<span class="ltx_bibblock">In <em id="bib.bib22.1.1" class="ltx_emph ltx_font_italic">WWW</em>.

</span>
</li>
<li id="bib.bib23" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kakol et al. (2017)</span>
<span class="ltx_bibblock">
Michal Kakol, Radoslaw Nielek, and Adam Wierzbicki. 2017.

</span>
<span class="ltx_bibblock">Understanding and predicting web content credibility using the
content credibility corpus.

</span>
<span class="ltx_bibblock"><em id="bib.bib23.1.1" class="ltx_emph ltx_font_italic">Information Processing &amp; Management</em>.

</span>
</li>
<li id="bib.bib24" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Karpukhin et al. (2020)</span>
<span class="ltx_bibblock">
Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu,
Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020.

</span>
<span class="ltx_bibblock">Dense passage retrieval for open-domain question answering.

</span>
<span class="ltx_bibblock">In <em id="bib.bib24.1.1" class="ltx_emph ltx_font_italic">EMNLP</em>.

</span>
</li>
<li id="bib.bib25" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kincaid et al. (1975)</span>
<span class="ltx_bibblock">
J Peter Kincaid, Robert P Fishburne Jr, Richard L Rogers, and Brad S Chissom.
1975.

</span>
<span class="ltx_bibblock">Derivation of new readability formulas (automated readability index,
fog count and flesch reading ease formula) for Navy enlisted personnel.

</span>
<span class="ltx_bibblock">Technical report, Naval Technical Training Command Research Branch.

</span>
</li>
<li id="bib.bib26" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Longpre et al. (2021)</span>
<span class="ltx_bibblock">
Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois,
and Sameer Singh. 2021.

</span>
<span class="ltx_bibblock">Entity-based knowledge conflicts in question answering.

</span>
<span class="ltx_bibblock">In <em id="bib.bib26.1.1" class="ltx_emph ltx_font_italic">EMNLP</em>.

</span>
</li>
<li id="bib.bib27" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Metzger et al. (2010)</span>
<span class="ltx_bibblock">
Miriam J. Metzger, Andrew J. Flanagin, and Ryan Bradley Medders. 2010.

</span>
<span class="ltx_bibblock">Social and heuristic approaches to credibility evaluation online.

</span>
<span class="ltx_bibblock"><em id="bib.bib27.1.1" class="ltx_emph ltx_font_italic">Journal of Communication</em>.

</span>
</li>
<li id="bib.bib28" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Mialon et al. (2023)</span>
<span class="ltx_bibblock">
Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis,
Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane
Dwivedi-Yu, Asli Celikyilmaz, et al. 2023.

</span>
<span class="ltx_bibblock">Augmented language models: A survey.

</span>
<span class="ltx_bibblock">In <em id="bib.bib28.1.1" class="ltx_emph ltx_font_italic">TMLR</em>.

</span>
</li>
<li id="bib.bib29" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Min et al. (2020)</span>
<span class="ltx_bibblock">
Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020.

</span>
<span class="ltx_bibblock">AmbigQA: Answering ambiguous open-domain questions.

</span>
<span class="ltx_bibblock">In <em id="bib.bib29.1.1" class="ltx_emph ltx_font_italic">EMNLP</em>.

</span>
</li>
<li id="bib.bib30" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Nakano et al. (2021)</span>
<span class="ltx_bibblock">
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina
Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders,
et al. 2021.

</span>
<span class="ltx_bibblock">WebGPT: Browser-assisted question-answering with human feedback.

</span>
<span class="ltx_bibblock"><em id="bib.bib30.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2112.09332</em>.

</span>
</li>
<li id="bib.bib31" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Pan et al. (2023)</span>
<span class="ltx_bibblock">
Liangming Pan, Wenhu Chen, Min-Yen Kan, and William Yang Wang. 2023.

</span>
<span class="ltx_bibblock">Attacking open-domain question answering by injecting misinformation.

</span>
<span class="ltx_bibblock">In <em id="bib.bib31.1.1" class="ltx_emph ltx_font_italic">AACL</em>.

</span>
</li>
<li id="bib.bib32" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Radford et al. (2019)</span>
<span class="ltx_bibblock">
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya
Sutskever, et al. 2019.

</span>
<span class="ltx_bibblock">Language models are unsupervised multitask learners.

</span>
<span class="ltx_bibblock"><em id="bib.bib32.1.1" class="ltx_emph ltx_font_italic">OpenAI blog</em>.

</span>
</li>
<li id="bib.bib33" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Raffel et al. (2020)</span>
<span class="ltx_bibblock">
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael
Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020.

</span>
<span class="ltx_bibblock">Exploring the limits of transfer learning with a unified text-to-text
transformer.

</span>
<span class="ltx_bibblock"><em id="bib.bib33.1.1" class="ltx_emph ltx_font_italic">JMLR</em>.

</span>
</li>
<li id="bib.bib34" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ram et al. (2023)</span>
<span class="ltx_bibblock">
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin
Leyton-Brown, and Yoav Shoham. 2023.

</span>
<span class="ltx_bibblock">In-context retrieval-augmented language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib34.1.1" class="ltx_emph ltx_font_italic">TACL</em>.

</span>
</li>
<li id="bib.bib35" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Rao and Daumé III (2018)</span>
<span class="ltx_bibblock">
Sudha Rao and Hal Daumé III. 2018.

</span>
<span class="ltx_bibblock">Learning to ask good questions: Ranking clarification questions using
neural expected value of perfect information.

</span>
<span class="ltx_bibblock">In <em id="bib.bib35.1.1" class="ltx_emph ltx_font_italic">ACL</em>.

</span>
</li>
<li id="bib.bib36" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Richards (2023)</span>
<span class="ltx_bibblock">
Toran Bruce Richards. 2023.

</span>
<span class="ltx_bibblock">AutoGPT.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://github.com/Significant-Gravitas/AutoGPT" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/Significant-Gravitas/AutoGPT</a>.

</span>
</li>
<li id="bib.bib37" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Santurkar et al. (2023)</span>
<span class="ltx_bibblock">
Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and
Tatsunori Hashimoto. 2023.

</span>
<span class="ltx_bibblock">Whose opinions do language models reflect?

</span>
<span class="ltx_bibblock">In <em id="bib.bib37.1.1" class="ltx_emph ltx_font_italic">ICML</em>.

</span>
</li>
<li id="bib.bib38" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Schick et al. (2023)</span>
<span class="ltx_bibblock">
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria
Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023.

</span>
<span class="ltx_bibblock">Toolformer: Language models can teach themselves to use tools.

</span>
<span class="ltx_bibblock"><em id="bib.bib38.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2302.04761</em>.

</span>
</li>
<li id="bib.bib39" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Sharma et al. (2019)</span>
<span class="ltx_bibblock">
Dushyant Sharma, Rishabh Shukla, Anil Kumar Giri, and Sumit Kumar. 2019.

</span>
<span class="ltx_bibblock">A brief review on search engine optimization.

</span>
<span class="ltx_bibblock">In <em id="bib.bib39.1.1" class="ltx_emph ltx_font_italic">Confluence</em>.

</span>
</li>
<li id="bib.bib40" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Shi et al. (2023)</span>
<span class="ltx_bibblock">
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis,
Luke Zettlemoyer, and Wen-tau Yih. 2023.

</span>
<span class="ltx_bibblock">REPLUG: Retrieval-augmented black-box language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib40.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2301.12652</em>.

</span>
</li>
<li id="bib.bib41" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Sun et al. (2023)</span>
<span class="ltx_bibblock">
Haitian Sun, William W Cohen, and Ruslan Salakhutdinov. 2023.

</span>
<span class="ltx_bibblock">Answering ambiguous questions with a database of questions, answers,
and revisions.

</span>
<span class="ltx_bibblock"><em id="bib.bib41.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2308.08661</em>.

</span>
</li>
<li id="bib.bib42" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Toledo et al. (2019)</span>
<span class="ltx_bibblock">
Assaf Toledo, Shai Gretz, Edo Cohen-Karlik, Roni Friedman, Elad Venezian, Dan
Lahav, Michal Jacovi, Ranit Aharonov, and Noam Slonim. 2019.

</span>
<span class="ltx_bibblock">Automatic argument quality assessment – new datasets and methods.

</span>
</li>
<li id="bib.bib43" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Touvron et al. (2023a)</span>
<span class="ltx_bibblock">
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne
Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro,
Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume
Lample. 2023a.

</span>
<span class="ltx_bibblock">LLaMA: Open and efficient foundation language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib43.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2302.13971</em>.

</span>
</li>
<li id="bib.bib44" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Touvron et al. (2023b)</span>
<span class="ltx_bibblock">
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine
Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale,
et al. 2023b.

</span>
<span class="ltx_bibblock">LLaMA 2: Open foundation and fine-tuned chat models.

</span>
<span class="ltx_bibblock"><em id="bib.bib44.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2307.09288</em>.

</span>
</li>
<li id="bib.bib45" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wei et al. (2022)</span>
<span class="ltx_bibblock">
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian
Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022.

</span>
<span class="ltx_bibblock">Finetuned language models are zero-shot learners.

</span>
<span class="ltx_bibblock">In <em id="bib.bib45.1.1" class="ltx_emph ltx_font_italic">ICLR</em>.

</span>
</li>
<li id="bib.bib46" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Xie et al. (2023)</span>
<span class="ltx_bibblock">
Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. 2023.

</span>
<span class="ltx_bibblock">Adaptive chameleon or stubborn sloth: Unraveling the behavior of
large language models in knowledge conflicts.

</span>
<span class="ltx_bibblock"><em id="bib.bib46.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2305.13300</em>.

</span>
</li>
<li id="bib.bib47" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Xu et al. (2023)</span>
<span class="ltx_bibblock">
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang
Tao, and Daxin Jiang. 2023.

</span>
<span class="ltx_bibblock">WizardLM: Empowering large language models to follow complex
instructions.

</span>
<span class="ltx_bibblock"><em id="bib.bib47.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2304.12244</em>.

</span>
</li>
<li id="bib.bib48" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zamani et al. (2020)</span>
<span class="ltx_bibblock">
Hamed Zamani, Susan T. Dumais, Nick Craswell, Paul N. Bennett, and Gord Lueck.
2020.

</span>
<span class="ltx_bibblock">Generating clarifying questions for information retrieval.

</span>
<span class="ltx_bibblock"><em id="bib.bib48.1.1" class="ltx_emph ltx_font_italic">The Web Conference</em>.

</span>
</li>
<li id="bib.bib49" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhang and Choi (2021)</span>
<span class="ltx_bibblock">
Michael JQ Zhang and Eunsol Choi. 2021.

</span>
<span class="ltx_bibblock">SituatedQA: Incorporating extra-linguistic contexts into QA.

</span>
<span class="ltx_bibblock">In <em id="bib.bib49.1.1" class="ltx_emph ltx_font_italic">EMNLP</em>.

</span>
</li>
</ul>
</section>
<div class="ltx_pagination ltx_role_newpage"></div>
<section id="A1" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix A </span>Additional Details on <span id="A1.1.1" class="ltx_text ltx_font_smallcaps">ConflictingQA</span>
</h2>

<div id="A1.p1" class="ltx_para">
<p id="A1.p1.1" class="ltx_p">Table <a href="#A1.T5" title="Table 5 ‣ Appendix A Additional Details on ConflictingQA ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a> lists each question category in the dataset and Table <a href="#A1.T6" title="Table 6 ‣ Appendix A Additional Details on ConflictingQA ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">6</span></a> contains the prompt used to generate these category. Table <a href="#A1.T7" title="Table 7 ‣ Appendix A Additional Details on ConflictingQA ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a> contains the prompt used to classify the stance of the retrieved websites.</p>
</div>
<figure id="A1.T5" class="ltx_table">
<p id="A1.T5.1" class="ltx_p">
<span id="A1.T5.1.1" class="ltx_inline-block ltx_minipage ltx_align_middle ltx_framed ltx_framed_rectangle" style="width:433.6pt;">
<span id="A1.T5.1.1.1" class="ltx_p"><span id="A1.T5.1.1.1.1" class="ltx_text ltx_font_typewriter">Publishing, Biodiversity, Religion, Endangered Species, Pomology, Odontology, Pharmacology, Diabetology, Lepidopterology, Horticulture, Paleoclimatology, Product Design, Sustainability, Genomics, Intellectual Property, Gemology, Biomathematics, Karyology, Biomechanics, Selenology, Meteoritics, Chronobiology, Online Learning, Sustainable Living, Mammalogy, Web Design, Cytogenetics, Politics, Veterinary Science, Informatics, Zoogeography, Organic Farming, Cryptocurrency, Ethnobotany, Petrology, Serology, Ethology, Seismology, Entrepreneurship, Zymology, Astronomy, Holistic Health, Ichthyology, Trichology, Hematology, Gerontology, Neurology, Aging, Heuristics, Nematology, Nuclear Energy, Conservation, Botany, Spelaeology, Urology, Virology, Ergonomics, Volcanology, Yoga, Dermatology, Stomatology, Paleopathology, Xenobiology, Anthropometry, Anthropology, Entertainment, Poetry, Animation, Archaeology, Ornithology, Radio, Etymology, World Religions, Oncology, Anthrozoology, Criminology, Herpetology, Television, Malacology, Paranormal, Philology, Forestry, Probabilistics, Aerospace, Somnology, Cardiology, Cognitive Science, Quantum Physics, Phylogenetics, Epistemology, Vulcanology, Epidemiology, Psychobiology, Kinematics, Telecommunications, Melittology, Otorhinolaryngology, Astronautics, Biophysics, Neuroscience, Paleo Diet, Endocrinology, Kinesiology, Constitutional Law, Pop Culture, Lexicology, Festivals, Evolution, Metallurgy, Pediatrics, Phonetics, Astrobiology, Pets, Multiculturalism, Veganism, Andragogy, Remote Work, Speleology, Telepathy, Marine Conservation, Human Geography, Creationism, Philosophy, Oceanography, Mycology, Work-Life Balance, Ethics, Bioethics, Viniculture, Pedagogy, Classical Music, Paleoethnobotany, Manuscripts, Paleobotany, Revolutions, Paleozoology</span></span>
</span>
</p>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 5: </span>The full list of the 136 categories from Table <a href="#S3.T1" title="Table 1 ‣ 3 The ConflictingQA Dataset ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>.</figcaption>
</figure>
<figure id="A1.T6" class="ltx_table">
<p id="A1.T6.1" class="ltx_p">
<span id="A1.T6.1.1" class="ltx_inline-block ltx_minipage ltx_align_middle ltx_framed ltx_framed_rectangle" style="width:433.6pt;">
<span id="A1.T6.1.1.1" class="ltx_p"><span id="A1.T6.1.1.1.1" class="ltx_text ltx_font_typewriter">I’m looking to create a list of trivia-style questions with contentious or disagreed about answers. The questions should be able to be answered with ‘‘yes’’ or ‘‘no’’. I want to be able to find sources arguing for both sides. 
<br class="ltx_break">
<br class="ltx_break">Here’s a list of example questions: 
<br class="ltx_break">
<br class="ltx_break">Are U.S. Railroad Gauges Based on Roman Chariots? 
<br class="ltx_break">Is Juice-Jacking a real threat? 
<br class="ltx_break">Did Coca-Cola Ever Contain Cocaine? 
<br class="ltx_break">Is red-wine good for the heart? 
<br class="ltx_break">Does red-meat cause heart disease? 
<br class="ltx_break">Is irregardless a real word? 
<br class="ltx_break">Should you take baby aspirin to prevent heart attacks? 
<br class="ltx_break">Is there an area in the Yellowstone where murder is legal? 
<br class="ltx_break"></span></span>
<span id="A1.T6.1.1.2" class="ltx_p"><span id="A1.T6.1.1.2.1" class="ltx_text ltx_font_typewriter">Generate a list of questions that are in the category of ‘‘category’’. Please continue this list in the same format. Do not repeat questions.</span></span>
</span>
</p>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 6: </span>The prompt used to generate the questions.</figcaption>
</figure>
<figure id="A1.T7" class="ltx_table">
<p id="A1.T7.1" class="ltx_p">
<span id="A1.T7.1.1" class="ltx_inline-block ltx_minipage ltx_align_middle ltx_framed ltx_framed_rectangle" style="width:433.6pt;">
<span id="A1.T7.1.1.1" class="ltx_p"><span id="A1.T7.1.1.1.1" class="ltx_text ltx_font_typewriter"># For Claude Instant v1
<br class="ltx_break">The following is text from a website: 
<br class="ltx_break">
<br class="ltx_break">"""
<br class="ltx_break">{TEXT}
<br class="ltx_break">"""
<br class="ltx_break"></span></span>
<span id="A1.T7.1.1.2" class="ltx_p"><span id="A1.T7.1.1.2.1" class="ltx_text ltx_font_typewriter">How would the author of this website respond to the question: "{search_query}". Answer with "Yes" or "No". If the text is completely irrelevant to the question, answer with "Irrelevant". Do not answer with anything other than "Yes", "No", or "Irrelevant".
<br class="ltx_break"></span></span>
<span id="A1.T7.1.1.3" class="ltx_p"><span id="A1.T7.1.1.3.1" class="ltx_text ltx_font_typewriter"># For GPT-4-1106-preview
<br class="ltx_break">The following is text from a website: 
<br class="ltx_break">
<br class="ltx_break">"""
<br class="ltx_break">{TEXT}
<br class="ltx_break">"""
<br class="ltx_break"></span></span>
<span id="A1.T7.1.1.4" class="ltx_p"><span id="A1.T7.1.1.4.1" class="ltx_text ltx_font_typewriter">Use ONLY the text to answer the question: "{search_query}". Answer ONLY based on the information given in the above excerpt. First explain your answer in 2-3 sentences. Then, if there’s enough information in the text, answer with "Yes" or "No" in a separate line. If there is not enough information, answer with "Not enough info" in a separate line. Do not say "Yes", "No", or "Not enough info" before you give your explanation. Only answer after your explanation. Do not answer with anything other than "Yes", "No", or "Not enough info".</span></span>
<span id="A1.T7.1.1.5" class="ltx_p"><span id="A1.T7.1.1.5.1" class="ltx_text ltx_font_typewriter">Answer with "Yes" if the answer is most likely Yes. Answer with "No" if the answer is most likely No. Answer with "Not enough info" if there is not enough info.</span></span>
</span>
</p>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 7: </span>The prompts used to determine the authors stance on each question.</figcaption>
</figure>
<figure id="A1.T8" class="ltx_table">
<p id="A1.T8.1" class="ltx_p">
<span id="A1.T8.1.1" class="ltx_inline-block ltx_minipage ltx_align_middle ltx_framed ltx_framed_rectangle" style="width:433.6pt;">
<span id="A1.T8.1.1.1" class="ltx_p"><span id="A1.T8.1.1.1.1" class="ltx_text ltx_font_typewriter">The following are two websites:

<br class="ltx_break">
<br class="ltx_break">Website 1:
<br class="ltx_break">"""
<br class="ltx_break">{TEXT_1}
<br class="ltx_break">"""
<br class="ltx_break">
<br class="ltx_break">Website 2:
<br class="ltx_break">"""
<br class="ltx_break">{TEXT_2}
<br class="ltx_break">"""
<br class="ltx_break">
<br class="ltx_break">Using these two sources, answer the following question:
"{search_query}"

<br class="ltx_break">Use only the information in the above text to answer the question. Answer with only "Yes" or "No". Do not answer with anything other than "Yes" or "No".</span></span>
</span>
</p>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 8: </span>The prompt used to make predictions based on conflicting pairs of documents. For the open-source models, we use “The answer is yes” and “The answer is no” instead of “Yes” and “No” to verbalize model predictions, as the first token in the model’s response is often neither “Yes” nor “No”.</figcaption>
</figure>
</section>
<section id="A2" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix B </span>Model-based Data Filtering</h2>

<div id="A2.p1" class="ltx_para">
<p id="A2.p1.1" class="ltx_p">We filter out any paragraph that the downstream LLM predicts a different stance for than the ensemble of <span id="A2.p1.1.1" class="ltx_text ltx_font_typewriter">GPT-4</span> and <span id="A2.p1.1.2" class="ltx_text ltx_font_typewriter">Claude v1 Instant</span>. We do this by taking the paragraph of interest and comparing it to a paragraph with the text “This website has no text”. We remove any paragraph where the model’s output differs from the predicted stance label.
We also balance the dataset such that each answer (<span id="A2.p1.1.3" class="ltx_text ltx_font_typewriter">Yes</span> or <span id="A2.p1.1.4" class="ltx_text ltx_font_typewriter">No</span>) to a question contains an equal number of convincing and unconvincing paragraphs.</p>
</div>
</section>
<section id="A3" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix C </span>Additional Results</h2>

<div id="A3.p1" class="ltx_para">
<p id="A3.p1.1" class="ltx_p">Figure <a href="#A3.F5" title="Figure 5 ‣ Appendix C Additional Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>–<a href="#A3.F8" title="Figure 8 ‣ Appendix C Additional Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a> contain the analogous plots for Figure <a href="#S4.F4" title="Figure 4 ‣ 4.2 Implementation Details ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a> across four other models.</p>
</div>
<figure id="A3.F5" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F5.1" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x10.png" id="A3.F5.1.g1" class="ltx_graphics ltx_img_square" width="145" height="117" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F5.2" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x11.png" id="A3.F5.2.g1" class="ltx_graphics ltx_img_square" width="145" height="118" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F5.3" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x12.png" id="A3.F5.3.g1" class="ltx_graphics ltx_img_landscape" width="145" height="116" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F5.4" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x13.png" id="A3.F5.4.g1" class="ltx_graphics ltx_img_square" width="145" height="117" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F5.5" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x14.png" id="A3.F5.5.g1" class="ltx_graphics ltx_img_landscape" width="145" height="115" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F5.6" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x15.png" id="A3.F5.6.g1" class="ltx_graphics ltx_img_square" width="145" height="118" alt="Refer to caption">
</figure>
</div>
</div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 5: </span>The analogous plots to Figure <a href="#S4.F4" title="Figure 4 ‣ 4.2 Implementation Details ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a> except it is for <span id="A3.F5.8.1" class="ltx_text ltx_font_typewriter">Claude v1 Instant</span>. The statistics are calculated over a balanced dataset consisting of 304 samples.</figcaption>
</figure>
<figure id="A3.F6" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F6.1" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x16.png" id="A3.F6.1.g1" class="ltx_graphics ltx_img_square" width="145" height="117" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F6.2" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x17.png" id="A3.F6.2.g1" class="ltx_graphics ltx_img_square" width="145" height="118" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F6.3" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x18.png" id="A3.F6.3.g1" class="ltx_graphics ltx_img_landscape" width="145" height="116" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F6.4" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x19.png" id="A3.F6.4.g1" class="ltx_graphics ltx_img_square" width="145" height="117" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F6.5" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x20.png" id="A3.F6.5.g1" class="ltx_graphics ltx_img_landscape" width="145" height="115" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F6.6" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x21.png" id="A3.F6.6.g1" class="ltx_graphics ltx_img_square" width="145" height="118" alt="Refer to caption">
</figure>
</div>
</div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 6: </span>The analogous plots to Figure <a href="#S4.F4" title="Figure 4 ‣ 4.2 Implementation Details ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a> except it is for <span id="A3.F6.8.1" class="ltx_text ltx_font_typewriter">GPT-4</span>. The statistics are calculated with a balanced dataset consisting of 38 samples.</figcaption>
</figure>
<figure id="A3.F7" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F7.1" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x22.png" id="A3.F7.1.g1" class="ltx_graphics ltx_img_square" width="145" height="117" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F7.2" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x23.png" id="A3.F7.2.g1" class="ltx_graphics ltx_img_square" width="145" height="118" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F7.3" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x24.png" id="A3.F7.3.g1" class="ltx_graphics ltx_img_landscape" width="145" height="116" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F7.4" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x25.png" id="A3.F7.4.g1" class="ltx_graphics ltx_img_square" width="145" height="117" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F7.5" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x26.png" id="A3.F7.5.g1" class="ltx_graphics ltx_img_landscape" width="145" height="115" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F7.6" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x27.png" id="A3.F7.6.g1" class="ltx_graphics ltx_img_square" width="145" height="118" alt="Refer to caption">
</figure>
</div>
</div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 7: </span>The analogous plots to Figure <a href="#S4.F4" title="Figure 4 ‣ 4.2 Implementation Details ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a> except it is for <span id="A3.F7.8.1" class="ltx_text ltx_font_typewriter">Vicuna 1.5 13B</span>. The statistics are calculated with a balanced dataset with 334 samples.</figcaption>
</figure>
<figure id="A3.F8" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F8.1" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x28.png" id="A3.F8.1.g1" class="ltx_graphics ltx_img_square" width="145" height="117" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F8.2" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x29.png" id="A3.F8.2.g1" class="ltx_graphics ltx_img_square" width="145" height="118" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F8.3" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x30.png" id="A3.F8.3.g1" class="ltx_graphics ltx_img_landscape" width="145" height="116" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F8.4" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x31.png" id="A3.F8.4.g1" class="ltx_graphics ltx_img_square" width="145" height="117" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F8.5" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x32.png" id="A3.F8.5.g1" class="ltx_graphics ltx_img_landscape" width="145" height="115" alt="Refer to caption">
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_3">
<figure id="A3.F8.6" class="ltx_figure ltx_figure_panel ltx_align_center"><img src="/html/2402.11782/assets/x33.png" id="A3.F8.6.g1" class="ltx_graphics ltx_img_square" width="145" height="118" alt="Refer to caption">
</figure>
</div>
</div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 8: </span>The analogous plots to Figure <a href="#S4.F4" title="Figure 4 ‣ 4.2 Implementation Details ‣ 4 Experimental Results ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a> except it is for <span id="A3.F8.8.1" class="ltx_text ltx_font_typewriter">WizardLM 1.2 13B.</span> The statistics are calculated with a balanced dataset with 318 samples.</figcaption>
</figure>
</section>
<section id="A4" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix D </span>Expressing Convincingness in Isolation</h2>

<div id="A4.p1" class="ltx_para">
<p id="A4.p1.1" class="ltx_p">We consider whether LLMs are able to express the convincingness of a paragraph in isolation. The model makes the rating using only the website. We prompt (Table <a href="#A4.T9" title="Table 9 ‣ Appendix D Expressing Convincingness in Isolation ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a>) asking the model to rate the credibility of the website from a scale of one to five. The rating of the model is then determined by an average of the ratings, weighted by the probability of each label. Following <cite class="ltx_cite ltx_citemacro_cite">Santurkar et al. (<a href="#bib.bib37" title="" class="ltx_ref">2023</a>)</cite>, we calculate probabilities by exponentiating and normalizing the logits for "one" through "five". We also give the model with examples of a "one" and "five" ratings from C3 <cite class="ltx_cite ltx_citemacro_cite">Kakol et al. (<a href="#bib.bib23" title="" class="ltx_ref">2017</a>)</cite>, a dataset for studying human credibility judgements. We use these few-shot examples as the model tended to be biased toward higher-ratings without them.</p>
</div>
<figure id="A4.T9" class="ltx_table">
<p id="A4.T9.1" class="ltx_p">
<span id="A4.T9.1.1" class="ltx_inline-block ltx_minipage ltx_align_middle ltx_framed ltx_framed_rectangle" style="width:433.6pt;">
<span id="A4.T9.1.1.1" class="ltx_p"><span id="A4.T9.1.1.1.1" class="ltx_text ltx_font_typewriter">From a scale of one to five, how credible is this source? Do not answer with anything other than one, two, three, four, or five.</span></span>
</span>
</p>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 9: </span>The prompt used to ask models directly the credibility of a website.</figcaption>
</figure>
</section>
<section id="A5" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix E </span>Counterfactual Perturbations</h2>

<div id="A5.p1" class="ltx_para">
<ol id="A5.I1" class="ltx_enumerate">
<li id="A5.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">1.</span> 
<div id="A5.I1.i1.p1" class="ltx_para">
<p id="A5.I1.i1.p1.1" class="ltx_p"><span id="A5.I1.i1.p1.1.1" class="ltx_text ltx_font_typewriter">Add Single Sentence</span>: We use <span id="A5.I1.i1.p1.1.2" class="ltx_text ltx_font_typewriter">claude-v1-instant</span> to add a single sentence to make the stance of the text obvious. For example, for “Does producing bottled water use more water than the bottle contains?”, we may add “In fact, producing a single bottle of water uses more water than the bottle contains.”</p>
</div>
</li>
<li id="A5.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">2.</span> 
<div id="A5.I1.i2.p1" class="ltx_para">
<p id="A5.I1.i2.p1.1" class="ltx_p"><span id="A5.I1.i2.p1.1.1" class="ltx_text ltx_font_typewriter">Rewrite Relevance</span>: We alter the text with <span id="A5.I1.i2.p1.1.2" class="ltx_text ltx_font_typewriter">claude-v1-instant</span> to make the text more relevant to the question.</p>
</div>
</li>
<li id="A5.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">3.</span> 
<div id="A5.I1.i3.p1" class="ltx_para">
<p id="A5.I1.i3.p1.1" class="ltx_p"><span id="A5.I1.i3.p1.1.1" class="ltx_text ltx_font_typewriter">Question Prefix</span>: We prefix the document with “The following text is about the question: [question]”.</p>
</div>
</li>
<li id="A5.I1.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">4.</span> 
<div id="A5.I1.i4.p1" class="ltx_para">
<p id="A5.I1.i4.p1.1" class="ltx_p"><span id="A5.I1.i4.p1.1.1" class="ltx_text ltx_font_typewriter">Keyword Stuffing</span>: We use <span id="A5.I1.i4.p1.1.2" class="ltx_text ltx_font_typewriter">claude-v1-instant</span> to add additional sentences that use keywords related to the question.</p>
</div>
</li>
<li id="A5.I1.i5" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">5.</span> 
<div id="A5.I1.i5.p1" class="ltx_para">
<p id="A5.I1.i5.p1.1" class="ltx_p"><span id="A5.I1.i5.p1.1.1" class="ltx_text ltx_font_typewriter">Add More Info</span>: We use <span id="A5.I1.i5.p1.1.2" class="ltx_text ltx_font_typewriter">claude-v1-instant</span> to add additional sentences of information that are unrelated to the question but related to the overall topic of the text. An example of this perturbation can be found in Figure <a href="#A5.F9" title="Figure 9 ‣ Appendix E Counterfactual Perturbations ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a>.</p>
</div>
</li>
<li id="A5.I1.i6" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">6.</span> 
<div id="A5.I1.i6.p1" class="ltx_para">
<p id="A5.I1.i6.p1.1" class="ltx_p"><span id="A5.I1.i6.p1.1.1" class="ltx_text ltx_font_typewriter">Add Science Reference</span>: We use <span id="A5.I1.i6.p1.1.2" class="ltx_text ltx_font_typewriter">claude-v1-instant</span> to add scientific references to the text.</p>
</div>
</li>
<li id="A5.I1.i7" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">7.</span> 
<div id="A5.I1.i7.p1" class="ltx_para">
<p id="A5.I1.i7.p1.1" class="ltx_p"><span id="A5.I1.i7.p1.1.1" class="ltx_text ltx_font_typewriter">Add Contact Info</span>: We suffix the text with the name and phone number of a fake author.</p>
</div>
</li>
<li id="A5.I1.i8" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">8.</span> 
<div id="A5.I1.i8.p1" class="ltx_para">
<p id="A5.I1.i8.p1.1" class="ltx_p"><span id="A5.I1.i8.p1.1.1" class="ltx_text ltx_font_typewriter">Rewrite Confidence</span>: We use <span id="A5.I1.i8.p1.1.2" class="ltx_text ltx_font_typewriter">claude-v1-instant</span> to make text sound more confident.</p>
</div>
</li>
<li id="A5.I1.i9" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">9.</span> 
<div id="A5.I1.i9.p1" class="ltx_para">
<p id="A5.I1.i9.p1.1" class="ltx_p"><span id="A5.I1.i9.p1.1.1" class="ltx_text ltx_font_typewriter">Rewrite Technical Language</span>: We use <span id="A5.I1.i9.p1.1.2" class="ltx_text ltx_font_typewriter">claude-v1-instant</span> to make the text more technical.</p>
</div>
</li>
<li id="A5.I1.i10" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">10.</span> 
<div id="A5.I1.i10.p1" class="ltx_para">
<p id="A5.I1.i10.p1.1" class="ltx_p"><span id="A5.I1.i10.p1.1.1" class="ltx_text ltx_font_typewriter">Rewrite Objective</span>: We use <span id="A5.I1.i10.p1.1.2" class="ltx_text ltx_font_typewriter">claude-v1-instant</span> to make the text more objective, e.g., Figure <a href="#A5.F9" title="Figure 9 ‣ Appendix E Counterfactual Perturbations ‣ What Evidence Do Language Models Find Convincing?" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a>.</p>
</div>
</li>
</ol>
</div>
<figure id="A5.F9" class="ltx_figure"><img src="/html/2402.11782/assets/x34.png" id="A5.F9.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="332" height="233" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 9: </span>We use the <span id="A5.F9.2.1" class="ltx_text ltx_font_typewriter">claude-v1-instant</span> model to make counterfactual edits to the text in order to change certain stylistic or relevancy features. Here, we show two examples of such perturbations.</figcaption>
</figure>
<div class="ltx_pagination ltx_role_newpage"></div>
</section>
</article>
</div>
<div class="ar5iv-footer"><a href="/html/2402.11780" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2402.11782" class="ar5iv-text-button ar5iv-severity-warning">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2402.11782">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2402.11782" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2402.11785" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Tue Mar  5 16:41:15 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2402.11782
Retrieval-augmented language models are being increasingly tasked with subjective, contentious, and conflicting queries such as “is aspartame linked to cancer”.
To resolve these ambiguous queries, one must search through a large range of websites and considerwhich, if any, of this evidence do I find convincing?In this work, we study how LLMs answer this question.
In particular, we constructConflictingQA, a dataset that pairs controversial queries with a series of real-world evidence documents that contain different facts (e.g., quantitative results) , argument styles (e.g., appeals to authority) , and answers (YesorNo) . We use this dataset to perform sensitivity and counterfactual analyses to explore which text features most affect LLM predictions. Overall, we find that current models rely heavily on therelevanceof a website to the query, while largely ignoringstylisticfeatures that humans find important such as whether a text contains scientific references or is written with a neutral tone.
Taken together, these results highlight the importance of RAG corpus quality (e.g., the need to filter misinformation) , and possibly even a shift in how LLMs are trained to better align with human judgements.
abstract
What Evidence Do Language Models Find Convincing?
Abstract
2402.11782
1IntroductionFigure 1:InConflictingQA, we create contentious questions such as “is aspartame linked to cancer”. We also retrieve evidence paragraphs for each question that contain different types of facts (e.g., quantitative results) , argument styles (e.g., appeals to authority) , and answers (YesorNo) . For example, in the figure above we show two evidence paragraphs with their key arguments highlighted. UsingConflictingQA, we studywhyLLMs trust certain types of evidence paragraphs and argument styles over others.LLMs are widely deployed in settings that require understanding context—from retrieval-augmented systems to web agents, models condition on sources that range from internet paragraphsto Python interpreters. At the same time, today’s models are also given tasks that are increasingly open-ended and controversial, such as “tell me if aspartame causes cancer”. To answer such questions, LLMs will read real-world paragraphs that are contradictory, noisy, and rife with misinformation.Humans have techniques to sift through such large quantities of complex and contradictory evidence by answering the question:which, if any, of this evidence do I find convincing?To do so, humans combine multiple strategies, including fact checking and evaluating a source’s credibility, harnessing prior knowledge and beliefs, and critically evaluating logical arguments.In this work, we explore how LLMs resolve similar ambiguities when faced with conflicting open-ended questions. To study this, we createConflictingQA, a dataset consisting of questions and real web documents that lead to conflicting answers. For instance, in Figure1we show an example where we pair the question “is aspartame linked to cancer” with a series of conflicting evidence documents collected from Google search. In our experiments, we evaluate theconvincingnessof each evidence document by computing the rate at which a model’s predictions align with that document’s viewpoint (i.e., its win-rate) .We perform sensitivity and counterfactual analyses to find in-the-wild features that correlate with document convincingness. We consider a mix of features that describe (1)  stylistic properties of a document and (2)  relevance of a document to the question. Many of these were inspired by results from studies of human credibility, for example, we consider whether adding scientific references makes text appear more convincing.Overall, we find that stylistic features play a considerably less impactful role in determining the convincingness of text than measures of relevance. Notably, we show that a simple perturbation targeting a website’s relevance—prefixing the page with “The following text is about the question:[question]”—is enough to substantially improves its win-rate. On the other hand, stylistic features like the informational content, whether a page contains references, or its confidence, tend to only have a neutral to negative effect on win-rate. These results show that LLM perceptions of convincingness, when grounded in real-world QA tasks, do not align with humans. Taken together, these results suggest there should be an increasing focus on thequalityof retrieved evidence and a shift in how LLMs are trained to align with human preferences. We release our code athttps://github.com/AlexWan0/rag-convincingness.
introduction
What Evidence Do Language Models Find Convincing?
1Introduction
2402.11782
2Background and MotivationsFigure 2:Models over-rely on document relevance. We study how the convincingness of a particular evidence paragraph (measured through win-rate)  changes when we modify it. We compare the effect of these changes to a baseline perturbation where we append “Thanks for reading!” to the end (indicated by the dotted line) . We find that many stylistic changes—inspired by factors that influence humans—have a neutral or even negative effect on models. On the other hand, perturbations that increase the texts relevance but minimally change its style have a substantial positive effect on models. Descriptions for each perturbation can be found in AppendixE.Standard LLMs can be used to solve tasks that do not require context, e.g., writing basic Python code or answering simple trivia questions. To give these models more knowledge, agency, and capabilities, recent efforts have augmented LLMs with retrieval, domain-specific tools, or even generic web access. These enhancements allow LLMs to answer more challenging open-domain questions (e.g., “is aspartame linked to cancer?”)  or accomplish open-ended tasks (e.g., “buy me a size 9 pair of blue running shoes”) .Handling conflicting evidence.A key question is how retrieval-augmented LLMs handle scenarios where their context is conflicting, ambiguous, or uncertain.
There has been a large body of work that studies howhumanshandle such conflicting evidence using HCI studiesor by trying to predict human argument preferences, but little work has been done on evaluating how AI models handle such conflicts.The existing work in AI has focused on conflicts between facts learned during pre-training and the evidence given during inference, finding that models are largely receptive to retrieved samples. However, these works focus on restricted settings such as QA over Wikipedia, where there are relatively uncontroversial factoid questions that have trusted evidence paragraphs.
Moreover, they do not focus onwhat typesof evidence models prefer. Our goal is to design a more realistic question answering benchmark to better analyze features about the evidence itself.
body
What Evidence Do Language Models Find Convincing?
2Background and Motivations
2402.11782
3TheConflictingQADatasetCategoryExample QuestionNum Evidence DocsPharmacologyAre antidepressants more effective than placebo?10Online LearningAre online degrees valued less by employers?10BiodiversityAre bees the most important pollinators?10Web DesignDoes longer website content rank better on Google?13SustainabilityAre electric cars really green?9PhilosophyAre humans fundamentally good or evil?7Nuclear EnergyCan nuclear power solve climate change?7Work-Life BalanceIs unlimited vacation time beneficial for employees?10SomnologyDo older people need less sleep?8BiomechanicsDo compression garments improve athletic performance?13Table 1:InConflictingQA, we create controversial questions for 136 different categories (see Table5for the complete list) . Above, we show an example question for ten different categories, as well as the number of evidence paragraphs for each one. The evidence paragraphs contain a mix ofYesandNoanswers.Here, we describe the construction ofConflictingQA, our dataset that evaluates what types of evidence are convincing for LLMs.
We designConflictingQAto emulate the common setup for deploying retrieval-augmented LLMs: we retrieve the most relevant documents for a particular user query and place them in the LLM’s context window.
To build our dataset, we tackle three challenges: collecting contentious questions, identifying relevant and diverse evidence paragraphs, and grouping evidence paragraphs together to create conflicting examples.Collecting contentious questions.We first create a series of realistic open-ended questions for which there exists conflicting evidence online. Critically, unlike past work on ambiguity in QA, we want to collectunambiguousquestions that still have answer conflicts. For example, in Figure1, we show a question “are artificial sweeteners linked to cancer?”, which is a widely-debated query in which there exist websites that support both answers. We design the questions to elicit binaries responses ofYesorNoto simplify evaluation.We create questions using GPT-4. To ensure that the model generates a diverse set of questions we take inspiration from previous work in synthetic dataset generationand stratify the generations by topic: we first generate question categories (e.g., climate change, robotics, oncology)  then generate sets of questions conditioned on each category (full prompt provided in Table6in AppendixA) . We qualitatively find that the questions are diverse and challenging; we show ten examples of them in Table1. We additionally manually remove duplicate questions in the dataset.Collecting evidence paragraphs.Given these questions, we want to find evidence paragraphs that support both the answers ofYesandNo. We also want these paragraphs to (1)  contain a diverse range of argument styles, factual information, etc., and (2)  be realistic inputs to an LLM. To handle this, we emulate running an real-world retrieval-augmented LLM system that uses the Google Search API as its retrieval engine. Concretely, we take the user’s query, reformulate it, and take the top-results from Google search for the answerYesand the top-for the answerNo.Number of questions238Number of question categories144Number of retrieved paragraphs2,208Average paragraph length (words) 365.01Number of paragraphs withcomparisons912Average number of comparisons per paragraph6.54Table 2:Basic statistics forConflictingQAwhen evaluating LLaMA-2 Chat. We start by collecting a set of controversial questions for different categories (top) . For each question, we retrieve a series of paragraphs from a variety of domains (middle) . To determine the convincingness of a paragraph, we compare it against at least five different paragraphs that have the opposite stance/viewpoint (bottom) .DomainCount.com527.org175.gov59.edu57.net12# unique39Table 3:The top five most common top-level domains found inConflictingQAfor evaluating LLaMA-2 Chat. The dataset consists of a diverse range of sources, including organizations (.org) , schools (.edu) , and governments (.gov) .We first turn each question into affirmative and negative statements, e.g., the question “is asparatame safe?” is converted to “asparatame is safe” and “asparatame is harmful” using GPT-4. We also put double quotes (to indicate to Google Search that we have exact-match keywords)  around any tokens that do not change after rephrasing the question into either statements (e.g., “aspartame”) . For both the affirmative and negative statements, we search the queries using the Google Search API and retrieve top-documents.111We setbecause qualitatively the relevancy of the results dropped off significantly after this point.As is common in many retrieval-augmented models, we do not consider any visual features of the web page. Instead, we extract the raw text from each document using jusText.222Package available athttps://github.com/miso-belica/jusText. Although humans use visual features when considering the credibility and trustworthiness of a source, we do not consider these features as most state-of-the-art LLMs do not use visual inputs.Additionally, we do not explicitly include metadata like source URL, publication date, or page headings.When searching queries such as “aspartame is safe”, we still retrieve documents that argue that aspartame is unsafe. To label the documents actual stance, we use an ensemble ofclaude-instant-v1andGPT-4-1106-previewand keep only the samples where the two models agree (see Table7in AppendixAfor the prompts) .333After identifying the stance, we also feed the paragraphs into the downstream LLM that we are testing and make sure that its answer aligns with the paragraphs predicted stances. This further filters and balances the data, accounting for mistakes in the downstream model. See AppendixBfor details.Furthermore, we allow the LLM to say that a document is irrelevant to the query; if so, we also filter it from the input.Finally, we want to isolateparagraphsfrom these larger documents to feed into the LLMs (as is common in RAG systems) . To do this, we extract the most relevant 512 token window of text inside the document. We run the TAS-B modelacross windows of 512 tokens with a 256 token stride, compute the dot product between the model’s embedding of that window and the model’s embedding of the question, and take the highest scoring window. We filter out any documents whose highest-scoring window has a dot product below 95.Creating conflicting examples.The end result of our data collection process is (1)  a set of controversial questions that (2)  have evidence paragraphs which contradict one another. This data can be used in a variety of ways to “stress test” RAG systems in order to understand how they behave under conflicting scenarios. One example of this is shown in Figure1, and the subsequent section will explore numerous possible uses ofConflictingQA.
Table3and Table3present basic statistics for our final data, accounting for specific filtering done for LLaMA-2 Chat.
body
What Evidence Do Language Models Find Convincing?
3TheConflictingQADataset
2402.11782
4Experimental ResultsIn this section, we useConflictingQAto evaluate what types of evidence models find convincing.4.1Convincingness as Paragraph Win RateWe mainly focus on usingConflictingQAin a setup where we ask an LLM a question while providing two conflicting evidence paragraphs (one that supportsYesand one that supportsNo) . Then, we measure which paragraph the model’s answer aligns with. By repeating this for all pairs of paragraphs, we can define the convincingness of a particular paragraph as itswin-rate, i.e., what percent of the time a model picks the answer in that paragraph over the other paragraphs.Concretely, letbe the set of top-paragraphs corresponding to a controversial questionwith stance. We take an LLM(e.g., LLaMA-2 Chat)  and ask it for a binary prediction for the question, based on two paragraphs selected from the larger set,and. The model makes a prediction:.For each paragraph, we define its win-rate as the empirical probability of the model’s prediction aligning with its stance when paired with a set of conflicting paragraphs, i.e.,Finally, as the ordering of the retrieved evidence is known to bias model predictions, we calculate win-rate based on both orderings of the retrieved paragraphs. We additionally filter our dataset to ensure that each win-rate calculation consists of comparisons with at least five unique paragraphs.Models Cannot Predict ConvincingnessWe designed the above experimental setting to emulate how production RAG models work. However, we could have instead just directly asked the LLM, “do you find paragraph X to be persuasive?”. This is how humans are typically asked to judge the convincingness of a piece of evidence. However, we find that LLMs are largely incapable of expressing the convincingness of a paragraph in words, e.g., there is little correlation in which paragraphs are marked as convincing in the two settings (Figure3) .444Our methodology for this setting is described in more detail in AppendixD.We thus focus on the more practically-grounded setting going forward.4.2Implementation DetailsWe evaluate a mix of open-source (LLaMA-2 Chat, Vicuna v1.5, and WizardLM v1.2)  and closed source (GPT-4, Anthropic Claude v1 Instant)  models. Importantly, we specify “Use only the information in the above text to answer the question” as we are looking to see how models judge stylistic differences in evidence, rather than their prior stances on the question.We extract binaryYes/Nopredictions from the model for each question. For open-source models, we compare the log-probabilities of the next-token. For the closed-source models, we prompt them to output onlyYesorNo. See Table8for the prompt used for question answering.Figure 3:Humans can read a paragraph in isolation and evaluate how convincing it is. For LLMs, when they are given a paragraph in isolation, they areunable to express its convincingness in words. Concretely, we plot the win rate of paragraphs versus what a model outputs when it is asked to judge the convincingness on a 1–5 Likert scale. The error bars show a 95% CI.(a) Flesh Kincaid Readability(b) Num Unique Tokens(c) Sentiment (FLAN confidence) (d) Perplexity (GPT-2 Medium) (e) -gram Overlap w/ Question(f) Question-Para Embedding Sim.Figure 4:Why do models prefer certain paragraphs over others?We test correlations between different features and paragraph win-rates. Here, we show LLaMA-2 Chat 13B (see all other models in AppendixC) , where the model tends to prefer samples with low-perplexity4(d) . In addition, paragraphs with high relevancy scores—particularly high question-paragraph embedding similarity are also highly convincing4(f) . See Figure2for additional analysis. The error bars show the 95% CI (n = 242) .4.3What Correlates With Convincingness?After collecting the win rates for each paragraph, we look to explainwhymodels pick some paragraphs over others. We first compute several automatic metrics and correlate them with the win-rate:•Readability: We use the Flesch-Kincaid readability test. This metric considers readability as a function of the average number of words per sentence and average number of syllables per word.•Number of unique tokens: We measure the number of unique lemmas in the text.555We use theWordNetLemmatizerfrom thenltklibrary.•Binary sentiment: We measure the probability of positive sentiment using the FLAN-large model.•Perplexity: We measure this using the GPT-2 medium model.•n-gram overlap: We measure the maximum length n-gram that is common to the question and paragraph.•Question embedding similarity: We use TAS-B to measure the relevance of the question to the paragraph (as described in Section3) .ResultsStylistic features are poor predictors of paragraph convincingness. Figure4shows the results for the LLaMA-2 Chat model and Figures5–8shows the results for other models. For example, across all models the Flesch-Kincaid score and number of unique tokens does not correlate with convincingness. Similarly, paragraphs with a more positive sentiment and lower perplexity tend to have some small impact on convincingness, with varying strengths from model to model.On the other hand, question-paragraph embedding similarity correlates strongly with win-rate across all models except for GPT-4. Similarly, a positive (but weaker)  correlational exists between n-gram overlap and win-rate.4.4Counterfactual AnalysisRather than a correlational study, we also test how win-rates change in a counterfactual setting where we directly edit paragraphs using an LLM. We make perturbations usingclaude-v1-instant, examples of which are shown in Figure9.Stylistic changesWe first consider changes inspired by factors that humans find important for the credibility of a text. For example, adding more information, adding scientific references, or making the text sound more objective. Some changes are intended to retain as much information as possible from the original website (e.g.,Add Science Reference,Add More Info) . Others involve significantly changing the entire paragraph (e.g.,Rewrite Objective,Rewrite Tech. Language) . All of the perturbations are described further in AppendixE.Relevancy changesBased on the results in Section4.3, we also consider several changes that make the text more relevant to the question. This includes rewriting the text (Rewrite Relevance) , adding keywords (Keyword Stuffing) , and prefixing the paragraph with “The following text is about the question:[question].” (Question Prefix) . Finally, we consider a perturbation inspired by the “AddSent” perturbation inwhere we useclaude-v1-instantto add a single sentence to make stance of a text obvious (Add Single Sentence) . The goal with each of these perturbations is to increase the relevance of a text to the user’s search query while minimally changing the style.We also compare these perturbations against a “control” perturbation where text is suffixed with “Thanks for reading!” This perturbation minimally influences both style and relevance. For simplicity we only perturb the paragraphs with theYesstance.Counterfactual resultsThe results for the counterfactual experiments are shown in Figure2: compared to the effect of the control perturbation, stylistic features tend to have a neutral to negative effect while relevancy-based features significantly improve win-rate. Note that many of these perturbations change a smaller amount of tokens than stylistic features—leaving thecontentof the website largely unchanged (e.g.,Add Single Sentence,Question Prefix) —but are still able to improve the convincingness of websites.Overall, we find that, as compared to typical finding from human experiments,LLMs tend to overindex on relevancy. They consider features such as the informational content or style of argumentation to be largely unimportant for deciding on an answer to a question. Instead, making simplistic changes like increasing the amount of-gram overlap between the question and the paragraph can substantially improve its convincingness.QuestionAffirmativeNegativeAre Coral snakes found in Africa?Old-world coral snakes are found in Africa, the Middle East, India, and parts of Southeast Asia. New World coral snakes can be found in North America, Central America, and South America.Coral snakes are found in scattered localities in the southern coastal plains from North Carolina to Louisiana, including all of Florida.Are Florida Panthers on the brink of extinction?As Florida’s panther numbers plummeted, the state’s human population nearly doubled over the past 30 years. Recent development patterns pose threats to panthers.Now, though, their population is on the upswing … Both the numbers and the genetic diversity of Florida panthers improved.Are artificial sweeteners safe for diabetics?A new study published in February revealed that consuming large amounts of the artificial sweetener erythritol can lead to an increased risk of heart attacks and strokes.Furthermore, xylitol does not need insulin to be metabolized, so it can be safely consumed by diabetics.Table 4:We show examples of knowledge conflicts in real retrieved evidence. For example, questions may be underspecified (e.g., “old-world” vs “new-world” coral snakes) . In other cases, the answer is dependent on the publication date (e.g.,currentlyon the brink vs recent upswing) . Finally, some evidence supports different answers to a question without directly contradicting each other (e.g., the safety of two different artificial sweeteners) .
body
What Evidence Do Language Models Find Convincing?
4Experimental Results
2402.11782
5Discussion & Related WorkHow should systems handle ambiguity?One reasonable suggestion is that agents should not make their own autonomous decisions when faced with ambiguous or conflicting evidence. For example, they may summarizebothsides of the aspartame argument, or they may ask the user to clarify their preferences. There is naturally a trade-off between autonomy and clarity. Past work has explored one side of this trade-off, for example by abstaining from answering in cases of ambiguity, by trying to provide multiple perspectives on the answer, or by asking clarification questions. Our work explores the other side of the trade-off: we analyze the behavior of models when they are expected to resolve ambiguity with more autonomy.Additionally, our dataset serves as a benchmark for exploring these questions as it reflects real-world ambiguities in question-answering. For example, in Table4)  to best answer “Are Coral snakes found in Africa?”, additional clarification questions would be needed from the user.Optimizing misinformation and SEO.In principle, our insights could also be used tooptimizeparagraphs to increase the chance that a QA model is convinced by it. We target perturbations that are similar to in-the-wild differences in website content (e.g., scientific references, informational content, etc.)  but past work has more directly created adversarial examples. Our counterfactual perturbations could also be used in a search engine optimization (SEO)  fashion to increase how often a certain product or company is mentioned in a RAG LLM’s answer. Indeed, concurrent work has explored ideas such as this, where they aim to optimize “impressions” in long-form answers by maximizing the number of tokens from a particular paragraph that appear in an output. We instead study how modelanswerscan be manipulated.Improving model judgements.Our work highlights the gap between human and model judgements of text credibility. This solution to this, however, is not clear cut. For one, it is not clear the level of discretion models should have when making predictions. Human judgements of website credibility differ from person to person, and users may not be comfortable with the idea that models are “choosing” for them what source to trust. One approach is to incorporate extraneous information about source trustworthiness. For example,propose aligning model predictions with that of known trustworthy sources via prompting. Another solution may be to limit retrieval to a set of trustworthy sources.
body
What Evidence Do Language Models Find Convincing?
5Discussion & Related Work
2402.11782
6ConclusionWe study how RAG model judge convincingness by collecting a diverse set of controversial questions and website text (ConflictingQA) , and designing a realistic evaluation framework based on how these models are used in practice. Our results show that today’s LLMs tend to overrely on relevancy and ignore many stylistic features of text that humans often deem important. Future work should explore how integrating other forms of information (e.g., metadata, visual content)  can influence these behaviors. In addition, given the possible flood of LLM-generated content on the internet, it is important to consider how these synthetic texts may influence LLM judgements of convincingness.
body
What Evidence Do Language Models Find Convincing?
6Conclusion
2402.11782
LimitationsWhileConflictingQAis diverse and simulates real-world uses of RAG models, it may not fully capture the complexity of how LLMs are used in practice. In particular, we may not evaluate all types of controversial questions and website text, and we focus on a setting with two paragraphs as input. We also only consider a binary Yes or No answer to contentious questions whereas LLM outputs in practice may be more nuanced. Moreover, we focus primarily on text-based content, and future work should consider the impact of metadata, visual content, and other forms of information that could influence LLM judgements of convincingness. Finally, we acknowledge that our study does not address the broader ethical and societal implications of LLMs both reading and generating most of the content on the web. Future research can help to explore some of these questions in further depth.AcknowledgmentsWe thank Sewon Min and the members of Berkeley NLP for useful feedback on this project. We thank Ruiqi Zhong for helping us run the D5 codebase. Eric Wallace is supported by the Apple Scholars in AI/ML Fellowship.
Part of this research was supported with Cloud TPUs from Google’s TPU Research Cloud (TRC) .
conclusion
What Evidence Do Language Models Find Convincing?
Limitations