<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">

<head>  <title>[2408.04614] Better Alignment with Instruction Back-and-Forth Translation</title>
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="apple-touch-icon" sizes="180x180" href="/static/browse/0.3.4/images/icons/apple-touch-icon.png">
  <link rel="icon" type="image/png" sizes="32x32" href="/static/browse/0.3.4/images/icons/favicon-32x32.png">
  <link rel="icon" type="image/png" sizes="16x16" href="/static/browse/0.3.4/images/icons/favicon-16x16.png">
  <link rel="manifest" href="/static/browse/0.3.4/images/icons/site.webmanifest">
  <link rel="mask-icon" href="/static/browse/0.3.4/images/icons/safari-pinned-tab.svg" color="#5bbad5">
  <meta name="msapplication-TileColor" content="#da532c">
  <meta name="theme-color" content="#ffffff">
  <link rel="stylesheet" type="text/css" media="screen" href="/static/browse/0.3.4/css/arXiv.css?v=20240822" />
  <link rel="stylesheet" type="text/css" media="print" href="/static/browse/0.3.4/css/arXiv-print.css?v=20200611" />
  <link rel="stylesheet" type="text/css" media="screen" href="/static/browse/0.3.4/css/browse_search.css" />
  <script language="javascript" src="/static/browse/0.3.4/js/accordion.js" /></script>
  
    <link rel="stylesheet" type="text/css" media="screen" href="/static/browse/0.3.4/css/slider.css?v=1.14" />
    <script src="//code.jquery.com/jquery-latest.min.js" type="text/javascript"></script>
    <script type="text/javascript" src="/static/browse/0.3.4/js/donate.js?v=1.11"></script><link rel="canonical" href="/abs/2408.04614"/><meta property="og:type" content="website" />
<meta property="og:site_name" content="arXiv.org" />
<meta property="og:title" content="Better Alignment with Instruction Back-and-Forth Translation" />
<meta property="og:url" content="https://arxiv.org/abs/2408.04614v2" />
<meta property="og:image" content="/static/browse/0.3.4/images/arxiv-logo-fb.png" />
<meta property="og:image:secure_url" content="/static/browse/0.3.4/images/arxiv-logo-fb.png" />
<meta property="og:image:width" content="1200" />
<meta property="og:image:height" content="700" />
<meta property="og:image:alt" content="arXiv logo"/>
<meta property="og:description" content="We propose a new method, instruction back-and-forth translation, to construct high-quality synthetic data grounded in world knowledge for aligning large language models (LLMs). Given documents from a web corpus, we generate and curate synthetic instructions using the backtranslation approach proposed by Li et al.(2023a), and rewrite the responses to improve their quality further based on the initial documents. Fine-tuning with the resulting (backtranslated instruction, rewritten response) pairs yields higher win rates on AlpacaEval than using other common instruction datasets such as Humpback, ShareGPT, Open Orca, Alpaca-GPT4 and Self-instruct. We also demonstrate that rewriting the responses with an LLM outperforms direct distillation, and the two generated text distributions exhibit significant distinction in embedding space. Further analysis shows that our backtranslated instructions are of higher quality than other sources of synthetic instructions, while our responses are more diverse and complex than those obtained from distillation. Overall we find that instruction back-and-forth translation combines the best of both worlds -- making use of the information diversity and quantity found on the web, while ensuring the quality of the responses which is necessary for effective alignment."/>
<meta name="twitter:site" content="@arxiv"/>
<meta name="twitter:card" content="summary"/>
<meta name="twitter:title" content="Better Alignment with Instruction Back-and-Forth Translation"/>
<meta name="twitter:description" content="We propose a new method, instruction back-and-forth translation, to construct high-quality synthetic data grounded in world knowledge for aligning large language models (LLMs). Given documents..."/>
<meta name="twitter:image" content="https://static.arxiv.org/icons/twitter/arxiv-logo-twitter-square.png"/>
<meta name="twitter:image:alt" content="arXiv logo"/>
  <link rel="stylesheet" media="screen" type="text/css" href="/static/browse/0.3.4/css/tooltip.css"/><link rel="stylesheet" media="screen" type="text/css" href="https://static.arxiv.org/js/bibex-dev/bibex.css?20200709"/>  <script src="/static/browse/0.3.4/js/mathjaxToggle.min.js" type="text/javascript"></script>  <script src="//code.jquery.com/jquery-latest.min.js" type="text/javascript"></script>
  <script src="//cdn.jsdelivr.net/npm/js-cookie@2/src/js.cookie.min.js" type="text/javascript"></script>
  <script src="//cdn.jsdelivr.net/npm/dompurify@2.3.5/dist/purify.min.js"></script>
  <script src="/static/browse/0.3.4/js/toggle-labs.js?20230622" type="text/javascript"></script>
  <script src="/static/browse/0.3.4/js/cite.js" type="text/javascript"></script><meta name="citation_title" content="Better Alignment with Instruction Back-and-Forth Translation" /><meta name="citation_author" content="Nguyen, Thao" /><meta name="citation_author" content="Li, Jeffrey" /><meta name="citation_author" content="Oh, Sewoong" /><meta name="citation_author" content="Schmidt, Ludwig" /><meta name="citation_author" content="Weston, Jason" /><meta name="citation_author" content="Zettlemoyer, Luke" /><meta name="citation_author" content="Li, Xian" /><meta name="citation_date" content="2024/08/08" /><meta name="citation_online_date" content="2024/08/13" /><meta name="citation_pdf_url" content="http://arxiv.org/pdf/2408.04614" /><meta name="citation_arxiv_id" content="2408.04614" /><meta name="citation_abstract" content="We propose a new method, instruction back-and-forth translation, to construct high-quality synthetic data grounded in world knowledge for aligning large language models (LLMs). Given documents from a web corpus, we generate and curate synthetic instructions using the backtranslation approach proposed by Li et al.(2023a), and rewrite the responses to improve their quality further based on the initial documents. Fine-tuning with the resulting (backtranslated instruction, rewritten response) pairs yields higher win rates on AlpacaEval than using other common instruction datasets such as Humpback, ShareGPT, Open Orca, Alpaca-GPT4 and Self-instruct. We also demonstrate that rewriting the responses with an LLM outperforms direct distillation, and the two generated text distributions exhibit significant distinction in embedding space. Further analysis shows that our backtranslated instructions are of higher quality than other sources of synthetic instructions, while our responses are more diverse and complex than those obtained from distillation. Overall we find that instruction back-and-forth translation combines the best of both worlds -- making use of the information diversity and quantity found on the web, while ensuring the quality of the responses which is necessary for effective alignment." />
</head>

<body  class="with-cu-identity">
  
  <aside class="slider-wrapper bps-banner forum grey">
    <a class="close-slider bps-banner" href="#"><img src="/static/browse/0.3.4/images/icons/close-slider.png" alt="close this message"></a>
  	<div class="columns">
      <img role="presentation" class="bps-banner-image" src="/static/browse/0.3.4/images/events/arxiv-forum-logotype-title.png" alt="arXiv Accessibility Forum 2024">
      <div class="copy-donation bps-banner">
        <h2 id="forum-session-title"></h2>
        <p id="forum-session-link-small"></p>
      </div>
      <div class="amount-donation bps-banner">
          <div id="forum-session-link-box" class="donate-cta"></div>
      </div>
    </div>
  </aside>
  <script src="/static/browse/0.3.4/js/rotate_forum_info.js" type="text/javascript"></script>
  <div class="flex-wrap-footer">
    <header>
      <a href="#content" class="is-sr-only">Skip to main content</a>
      <!-- start desktop header -->
      <div class="columns is-vcentered is-hidden-mobile" id="cu-identity">
        <div class="column" id="cu-logo">
          <a href="https://www.cornell.edu/"><img src="/static/browse/0.3.4/images/icons/cu/cornell-reduced-white-SMALL.svg" alt="Cornell University" /></a>
        </div><div class="column banner-minimal forum">
            <!-- /from July 7 at 1:00 AM to Sept 2 at 1:00 AM --><p>This week: the arXiv Accessibility Forum</p>
            <a href="https://accessibility2024.arxiv.org/schedule" target="_blank">Forum Schedule</a></div><div class="column" id="support-ack">
          <span id="support-ack-url">We gratefully acknowledge support from the Simons Foundation, <a href="https://info.arxiv.org/about/ourmembers.html">member institutions</a>, and all contributors.</span>
          <a href="https://info.arxiv.org/about/donate.html" class="btn-header-donate">Donate</a>
        </div>
      </div>

      <div id="header" class="is-hidden-mobile">
<a aria-hidden="true" href="{url_path('ignore_me')}"></a>
  <div class="header-breadcrumbs is-hidden-mobile">
    <a href="/"><img src="/static/browse/0.3.4/images/arxiv-logo-one-color-white.svg" alt="arxiv logo" style="height:40px;"/></a> <span>&gt;</span> <a href="/list/cs/recent">cs</a> <span>&gt;</span> arXiv:2408.04614
  </div>
          <div class="search-block level-right">
    <form class="level-item mini-search" method="GET" action="https://arxiv.org/search">
      <div class="field has-addons">
        <div class="control">
          <input class="input is-small" type="text" name="query" placeholder="Search..." aria-label="Search term or terms" />
          <p class="help"><a href="https://info.arxiv.org/help">Help</a> | <a href="https://arxiv.org/search/advanced">Advanced Search</a></p>
        </div>
        <div class="control">
          <div class="select is-small">
            <select name="searchtype" aria-label="Field to search">
              <option value="all" selected="selected">All fields</option>
              <option value="title">Title</option>
              <option value="author">Author</option>
              <option value="abstract">Abstract</option>
              <option value="comments">Comments</option>
              <option value="journal_ref">Journal reference</option>
              <option value="acm_class">ACM classification</option>
              <option value="msc_class">MSC classification</option>
              <option value="report_num">Report number</option>
              <option value="paper_id">arXiv identifier</option>
              <option value="doi">DOI</option>
              <option value="orcid">ORCID</option>
              <option value="author_id">arXiv author ID</option>
              <option value="help">Help pages</option>
              <option value="full_text">Full text</option>
            </select>
          </div>
        </div>
        <input type="hidden" name="source" value="header">
        <button class="button is-small is-cul-darker">Search</button>
      </div>
    </form>
  </div>
     </div><!-- /end desktop header -->

      <div class="mobile-header">
        <div class="columns is-mobile">
          <div class="column logo-arxiv"><a href="https://arxiv.org/"><img src="/static/browse/0.3.4/images/arxiv-logomark-small-white.svg" alt="arXiv logo" style="height:60px;" /></a></div>
          <div class="column logo-cornell"><a href="https://www.cornell.edu/">
            <picture>
              <source media="(min-width: 501px)"
                srcset="/static/browse/0.3.4/images/icons/cu/cornell-reduced-white-SMALL.svg  400w"
                sizes="400w" />
              <source srcset="/static/browse/0.3.4/images/icons/cu/cornell_seal_simple_black.svg 2x" />
              <img src="/static/browse/0.3.4/images/icons/cu/cornell-reduced-white-SMALL.svg" alt="Cornell University Logo" />
            </picture>
          </a></div>
          <div class="column nav" id="toggle-container" role="menubar">
            <button class="toggle-control"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" class="icon filter-white"><title>open search</title><path d="M505 442.7L405.3 343c-4.5-4.5-10.6-7-17-7H372c27.6-35.3 44-79.7 44-128C416 93.1 322.9 0 208 0S0 93.1 0 208s93.1 208 208 208c48.3 0 92.7-16.4 128-44v16.3c0 6.4 2.5 12.5 7 17l99.7 99.7c9.4 9.4 24.6 9.4 33.9 0l28.3-28.3c9.4-9.4 9.4-24.6.1-34zM208 336c-70.7 0-128-57.2-128-128 0-70.7 57.2-128 128-128 70.7 0 128 57.2 128 128 0 70.7-57.2 128-128 128z"/></svg></button>
            <div class="mobile-toggle-block toggle-target">
              <form class="mobile-search-form" method="GET" action="https://arxiv.org/search">
                <div class="field has-addons">
                  <input class="input" type="text" name="query" placeholder="Search..." aria-label="Search term or terms" />
                  <input type="hidden" name="source" value="header">
                  <input type="hidden" name="searchtype" value="all">
                  <button class="button">GO</button>
                </div>
              </form>
            </div>

            <button class="toggle-control"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512" class="icon filter-white" role="menu"><title>open navigation menu</title><path d="M16 132h416c8.837 0 16-7.163 16-16V76c0-8.837-7.163-16-16-16H16C7.163 60 0 67.163 0 76v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16z"/ ></svg></button>
            <div class="mobile-toggle-block toggle-target">
              <nav class="mobile-menu" aria-labelledby="mobilemenulabel">
                <h2 id="mobilemenulabel">quick links</h2>
                <ul>
                    <li><a href="https://arxiv.org/login">Login</a></li>
                    <li><a href="https://info.arxiv.org/help">Help Pages</a></li>
                    <li><a href="https://info.arxiv.org/about">About</a></li>
                </ul>
              </nav>
            </div>
          </div>
        </div>
      </div><!-- /end mobile-header -->
    </header>

    <main>
      <div id="content">
<!--
rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:dc="http://purl.org/dc/elements/1.1/"
         xmlns:trackback="http://madskills.com/public/xml/rss/module/trackback/">
    <rdf:Description
        rdf:about="/abs/2408.04614"
        dc:identifier="/abs/2408.04614"
        dc:title="Better Alignment with Instruction Back-and-Forth Translation"
        trackback:ping="/trackback/2408.04614" />
    </rdf:RDF>
--><div id="abs-outer">

  <div class="leftcolumn">
    <div class="subheader">
      <h1>Computer Science > Computation and Language</h1>
    </div>

    <div class="header-breadcrumbs-mobile">
      <strong>arXiv:2408.04614</strong> (cs)
    </div>
<link rel="stylesheet" type="text/css" href="/static/base/1.0.1/css/abs.css">
<div id="content-inner">
  <div id="abs">
    <div class="dateline">
  [Submitted on 8 Aug 2024 (<a href="https://arxiv.org/abs/2408.04614v1">v1</a>), last revised 13 Aug 2024 (this version, v2)]</div>
    <h1 class="title mathjax"><span class="descriptor">Title:</span>Better Alignment with Instruction Back-and-Forth Translation</h1>
    <div class="authors"><span class="descriptor">Authors:</span><a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Nguyen,+T">Thao Nguyen</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Li,+J">Jeffrey Li</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Oh,+S">Sewoong Oh</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Schmidt,+L">Ludwig Schmidt</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Weston,+J">Jason Weston</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Zettlemoyer,+L">Luke Zettlemoyer</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Li,+X">Xian Li</a></div>            <div id="download-button-info" hidden>View a PDF of the paper titled Better Alignment with Instruction Back-and-Forth Translation, by Thao Nguyen and 6 other authors</div>
    <a class="mobile-submission-download" href="/pdf/2408.04614">View PDF</a>
    <a class="mobile-submission-download" href="https://arxiv.org/html/2408.04614v2">HTML (experimental)</a>



    <blockquote class="abstract mathjax">
            <span class="descriptor">Abstract:</span>We propose a new method, instruction back-and-forth translation, to construct high-quality synthetic data grounded in world knowledge for aligning large language models (LLMs). Given documents from a web corpus, we generate and curate synthetic instructions using the backtranslation approach proposed by Li et al.(2023a), and rewrite the responses to improve their quality further based on the initial documents. Fine-tuning with the resulting (backtranslated instruction, rewritten response) pairs yields higher win rates on AlpacaEval than using other common instruction datasets such as Humpback, ShareGPT, Open Orca, Alpaca-GPT4 and Self-instruct. We also demonstrate that rewriting the responses with an LLM outperforms direct distillation, and the two generated text distributions exhibit significant distinction in embedding space. Further analysis shows that our backtranslated instructions are of higher quality than other sources of synthetic instructions, while our responses are more diverse and complex than those obtained from distillation. Overall we find that instruction back-and-forth translation combines the best of both worlds -- making use of the information diversity and quantity found on the web, while ensuring the quality of the responses which is necessary for effective alignment.
    </blockquote>

    <!--CONTEXT-->
    <div class="metatable">
      <table summary="Additional metadata"><tr>
          <td class="tablecell label">Subjects:</td>
          <td class="tablecell subjects">
            <span class="primary-subject">Computation and Language (cs.CL)</span>; Artificial Intelligence (cs.AI); Machine Learning (cs.LG)</td>
        </tr><tr>
          <td class="tablecell label">Cite as:</td>
          <td class="tablecell arxivid"><span class="arxivid"><a href="https://arxiv.org/abs/2408.04614">arXiv:2408.04614</a> [cs.CL]</span></td>
        </tr>
        <tr>
          <td class="tablecell label">&nbsp;</td>
          <td class="tablecell arxividv">(or <span class="arxivid">
              <a href="https://arxiv.org/abs/2408.04614v2">arXiv:2408.04614v2</a> [cs.CL]</span> for this version)
          </td>
        </tr>
        <tr>
          <td class="tablecell label">&nbsp;</td>
          <td class="tablecell arxivdoi">              <a href="https://doi.org/10.48550/arXiv.2408.04614"  id="arxiv-doi-link">https://doi.org/10.48550/arXiv.2408.04614</a><div class="button-and-tooltip">
              <button class="more-info" aria-describedby="more-info-desc-1">
                <svg height="15" role="presentation" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path fill="currentColor" d="M256 8C119.043 8 8 119.083 8 256c0 136.997 111.043 248 248 248s248-111.003 248-248C504 119.083 392.957 8 256 8zm0 110c23.196 0 42 18.804 42 42s-18.804 42-42 42-42-18.804-42-42 18.804-42 42-42zm56 254c0 6.627-5.373 12-12 12h-88c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h12v-64h-12c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h64c6.627 0 12 5.373 12 12v100h12c6.627 0 12 5.373 12 12v24z" class=""></path></svg>
                <span class="visually-hidden">Focus to learn more</span>
              </button>
              <!-- tooltip description -->
              <div role="tooltip" id="more-info-desc-1">
                <span class="left-corner"></span>                  arXiv-issued DOI via DataCite</div>
            </div>
          </td>
        </tr></table>
    </div>
  </div>
</div>
    <div class="submission-history">
      <h2>Submission history</h2> From: Xian Li [<a href="/show-email/b85538ae/2408.04614">view email</a>]      <br/>            <strong><a href="/abs/2408.04614v1">[v1]</a></strong>
        Thu, 8 Aug 2024 17:42:32 UTC (880 KB)<br/>
    <strong>[v2]</strong>
        Tue, 13 Aug 2024 18:00:57 UTC (881 KB)<br/>
</div>
  </div>
  <!--end leftcolumn-->
<div class="extra-services">    <div class="full-text">
      <a name="other"></a>
      <span class="descriptor">Full-text links:</span>
      <h2>Access Paper:</h2>
      <ul>
  <div id="download-button-info" hidden>
View a PDF of the paper titled Better Alignment with Instruction Back-and-Forth Translation, by Thao Nguyen and 6 other authors</div><li><a href="/pdf/2408.04614" aria-describedby="download-button-info" accesskey="f" class="abs-button download-pdf">View PDF</a></li><li><a href="https://arxiv.org/html/2408.04614v2" class="abs-button" id="latexml-download-link">HTML (experimental)</a></li><li><a href="/src/2408.04614" class="abs-button download-eprint">TeX Source</a></li><li><a href="/format/2408.04614" class="abs-button download-format">Other Formats</a></li></ul>
      <div class="abs-license"><a href="http://creativecommons.org/licenses/by/4.0/" title="Rights to this article" class="has_license">
          <img alt="license icon" role="presentation" src="https://arxiv.org/icons/licenses/by-4.0.png"/>
          <span>view license</span>
        </a></div>
    </div>
    <!--end full-text-->    <div class="browse">
    Current browse context: <div class="current">cs.CL</div>

  <div class="prevnext">
<span class="arrow">
      <a class="abs-button prev-url" href="/prevnext?id=2408.04614&amp;function=prev&amp;context=cs.CL"
         accesskey="p" title="previous in cs.CL (accesskey p)">&lt;&nbsp;prev</a>
    </span>
    <span class="is-hidden-mobile">&nbsp; | &nbsp;</span>    <span class="arrow">
      <a class="abs-button next-url" href="/prevnext?id=2408.04614&amp;function=next&amp;context=cs.CL" accesskey="n"
         title="next in cs.CL (accesskey n)">next&nbsp;&gt;</a>
    </span><br/>
  </div><div class="list">
    <a class="abs-button abs-button-grey abs-button-small context-new" href="/list/cs.CL/new">new</a>
    <span class="is-hidden-mobile"> | </span>
    <a class="abs-button abs-button-grey abs-button-small context-recent" href="/list/cs.CL/recent">recent</a>
    <span class="is-hidden-mobile"> | </span><a class="abs-button abs-button-grey abs-button-small context-id" href="/list/cs.CL/2024-08">2024-08</a>
  </div><div class="abs-switch-cat">
    Change to browse by:
    <div class="switch context-change">
        <a href="/abs/2408.04614?context=cs">cs</a><br class="is-hidden-mobile">
        <a class="subclass" href="/abs/2408.04614?context=cs.AI">cs.AI</a><br class="is-hidden-mobile">
        <a class="subclass" href="/abs/2408.04614?context=cs.LG">cs.LG</a><br class="is-hidden-mobile">
    </div>
  </div>

    </div>

    <div class="extra-ref-cite">
      <h3>References &amp; Citations</h3>
      <ul>
        <li><a  class="abs-button abs-button-small cite-ads" href="https://ui.adsabs.harvard.edu/abs/arXiv:2408.04614">NASA ADS</a></li><li><a  class="abs-button abs-button-small cite-google-scholar" href="https://scholar.google.com/scholar_lookup?arxiv_id=2408.04614" target="_blank" rel="noopener">Google Scholar</a></li>
        <li><a  class="abs-button abs-button-small cite-semantic-scholar" href="https://api.semanticscholar.org/arXiv:2408.04614" target="_blank" rel="noopener">Semantic Scholar</a></li>
      </ul>
      <div style="clear:both;"></div>
    </div>

<div class='extra-ref-cite'>
    <a id='bib-cite-css' hidden='true' href='/static/browse/0.3.4/css/cite.css'>a</a>

    <span id='bib-cite-trigger' class="bib-cite-button abs-button">export BibTeX citation</span>
    <span id='bib-cite-loading' hidden='true'>Loading...</span>
</div>

<div id='bib-cite-modal' class='bib-modal' hidden='true'>
    <div class='bib-modal-content'>
        <div class='bib-modal-title'>
            <h2>BibTeX formatted citation</h2>
            <span class='bib-modal-close' >&times;</span>
        </div>
        <div>
            <textarea id='bib-cite-target' class="bib-citation-content" aria-label="loading the citation">loading...</textarea>
        </div>
        <div>
            <span>Data provided by: </span>
            <a id='bib-cite-source-api'></a>
        </div>
    </div>
</div><div class="bookmarks">
  <div><h3>Bookmark</h3></div><a class="abs-button abs-button-grey abs-button-small" href="http://www.bibsonomy.org/BibtexHandler?requTask=upload&amp;url=https://arxiv.org/abs/2408.04614&amp;description=Better Alignment with Instruction Back-and-Forth Translation"
     title="Bookmark on BibSonomy">
    <img src="/static/browse/0.3.4/images/icons/social/bibsonomy.png"
         alt="BibSonomy logo"/>
  </a>
  <a class="abs-button abs-button-grey abs-button-small" href="https://reddit.com/submit?url=https://arxiv.org/abs/2408.04614&amp;title=Better Alignment with Instruction Back-and-Forth Translation"
     title="Bookmark on Reddit">
    <img src="/static/browse/0.3.4/images/icons/social/reddit.png"
         alt="Reddit logo"/>
  </a>
</div>  </div>
  <!--end extra-services-->
<!-- LABS AREA -->
<div id="labstabs">
  <div class="labstabs"><input type="radio" name="tabs" id="tabone"checked="checked">
    <label for="tabone">Bibliographic Tools</label>
    <div class="tab labs-display-bib">
      <h1>Bibliographic and Citation Tools</h1>
      <div class="toggle">
        <div class="columns is-mobile lab-row">
          <div class="column lab-switch">
            <label class="switch">
              <input id="bibex-toggle" type="checkbox" class="lab-toggle">
              <span class="slider"></span>
              <span class="is-sr-only">Bibliographic Explorer Toggle</span>
            </label>
          </div>
          <div class="column lab-name">
            <span id="label-for-bibex">Bibliographic Explorer</span> <em>(<a href="https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer">What is the Explorer?</a>)</em>
          </div>
        </div>
<div class="columns is-mobile lab-row">
          <div class="column lab-switch">
            <label class="switch">
              <input
                id="litmaps-toggle"
                type="checkbox"
                class="lab-toggle"
                data-script-url="/static/browse/0.3.4/js/litmaps.js?20210617"
                aria-labelledby="label-for-litmaps">
              <span class="slider"></span>
              <span class="is-sr-only">Litmaps Toggle</span>
            </label>
          </div>
          <div class="column lab-name">
            <span id="label-for-litmaps">Litmaps</span> <em>(<a href="https://www.litmaps.co/" target="_blank">What is Litmaps?</a>)</em>
          </div>
        </div>
        <div class="columns is-mobile lab-row">
          <div class="column lab-switch">
            <label class="switch">
              <input
                id="scite-toggle"
                type="checkbox"
                class="lab-toggle"
                data-script-url="/static/browse/0.3.4/js/scite.js?20210617"
                aria-labelledby="label-for-scite">
              <span class="slider"></span>
              <span class="is-sr-only">scite.ai Toggle</span>
            </label>
          </div>
          <div class="column lab-name">
            <span id="label-for-scite">scite Smart Citations</span> <em>(<a href="https://www.scite.ai/" target="_blank">What are Smart Citations?</a>)</em>
          </div>
        </div>
      </div>
        <div class="labs-content-placeholder labs-display" style="display: none;"></div>
        <div style="min-height: 15px" id="litmaps-open-in"></div>
        <div style="min-height: 15px" id="scite-open-in"></div>
    </div>


    <input type="radio" name="tabs" id="tabtwo">
    <label for="tabtwo">Code, Data, Media</label>
    <div class="tab">
      <h1>Code, Data and Media Associated with this Article</h1>
      <div class="toggle">
        <div class="columns is-mobile lab-row">
          <div class="column lab-switch">
            <label class="switch">
              <input        
                id="catalyzex-toggle"
                data-script-url="/static/browse/0.3.4/js/catalyzex.js"
                type="checkbox" class="lab-toggle" aria-labelledby="label-for-cx">
              <span class="slider"></span>
              <span class="is-sr-only">Links to Code Toggle</span>
            </label>
          </div>
          <div class="column lab-name">
            <span id="label-for-cx">CatalyzeX Code Finder for Papers</span> <em>(<a href="https://www.catalyzex.com" target="_blank">What is CatalyzeX?</a>)</em>
          </div>
        </div>

        <div class="columns is-mobile lab-row">
          <div class="column lab-switch">
            <label class="switch">
              <input
                id="dagshub-toggle"
                data-script-url="/static/browse/0.3.4/js/dagshub.js"
                type="checkbox" class="lab-toggle" aria-labelledby="label-for-dagshub">
              <span class="slider"></span>
              <span class="is-sr-only">DagsHub Toggle</span>
            </label>
          </div>
          <div class="column lab-name">
            <span id="label-for-dagshub">DagsHub</span> <em>(<a href="https://dagshub.com/" target="_blank">What is DagsHub?</a>)</em>
          </div>
        </div>
  
        <div class="columns is-mobile lab-row">
          <div class="column lab-switch">
            <label class="switch">
              <input
                id="gotitpub-toggle"
                data-script-url="/static/browse/0.3.4/js/gotitpub.js"
                type="checkbox" class="lab-toggle" aria-labelledby="label-for-gotitpub">
              <span class="slider"></span>
              <span class="is-sr-only">GotitPub Toggle</span>
            </label>
          </div>
          <div class="column lab-name">
            <span id="label-for-gotitpub">Gotit.pub</span> <em>(<a href="http://gotit.pub/faq" target="_blank">What is GotitPub?</a>)</em>
          </div>
        </div>

        <div class="columns is-mobile lab-row">
          <div class="column lab-switch">
            <label class="switch">
              <input
                id="paperwithcode-toggle"
                data-script-url="/static/browse/0.3.4/js/paperswithcode.js"
                type="checkbox" class="lab-toggle" aria-labelledby="label-for-pwc">
              <span class="slider"></span>
              <span class="is-sr-only">Links to Code Toggle</span>
            </label>
          </div>
          <div class="column lab-name">
            <span id="label-for-pwc">Papers with Code</span> <em>(<a href="https://paperswithcode.com/" target="_blank">What is Papers with Code?</a>)</em>
          </div>
        </div>


        <div class="columns is-mobile lab-row">
          <div class="column lab-switch">
            <label class="switch">
              <input
                id="sciencecast-toggle"
                data-script-url="/static/browse/0.3.4/js/sciencecast.js"
                type="checkbox" class="lab-toggle" aria-labelledby="label-for-sciencecast">
              <span class="slider"></span>
              <span class="is-sr-only">ScienceCast Toggle</span>
            </label>
          </div>
          <div class="column lab-name">
            <span id="label-for-sciencecast">ScienceCast</span> <em>(<a href="https://sciencecast.org/welcome" target="_blank">What is ScienceCast?</a>)</em>
          </div>
        </div>
      </div>

      <div id="catalyzex-output" style="display:none"></div>
      <div id="dagshub-output" style="display:none"></div>
      <div id="gotitpub-output" style="display:none"></div>
      <div id="pwc-output" style="display:none"></div>
      <div id="pwc-data-output" style="display:none"></div>
      <div id="sciencecast-output" style="display:none"></div>
    </div>


      <input type="radio" name="tabs" id="labstabs-demos-input">
      <label for="labstabs-demos-input" id="labstabs-demos-label">Demos</label>
      <div class="tab">
        <h1>Demos</h1>
        <div class="toggle">
          <div class="columns is-mobile lab-row">
            <div class="column lab-switch">
              <label class="switch">
                <input
                  id="replicate-toggle"
                  data-script-url="/static/browse/0.3.4/js/replicate.js"
                  type="checkbox" class="lab-toggle" aria-labelledby="label-for-replicate">
                <span class="slider"></span>
                <span class="is-sr-only">Replicate Toggle</span>
              </label>
            </div>
            <div class="column lab-name">
              <span id="label-for-replicate">Replicate</span> <em>(<a href="https://replicate.com/docs/arxiv/about" target="_blank">What is Replicate?</a>)</em>
            </div>
          </div>
          <div class="columns is-mobile lab-row">
            <div class="column lab-switch">
              <label class="switch">
                <input
                  id="spaces-toggle"
                  data-script-url="/static/browse/0.3.4/js/spaces.js"
                  type="checkbox" class="lab-toggle" aria-labelledby="label-for-spaces">
                <span class="slider"></span>
                <span class="is-sr-only">Spaces Toggle</span>
              </label>
            </div>
            <div class="column lab-name">
              <span id="label-for-spaces">Hugging Face Spaces</span> <em>(<a href="https://huggingface.co/docs/hub/spaces" target="_blank">What is Spaces?</a>)</em>
            </div>
          </div>
          <div class="columns is-mobile lab-row">
            <div class="column lab-switch">
              <label class="switch">
                <input
                  id="txyz-toggle"
                  data-script-url="/static/browse/0.3.4/js/txyz.js"
                  type="checkbox" class="lab-toggle" aria-labelledby="label-for-txyz">
                <span class="slider"></span>
                <span class="is-sr-only">Spaces Toggle</span>
              </label>
            </div>
            <div class="column lab-name">
              <span id="label-for-txyz">TXYZ.AI</span> <em>(<a href="https://txyz.ai" target="_blank">What is TXYZ.AI?</a>)</em>
            </div>
          </div>
        </div>
        <div id="replicate-output"></div>
        <div id="spaces-output"></div>
        <div id="txyz-output"></div>
      </div>
      <input type="radio" name="tabs" id="tabfour">
      <label for="tabfour">Related Papers</label>
      <div class="tab">
        <h1>Recommenders and Search Tools</h1>
        <div class="toggle">
          <div class="columns is-mobile lab-row">
            <div class="column lab-switch">
              <label class="switch">
                <input id="influenceflower-toggle"
                data-script-url="/static/browse/0.3.4/js/influenceflower.js"
                type="checkbox" class="lab-toggle" aria-labelledby="label-for-influenceflower">
                <span class="slider"></span>
                <span class="is-sr-only">Link to Influence Flower</span>
              </label>
            </div>
            <div class="column lab-name">
              <span id="label-for-influenceflower">Influence Flower</span> <em>(<a href="https://influencemap.cmlab.dev/" target="_blank">What are Influence Flowers?</a>)</em>
            </div>
          </div>
          <div class="columns is-mobile lab-row">
            <div class="column lab-switch">
              <label class="switch">
                <input id="connectedpapers-toggle" type="checkbox" class="lab-toggle"
                       data-script-url="/static/browse/0.3.4/js/connectedpapers.js"
                       aria-labelledby="label-for-connected-papers" >
                <span class="slider"></span>
                <span class="is-sr-only">Connected Papers Toggle</span>
              </label>
            </div>
            <div class="column lab-name">
              <span id="label-for-connected-papers">Connected Papers</span> <em>(<a href="https://www.connectedpapers.com/about" target="_blank">What is Connected Papers?</a>)</em>
            </div>
          </div>
          <div class="columns is-mobile lab-row">
            <div class="column lab-switch">
              <label class="switch">
                <input id="core-recommender-toggle" type="checkbox" class="lab-toggle" aria-labelledby="label-for-core">
                <span class="slider"></span>
                <span class="is-sr-only">Core recommender toggle</span>
              </label>
            </div>
            <div class="column lab-name">
              <span id="label-for-core">CORE Recommender</span> <em>(<a href="https://core.ac.uk/services/recommender">What is CORE?</a>)</em>
            </div>
          </div></div>
        <div id="influenceflower-output"></div>
        <div id="influenceflower-output-graph" style="display:none">
          <ul class="flower-tabs">
            <li class="active"><a class="btn tab-btn" onclick="openTab(event, 'tab-author')">Author</a></li>
            <li><a class="btn tab-btn" onclick="openTab(event, 'tab-venue')">Venue</a></li>
            <li><a class="btn tab-btn" onclick="openTab(event, 'tab-inst')">Institution</a></li>
            <li><a class="btn tab-btn" onclick="openTab(event, 'tab-topic')">Topic</a></li>
          </ul>
          <div class="flower-tab-content">
            <div class="tab-flower active" id="tab-author"><svg id="flower-graph-author"></svg></div>
            <div class="tab-flower" id="tab-venue"><svg id="flower-graph-venue"></svg></div>
            <div class="tab-flower" id="tab-inst"><svg id="flower-graph-inst"></svg></div>
            <div class="tab-flower" id="tab-topic"><svg id="flower-graph-topic"></svg></div>
          </div>
        </div>
        <div style="min-height: 15px" id="connectedpapers-output"></div>
        <div id="coreRecommenderOutput"></div>
        <div id="iarxivOutput"></div>
      </div>

      <input type="radio" name="tabs" id="tabfive">
      <label for="tabfive">
        About arXivLabs
      </label>
      <div class="tab">
        <div class="columns">
          <div class="column">
            <h1>arXivLabs: experimental projects with community collaborators</h1>
            <p>arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.</p>
            <p>Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.</p>
            <p>Have an idea for a project that will add value for arXiv's community? <a href="https://info.arxiv.org/labs/index.html"><strong>Learn more about arXivLabs</strong></a>.</p>
          </div>
          <div class="column is-narrow is-full-mobile">
            <p class="icon-labs"><svg xmlns="http://www.w3.org/2000/svg" role="presentation" viewBox="0 0 635.572 811"><path d="M175.6 676v27h-27v-27zm-54 27v27h27v-27zm-27 27v27h27v-27zm396-54v27h-27v-27zm0 27v27h27v-27zm27 27v27h27v-27zm-27-414h27v27h-27zm27 0h27v-27h-27zm27-27h27v-27h-27zm-396 45h-27v-27h27zm-27-54h-27v27h27zm-27-27h-27v27h27z"/><path d="M94.6 730v27h-27v-27zm477 0v27h-27v-27zm-27-495h27v27h-27zm-450 18h-27v-27h27zm477 9h27v27h-27zm-54 495h27v27h-27zm-423 0h27v27h-27zm-54-504h27v27h-27z" fill="#666"/><path d="M67.6 730v27h-27v-27zm54 54v27h-27v-27zm0-108v27h27v-27zm-27 27v27h27v-27zm-81 0v27h27v-27zm585 27v27h-27v-27zm-108-54v27h27v-27zm27 27v27h27v-27zm81 0v27h27v-27zm-54-495h27v27h-27zm-54 108h27v-27h-27zm27-27h27v-27h-27zm0-81h27v-27h-27zm-423 18h-27v-27h27zm54 54h-27v27h27zm-27-27h-27v27h27zm0-81h-27v27h27zm423 612v27h-27v-27zm81-522v27h-27v-27zm-585-9v27h-27v-27z" fill="#999"/><path d="M94.6 784v27h-27v-27zm-27-27v27h27v-27zm-27-54v27h27v-27zm27 0v27h27v-27zm0-27v27h27v-27zm27 0v27h27v-27zm0-27v27h27v-27zm27 0v27h27v-27zm-108 81v27h27v-27zm558 54v27h-27v-27zm-27-27v27h27v-27zm27-54v27h27v-27zm-27 0v27h27v-27zm0-27v27h27v-27zm-27 0v27h27v-27zm0-27v27h27v-27zm-27 0v27h27v-27zm108 81v27h27v-27zm0-495h27v27h-27zm-27 27h27v-27h-27zm-54-27h27v-27h-27zm0 27h27v-27h-27zm-27 0h27v-27h-27zm0 27h27v-27h-27zm-27 0h27v-27h-27zm0 27h27v-27h-27zm81-108h27v-27h-27zm-504 45h-27v-27h27zm27-27h-27v27h27zm54-27h-27v27h27zm0 27h-27v27h27zm27 0h-27v27h27zm0 27h-27v27h27zm27 0h-27v27h27zm0 27h-27v27h27zm-81-108h-27v27h27z" fill="#ccc"/><path d="M598.6 665.1H41.5C-76.5 667 176 280.2 176 280.2h53a46.5 46.5 0 0162.8-56.3 29.2 29.2 0 1128.5 35.9h-1a46.5 46.5 0 01-1.5 20.3l142.5-.1s255.3 387 138.3 385.1zM291 181a29.3 29.3 0 10-29.2-29.3A29.3 29.3 0 00291 181zm65.4-66.8a22.4 22.4 0 10-22.5-22.4 22.4 22.4 0 0022.5 22.4z" fill="#fc0"/><path d="M245.5 172V10h153v162s324 495 198 495h-558c-126 0 207-495 207-495zm126 54h56m-13 72h56m-9 72h56m-20 72h56m-22 72h56m-29 72h56m-457-45c20.8 41.7 87.3 81 160.7 81 72.1 0 142.1-38.2 163.4-81" fill="none" stroke="#000" stroke-miterlimit="10" stroke-width="20"/><path d="M273.3 421.7c0 31-9.8 56.3-21.9 56.3s-21.8-25.2-21.8-56.3 9.8-56.3 21.8-56.3 21.9 25.2 21.9 56.3zm114.4-56.3c-12 0-21.8 25.2-21.8 56.3s9.7 56.3 21.8 56.3 21.9-25.2 21.9-56.3-9.8-56.3-21.9-56.3zM150.1 526.6c-18.2 6.7-27.5 22.9-23.2 30.2s14.8-5.5 33-12.2 37.4-4.9 33-12.2-24.5-12.6-42.8-5.8zm296 5.8c-4.2 7.3 14.9 5.5 33.1 12.2s28.7 19.5 33 12.2-5-23.5-23.2-30.2-38.5-1.5-42.8 5.8z"/></svg></p>
          </div>
        </div>
      </div>

    </div>
</div>
<!-- END LABS AREA -->
  <div class="endorsers">
    <a href="/auth/show-endorsers/2408.04614" class="endorser-who">Which authors of this paper are endorsers?</a> |
    <a id="mathjax_toggle" href="javascript:setMathjaxCookie()">Disable MathJax</a> (<a href="https://info.arxiv.org/help/mathjax.html">What is MathJax?</a>)
    <span class="help" style="font-style: normal; float: right; margin-top: 0; margin-right: 1em;"></span>
  </div>
  <script type="text/javascript" language="javascript">mathjaxToggle();</script>
</div>
      </div>
    </main>

    <footer style="clear: both;">
      <div class="columns is-desktop" role="navigation" aria-label="Secondary" style="margin: -0.75em -0.75em 0.75em -0.75em">
        <!-- Macro-Column 1 -->
        <div class="column" style="padding: 0;">
          <div class="columns">
            <div class="column">
              <ul style="list-style: none; line-height: 2;">
                <li><a href="https://info.arxiv.org/about">About</a></li>
                <li><a href="https://info.arxiv.org/help">Help</a></li>
              </ul>
            </div>
            <div class="column">
              <ul style="list-style: none; line-height: 2;">
                <li>
                  <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" class="icon filter-black" role="presentation"><title>contact arXiv</title><desc>Click here to contact arXiv</desc><path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"/></svg>
                  <a href="https://info.arxiv.org/help/contact.html"> Contact</a>
                </li>
                <li>
                  <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" class="icon filter-black" role="presentation"><title>subscribe to arXiv mailings</title><desc>Click here to subscribe</desc><path d="M476 3.2L12.5 270.6c-18.1 10.4-15.8 35.6 2.2 43.2L121 358.4l287.3-253.2c5.5-4.9 13.3 2.6 8.6 8.3L176 407v80.5c0 23.6 28.5 32.9 42.5 15.8L282 426l124.6 52.2c14.2 6 30.4-2.9 33-18.2l72-432C515 7.8 493.3-6.8 476 3.2z"/></svg>
                  <a href="https://info.arxiv.org/help/subscribe"> Subscribe</a>
                </li>
              </ul>
            </div>
          </div>
        </div>
        <!-- End Macro-Column 1 -->
        <!-- Macro-Column 2 -->
        <div class="column" style="padding: 0;">
          <div class="columns">
            <div class="column">
              <ul style="list-style: none; line-height: 2;">
                <li><a href="https://info.arxiv.org/help/license/index.html">Copyright</a></li>
                <li><a href="https://info.arxiv.org/help/policies/privacy_policy.html">Privacy Policy</a></li>
              </ul>
            </div>
            <div class="column sorry-app-links">
              <ul style="list-style: none; line-height: 2;">
                <li><a href="https://info.arxiv.org/help/web_accessibility.html">Web Accessibility Assistance</a></li>
                <li>
                  <p class="help">
                    <a class="a11y-main-link" href="https://status.arxiv.org" target="_blank">arXiv Operational Status <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 256 512" class="icon filter-dark_grey" role="presentation"><path d="M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z"/></svg></a><br>
                    Get status notifications via
                    <a class="is-link" href="https://subscribe.sorryapp.com/24846f03/email/new" target="_blank"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" class="icon filter-black" role="presentation"><path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"/></svg>email</a>
                    or <a class="is-link" href="https://subscribe.sorryapp.com/24846f03/slack/new" target="_blank"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512" class="icon filter-black" role="presentation"><path d="M94.12 315.1c0 25.9-21.16 47.06-47.06 47.06S0 341 0 315.1c0-25.9 21.16-47.06 47.06-47.06h47.06v47.06zm23.72 0c0-25.9 21.16-47.06 47.06-47.06s47.06 21.16 47.06 47.06v117.84c0 25.9-21.16 47.06-47.06 47.06s-47.06-21.16-47.06-47.06V315.1zm47.06-188.98c-25.9 0-47.06-21.16-47.06-47.06S139 32 164.9 32s47.06 21.16 47.06 47.06v47.06H164.9zm0 23.72c25.9 0 47.06 21.16 47.06 47.06s-21.16 47.06-47.06 47.06H47.06C21.16 243.96 0 222.8 0 196.9s21.16-47.06 47.06-47.06H164.9zm188.98 47.06c0-25.9 21.16-47.06 47.06-47.06 25.9 0 47.06 21.16 47.06 47.06s-21.16 47.06-47.06 47.06h-47.06V196.9zm-23.72 0c0 25.9-21.16 47.06-47.06 47.06-25.9 0-47.06-21.16-47.06-47.06V79.06c0-25.9 21.16-47.06 47.06-47.06 25.9 0 47.06 21.16 47.06 47.06V196.9zM283.1 385.88c25.9 0 47.06 21.16 47.06 47.06 0 25.9-21.16 47.06-47.06 47.06-25.9 0-47.06-21.16-47.06-47.06v-47.06h47.06zm0-23.72c-25.9 0-47.06-21.16-47.06-47.06 0-25.9 21.16-47.06 47.06-47.06h117.84c25.9 0 47.06 21.16 47.06 47.06 0 25.9-21.16 47.06-47.06 47.06H283.1z"/></svg>slack</a>
                  </p>
                </li>
              </ul>
            </div>
          </div>
        </div> <!-- end MetaColumn 2 -->
        <!-- End Macro-Column 2 -->
      </div>
    </footer>
  </div>

  <script src="/static/base/1.0.1/js/member_acknowledgement.js"></script>

</body>

</html>
2408.04614
 We propose a new method, instruction backand-forth translation, to construct high-quality synthetic data grounded in world knowledge for aligning large language models (LLMs).  Given documents from a web corpus, we generate and curate synthetic instructions using the backtranslation approach proposed by Li et al.  (2023a), and rewrite the responses to improve their quality further based on the initial documents. Fine-tuning with the resulting (backtranslated instruction, rewritten response) pairs yields higher win rates on AlpacaEval than using other common instruction datasets such as Humpback, ShareGPT, Evol-Instruct, Open Orca, Alpaca-GPT4 and Self-instruct. We also demonstrate that rewriting the responses with an LLM outperforms direct distillation, and the two generated text distributions exhibit significant distinction in embedding space. Further analysis shows that our backtranslated instructions are of higher quality than other sources of synthetic instructions, while our responses are more diverse and complex than those obtained from distillation. Overall we find that instruction back-and-forth translation combines the best of both worlds—making use of the information diversity and quantity found on the web, while ensuring the quality of the responses which is necessary for effective alignment. 
abstract
Better Alignment With Instruction Back-And-Forth Translation
Abstract
2408.04614
 In recent years, it is increasingly common for large language models (LLMs) to be deployed through a chat interface to interact with users' queries.  This capability is achieved by taking models that have been pre-trained on massive amounts of webcrawled text and fine-tuning them on a relatively smaller set of instruction-response pairs or preferences (Ouyang et al., 2022). Popular instructiontuning corpora are often constructed by (i) human annotation and curation (Köpf et al., 2024; Conover et al., 2023; Zhou et al., 2024), *(ii)* converting existing texts, e.g. from other NLP tasks (Longpre et al., 2023) or web crawls (Li et al., 2023a; Köksal et al., 2023) to instruction-response pairs, and *(iii)* distilling knowledge from a model (Chiang et al., 2023; Mukherjee et al., 2023).  There are benefits and disadvantages to each of these approaches. While (i) can offer strong baselines (Zhou et al., 2024; Köpf et al., 2024), the reliance on human annotations makes scaling up these datasets difficult. The success of (i) and *(ii)* is also more dependent on having an effective filtering mechanism to remove noisy data from humans and the web. On the other hand, *(iii)* is more costeffective and scalable; many current state-of-the-art instruction datasets employ distillation. However, distillation alone has also been claimed to be a "false promise"—Gudibande et al. (2023) shows that specific capability improvements depend on the coverage of the distilled data and that the performance gains from distilling can plateau quickly.  These findings suggest that less noisy and more diverse data is crucial for instruction-tuning to close the gap between open and closed LLMs.  Given the knowledge breadth that can be found on the internet, generating instruction-response data with backtranslation has been shown to be a promising alternative to distillation (Li et al., 2023a). In particular, the authors parse the ClueWeb corpus (Overwijk et al., 2022) for selfcontained text segments, train a model specifically for instruction generation on Open Assistant data (Köpf et al., 2024), and generate instructions to go with the text segments. However, this approach relies on high-quality text data provided by ClueWeb, a paid-access corpus containing only the most popular web pages from search results, as well as a list of manually specified rules for parsing raw HTML files into structured responses (which are then used directly for fine-tuning).  In this work, we also adopt the instruction backtranslation technique but make it more accessible and scalable. We make direct use of preprocessed  ![1_image_0.png](1_image_0.png)  ![1_image_1.png](1_image_1.png)  documents from a large-scale *open-source* corpus like Dolma (Soldaini et al., 2024) and generate instructions via backtranslation accordingly. We find that the quality of our instructions are comparable to those backtranslated from ClueWeb. To make up for the lack of manually designed rules for structuring the response, we experiment with using an LLM to rewrite the response based on the generated instruction and the initial web text. This also allows us to avoid directly distilling and overfitting to an LLM's knowledge. An overview of our pipeline, which we call *back-and-forth translation*, can be found in Figure 1.  Given the same data quantity, fine-tuning Llama2-70B on the instruction-response pairs from our data generation pipeline improves the AlpacaEval win rate by 3.6% compared to using the backtranslation data from previous work (Li et al., 2023a), and by at least 3.2% compared to using other existing distillation datasets such as OpenOrca (Lian et al., 2023), ShareGPT (Chiang et al., 2023), EvolInstruct (Xu et al., 2023), Alpaca-GPT4 (Peng et al., 2023) and Self-instruct (Wang et al., 2022a) (Section 4). By asking a model to rewrite responses based on initial texts extracted from Dolma, we obtain a distribution of responses that interpolates between the original web text distribution and the distribution of outputs distilled from the same model.  Fine-tuning on the rewritten responses in turn outperforms fine-tuning on the distilled responses corresponding to the same instructions (Section 5.1).  In addition, we offer some insights into how instruction backtranslation and response rewriting affect the quality of instructions and responses respectively, especially in comparison with existing data generation methods (Sections 5.2 and 5.3). Overall our results suggest that back-andforth translation offers an effective way to generate instruction-tuning data enriched with diverse information found on the web, while ensuring the quality of the response annotations by having aligned LLMs in the loop. 
introduction
Better Alignment With Instruction Back-And-Forth Translation
Introduction
2408.04614
 Figure 1 shows an overview of our pipeline. Here we describe each step in more detail. 
body
Better Alignment With Instruction Back-And-Forth Translation
Method
2408.04614
 Our work is inspired by the backtranslation method from Li et al. (2023a). In this previous work, the authors fine-tune a base language model on some seed instruction-response pairs (xs, ys) (e.g. from Open Assistant (Köpf et al., 2024)) to obtain a backward model Myx := p(x|y) that learns to generate instructions. The authors then extract candidate responses yi from .warc files of a web corpus, ClueWeb, using carefully constructed HTMLparsing rules, and augment the responses with corresponding instructions output by the backward model. This yields a set of candidate (ˆxi, yi) pairs.  The authors also separately fine-tune the same base language model on the same seed data (xs, ys) to obtain a forward model Mxy := p(y|x) that can follow instructions. This model is then prompted to score candidate (ˆxi, yi) pairs on a 5-point scale.  The final instruction-tuning dataset consists of only score-5 examples. The paper provides ablations to show that this curation step is critical to achieving performance gains, especially with increasing instruction data quantity.  The two steps derived from Li et al. (2023a) are denoted as (1) Backtranslation and (2) Filtering in Figure 1. 
body
Better Alignment With Instruction Back-And-Forth Translation
Background: Instruction Backtranslation
2408.04614
 A major limitation of previous work (Li et al., 2023a) is the limited availability of high-quality candidate responses. The authors rely on Clueweb (Overwijk et al., 2022) as the source for unlabelled responses {yi}, using only highly linked websites visited by a search engine (e.g. Wikipedia, popular news sites) as this offers quality control over the extracted texts {yi}. However, (i) ClueWeb requires paid access, (ii) the text segments were extracted from raw HTML format, which requires specific preprocessing and may still result in segments that are sub-optimal as responses.  Our work removes the data access restrictions and preprocessing steps by using cleaned documents from an open-source corpus, Dolma (Soldaini et al., 2024), for the initial web-scraped responses. Since these documents come preextracted (with all HTML structures removed) and were written for various purposes, they may contain redundant information and the content presentation could also be improved (e.g. by being split into paragraphs). Consequently, we use an LLM to improve these documents to better resemble responses from AI Assistants; this step is denoted as (3) Rewriting in Figure 1. Conditioned on initial text yi from Dolma and the corresponding backtranslated instruction xˆi, we prompt an aligned LLM, Llama-2-70B-chat, to rewrite the response to improve its quality (y ′i ). The full prompt can be found in Appendix A.2.  By default, we apply rewriting to (ˆxi, yi) pairs that have passed the filtering stage. However, we also experiment with skipping the filtering step, i.e.  rewrite responses for any pair even if the forward model finds some web responses and corresponding generated instructions not properly aligned.  Overall we find that step (3) Rewriting is more effective compared to (2) Filtering, though using both offers complementary performance benefits. We will elaborate on this in Section 4. 
body
Better Alignment With Instruction Back-And-Forth Translation
The Rewriting Process
2408.04614
 Data. To source the initial web-crawled responses, we use the Common Crawl subset of Dolma v1 (Soldaini et al., 2024). This subset has been preprocessed with quality and content filters, in addition to undergoing deduplication. We additionally filter out documents whose lengths are close to exceeding the context length of Llama-2; this removes about 25% of the Common Crawl subset. Besides, we only use data from the head split (which consists of documents with the best perplexity scores), as preliminary experiments show that this split offers better candidate responses than the middle split (see Table 5 in Appendix).  For the seed data used to train the forward and backward models, we follow previous work (Li et al., 2023a) and use 3200 examples from the Open Assistant dataset (Köpf et al., 2024), chosen from the first turn of each conversation tree. Note that the seed data only consists of English language responses that are considered high-quality, based on their human annotated rank (rank 0). Model. We fine-tune a Llama-2-70B base model (Touvron et al., 2023) on the seed data to obtain the forward and backward models used in steps (1) and (2) of our pipeline. The rewriting step employs Llama-2-70B-chat by default. We also experiment with using a smaller model (Llama-2-7B-chat) as well as the forward model from step (2) for rewriting, but we observe that the output quality is worse (Appendix D). For performance evaluation, we finetune both the 7B and 70B scales of the Llama-2 base model on the resulting instruction-response pairs in a supervised manner. Specific hyperparameters can be found in Appendix A.  Evaluation. Given a fine-tuned Llama-2 model, we prompt it to respond to 805 questions from the AlpacaEval benchmark (Li et al., 2023b) and report the model's win rate over text-davinci-003 as evaluated by GPT-4 model. We also adopt the lengthcontrolled win rate evaluation from AlpacaEval 2.0 (Dubois et al., 2024), see Appendix E for more details. Performance on other NLP tasks can be found in Appendix F.  3 
body
Better Alignment With Instruction Back-And-Forth Translation
Experiment Setup 3.1 Training Details
2408.04614
 The other fine-tuning data sources we compare to include: - **Open Orca** (Lian et al., 2023; Mukherjee et al., 2023): contains GPT-4-distilled outputs to FLAN tasks (Longpre et al., 2023), which are converted from existing NLP datasets using manually crafted templates. The tasks have been augmented with prompting to elicit some form of reasoning during distillation.  - **ShareGPT** (Chiang et al., 2023): the data comes from ShareGPT.com, where users shared their own conversation logs with ChatGPT. We only take the first instruction and first response from each conversation for fine-tuning.  - **ClueWeb + filtering** (Li et al., 2023a): responses are parsed from HTML files in the ClueWeb corpus (Overwijk et al., 2022) and do not undergo rewriting. Instructions are generated with the backtranslation approach. After preprocessing and two rounds of curation, previous work produces 41.8K instruction-response pairs in total.  - **Self-instruct** (Wang et al., 2022a): the instructions, inputs and outputs are generated by GPT-3, bootstrapped from a small set of seed tasks.  - **Alpaca-GPT4** (Peng et al., 2023): contains GPT4-distilled responses to instructions from Alpaca dataset (Taori et al., 2023). Alpaca's instruction generation seeks to improve over the Self-instruct framework by using different prompts and a more advanced model (text-davinci-003).  - **Evol-Instruct** (Xu et al., 2023): starting from the Alpaca instruction set, this work uses a set of evolution prompts to rewrite the instructions to improve their complexity. This evolution process is repeated multiple times, using an eliminator in between to filter out the failed instructions.  Responses are then distilled from ChatGPT.  We note that except for the first two, the rest of the baselines employ synthetic instructions. Besides, the majority of these datasets (other than ClueWeb) distill responses from different existing LLMs. We use these GPT-distilled datasets for research-only, non-commercial purposes (i.e. to serve as competitive baselines to compare our method against). 
body
Better Alignment With Instruction Back-And-Forth Translation
Baselines
2408.04614
 We validate the effectiveness of our data generation method, by examining the AlpacaEval performance of Llama-2 models fine-tuned on our data versus on other datasets, given the same instruction quantity. 
body
Better Alignment With Instruction Back-And-Forth Translation
Fine-Tuning Results
2408.04614
 We fix the number of samples to be 12.8K and compare fine-tuning Llama-2-7B base model on our backtranslation and rewritten data to fine-tuning on other common baselines described in Section 3.2. Results are shown in Table 1. Below we describe our different data variations in more detail: - *Dolma + filtering:* Data obtained from steps (1) and (2) of our pipeline (Figure 1). We gather initial responses from the Dolma corpus, generate corresponding instructions with backtranslation, and filter out instruction-response pairs that do not get a score 5 from our forward model (i.e.  not well-aligned). Responses do not undergo rewriting for this baseline.  - *Dolma + rewriting:* Data obtained from steps (1) and (3) of our pipeline (Figure 1). We source candidate responses from Dolma, generate instructions with backtranslation and rewrite the responses with Llama-2-70B-chat. No intermediate filtering is done in this case.  - *Dolma + filtering + rewriting:* Data obtained from going through all the steps of our pipeline (Figure 1) as described in Section 2.  We find that at this scale, our filtered backtranslation data (*Dolma + filtering*) outperforms similarly constructed data from previous work (Li et al., 2023a) (*ClueWeb + filtering*). Our best dataset that undergoes both filtering and rewriting yields better win rate than all other baselines. It is worth noting that fine-tuning on rewritten responses from unfiltered instruction-response pairs (*Dolma + rewriting*) outperforms fine-tuning on initial web-scraped responses that have passed the filtering stage but have not been rewritten (*Dolma + filtering*). This signals that the rewriting step is more effective than filtering at improving the quality of instructiontuning data. 
body
Better Alignment With Instruction Back-And-Forth Translation
7B Scale
2408.04614
 We also experiment with fine-tuning the Llama-270B base model on different variants of the backtranslation data described in the previous section.  While the instruction-tuning data generated by previous work (Li et al., 2023a) is limited by the amount of high-quality text from the initial web corpus (i.e. ClueWeb), our approach overcomes this limitation with response rewriting. We generate 51.2K instruction-response pairs with our pipeline.  In Table 2, we find that by simply doing backtranslation on Dolma texts and filtering like previ4  Data source **Data size Win rate (%)**  Dolma + filtering 12.8K 71.70  Dolma + rewriting 12.8K 73.44  Dolma + filtering + rewriting 12.8K **74.38**  ClueWeb + filtering 12.8K 70.77  Open Orca 12.8K 74.20 ShareGPT 12.8K 72.69  Evol-Instruct 12.8K 72.32  Alpaca-GPT4 12.8K 71.17  Self-instruct 12.8K 65.11  Table 1: **Performance of fine-tuning Llama-2-7B.**  Given the same data quantity (12.8K), fine-tuning on  the instruction-response pairs obtained from instruction  back-and-forth translation outperforms fine-tuning on  the backtranslated ClueWeb data from previous work  (Li et al., 2023a), as well as other common instruction  datasets. We also find that the rewriting step is more  effective than the filtering step at improving the data  quality, and subsequently, the model win rate.  Data source **Data size Win rate (%)**  Dolma + filtering 51.2K 87.42  Dolma + rewriting 51.2K 90.52  Dolma + filtering + rewriting 51.2K **91.74**  ClueWeb + filtering 41.8K 88.18  Open Orca 51.2K 87.31 ShareGPT 51.2K 88.56  Evol-Instruct 51.2K 86.05  Alpaca-GPT4 51.2K 86.18  Self-instruct 51.2K 78.48  Dolma + filtering + rewriting 25.6K 90.22 Dolma + filtering + distilling 25.6K 87.58  Table 2: **Performance of fine-tuning Llama-2-70B.**  While backtranslation data from previous work (Li et al.,  2023a) is limited by the number of high-quality web  pages in Clueweb, our approach relies on Dolma texts  and thus has access to many more candidate responses.  Similar to the 7B scale results, (i) rewriting is more  effective at improving data quality than filtering, (ii) filtering backtranslated instructions and then rewriting  the responses does the best and outperforms previous  work. We also observe that using an aligned LLM for  response rewriting yields better data, and subsequently,  win rate, than using the same model for distillation.  ous work (*Dolma + filtering*), the resulting model slightly lags behind the Humpback model from (Li et al., 2023a) (*ClueWeb + filtering*) in terms of win rate. However, after rewriting responses in the filtered subset (*Dolma + filtering + rewriting*), we manage to outperform previous work by 3.6%.  Similar to the 7B scale, we also observe at the 70B scale that fine-tuning on rewritten responses from unfiltered instruction-response pairs (*Dolma* + rewriting) is more effective than fine-tuning on web-scraped responses that have passed the filter but have not been rewritten (*Dolma + filtering*). 
body
Better Alignment With Instruction Back-And-Forth Translation
70B Scale
2408.04614
 Given the performance benefits of rewritten data, we analyze how the outputs obtained from rewriting are different from those obtained from distillation. We also analyze the characteristics of the instruction-response pairs resulting from our pipeline compared to other existing datasets. 
body
Better Alignment With Instruction Back-And-Forth Translation
Understanding Rewritten Data Quality
2408.04614
 As rewriting involves asking Llama-2-70B-chat to improve the response quality, conditioned on an initial web-crawled response and a backtranslated instruction, a fundamental question arises: *does the* rewriting process leverage information in the raw text or does it simply distill knowledge stored in Llama-2-70B-chat?  Given the same set of backtranslated instructions, we use MAUVE score (Pillutla et al., 2021) to quantify the distributional differences among three sets of responses: initial web-scraped responses (from Dolma), rewritten responses, and responses distilled from Llama-2-70B-chat. MAUVE was originally designed to measure the gap between machine- and human-generated texts. This metric computes the area under the curve of divergence frontiers in a quantized space, after embedding text samples with a language model (by default, GPT-2). MAUVE score ranges between 0 and 1; the higher it is, the more similar the text distributions are. In the first row of Table 3, we sample two disjoint sets of 10K distilled responses and find that they exhibit high MAUVE score (0.960) as expected, since they are from the same distribution. Comparing 10K initial responses sourced from Dolma to 10K responses distilled from Llama2-70B-chat, we observe that these two sets of texts differ significantly, even though they are supposedly responses to the same (backtranslated) queries (MAUVE score = 0.0338). Rewritten responses exhibit some similarity with distilled outputs but there still exists a significant gap between them (MAUVE score = 0.340). This suggests that the rewriting process is sufficiently distinct from distillation. We provide some examples of rewritten and distilled responses in Appendix C. We also compare empirical performance of fine-tuning on rewritten data versus distilled data. For the latter, we feed 25.6K instructions randomly sampled from our filtered backtranslated dataset to Llama-2-70Bchat and let the model answer directly. For the former, we use the same set of 25.6K instructions and prompt Llama-2-70B-chat to rewrite the cor5  ![5_image_0.png](5_image_0.png)  responding web-scraped responses. Fine-tuning a Llama-2-70B model on the distilled responses yields lower win rate compared to fine-tuning on the rewritten texts (bottom two rows of Table 2).  This demonstrates that the rewriting process improves the quality of response data in general, beyond just extracting what an LLM already knows, possibly because rewriting incorporates the information diversity found in web-scraped texts. 
body
Better Alignment With Instruction Back-And-Forth Translation
Rewriting Versus Distilling
2408.04614
 Measuring instruction quality empirically.  We attempt to isolate the quality of instructions from different datasets by unifying the response distribution to be outputs distilled from the same model. More specifically, we randomly sample 12.8K instructions from our backtranslation data (with and without filtering) as well as from each of the baseline datasets described in Section 3.2. We discard all existing answers, and feed each set of 12.8K queries to Llama-2-70B-chat to obtain distilled responses. We then fine-tune a Llama-2-7B model on each set of (instruction, new response) pairs and evaluate the AlpacaEval win rate of the resulting model. In Figure 2, we find that (i) filtered backtranslation instructions (i.e. from Dolma or ClueWeb) outperform other synthetic instruction generation methods (i.e. Alpaca (Taori et al., 2023) and Self-instruct (Wang et al., 2022a)), *(ii)* however, synthetic instructions still lag behind humanwritten ones, obtained via user interactions with a chat interface (ChatGPT), *(iii)* instructions constructed from transforming existing NLP datasets (i.e. Open Orca (Lian et al., 2023; Mukherjee et al., 2023)) do not yield high distillation performance, possibly due to limited and repetitive task types.  ![5_image_1.png](5_image_1.png)  Diversity. As a proxy for diversity, we measure the number of unique trigrams across different instruction sets, given the same data quantity. In Figure 3 (left), we observe that while our instruction set (backtranslated from Dolma) is more diverse than other synthetic instruction setsincluding ClueWeb, Alpaca and Self-instruct, there is still a significant gap in diversity between modelgenerated instructions and manually-crafted ones (e.g., Open Orca and ShareGPT).  It is worth noting that the human-written instructions (e.g. during their interactions with ChatGPT) tend to be substantially longer than synthetic ones, see Table 6 in the appendix. We thus conduct an ablation study to study how much the length factor contributes to the quality of instructions. We filter out instructions in ShareGPT that exceed the maximum length of our backtranslated instructions (i.e. 514 tokens) and among the remaining datapoints, randomly sample 12.8K instructions to repeat the distillation experiment described earlier (Figure 2). This yields a shorter version of ShareGPT ("ShareGPT (short)") in which the instructions have about the same average length as our backtranslated instructions (i.e. 69 tokens). We observe that when fixing the output distribution to be distilled responses from Llama-2-70B-chat,  ![6_image_0.png](6_image_0.png)  fine-tuning on "ShareGPT (short)" is less effective than using the original ShareGPT instructions, but still outperforms backtranslated instructions (Figure 2). This suggests that independent of length, human-written instructions are still of higher quality overall compared to synthetic instructions.  Complexity. Following the InsTag Complexity metric employed by Liu et al. (2023), we use the InsTag public tagger (Lu et al., 2023), which is a Llama-2-7B model fine-tuned on ChatGPTgenerated tags, to automatically tag our text data with all detectable semantics and intentions. We use the average number of unique tags uncovered in instructions, and separately, responses, from each dataset as a proxy for complexity. In Table 4, we observe that our Dolma-backtranslated instructions offers higher InsTag complexity than most instruction sets from previous work, including ClueWebbacktranslated data (Li et al., 2023a) and Open Orca (Lian et al., 2023). Human-written instructions, i.e. from user interactions with ChatGPT, still yield the highest number of tags on average. 
body
Better Alignment With Instruction Back-And-Forth Translation
Instruction Quality Analysis
2408.04614
 Diversity. We apply the same diversity analysis as in Section 5.2 to response data. In Figure 3 (right), we observe that the initial web responses sourced from Dolma are substantially more diverse than outputs distilled from existing models (e.g. ChatGPT, GPT-4, GPT-3), as well as web  Data source Instruction **Response** Dolma + filtering 5.6 8.3 Dolma + rewriting 4.8 6.8  Dolma + filtering + rewriting 5.1 6.6  ClueWeb 3.7 6.6  Open Orca 3.5 4.9  Alpaca-GPT4 3.6 4.4 Self-instruct 3.1 5.1 ShareGPT 6.2 5.2  Table 4: **Average number of semantic and intention tags uncovered in different instruction-tuning**  datasets by the InsTag model (Lu et al., **2023).** While  our backtranslated instructions yield fewer tags than  human-written ones (i.e. ShareGPT), they exhibit  higher complexity than synthetic instructions from other datasets. Besides, we find that responses sourced from or are based on web texts generally have higher InsTag  complexity than those obtained via distillation.  texts from ClueWeb. The rewriting process reduces information diversity of these web-scraped responses slightly, but still leads to much more diverse responses than distillation. Overall this analysis demonstrates the importance of including web sources in the data construction process, in order to encourage more diverse instructions and responses, compared to just distilling knowledge from existing LLMs.  Complexity. Applying the same analysis as in Section 5.2, using the average number of semantic and intention tags uncovered by InsTag tagger (Lu et al., 2023) as a proxy for complexity, we compare the responses from our pipeline (with and without rewriting) to responses from other baseline datasets.  In Table 4, we find that the initial web-scraped responses from Dolma yield the highest complexity.  Rewriting generally reduces the InsTag complexity of the response. However, our rewritten responses are still substantially more complex than most existing response data, which is commonly distilled from high-performing LLMs.  We also provide an analysis of instruction and response lengths as another quality metric. Refer to Appendix B for more details. 
body
Better Alignment With Instruction Back-And-Forth Translation
Response Quality Analysis
2408.04614
 We discuss related papers that construct new instruction-tuning datasets or propose methods to improve existing ones. More in-depth review can be found in Zhang et al. (2023).  Human-crafted data. Open Assistant (Köpf et al., 2024), Dolly (Conover et al., 2023) and Super-NI (Wang et al., 2022b) are some examples of datasets that contain solely human-generated and human-annotated conversations, covering a range of topics and NLP tasks. These datasets tend to be relatively small in scale due to the expensive costs of manual annotation and verification.  Other papers do not explicitly ask humans to create questions and answer them, but instead repurpose existing datasets. For example, FLAN (Longpre et al., 2023) and Natural instructions (Mishra et al., 2021) transform inputs and outputs of more than 60 NLP tasks into instruction-tuning data. This suffers from the same scalability issue as human-annotated datasets. Synthetic instruction generation. In contrast to sourcing manually written instructions, which may be expensive to scale, some papers propose ways to automatically generate large quantities of instructions (Wang et al., 2022a; Taori et al., 2023). In particular, our work is inspired by the backtranslation technique proposed in Li et al. (2023a), which fine-tunes an LLM specifically for the task of instruction generation, and then applies the model to augment text segments extracted from the web with corresponding instructions. The paper suggests that this approach allows the resulting instructiontuning data to be more diverse especially in the long tail. Another prior work, LongForm (Köksal et al., 2023), introduces a similar approach for generating instructions.  Most related to our approach is the work by Chen et al. (2023b), who train an LLM to generate *both* instructions and responses from web-scraped documents. In contrast to their method, we (i) generate instructions separately with backtranslation and then ask an LLM to improve the existing responses, (ii) obtain better performance with much fewer data (Table 5) (iii) generate more data (51.2K compared to 12.4K), (iv) offer more insights into the quality of our instructions and responses in comparison to other existing datasets. In addition, concurrent work by Zheng et al. (2024) also proposes more detailed scoring and refinement prompts to improve the instruction curation and response formatting of the backtranslation pipeline from Li et al. (2023a), applying it to Chinese text data. Distillation. Perhaps the most common approach in instruction-tuning data generation, distillation seeks to mimic the capabilities of powerful LLMs (e.g. GPT-4) by feeding queries to these models and using the outputs to fine-tune subsequent LLMs.  Datasets that are built this way include ShareGPT (Chiang et al., 2023), OpenInstruct (Wang et al., 2023), Alpaca-GPT4 (Peng et al., 2023) and UltraFeedback (Tunstall et al., 2023).  Improving instruction-tuning data quality.  Some prior work studies characteristics of highquality instruction-tuning data (Liu et al., 2023) and proposes curation techniques accordingly. LIMA (Zhou et al., 2024) carefully collects 1K fine-tuning samples via both internet sourcing and human annotation, and shows that strong performance can be achieved despite the small data quantity. Similarly, Chen et al. (2023a) demonstrates that performance gain is possible by fine-tuning on only a small subset of the original dataset (Alpaca), using ChatGPT as the quality evaluator. Zhao et al.  (2024) finds that selecting only the 1K longest responses from existing datasets offers a very strong baseline, independent of GPT-4's preference for longer texts. Evol-Instruct (Xu et al., 2023) and Orca (Mukherjee et al., 2023) manually prompt models to enhance the complexity of instructions, and subsequently, data generation (e.g. by asking for justification). Fan et al. (2024) reformats the responses of existing instruction data to augment them with relevant information and align them with pre-determined criteria set by humans.  7 Discussion We propose instruction back-and-forth translation: combining instruction backtranslation method from Li et al. (2023a) with response rewriting, in order to benefit from both the information diversity found on the internet and the quality of model annotations, while enabling scalability owing to the size of the web corpus where we source initial responses from.  Future work. Our findings motivate a number of interesting future directions. One concrete question is whether applying other existing curation techniques—e.g. quality filters proposed by Liu et al. (2023)—to our pool of (synthetic instructions, rewritten response) pairs would lead to further performance gains. In addition, we also look forward to scaling up our data generation pipeline and studying the implication of the rewritten data on the pre-training process, given concurrent work (Maini et al., 2024) that explores paraphrasing pretraining data into the question-answering format.  Limitations. Although we try to control for confounding factors (e.g. data quantity), our findings are only obtained from using one model family, i.e. Llama-2 (Touvron et al., 2023). Besides, our pipeline revolves around general-purpose English instructions, with limited coding or science-related tasks. Nevertheless, it is possible to extend our method to more domain-specific data, e.g. by crawling texts from StackOverflow, generating instructions and rewriting the responses with Code Llama (Roziere et al., 2023). 
body
Better Alignment With Instruction Back-And-Forth Translation
Related Work
2408.04614
 possible that sourcing response data from the web could affect the factuality of the fine-tuned model and/ or make it more prone to hallucination. The same risks apply to the response rewriting process. Future work could include additional steps to verify the quality of the information in the responses, and check whether it overlaps with what the model already knows, before using the data for fine-tuning. 
body
Better Alignment With Instruction Back-And-Forth Translation
Ethical Considerations & Potential Risks. It Is
2408.04614
 We are grateful to Weijia Shi and Mike Lewis for helpful discussion as well as feedback on the manuscript. TN is supported by the UW-Meta AI Mentorship Program. This work is supported in part by NSF grant 2019844. 
body
Better Alignment With Instruction Back-And-Forth Translation
Acknowledgments
2408.04614
 Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. Piqa: Reasoning about physical commonsense in natural language. In *Proceedings of the* AAAI conference on artificial intelligence, volume 34, pages 7432–7439.  Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. 2023a. Alpagasus: Training a better alpaca with fewer data.  arXiv preprint arXiv:2307.08701.  Yongrui Chen, Haiyun Jiang, Xinting Huang, Shuming Shi, and Guilin Qi. 2023b. Tegit: Generating highquality instruction-tuning data with text-grounded task design. *arXiv preprint arXiv:2309.05447*.  Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An opensource chatbot impressing gpt-4 with 90%* chatgpt quality.  Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. *arXiv* preprint arXiv:1803.05457.  Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world's first truly open instructiontuned llm.  Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. 2024. Length-controlled alpacaeval: A simple way to debias automatic evaluators. *arXiv preprint arXiv:2404.04475*.  Run-Ze Fan, Xuefeng Li, Haoyang Zou, Junlong Li, Shwai He, Ethan Chern, Jiewen Hu, and Pengfei Liu. 2024. Reformatted alignment. arXiv preprint arXiv:2402.12219.  Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. The false promise of imitating proprietary llms. *arXiv preprint arXiv:2305.15717*.  Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt.  2020. Measuring massive multitask language understanding. *arXiv preprint arXiv:2009.03300*.  Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. *arXiv preprint arXiv:1904.09751*.  Abdullatif Köksal, Timo Schick, Anna Korhonen, and Hinrich Schütze. 2023. Longform: Optimizing instruction tuning for long text generation with corpus extraction. *arXiv preprint arXiv:2304.08460*.  Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Richárd Nagyfi, et al. 2024. Openassistant conversations-democratizing large language model alignment. *Advances in Neural Information Processing Systems*, 36.  Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, Jason Weston, and Mike Lewis. 2023a. Self-alignment with instruction backtranslation. *arXiv preprint arXiv:2308.06259*.  Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023b. Alpacaeval: An automatic evaluator of instruction-following models.  https://github.com/tatsu-lab/alpaca_eval.  Wing Lian, Bleys Goodson, Eugene Pentland, Austin Cook, Chanvichet Vong, and "Teknium". 2023. Openorca: An open dataset of gpt augmented flan reasoning traces. https://https://huggingface. co/Open-Orca/OpenOrca. Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. 2023. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. *arXiv preprint* arXiv:2312.15685.  Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. 2023. The flan collection: Designing data and methods for effective instruction tuning. In *International Conference on* Machine Learning, pages 22631–22648. PMLR.  Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. 2023. \# instag: Instruction tagging for analyzing supervised fine-tuning of large language models.  In *The Twelfth International Conference on Learning* Representations.  Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, and Navdeep Jaitly. 2024. Rephrasing the web: A recipe for compute and data-efficient language modeling. *arXiv preprint arXiv:2401.16380*.  Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021. Cross-task generalization via natural language crowdsourcing instructions.  arXiv preprint arXiv:2104.08773. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707.  Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.  2022. Training language models to follow instructions with human feedback. *Advances in neural information processing systems*, 35:27730–27744.  Arnold Overwijk, Chenyan Xiong, and Jamie Callan.  2022. Clueweb22: 10 billion web documents with rich information. In *Proceedings of the 45th International ACM SIGIR Conference on Research and* Development in Information Retrieval, pages 3360– 3362. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. Instruction tuning with gpt-4. *arXiv preprint arXiv:2304.03277*.  Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. Mauve: Measuring the gap between neural text and human text using divergence frontiers. *Advances in Neural Information Processing Systems*, 34:4816–4828.  Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. 2023.  Code llama: Open foundation models for code. *arXiv* preprint arXiv:2308.12950.  Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, and Kyle Lo. 2024. Dolma: An Open Corpus of Three Trillion Tokens for Language Model Pretraining Research. *arXiv preprint*. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Alpaca: A strong, replicable instruction-following model. *Stanford Center for Research on Foundation Models.* https://crfm. stanford. edu/2023/03/13/alpaca. html, 3(6):7.  Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.  Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2023.  Zephyr: Direct distillation of lm alignment. *Preprint*, arXiv:2310.16944.  Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. 2023. How far can camels go? exploring the state of instruction tuning on open resources. *Preprint*, arXiv:2306.04751.  Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022a. Self-instruct: Aligning language models with self-generated instructions. *arXiv* preprint arXiv:2212.10560.  Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al.  2022b. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. arXiv preprint arXiv:2204.07705.  Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244.  Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? *arXiv preprint* arXiv:1905.07830.  Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. 2023. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792.  Hao Zhao, Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. 2024. Long is more for alignment: A simple but tough-to-beat baseline for instruction fine-tuning. *arXiv preprint* arXiv:2402.04833.  Tianyu Zheng, Shuyue Guo, Xingwei Qu, Jiawei Guo, Weixu Zhang, Xinrun Du, Chenghua Lin, Wenhao Huang, Wenhu Chen, Jie Fu, et al. 2024.  Kun: Answer polishment for chinese self-alignment with instruction back-translation. *arXiv preprint* arXiv:2401.06477.  Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2024. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36. 
references
Better Alignment With Instruction Back-And-Forth Translation
References
2408.04614
 We randomly sample text segments from different Common-Crawl-derived corpora, generate the corresponding instructions with backtranslation, and filter the candidate pairs to obtain 3200 highquality samples for fine-tuning Llama-2-7B. In Table 5, even though Dolma head split still slightly lags behind ClueWeb when it comes to producing high-quality responses for fine-tuning, it still outperforms other web data sources such as C4 and Dolma middle split.  | Data source                                      | Data size   | Win rate (%)   | |--------------------------------------------------|-------------|----------------| | C4                                               | 3.2K        | 66.83          | | Dolma (head)                                     | 3.2K        | 67.92          | | Dolma (middle)                                   | 3.2K        | 67.21          | | ClueWeb                                          | 3.2K        | 68.04          | | Table 5: Comparison of text quality from various |             |                | 
body
Better Alignment With Instruction Back-And-Forth Translation
A More Training Details A.1 Web Corpus Ablation
2408.04614
 By default, we use nucleus sampling (Holtzman et al., 2019) for our data generation.  (1) Backtranslation We prompt the fine-tuned backward model to generate instruction that can go with a given web text response, with T = 1.0, p = 0.9. The prompt follows from previous work (Li et al., 2023a): [INST] Below is a candidate answer to a question or instruction from an user. Write the most likely question to which the text below would be a great answer.  <response> Answer in the style of an AI Assistant.  [/INST] (2) Filtering We prompt the fine-tuned forward model to score instruction-response pairs with T = 1.0, p = 0.9. The prompt for scoring could be found in Table 19 of previous work (Li et al., 2023a).  (3) Rewriting Given <response>, which is a cleaned text document from Dolma, and the corresponding <instruction> generated by our backward model, we ask Llama-2-70B-chat to improve the response with T = 1.0, p = 0.9 and the following prompt: [INST] Given the draft response to the provided question below, rewrite the draft to improve it, so it is a high quality response to the given question.  Draft Response: <response> 
body
Better Alignment With Instruction Back-And-Forth Translation
A.2 Data Generation Hyperparameters
2408.04614
 Given the above question, rewrite the draft response to be an improvement over the draft response. It should be as similar as possible, copying text where possible, while making the flow more clear, useful, relevant and providing a direct answer to the question. It should be written to be impeccably tailored to the user's question as if written by an AI Assistant, without extraneous information, reflecting expert knowledge, and demonstrating a high-quality, engaging, and insightful answer. Try not to add new facts that are not already in the draft response.  Return the rewritten response between [RES] and [/RES]. [/INST] 
body
Better Alignment With Instruction Back-And-Forth Translation
Question: <Instruction>
2408.04614
 We fine-tune Llama-2 model (Touvron et al., 2023) with 7B and 70B parameters. By default, we use a cosine learning rate schedule with batch size 32, weight decay 0.1 and dropout 0.1. For the 7B scale, we use learning rate 1e-5. With any dataset size smaller than or equal to 25.6K, we fine-tune for {600, 900, 1200} steps and report the highest win rate. For the 70B scale, we use learning rate 5.5e-6 and fine-tune on all 51.2K-size datasets for 1600 steps, and all 25.6K-size datasets for 1200 steps.  Following previous work (Li et al., 2023a), in each of our fine-tuning experiments, we combine both seed data (3.2K samples from Open Assistant) and the dataset of interest, tagging the former distribution with "Answer in the style of an AI Assistant." and the latter one with "Answer with knowledge from web search." to distinguish the two data sources.  Each 7B-scale fine-tuning run takes 4 hours with 8 A100 GPUs and 12.8K examples, while each 70B-scale run takes 16 hours with 64 A100 GPUs and 51.2K examples. 
body
Better Alignment With Instruction Back-And-Forth Translation
A.3 Training Hyperparameters
2408.04614
 Similar to Li et al. (2023a), we use nucleus sampling (Holtzman et al., 2019) with temperature T = 0.7, p = 0.9 for generation. When evaluating on the AlpacaEval set (Li et al., 2023b), we append the questions with the prompt "Answer in the style of an AI Assistant." or "Answer in the style of an AI Assistant, with knowledge from web search if needed.", in accordance with the tags used during training, and pick whichever prompt that leads to higher win rate on average. 
body
Better Alignment With Instruction Back-And-Forth Translation
A.4 Evaluation Hyperparameters
2408.04614
 Average length In Table 6, we report the average length, measured in number of tokens, for instructions and responses from our datasets and other baselines described in Section 3.2.  | Data source                                                                                             | Instruction   | Response   | |---------------------------------------------------------------------------------------------------------|---------------|------------| | length                                                                                                  | length        |            | | Dolma + filtering                                                                                       | 69            | 499        | | Dolma + rewriting                                                                                       | 58            | 449        | | Dolma + filtering + rewriting                                                                           | 69            | 468        | | Dolma + filtering + distilling                                                                          | 69            | 567        | | ClueWeb                                                                                                 | 31            | 442        | | Open Orca                                                                                               | 248           | 181        | | ShareGPT                                                                                                | 264           | 318        | | Alpaca-GPT4                                                                                             | 25            | 163        | | Self-instruct                                                                                           | 38            | 37         | | Open Assistant                                                                                          | 40            | 273        | | Table 6: Average token length of instructions and responses from different instruction-tuning datasets. |               |            |  When it comes to instructions, we find that those generated by backtranslation tend to be longer than those obtained from other synthetic instruction generation methods (i.e. Alpaca (Taori et al., 2023) or Self-instruct (Wang et al., 2022a)). However, our backtranslation instructions are substantially shorter than manually crafted instructions (i.e. Open Orca (Mukherjee et al., 2023; Lian et al., 2023) or ShareGPT (Chiang et al., 2023)). We hypothesize that this property of our generated instructions is influenced by the length of instructions in the seed dataset (i.e. Open Assistant) used to train the backward model. Future work could explore using datasets with longer instructions for training the backward model, or enhance the complexity of synthetic instructions further with specifically crafted prompts, such as via the method proposed in Evol-Instruct (Xu et al., 2023).  In terms of responses, those obtained from instruction back-and-forth translation are overall longer than those found in other existing datasets.  We observe that rewriting reduces the response length, possibly due to the removal of irrelevant information in the initial web-crawled responses (468 tokens versus 499 tokens on average). On the other hand, given the same backtranslated instructions, distilling from an existing model yields significantly longer responses than the initial Dolma texts (567 tokens versus 468 tokens on average).  Score distribution In step (2) Filtering of our pipeline, we ask the fine-tuned forward model to score (backtranslated instruction, web response from Dolma) pairs. We use a set of regex expressions to extract the scores from the model's responses. We find that the score is valid (i.e. being in the range of 1 to 5) 88% of the time, with 5-score examples making up 13.6% of the data pool. 
body
Better Alignment With Instruction Back-And-Forth Translation
B Data Statistics
2408.04614
 Refer to Tables 7, 8 and 9.  | Backtranslated instruction   | write a press release for apple announcing the acquisition of shazam                                                                                                                                     | |------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Response (from Dolma)        | Cupertino, California - Apple today announced it has completed its acquisition of Shazam, one of the world's most popular and highly-rated music apps, used by hundreds of millions of people worldwide. | | Rewritten response           | Apple Acquires Shazam: A New Chapter in Music Recognition Technology Cupertino, California - Apple is thrilled to announce the completion of its acquisition of Shazam, the world-renowned music app used by hundreds of millions of people globally. This strategic move marks a significant milestone in Apples mission to revolutionize the way ´ people discover, enjoy, and interact with music. With Shazam, users can effortlessly identify songs, artists, and albums in a matter of seconds, making it an indispensable tool for music lovers everywhere. By integrating Shazams´ innovative technology into our ecosystem, we can further enhance the music experience for our customers, providing them with seamless access to the songs and artists they love. "We are delighted to welcome Shazam to the Apple family," said Tim Cook, CEO of Apple. "This acquisition is a testament to our commitment to delivering the best possible music experiences to our customers. We look forward to exploring the exciting possibilities that this partnership presents, and we are eager to see where this new chapter will take us." Shazam has been a game-changer in the music industry since its inception, using audio fingerprinting technology to quickly and accurately recognize songs. This cutting-edge tech has made it possible for users to identify music playing in the background, whether in a store, on the radio, or even in a movie. Shazam has also expanded its offerings to include features like lyrics, recommendations, and playlists, cementing its place as a leader in the music recognition space. The union between Apple and Shazam is a natural one. Both companies share a passion for delivering exceptional user experiences, and we are confident that this collaboration will bring even more joy and convenience to music fans everywhere. "We are incredibly excited to join forces with Apple, a company that shares our dedication to delivering the best possible user experience," said Rich Riley, CEO of Shazam. "We look forward to working together to create new and innovative ways for people to enjoy and engage with music." Apple and Shazam share a rich history, dating back to the early days of the App Store. Shazam was one of the first apps to be featured on the App Store, and it quickly became a fan favorite. Today, this partnership represents the next step in that journey, as we continue to push the boundaries of what is possible with music recognition technology. At Apple, we believe that music has the power to bring people together and enrich their lives. With the addition of Shazam to our family, we are one step closer to making that vision a reality. So, let the music play, and let's see where this journey takes us next.                                                                                                                                                                                                          |  Distilled response Sure! Here's a possible press release for Apple announcing the acquisition of Shazam: Apple Acquires Shazam, Bringing Music Recognition Technology In-House Cupertino, California - December 17, 2022 - Apple today announced that it has acquired Shazam, a leading music recognition platform that has been a popular choice for music fans around the world. The acquisition brings Shazam's innovative audio fingerprinting technology and extensive music database into the Apple fold, further enhancing the companys´ capabilities in the music and audio space.  "We´re thrilled to welcome Shazam to the Apple family," said Tim Cook, CEO of Apple. "Shazam's technology and talented team will play a key role in our efforts to continue delivering the best music experiences to our customers. We're looking forward to exploring the many ways that Shazam's audio fingerprinting tech can be integrated into our products and services, and we're excited to see what the future holds for this partnership." Shazam's proprietary audio fingerprinting technology allows users to identify songs and artists playing around them simply by holding their device near the source of the sound. The platform has become a go-to destination for music discovery and identification, with over 1 billion downloads worldwide and counting.  "We´re proud of the Shazam team for building such a powerful and beloved platform," said Oliver Schusser, Vice President of Apple Music. "We're excited to integrate Shazam's technology into our own products and services, and we're looking forward to working with the Shazam team to take music recognition and discovery to the next level." The acquisition follows Apple's purchase of Beats Electronics in 2014, which brought the popular Beats headphones and streaming service into the Apple family. Like that deal, the Shazam acquisition underscores Apple's commitment to delivering the best possible music experiences to its customers.  Terms of the deal were not disclosed. Shazam will continue operating independently, with the same leadership and team that have driven its success to date.  About Apple Apple revolutionized personal technology with the introduction of the Macintosh computer in 1984. Today, Apple leads the world in innovation with iPhone, iPad, Mac, Apple Watch, and Apple TV. Apple's four software platforms—iOS, macOS, watchOS, and tvOS—provide seamless experiences across all Apple devices and empower people with breakthrough services including the App Store, Apple Music, Apple Pay, and iCloud. Apple's 100,000 employees are dedicated to making the best products on earth, and to leaving the world better than they found it.  About Shazam Shazam is a popular music recognition platform that has been a go-to destination for music fans around the world. With over 1 billion downloads worldwide, Shazam lets users identify songs and artists playing around them simply by holding their device near the source of the sound. Shazam's proprietary audio fingerprinting technology has made it a leader in music recognition and discovery, and its extensive music database has made it a valuable resource for music fans everywhere.  Contacts: Apple Media Relations: [media.relations@apple.com](mailto:media.relations@apple.com) Shazam Press Office: [press@shazam.com](mailto:press@shazam.com) Of course, this is just one possible version of a press release announcing Apple's acquisition of Shazam. The actual press release could be different, depending on the specifics of the deal and the messaging that Apple and Shazam want to convey.  Table 7: **Example of a score-5 (backtranslated instruction, web-scraped response) pair**  | Backtranslated instruction   | Why doesn't anybody ride the train in San Jose?                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | |------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Response (from Dolma)        | Less than 1 percent of Santa Clara County residents ride VTA light rail, and taxpayers subsidize 85 percent of the operating costs. Light rail recently celebrated its 25th anniversary in Silicon Valley, but most people in the area were probably too busy driving to celebrate. Mike Rosenberg of the Mercury News reports that the system, run by the Valley Transportation Authority, is among the country's worst by just about every common metric of success. To wit: Less than 1 percent of Santa Clara County residents ride VTA light rail; the per-passenger round-trip operating cost is $11.74 and taxpayers subsidize 85 percent of costs - third and second worst in the country, respectively. There are problems with measuring costs per passenger mile on light rail, but ouch. Light-rail agencies in Minneapolis, Houston, Newark, N.J., and Phoenix each run less service than VTA yet carry more passengers than the South Bay's network. Several cities that are much smaller than San Jose - from St. Louis to Salt Lake City to Portland, Ore. - also feature light-rail systems with more riders than VTA. Sacramento - which also opened its light-rail network in 1987, operates with approximately the same level of service and runs through a similarly sprawled-out region - carries nearly 40 percent more passengers per day than VTA. When I first encountered VTA light rail in San Jose (briefly) in 1990. I noted how slowly it moved thru the downtown area, intermingling amongst pedestrian traffic. I thought it then to be weird at best for a city to run light rail on a sidewalk where pedestrians normally are. Fast forward to today (after returning to New Jersey then moving back to California), even with system-wide expansion to Mountain View, South San Jose and East San Jose, and its still the same, sad slowness I remember. Silicon Valley must be getting used to bad transit news at this point. In November, Rosenberg reported that a VTA plan to extend a light rail line 1.6 miles to Los Gatos, home of Netflix, will cost $175 million while drawing only about 200 new riders. Back in May, a local news station found a culture of fare evasion on VTA that gives the system a rate of 7.2 percent - highest in the region. In April, construction began on an extension of BART from the Bay Area to Santa Clara, ending at the Berryessa station, where it will meet the VTA light rail system. Among that project's shortcomings are the fact that the stop falls short of downtown San Jose, and that Berryessa seems poorly suited to transit-oriented development. An extension to San Jose is still in the plans, though BART service to the city would duplicate the Caltrain commuter rail that already goes there. Not that anyone in Silicon Valley uses Caltrain anyway. In October, Stamen Designs released a map of private bus lines that serve the campuses of Google, Apple, Facebook, and the like for employees living in downtown San Francisco. These buses are seen as a better option than Caltrain, which runs infrequently and requires a number of transfers from the city to reach Silicon Valley... | | Rewritten response           | San Jose's VTA light rail system has struggled with low ridership, high costs, and poor performance, raising the question of why nobody rides the train in San Jose. According to Mike Rosenberg of the Mercury News, the system has some of the worst metrics in the country, with fewer than 1% of Santa Clara County residents using the train and taxpayers subsidizing 85% of the operating costs. One reason for the low ridership may be the limited coverage area, as the system primarily serves the downtown area and nearby neighborhoods. Additionally, the trains move slowly through the downtown area, which may not appeal to commuters looking for a quick and efficient ride. Moreover, Silicon Valley's tech industry leaders seem to prioritize corporate campuses over accessible public transportation. Private bus lines for employees living in downtown San Francisco are viewed as a better alternative to Caltrain, which runs infrequently and requires multiple transfers to reach Silicon Valley. Despite efforts to expand the VTA light rail system, including a planned extension to Los Gatos that will cost $175 million and draw only 200 new riders, the system still falls short of meeting the region's transportation needs. The Berryessa station, where the VTA light rail system will meet the BART, is not located in downtown San Jose, and the extension to San Jose is still in the plans but will duplicate the Caltrain commuter rail that already exists. In conclusion, the low ridership of the VTA light rail system in San Jose can be attributed to a combination of factors, including limited coverage, slow travel times, and a lack of investment in accessible public transportation by the tech industry. Addressing these issues could potentially improve the system's performance and encourage more people to use the train.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |  | Distilled response                                                                    | There could be several reasons why people may not ride the train in San Jose, California. Here are a few possible reasons: 1. Limited public transportation options: San Jose is a sprawling city with a car-centric culture, and public transportation options may be limited, especially outside of downtown areas. This can make it difficult for people to access the train or other forms of public transportation, especially if they live in suburban or rural areas. 2. Lack of convenient routes: The train routes in San Jose may not be convenient for many residents, especially if they live in areas that are not serviced by the train. This can make it difficult for people to use the train for their daily commutes or other trips. 3. Limited frequency of service: The frequency of train service in San Jose may be limited, with few trains running during off-peak hours or on weekends. This can make it difficult for people to use the train for spontaneous trips or for traveling outside of peak commute times. 4. Competition from other modes of transportation: San Jose is located in the heart of Silicon Valley, and many people may prefer to use their own cars or ride-hailing services like Uber or Lyft to get around. This can make it difficult for the train to compete for passengers, especially in areas with heavy traffic or limited public transportation options. 5. Lack of awareness or promotion: The train service in San Jose may not be well-promoted or well-known, especially among visitors or new residents. This can make it difficult for people to learn about the train and its routes, schedules, and fares. 6. Affordability: The cost of taking the train in San Jose may be prohibitively expensive for some people, especially for those who are used to cheaper transportation options like driving or ride-hailing services. This can make it difficult for the train to attract a diverse range of passengers. 7. Safety concerns: Some people may be hesitant to ride the train in San Jose due to safety concerns, such as concerns about crime or safety on the trains or in the surrounding areas. 8. Lack of amenities: The train service in San Jose may not offer the same level of amenities as other modes of transportation, such as comfortable seating, Wi-Fi, or food and beverage options.   | |---------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Table 8: Example of a score-3 (backtranslated instruction, web-scraped response) pair |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |  | Backtranslated instruction                                                            | When was the Chicago Symphony Orchestra's first tour of Russia and Eastern Europe, and were any recordings made during that tour?                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | |---------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Response (from Dolma)                                                                 | You are currently browsing the tag archive for the 'András Schiff' tag. Congratulations to Bernard Haitink—the Chicago Symphony Orchestra's principal conductor from 2006 until 2010 and a frequent guest conductor—the recipient of this year's Gramophone magazine award for lifetime achievement! Congratulations, Maestro Haitink! Haitink returns to Chicago in April 2016, leading the Orchestra in Mozart's Piano Concerto no. 22 with Till Fellner and Strauss's An Alpine Symphony. Sir Georg Solti led the Chicago Symphony Orchestra's first trip to the Soviet Union and Hungary in November 1990, also including a single stop in Vienna. Two recordings were made during the tour, both for London Records. The performance of Bruckner's Eighth Symphony was recorded live in Leningrad on November 22 (London's first recording venture in the Soviet Union); Michael Haas was the producer, James Lock and Colin Moorfoot were the engineers, and Sally Drew was the tape editor. Mahler's Fifth Symphony was recorded live in Vienna on November 30, Michael Haas was the producer, Stan Goodall was the engineer, and Matthew Hutchinson was the tape editor.                           | | Rewritten response                                                                    | The Chicago Symphony Orchestra's first tour of Russia and Eastern Europe took place in November 1990, led by Sir Georg Solti. During this tour, the orchestra made two recordings for London Records. The first recording was of Bruckner's Eighth Symphony, which was recorded live in Leningrad on November 22, with Michael Haas as producer, James Lock and Colin Moorfoot as engineers, and Sally Drew as tape editor. The second recording was of Mahler's Fifth Symphony, recorded live in Vienna on November 30, with Michael Haas as producer, Stan Goodall as engineer, and Matthew Hutchinson as tape editor. Congratulations are in order for Bernard Haitink, who will be returning to Chicago in April 2016 to lead the orchestra in Mozart's Piano Concerto No. 22 with Till Fellner and Strauss's An Alpine Symphony. Haitink, who served as the principal conductor of the Chicago Symphony Orchestra from 2006 to 2010, is this year's recipient of Gramophone magazine's lifetime achievement award.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | | Distilled response                                                                    | The Chicago Symphony Orchestra's first tour of Russia and Eastern Europe took place in 1971. The orchestra, led by music director Georg Solti, performed in several cities, including Moscow, Leningrad (now St. Petersburg), Warsaw, and Prague. During the tour, the orchestra made several recordings, including a live recording of Tchaikovsky's Symphony No. 6, which was recorded at the Great Hall of the Moscow Conservatory. This recording was later released on the RCA Red Seal label and won a Grammy Award for Best Classical Performance - Orchestra. Additionally, the orchestra recorded a live performance of Shostakovich's Symphony No. 5 in Leningrad, which was also released on RCA Red Seal. These recordings were significant not only for their artistic quality but also for their historical significance, as they represented one of the first times a Western orchestra had toured the Soviet Union and Eastern Europe since the end of World War II. The tour and recordings helped to establish the Chicago Symphony Orchestra as one of the leading ensembles in the world and paved the way for future cultural exchanges between the United States and Eastern Europe. | | Table 9: Example of a score-1 (backtranslated instruction, web-scraped response) pair |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | 
body
Better Alignment With Instruction Back-And-Forth Translation
C Examples Of Our Instruction-Tuning Data
2408.04614
 Besides Llama-2-70B-chat, we also experiment with rewriting web-crawled responses with: (i) a smaller scale model, Llama-2-7B-chat, (ii) a less aligned model, Llama-2 base fine-tuned on Open Assistant seed data (i.e. the forward model used in our filtering step). Ablation results are shown in Figure 4.  In our small-scale experiments fine-tuning a Llama-2-7B model with 3.2K and 6.4K (instruction, rewritten response) pairs, we find that the AlpacaEval win rate is lower when using data rewritten by Llama-2-7B-chat and OA-finetuned Llama2. This suggests that effectively structuring and enriching the initial raw text response may require a sufficiently aligned model that also contains significant knowledge itself.  ![18_image_0.png](18_image_0.png) 
body
Better Alignment With Instruction Back-And-Forth Translation
D Rewriting Ablations
2408.04614
 AlpacaEval 2.0 (Dubois et al., 2024) upgrades the baseline model from text-davinci-003 to GPT-4 Turbo, and uses GPT-4 Turbo as the evaluator by default. The evaluation also debiases the raw win rate via a fitted logistic regression model, in order to control for the length of the outputs.  As noted by the Alpaca team, length-controlled (LC) win rates alleviate length biases of GPT-4, but may favor models fine-tuned on its outputs.  Our baselines of choice (described in Section 3.2) mostly distill outputs from GPT4 (in the case of Alpaca-GPT4 and Open Orca) and GPT-related models (ChatGPT for ShareGPT and Evol-Instruct, GPT-3 for Self-instruct).  We do not emphasize AlpacaEval 2.0 results in the main text, given that the win rates against GPT-4 Turbo obtained from our baselines are close together within a limited range, leading to their performance gaps being not sufficiently significant.  With the current results in Table 10, we find that similar to the observations in Table 2: (i) rewriting is more effective at improving data quality than filtering (looking at the first two rows), and (ii) using an aligned LLM for response rewriting yields better win rate than using the same model for distillation (looking at the bottom two rows). However, with this evaluation, rewriting all candidate instruction-response pairs without filtering (Dolma + rewriting) yields the best performance, when all baselines are fine-tuned with same data quantity (51.2K). 
body
Better Alignment With Instruction Back-And-Forth Translation
E Alpacaeval 2.0 Results
2408.04614
 We also evaluate the models described in Section 4 on some common NLP benchmarks: - HellaSwag (Zellers et al., 2019) : consists of 70K multiple-choice questions designed to test grounded commonsense inference. Each question comes from either activitynet or wikihow, along with four answer choices about what might happen next in the scene. The correct answer is the actual sentence for the next event, while the other three are adversarially generated and human verified.  - ARC (Clark et al., 2018) : aims to test advanced question-answering capabilities with 7787 gradeschool level, multiple-choice science questions.  The dataset consists of Challenge and Easy Sets, with the former containing only questions answered incorrectly by both a retrieval-based algorithm and a word co-occurrence algorithm.  - PIQA (Bisk et al., 2020) : another multiple-choice dataset, created to test an NLP model's understanding of the physics model of the world. Questions are inspired by how-to instructions and the model is supposed to pick the correct answer out of two choices.  - MMLU (Hendrycks et al., 2020) : covers 57 subjects across STEM, the humanities, the social sciences, etc. The question difficulty ranges from an elementary level to an advanced professional level, testing for both world knowledge and problem solving ability. Questions are presented in a  | Data source                                                                                                 | Data size    | GPT-4-Turbo       | GPT-4-Turbo   | Claude-3-Opus   | Claude-3-Opus   | |-------------------------------------------------------------------------------------------------------------|--------------|-------------------|---------------|-----------------|-----------------| | Win rate - LC (%)                                                                                            | Win rate (%)  | Win rate - LC (%)  | Win rate (%)   |                 |                 | | Dolma + filtering                                                                                           | 51.2K        | 16.71             | 11.90         | 17.66           | 9.63            | | Dolma + rewriting                                                                                           | 51.2K        | 17.44             | 12.66         | 21.54           | 15.34           | | Dolma + filtering + rewriting                                                                               | 51.2K        | 16.68             | 12.07         | 19.09           | 12.86           | | ClueWeb + filtering                                                                                         | 41.8K        | 15.93             | 11.89         | 18.39           | 10.26           | | Open Orca                                                                                                   | 51.2K        | 16.96             | 11.82         | 16.35           | 9.63            | | ShareGPT                                                                                                    | 51.2K        | 18.42             | 11.62         | 19.14           | 9.81            | | Evol-Instruct                                                                                               | 51.2K        | 15.65             | 9.51          | 17.89           | 8.94            | | Alpaca-GPT4                                                                                                 | 51.2K        | 17.19             | 11.48         | 17.04           | 8.63            | | Self-instruct                                                                                               | 51.2K        | 12.20             | 7.09          | 13.24           | 6.58            | | Dolma + filtering + rewriting                                                                               | 25.6K        | 17.90             | 13.05         | 17.55           | 11.86           | | Dolma + filtering + distilling                                                                              | 25.6K        | 17.80             | 11.95         | 17.50           | 10.50           | | Table 10: Win rates against GPT-4-Turbo of Llama-2-70B models fine-tuned on different instruction datasets, |              |                   |               |                 |                 |  | Data source                                                            | Data size   | HellaSwag   | ARC   | PIQA   | MMLU   | |------------------------------------------------------------------------|-------------|-------------|-------|--------|--------| | 7B scale                                                               |             |             |       |        |        | | Dolma + filtering                                                      | 12.8K       | 60.6        | 59.1  | 76.1   | 35.6   | | Dolma + rewriting                                                      | 12.8K       | 56.4        | 56.8  | 74.6   | 33.8   | | Dolma + filtering + rewriting                                          | 12.8K       | 56.8        | 56.7  | 74.5   | 36.6   | | ClueWeb + filtering                                                    | 12.8K       | 60.3        | 57.5  | 75.7   | 35.1   | | Open Orca                                                              | 12.8K       | 57.3        | 59.1  | 76.0   | 45.7   | | ShareGPT                                                               | 12.8K       | 56.3        | 59.2  | 75.1   | 40.4   | | Evol-Instruct                                                          | 12.8K       | 54.2        | 56.1  | 75.3   | 35.9   | | Alpaca-GPT4                                                            | 12.8K       | 57.3        | 57.7  | 74.7   | 35.0   | | Self-instruct                                                          | 12.8K       | 56.7        | 55.7  | 74.9   | 35.2   | | 70B scale                                                              |             |             |       |        |        | | Dolma + filtering                                                      | 51.2K       | 68.1        | 69.4  | 81.8   | 62.8   | | Dolma + rewriting                                                      | 51.2K       | 66.4        | 69.3  | 81.2   | 61.9   | | Dolma + filtering + rewriting                                          | 51.2K       | 66.8        | 69.1  | 81.4   | 61.4   | | ClueWeb + filtering                                                    | 41.8K       | 68.8        | 68.4  | 81.6   | 63.3   | | Open Orca                                                              | 51.2K       | 65.8        | 66.6  | 78.6   | 61.8   | | ShareGPT                                                               | 51.2K       | 65.7        | 68.4  | 79.7   | 60.0   | | Evol-Instruct                                                          | 51.2K       | 66.8        | 68.5  | 80.4   | 60.9   | | Dolma + filtering + rewriting                                          | 25.6K       | 66.8        | 69.7  | 82.2   | 62.7   | | Dolma + filtering + distilling                                         | 25.6K       | 65.7        | 70.9  | 82.0   | 60.8   | | Table 11: Performance of our fine-tuned models on different NLP tasks. |             |             |       |        |        |  multiple-choice format.  We report the results on these tasks in Table 1.
body
Better Alignment With Instruction Back-And-Forth Translation
F Other Nlp Evaluations