<!DOCTYPE html><html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2406.16797] Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs</title><meta property="og:description" content="Existing methods for adapting large language models (LLMs) to new tasks are not suited to multi-task adaptation because they modify all the model weights–causing destructive interference between tasks. The resulting ef…">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2406.16797">

<!--Generated on Fri Jul  5 22:24:41 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document ltx_authors_1line">
<h1 class="ltx_title ltx_title_document">Lottery Ticket Adaptation: 
<br class="ltx_break">Mitigating Destructive Interference in LLMs</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname"><span id="id1.1.1" class="ltx_text ltx_font_bold">Ashwinee Panda<sup id="id1.1.1.1" class="ltx_sup"><span id="id1.1.1.1.1" class="ltx_text ltx_font_medium ltx_font_italic">p</span></sup></span>  <span id="id2.2.2" class="ltx_text ltx_font_bold">Berivan Isik<sup id="id2.2.2.1" class="ltx_sup"><span id="id2.2.2.1.1" class="ltx_text ltx_font_medium ltx_font_italic">s</span></sup></span>  <span id="id3.3.3" class="ltx_text ltx_font_bold">Xiangyu Qi<sup id="id3.3.3.1" class="ltx_sup"><span id="id3.3.3.1.1" class="ltx_text ltx_font_medium ltx_font_italic">p</span></sup></span> 
<br class="ltx_break"><span id="id4.4.4" class="ltx_text ltx_font_bold">Sanmi Koyejo<sup id="id4.4.4.1" class="ltx_sup"><span id="id4.4.4.1.1" class="ltx_text ltx_font_medium ltx_font_italic">s</span></sup></span>  <span id="id5.5.5" class="ltx_text ltx_font_bold">Tsachy Weissman<sup id="id5.5.5.1" class="ltx_sup"><span id="id5.5.5.1.1" class="ltx_text ltx_font_medium ltx_font_italic">s</span></sup></span>  <span id="id6.6.6" class="ltx_text ltx_font_bold">Prateek Mittal<sup id="id6.6.6.1" class="ltx_sup"><span id="id6.6.6.1.1" class="ltx_text ltx_font_medium ltx_font_italic">p</span></sup></span> 
<br class="ltx_break"><sup id="id9.9.id1" class="ltx_sup"><span id="id9.9.id1.1" class="ltx_text ltx_font_italic">p</span></sup>Princeton University, <sup id="id10.10.id2" class="ltx_sup"><span id="id10.10.id2.1" class="ltx_text ltx_font_italic">s</span></sup>Stanford University
</span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id11.id1" class="ltx_p">Existing methods for adapting large language models (LLMs) to new tasks are not suited to multi-task adaptation because they modify all the model weights–causing destructive interference between tasks. The resulting effects, such as catastrophic forgetting of earlier tasks, make it challenging to obtain good performance on multiple tasks at the same time.
To mitigate this, we propose Lottery Ticket Adaptation (LoTA), a sparse adaptation method that identifies and optimizes only a sparse subnetwork of the model. We evaluate LoTA on a wide range of challenging tasks such as instruction following, reasoning, math, and summarization. LoTA obtains better performance than full fine-tuning and low-rank adaptation (LoRA), and maintains good performance even after training on other tasks – thus, avoiding catastrophic forgetting. By extracting and fine-tuning over <em id="id11.id1.1" class="ltx_emph ltx_font_italic">lottery tickets</em> (or <em id="id11.id1.2" class="ltx_emph ltx_font_italic">sparse task vectors</em>), LoTA also enables model merging over highly dissimilar tasks. Our code is made <a target="_blank" href="https://github.com/kiddyboots216/lottery-ticket-adaptation" title="" class="ltx_ref ltx_href">publicly available</a>. <span id="footnote1" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_tag ltx_tag_note">1</span>A preliminary version of this work has appeared in <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib1" title="" class="ltx_ref">1</a>]</cite>.</span></span></span></p>
</div>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Introduction</h2>

<div id="S1.p1" class="ltx_para">
<p id="S1.p1.1" class="ltx_p">Large language models (LLMs) <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib2" title="" class="ltx_ref">2</a>]</cite> have seen an explosion of applications to real-world problems <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib3" title="" class="ltx_ref">3</a>, <a href="#bib.bib4" title="" class="ltx_ref">4</a>]</cite> via adaptation <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib5" title="" class="ltx_ref">5</a>]</cite> to new tasks. Three major <em id="S1.p1.1.1" class="ltx_emph ltx_font_italic">multi-task adaptation</em> paradigms have emerged: storing and loading task-specific <em id="S1.p1.1.2" class="ltx_emph ltx_font_italic">adapters</em> <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib6" title="" class="ltx_ref">6</a>, <a href="#bib.bib7" title="" class="ltx_ref">7</a>]</cite>, continuing to train instruction-tuned models on new tasks in serial via <em id="S1.p1.1.3" class="ltx_emph ltx_font_italic">sequential training</em> <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib5" title="" class="ltx_ref">5</a>]</cite>, and combining the adaptations to tasks learned in parallel via <em id="S1.p1.1.4" class="ltx_emph ltx_font_italic">model merging</em> <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib8" title="" class="ltx_ref">8</a>]</cite>.
Each paradigm has its own associated challenges, such as catastrophic forgetting during sequential training <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib9" title="" class="ltx_ref">9</a>, <a href="#bib.bib10" title="" class="ltx_ref">10</a>, <a href="#bib.bib11" title="" class="ltx_ref">11</a>, <a href="#bib.bib12" title="" class="ltx_ref">12</a>, <a href="#bib.bib13" title="" class="ltx_ref">13</a>]</cite>, and methods that have been proposed to mitigate these challenges <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib14" title="" class="ltx_ref">14</a>, <a href="#bib.bib15" title="" class="ltx_ref">15</a>]</cite>. In this work, we propose a new LLM adaptation method, called <span id="S1.p1.1.5" class="ltx_text ltx_font_bold">Lottery Ticket Adaptation (LoTA)</span>, that (1) provides sparse adaptation by freezing a majority of the parameters and updating only a sparse subnetwork of the base model and (2) resolves the challenges in common <em id="S1.p1.1.6" class="ltx_emph ltx_font_italic">multi-task adaptation</em> paradigms. (More details in Section <a href="#S3" title="3 Lottery Ticket Adaptation (LoTA) ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>.) We summarize our contributions:</p>
<ul id="S1.I1" class="ltx_itemize">
<li id="S1.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i1.p1" class="ltx_para">
<p id="S1.I1.i1.p1.2" class="ltx_p">We train <em id="S1.I1.i1.p1.2.1" class="ltx_emph ltx_font_italic">lottery tickets</em> (or <em id="S1.I1.i1.p1.2.2" class="ltx_emph ltx_font_italic">sparse task vectors</em>) that can be stored efficiently and obtain performance similar to full fine-tuning (FFT) and higher than LoRA across a range of tasks spanning reasoning, math, code generation, and instruction following. When adapting Mistral for instruction following, FFT and LoTA both get a length-controlled AlpacaEval 2 winrate <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib16" title="" class="ltx_ref">16</a>]</cite>(how often GPT-4 prefers the outputs of our model over its own) of <math id="S1.I1.i1.p1.1.m1.1" class="ltx_Math" alttext="19.0\%" display="inline"><semantics id="S1.I1.i1.p1.1.m1.1a"><mrow id="S1.I1.i1.p1.1.m1.1.1" xref="S1.I1.i1.p1.1.m1.1.1.cmml"><mn id="S1.I1.i1.p1.1.m1.1.1.2" xref="S1.I1.i1.p1.1.m1.1.1.2.cmml">19.0</mn><mo id="S1.I1.i1.p1.1.m1.1.1.1" xref="S1.I1.i1.p1.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S1.I1.i1.p1.1.m1.1b"><apply id="S1.I1.i1.p1.1.m1.1.1.cmml" xref="S1.I1.i1.p1.1.m1.1.1"><csymbol cd="latexml" id="S1.I1.i1.p1.1.m1.1.1.1.cmml" xref="S1.I1.i1.p1.1.m1.1.1.1">percent</csymbol><cn type="float" id="S1.I1.i1.p1.1.m1.1.1.2.cmml" xref="S1.I1.i1.p1.1.m1.1.1.2">19.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.I1.i1.p1.1.m1.1c">19.0\%</annotation></semantics></math>, but LoRA only gets a winrate of <math id="S1.I1.i1.p1.2.m2.1" class="ltx_Math" alttext="15.3\%" display="inline"><semantics id="S1.I1.i1.p1.2.m2.1a"><mrow id="S1.I1.i1.p1.2.m2.1.1" xref="S1.I1.i1.p1.2.m2.1.1.cmml"><mn id="S1.I1.i1.p1.2.m2.1.1.2" xref="S1.I1.i1.p1.2.m2.1.1.2.cmml">15.3</mn><mo id="S1.I1.i1.p1.2.m2.1.1.1" xref="S1.I1.i1.p1.2.m2.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S1.I1.i1.p1.2.m2.1b"><apply id="S1.I1.i1.p1.2.m2.1.1.cmml" xref="S1.I1.i1.p1.2.m2.1.1"><csymbol cd="latexml" id="S1.I1.i1.p1.2.m2.1.1.1.cmml" xref="S1.I1.i1.p1.2.m2.1.1.1">percent</csymbol><cn type="float" id="S1.I1.i1.p1.2.m2.1.1.2.cmml" xref="S1.I1.i1.p1.2.m2.1.1.2">15.3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.I1.i1.p1.2.m2.1c">15.3\%</annotation></semantics></math>.</p>
</div>
</li>
<li id="S1.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i2.p1" class="ltx_para">
<p id="S1.I1.i2.p1.3" class="ltx_p">We apply LoTA to mitigate <span id="S1.I1.i2.p1.3.1" class="ltx_text ltx_font_bold">catastrophic forgetting</span> <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib9" title="" class="ltx_ref">9</a>]</cite> of earlier tasks, enabling sequential adaptation to new tasks. When adapting an instruction tuned model to a mix of new tasks, the winrate of the FFT model drops from <math id="S1.I1.i2.p1.1.m1.1" class="ltx_Math" alttext="19.0\%" display="inline"><semantics id="S1.I1.i2.p1.1.m1.1a"><mrow id="S1.I1.i2.p1.1.m1.1.1" xref="S1.I1.i2.p1.1.m1.1.1.cmml"><mn id="S1.I1.i2.p1.1.m1.1.1.2" xref="S1.I1.i2.p1.1.m1.1.1.2.cmml">19.0</mn><mo id="S1.I1.i2.p1.1.m1.1.1.1" xref="S1.I1.i2.p1.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S1.I1.i2.p1.1.m1.1b"><apply id="S1.I1.i2.p1.1.m1.1.1.cmml" xref="S1.I1.i2.p1.1.m1.1.1"><csymbol cd="latexml" id="S1.I1.i2.p1.1.m1.1.1.1.cmml" xref="S1.I1.i2.p1.1.m1.1.1.1">percent</csymbol><cn type="float" id="S1.I1.i2.p1.1.m1.1.1.2.cmml" xref="S1.I1.i2.p1.1.m1.1.1.2">19.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.I1.i2.p1.1.m1.1c">19.0\%</annotation></semantics></math> to <math id="S1.I1.i2.p1.2.m2.1" class="ltx_Math" alttext="0.5\%" display="inline"><semantics id="S1.I1.i2.p1.2.m2.1a"><mrow id="S1.I1.i2.p1.2.m2.1.1" xref="S1.I1.i2.p1.2.m2.1.1.cmml"><mn id="S1.I1.i2.p1.2.m2.1.1.2" xref="S1.I1.i2.p1.2.m2.1.1.2.cmml">0.5</mn><mo id="S1.I1.i2.p1.2.m2.1.1.1" xref="S1.I1.i2.p1.2.m2.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S1.I1.i2.p1.2.m2.1b"><apply id="S1.I1.i2.p1.2.m2.1.1.cmml" xref="S1.I1.i2.p1.2.m2.1.1"><csymbol cd="latexml" id="S1.I1.i2.p1.2.m2.1.1.1.cmml" xref="S1.I1.i2.p1.2.m2.1.1.1">percent</csymbol><cn type="float" id="S1.I1.i2.p1.2.m2.1.1.2.cmml" xref="S1.I1.i2.p1.2.m2.1.1.2">0.5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.I1.i2.p1.2.m2.1c">0.5\%</annotation></semantics></math>, but by using LoTA we can limit the drop to <math id="S1.I1.i2.p1.3.m3.1" class="ltx_Math" alttext="15.9\%" display="inline"><semantics id="S1.I1.i2.p1.3.m3.1a"><mrow id="S1.I1.i2.p1.3.m3.1.1" xref="S1.I1.i2.p1.3.m3.1.1.cmml"><mn id="S1.I1.i2.p1.3.m3.1.1.2" xref="S1.I1.i2.p1.3.m3.1.1.2.cmml">15.9</mn><mo id="S1.I1.i2.p1.3.m3.1.1.1" xref="S1.I1.i2.p1.3.m3.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S1.I1.i2.p1.3.m3.1b"><apply id="S1.I1.i2.p1.3.m3.1.1.cmml" xref="S1.I1.i2.p1.3.m3.1.1"><csymbol cd="latexml" id="S1.I1.i2.p1.3.m3.1.1.1.cmml" xref="S1.I1.i2.p1.3.m3.1.1.1">percent</csymbol><cn type="float" id="S1.I1.i2.p1.3.m3.1.1.2.cmml" xref="S1.I1.i2.p1.3.m3.1.1.2">15.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.I1.i2.p1.3.m3.1c">15.9\%</annotation></semantics></math>.</p>
</div>
</li>
<li id="S1.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i3.p1" class="ltx_para">
<p id="S1.I1.i3.p1.2" class="ltx_p">We can use LoTA to <span id="S1.I1.i3.p1.2.1" class="ltx_text ltx_font_bold">merge models in parallel</span> <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib17" title="" class="ltx_ref">17</a>, <a href="#bib.bib18" title="" class="ltx_ref">18</a>, <a href="#bib.bib19" title="" class="ltx_ref">19</a>]</cite> across dramatically different tasks, achieving better performance than existing merging methods that rely on post hoc sparsification <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib20" title="" class="ltx_ref">20</a>]</cite> (which degrades performance for FFT models) because it naturally trains <em id="S1.I1.i3.p1.2.2" class="ltx_emph ltx_font_italic">sparse task vectors</em>. When merging instruction following and math models with LoTA, we get a task-average performance of <math id="S1.I1.i3.p1.1.m1.1" class="ltx_Math" alttext="38.5\%" display="inline"><semantics id="S1.I1.i3.p1.1.m1.1a"><mrow id="S1.I1.i3.p1.1.m1.1.1" xref="S1.I1.i3.p1.1.m1.1.1.cmml"><mn id="S1.I1.i3.p1.1.m1.1.1.2" xref="S1.I1.i3.p1.1.m1.1.1.2.cmml">38.5</mn><mo id="S1.I1.i3.p1.1.m1.1.1.1" xref="S1.I1.i3.p1.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S1.I1.i3.p1.1.m1.1b"><apply id="S1.I1.i3.p1.1.m1.1.1.cmml" xref="S1.I1.i3.p1.1.m1.1.1"><csymbol cd="latexml" id="S1.I1.i3.p1.1.m1.1.1.1.cmml" xref="S1.I1.i3.p1.1.m1.1.1.1">percent</csymbol><cn type="float" id="S1.I1.i3.p1.1.m1.1.1.2.cmml" xref="S1.I1.i3.p1.1.m1.1.1.2">38.5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.I1.i3.p1.1.m1.1c">38.5\%</annotation></semantics></math> where a merge of FFT models obtains <math id="S1.I1.i3.p1.2.m2.1" class="ltx_Math" alttext="36.7\%" display="inline"><semantics id="S1.I1.i3.p1.2.m2.1a"><mrow id="S1.I1.i3.p1.2.m2.1.1" xref="S1.I1.i3.p1.2.m2.1.1.cmml"><mn id="S1.I1.i3.p1.2.m2.1.1.2" xref="S1.I1.i3.p1.2.m2.1.1.2.cmml">36.7</mn><mo id="S1.I1.i3.p1.2.m2.1.1.1" xref="S1.I1.i3.p1.2.m2.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S1.I1.i3.p1.2.m2.1b"><apply id="S1.I1.i3.p1.2.m2.1.1.cmml" xref="S1.I1.i3.p1.2.m2.1.1"><csymbol cd="latexml" id="S1.I1.i3.p1.2.m2.1.1.1.cmml" xref="S1.I1.i3.p1.2.m2.1.1.1">percent</csymbol><cn type="float" id="S1.I1.i3.p1.2.m2.1.1.2.cmml" xref="S1.I1.i3.p1.2.m2.1.1.2">36.7</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.I1.i3.p1.2.m2.1c">36.7\%</annotation></semantics></math>.</p>
</div>
</li>
</ul>
</div>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Background: Multi-Task Adaptation</h2>

<div id="S2.p1" class="ltx_para">
<p id="S2.p1.1" class="ltx_p">In this section, we go over common multi-task adaptation paradigms and discuss the challenges existing fine-tuning methods, such as FFT and LoRA, bring in each paradigm.</p>
</div>
<figure id="S2.F1" class="ltx_figure"><img src="/html/2406.16797/assets/x1.png" id="S2.F1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="181" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure"><span id="S2.F1.2.1.1" class="ltx_text" style="font-size:90%;">Figure 1</span>: </span><span id="S2.F1.3.2" class="ltx_text" style="font-size:90%;">Multi-task adaptation: storing and loading adapters, sequential training, model merging.
</span></figcaption>
</figure>
<section id="S2.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Storing and Loading Adapters.</h4>

<div id="S2.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="S2.SS0.SSS0.Px1.p1.1" class="ltx_p">As illustrated in the first row of Figure <a href="#S2.F1" title="Figure 1 ‣ 2 Background: Multi-Task Adaptation ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>, an emerging paradigm for multi-task adaptation is to store an adapter for each desired task and load a particular adapter at inference time <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib21" title="" class="ltx_ref">21</a>, <a href="#bib.bib7" title="" class="ltx_ref">7</a>, <a href="#bib.bib22" title="" class="ltx_ref">22</a>]</cite>, depending on the need <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib23" title="" class="ltx_ref">23</a>]</cite>. Notably, Apple stores and loads adapters to enable their on-device models <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib24" title="" class="ltx_ref">24</a>]</cite>.
This approach, while avoiding any interference between tasks, increases the memory and compute cost as it requires storing and loading an additional adapter per task. To mitigate these costs, a number of PEFT methods have been developed <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib25" title="" class="ltx_ref">25</a>, <a href="#bib.bib26" title="" class="ltx_ref">26</a>, <a href="#bib.bib27" title="" class="ltx_ref">27</a>, <a href="#bib.bib28" title="" class="ltx_ref">28</a>]</cite>.
Among them, LoRA <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib6" title="" class="ltx_ref">6</a>]</cite> (training adapters in the low-rank space) has received notable attention due to its simplicity. For instance, services such as Punica and S-LoRA allow developers to use this approach to serve large numbers of task-specific adapters for specific requests <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib29" title="" class="ltx_ref">29</a>, <a href="#bib.bib30" title="" class="ltx_ref">30</a>]</cite>.
However, a persistent gap in capacity between PEFT methods and FFT has presented a tradeoff between adapter overhead and performance <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib6" title="" class="ltx_ref">6</a>, <a href="#bib.bib31" title="" class="ltx_ref">31</a>, <a href="#bib.bib32" title="" class="ltx_ref">32</a>, <a href="#bib.bib33" title="" class="ltx_ref">33</a>, <a href="#bib.bib34" title="" class="ltx_ref">34</a>]</cite>.</p>
</div>
</section>
<section id="S2.SS0.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Sequential Training.</h4>

<div id="S2.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="S2.SS0.SSS0.Px2.p1.1" class="ltx_p">When it is desired to have a single model with multi-task abilities (as opposed to storing and loading adapters per task), one common approach is to fine-tune the model on different tasks sequentially <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib35" title="" class="ltx_ref">35</a>]</cite>, e.g., first fine-tune on task A, then fine-tune on task B. This is summarized in the second row of Figure <a href="#S2.F1" title="Figure 1 ‣ 2 Background: Multi-Task Adaptation ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>. Note that sequential training is distinct from continual pre-training, because sequential training uses the instruction tuning objective whereas continual pre-training typically uses the pre-training objective of next-word-prediction <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib36" title="" class="ltx_ref">36</a>]</cite>.</p>
</div>
<div id="S2.SS0.SSS0.Px2.p2" class="ltx_para">
<p id="S2.SS0.SSS0.Px2.p2.1" class="ltx_p">Fine-tuning the LLM for new tasks with FFT or existing PEFT methods leads to catastrophic forgetting of earlier tasks. This is problematic, especially for <em id="S2.SS0.SSS0.Px2.p2.1.1" class="ltx_emph ltx_font_italic">safety</em> alignment, since we can fine-tune an LLM to be safe but later get this feature erased during future fine-tuning on new tasks <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib37" title="" class="ltx_ref">37</a>]</cite>. In fact, a number of works have aimed to mitigate this vulnerability <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib9" title="" class="ltx_ref">9</a>, <a href="#bib.bib10" title="" class="ltx_ref">10</a>, <a href="#bib.bib11" title="" class="ltx_ref">11</a>, <a href="#bib.bib12" title="" class="ltx_ref">12</a>, <a href="#bib.bib13" title="" class="ltx_ref">13</a>]</cite>, leaving an open research question: <em id="S2.SS0.SSS0.Px2.p2.1.2" class="ltx_emph ltx_font_italic">Can model developers allow users to finetune their aligned models on custom datasets while retaining safety?</em></p>
</div>
</section>
<section id="S2.SS0.SSS0.Px3" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Model Merging.</h4>

<div id="S2.SS0.SSS0.Px3.p1" class="ltx_para">
<p id="S2.SS0.SSS0.Px3.p1.1" class="ltx_p">There has been a recent interest in model arithmetic and editing methods, including merging multiple models adapted to different individual tasks to have a single model adapted to multiple tasks simultaneously <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib8" title="" class="ltx_ref">8</a>]</cite>. As the third row of Figure <a href="#S2.F1" title="Figure 1 ‣ 2 Background: Multi-Task Adaptation ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> shows, this is typically done via aggregating <em id="S2.SS0.SSS0.Px3.p1.1.1" class="ltx_emph ltx_font_italic">task vectors</em> (or adapters) of different tasks. The existing model merging techniques either require post-processing the task vectors through sparsification <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib38" title="" class="ltx_ref">38</a>, <a href="#bib.bib39" title="" class="ltx_ref">39</a>, <a href="#bib.bib20" title="" class="ltx_ref">20</a>]</cite>, degrading the performance on the task, and/or require extensive hyperparameter tuning for a weighted aggregation of task vectors <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib40" title="" class="ltx_ref">40</a>, <a href="#bib.bib41" title="" class="ltx_ref">41</a>]</cite>.</p>
</div>
<div id="S2.SS0.SSS0.Px3.p2" class="ltx_para">
<p id="S2.SS0.SSS0.Px3.p2.1" class="ltx_p">Each paradigm of multi-task learning poses different challenges, and different methods have been proposed to address these challenges. We defer the more in-depth analysis of these proposed methods in <a href="#S6" title="6 Related Work ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">6</span></a> because of the sheer quantity of related work that must be covered. The number of methods itself poses challenges for studying their drawbacks, especially when these methods are adopted in settings orthogonal to those they were originally developed for (i.e., LoRA leading to catastrophic forgetting of safety alignment <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib37" title="" class="ltx_ref">37</a>]</cite>). <em id="S2.SS0.SSS0.Px3.p2.1.1" class="ltx_emph ltx_font_italic">Rather than proposing tailored solutions for each paradigm of multi-task adaptation, we want to propose a simple algorithm that can serve as an effective foundation across all three paradigms and evaluate it on a wide range of challenging tasks.</em></p>
</div>
</section>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>Lottery Ticket Adaptation (LoTA)</h2>

<figure id="S3.fig1" class="ltx_figure">
<figure id="alg1" class="ltx_float ltx_minipage ltx_align_middle ltx_framed ltx_framed_top" style="width:232.0pt;">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_float"><span id="alg1.2.1.1" class="ltx_text ltx_font_bold">Algorithm 1</span> </span> Lottery Ticket Adaptation (LoTA)</figcaption>
<div id="alg1.3" class="ltx_listing ltx_listing">
<div id="alg1.l0" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l0.1.1.1" class="ltx_text" style="font-size:80%;">0:</span></span>  Adaptation algorithm <math id="alg1.l0.m1.1" class="ltx_Math" alttext="\mathbb{A}" display="inline"><semantics id="alg1.l0.m1.1a"><mi id="alg1.l0.m1.1.1" xref="alg1.l0.m1.1.1.cmml">𝔸</mi><annotation-xml encoding="MathML-Content" id="alg1.l0.m1.1b"><ci id="alg1.l0.m1.1.1.cmml" xref="alg1.l0.m1.1.1">𝔸</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l0.m1.1c">\mathbb{A}</annotation></semantics></math>, alignment dataset <math id="alg1.l0.m2.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="alg1.l0.m2.1a"><mi id="alg1.l0.m2.1.1" xref="alg1.l0.m2.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="alg1.l0.m2.1b"><ci id="alg1.l0.m2.1.1.cmml" xref="alg1.l0.m2.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l0.m2.1c">\mathbb{D}</annotation></semantics></math>, pre-trained weights <math id="alg1.l0.m3.1" class="ltx_Math" alttext="w_{P}" display="inline"><semantics id="alg1.l0.m3.1a"><msub id="alg1.l0.m3.1.1" xref="alg1.l0.m3.1.1.cmml"><mi id="alg1.l0.m3.1.1.2" xref="alg1.l0.m3.1.1.2.cmml">w</mi><mi id="alg1.l0.m3.1.1.3" xref="alg1.l0.m3.1.1.3.cmml">P</mi></msub><annotation-xml encoding="MathML-Content" id="alg1.l0.m3.1b"><apply id="alg1.l0.m3.1.1.cmml" xref="alg1.l0.m3.1.1"><csymbol cd="ambiguous" id="alg1.l0.m3.1.1.1.cmml" xref="alg1.l0.m3.1.1">subscript</csymbol><ci id="alg1.l0.m3.1.1.2.cmml" xref="alg1.l0.m3.1.1.2">𝑤</ci><ci id="alg1.l0.m3.1.1.3.cmml" xref="alg1.l0.m3.1.1.3">𝑃</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l0.m3.1c">w_{P}</annotation></semantics></math>, sparsity ratio <math id="alg1.l0.m4.1" class="ltx_Math" alttext="s" display="inline"><semantics id="alg1.l0.m4.1a"><mi id="alg1.l0.m4.1.1" xref="alg1.l0.m4.1.1.cmml">s</mi><annotation-xml encoding="MathML-Content" id="alg1.l0.m4.1b"><ci id="alg1.l0.m4.1.1.cmml" xref="alg1.l0.m4.1.1">𝑠</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l0.m4.1c">s</annotation></semantics></math>, learning rate <math id="alg1.l0.m5.1" class="ltx_Math" alttext="\eta" display="inline"><semantics id="alg1.l0.m5.1a"><mi id="alg1.l0.m5.1.1" xref="alg1.l0.m5.1.1.cmml">η</mi><annotation-xml encoding="MathML-Content" id="alg1.l0.m5.1b"><ci id="alg1.l0.m5.1.1.cmml" xref="alg1.l0.m5.1.1">𝜂</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l0.m5.1c">\eta</annotation></semantics></math>, number of calibration iterations <math id="alg1.l0.m6.1" class="ltx_Math" alttext="\mathcal{T}" display="inline"><semantics id="alg1.l0.m6.1a"><mi class="ltx_font_mathcaligraphic" id="alg1.l0.m6.1.1" xref="alg1.l0.m6.1.1.cmml">𝒯</mi><annotation-xml encoding="MathML-Content" id="alg1.l0.m6.1b"><ci id="alg1.l0.m6.1.1.cmml" xref="alg1.l0.m6.1.1">𝒯</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l0.m6.1c">\mathcal{T}</annotation></semantics></math>, number of sparse training iterations <math id="alg1.l0.m7.1" class="ltx_Math" alttext="T" display="inline"><semantics id="alg1.l0.m7.1a"><mi id="alg1.l0.m7.1.1" xref="alg1.l0.m7.1.1.cmml">T</mi><annotation-xml encoding="MathML-Content" id="alg1.l0.m7.1b"><ci id="alg1.l0.m7.1.1.cmml" xref="alg1.l0.m7.1.1">𝑇</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l0.m7.1c">T</annotation></semantics></math>.

</div>
<div id="alg1.l1" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l1.1.1.1" class="ltx_text" style="font-size:80%;">1:</span></span>  <span id="alg1.l1.2" class="ltx_text ltx_font_bold">Mask Calibration:</span>

</div>
<div id="alg1.l2" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l2.1.1.1" class="ltx_text" style="font-size:80%;">2:</span></span>  <math id="alg1.l2.m1.1" class="ltx_Math" alttext="w_{F}\leftarrow w_{P}" display="inline"><semantics id="alg1.l2.m1.1a"><mrow id="alg1.l2.m1.1.1" xref="alg1.l2.m1.1.1.cmml"><msub id="alg1.l2.m1.1.1.2" xref="alg1.l2.m1.1.1.2.cmml"><mi id="alg1.l2.m1.1.1.2.2" xref="alg1.l2.m1.1.1.2.2.cmml">w</mi><mi id="alg1.l2.m1.1.1.2.3" xref="alg1.l2.m1.1.1.2.3.cmml">F</mi></msub><mo stretchy="false" id="alg1.l2.m1.1.1.1" xref="alg1.l2.m1.1.1.1.cmml">←</mo><msub id="alg1.l2.m1.1.1.3" xref="alg1.l2.m1.1.1.3.cmml"><mi id="alg1.l2.m1.1.1.3.2" xref="alg1.l2.m1.1.1.3.2.cmml">w</mi><mi id="alg1.l2.m1.1.1.3.3" xref="alg1.l2.m1.1.1.3.3.cmml">P</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="alg1.l2.m1.1b"><apply id="alg1.l2.m1.1.1.cmml" xref="alg1.l2.m1.1.1"><ci id="alg1.l2.m1.1.1.1.cmml" xref="alg1.l2.m1.1.1.1">←</ci><apply id="alg1.l2.m1.1.1.2.cmml" xref="alg1.l2.m1.1.1.2"><csymbol cd="ambiguous" id="alg1.l2.m1.1.1.2.1.cmml" xref="alg1.l2.m1.1.1.2">subscript</csymbol><ci id="alg1.l2.m1.1.1.2.2.cmml" xref="alg1.l2.m1.1.1.2.2">𝑤</ci><ci id="alg1.l2.m1.1.1.2.3.cmml" xref="alg1.l2.m1.1.1.2.3">𝐹</ci></apply><apply id="alg1.l2.m1.1.1.3.cmml" xref="alg1.l2.m1.1.1.3"><csymbol cd="ambiguous" id="alg1.l2.m1.1.1.3.1.cmml" xref="alg1.l2.m1.1.1.3">subscript</csymbol><ci id="alg1.l2.m1.1.1.3.2.cmml" xref="alg1.l2.m1.1.1.3.2">𝑤</ci><ci id="alg1.l2.m1.1.1.3.3.cmml" xref="alg1.l2.m1.1.1.3.3">𝑃</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l2.m1.1c">w_{F}\leftarrow w_{P}</annotation></semantics></math>

</div>
<div id="alg1.l3" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l3.1.1.1" class="ltx_text" style="font-size:80%;">3:</span></span>  <span id="alg1.l3.2" class="ltx_text ltx_font_bold">for</span> <math id="alg1.l3.m1.3" class="ltx_Math" alttext="\tau\in{0,\dots,\mathcal{T}}" display="inline"><semantics id="alg1.l3.m1.3a"><mrow id="alg1.l3.m1.3.4" xref="alg1.l3.m1.3.4.cmml"><mi id="alg1.l3.m1.3.4.2" xref="alg1.l3.m1.3.4.2.cmml">τ</mi><mo id="alg1.l3.m1.3.4.1" xref="alg1.l3.m1.3.4.1.cmml">∈</mo><mrow id="alg1.l3.m1.3.4.3.2" xref="alg1.l3.m1.3.4.3.1.cmml"><mn id="alg1.l3.m1.1.1" xref="alg1.l3.m1.1.1.cmml">0</mn><mo id="alg1.l3.m1.3.4.3.2.1" xref="alg1.l3.m1.3.4.3.1.cmml">,</mo><mi mathvariant="normal" id="alg1.l3.m1.2.2" xref="alg1.l3.m1.2.2.cmml">…</mi><mo id="alg1.l3.m1.3.4.3.2.2" xref="alg1.l3.m1.3.4.3.1.cmml">,</mo><mi class="ltx_font_mathcaligraphic" id="alg1.l3.m1.3.3" xref="alg1.l3.m1.3.3.cmml">𝒯</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l3.m1.3b"><apply id="alg1.l3.m1.3.4.cmml" xref="alg1.l3.m1.3.4"><in id="alg1.l3.m1.3.4.1.cmml" xref="alg1.l3.m1.3.4.1"></in><ci id="alg1.l3.m1.3.4.2.cmml" xref="alg1.l3.m1.3.4.2">𝜏</ci><list id="alg1.l3.m1.3.4.3.1.cmml" xref="alg1.l3.m1.3.4.3.2"><cn type="integer" id="alg1.l3.m1.1.1.cmml" xref="alg1.l3.m1.1.1">0</cn><ci id="alg1.l3.m1.2.2.cmml" xref="alg1.l3.m1.2.2">…</ci><ci id="alg1.l3.m1.3.3.cmml" xref="alg1.l3.m1.3.3">𝒯</ci></list></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l3.m1.3c">\tau\in{0,\dots,\mathcal{T}}</annotation></semantics></math> <span id="alg1.l3.3" class="ltx_text ltx_font_bold">do</span>



</div>
<div id="alg1.l4" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l4.1.1.1" class="ltx_text" style="font-size:80%;">4:</span></span>     <math id="alg1.l4.m1.1" class="ltx_Math" alttext="\nabla=\mathbb{A}_{\mathbb{D}}(w_{F})" display="inline"><semantics id="alg1.l4.m1.1a"><mrow id="alg1.l4.m1.1.1" xref="alg1.l4.m1.1.1.cmml"><mo id="alg1.l4.m1.1.1.3" xref="alg1.l4.m1.1.1.3.cmml">∇</mo><mo id="alg1.l4.m1.1.1.2" xref="alg1.l4.m1.1.1.2.cmml">=</mo><mrow id="alg1.l4.m1.1.1.1" xref="alg1.l4.m1.1.1.1.cmml"><msub id="alg1.l4.m1.1.1.1.3" xref="alg1.l4.m1.1.1.1.3.cmml"><mi id="alg1.l4.m1.1.1.1.3.2" xref="alg1.l4.m1.1.1.1.3.2.cmml">𝔸</mi><mi id="alg1.l4.m1.1.1.1.3.3" xref="alg1.l4.m1.1.1.1.3.3.cmml">𝔻</mi></msub><mo lspace="0em" rspace="0em" id="alg1.l4.m1.1.1.1.2" xref="alg1.l4.m1.1.1.1.2.cmml">​</mo><mrow id="alg1.l4.m1.1.1.1.1.1" xref="alg1.l4.m1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="alg1.l4.m1.1.1.1.1.1.2" xref="alg1.l4.m1.1.1.1.1.1.1.cmml">(</mo><msub id="alg1.l4.m1.1.1.1.1.1.1" xref="alg1.l4.m1.1.1.1.1.1.1.cmml"><mi id="alg1.l4.m1.1.1.1.1.1.1.2" xref="alg1.l4.m1.1.1.1.1.1.1.2.cmml">w</mi><mi id="alg1.l4.m1.1.1.1.1.1.1.3" xref="alg1.l4.m1.1.1.1.1.1.1.3.cmml">F</mi></msub><mo stretchy="false" id="alg1.l4.m1.1.1.1.1.1.3" xref="alg1.l4.m1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l4.m1.1b"><apply id="alg1.l4.m1.1.1.cmml" xref="alg1.l4.m1.1.1"><eq id="alg1.l4.m1.1.1.2.cmml" xref="alg1.l4.m1.1.1.2"></eq><ci id="alg1.l4.m1.1.1.3.cmml" xref="alg1.l4.m1.1.1.3">∇</ci><apply id="alg1.l4.m1.1.1.1.cmml" xref="alg1.l4.m1.1.1.1"><times id="alg1.l4.m1.1.1.1.2.cmml" xref="alg1.l4.m1.1.1.1.2"></times><apply id="alg1.l4.m1.1.1.1.3.cmml" xref="alg1.l4.m1.1.1.1.3"><csymbol cd="ambiguous" id="alg1.l4.m1.1.1.1.3.1.cmml" xref="alg1.l4.m1.1.1.1.3">subscript</csymbol><ci id="alg1.l4.m1.1.1.1.3.2.cmml" xref="alg1.l4.m1.1.1.1.3.2">𝔸</ci><ci id="alg1.l4.m1.1.1.1.3.3.cmml" xref="alg1.l4.m1.1.1.1.3.3">𝔻</ci></apply><apply id="alg1.l4.m1.1.1.1.1.1.1.cmml" xref="alg1.l4.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg1.l4.m1.1.1.1.1.1.1.1.cmml" xref="alg1.l4.m1.1.1.1.1.1">subscript</csymbol><ci id="alg1.l4.m1.1.1.1.1.1.1.2.cmml" xref="alg1.l4.m1.1.1.1.1.1.1.2">𝑤</ci><ci id="alg1.l4.m1.1.1.1.1.1.1.3.cmml" xref="alg1.l4.m1.1.1.1.1.1.1.3">𝐹</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l4.m1.1c">\nabla=\mathbb{A}_{\mathbb{D}}(w_{F})</annotation></semantics></math> {Compute gradient for weights}

</div>
<div id="alg1.l5" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l5.1.1.1" class="ltx_text" style="font-size:80%;">5:</span></span>     <math id="alg1.l5.m1.1" class="ltx_Math" alttext="w_{F}=w_{F}-\eta\cdot\nabla" display="inline"><semantics id="alg1.l5.m1.1a"><mrow id="alg1.l5.m1.1.1" xref="alg1.l5.m1.1.1.cmml"><msub id="alg1.l5.m1.1.1.2" xref="alg1.l5.m1.1.1.2.cmml"><mi id="alg1.l5.m1.1.1.2.2" xref="alg1.l5.m1.1.1.2.2.cmml">w</mi><mi id="alg1.l5.m1.1.1.2.3" xref="alg1.l5.m1.1.1.2.3.cmml">F</mi></msub><mo id="alg1.l5.m1.1.1.1" xref="alg1.l5.m1.1.1.1.cmml">=</mo><mrow id="alg1.l5.m1.1.1.3" xref="alg1.l5.m1.1.1.3.cmml"><msub id="alg1.l5.m1.1.1.3.2" xref="alg1.l5.m1.1.1.3.2.cmml"><mi id="alg1.l5.m1.1.1.3.2.2" xref="alg1.l5.m1.1.1.3.2.2.cmml">w</mi><mi id="alg1.l5.m1.1.1.3.2.3" xref="alg1.l5.m1.1.1.3.2.3.cmml">F</mi></msub><mo id="alg1.l5.m1.1.1.3.1" xref="alg1.l5.m1.1.1.3.1.cmml">−</mo><mrow id="alg1.l5.m1.1.1.3.3" xref="alg1.l5.m1.1.1.3.3.cmml"><mi id="alg1.l5.m1.1.1.3.3.2" xref="alg1.l5.m1.1.1.3.3.2.cmml">η</mi><mo lspace="0.222em" rspace="0.222em" id="alg1.l5.m1.1.1.3.3.1" xref="alg1.l5.m1.1.1.3.3.1.cmml">⋅</mo><mo id="alg1.l5.m1.1.1.3.3.3" xref="alg1.l5.m1.1.1.3.3.3.cmml">∇</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l5.m1.1b"><apply id="alg1.l5.m1.1.1.cmml" xref="alg1.l5.m1.1.1"><eq id="alg1.l5.m1.1.1.1.cmml" xref="alg1.l5.m1.1.1.1"></eq><apply id="alg1.l5.m1.1.1.2.cmml" xref="alg1.l5.m1.1.1.2"><csymbol cd="ambiguous" id="alg1.l5.m1.1.1.2.1.cmml" xref="alg1.l5.m1.1.1.2">subscript</csymbol><ci id="alg1.l5.m1.1.1.2.2.cmml" xref="alg1.l5.m1.1.1.2.2">𝑤</ci><ci id="alg1.l5.m1.1.1.2.3.cmml" xref="alg1.l5.m1.1.1.2.3">𝐹</ci></apply><apply id="alg1.l5.m1.1.1.3.cmml" xref="alg1.l5.m1.1.1.3"><minus id="alg1.l5.m1.1.1.3.1.cmml" xref="alg1.l5.m1.1.1.3.1"></minus><apply id="alg1.l5.m1.1.1.3.2.cmml" xref="alg1.l5.m1.1.1.3.2"><csymbol cd="ambiguous" id="alg1.l5.m1.1.1.3.2.1.cmml" xref="alg1.l5.m1.1.1.3.2">subscript</csymbol><ci id="alg1.l5.m1.1.1.3.2.2.cmml" xref="alg1.l5.m1.1.1.3.2.2">𝑤</ci><ci id="alg1.l5.m1.1.1.3.2.3.cmml" xref="alg1.l5.m1.1.1.3.2.3">𝐹</ci></apply><apply id="alg1.l5.m1.1.1.3.3.cmml" xref="alg1.l5.m1.1.1.3.3"><ci id="alg1.l5.m1.1.1.3.3.1.cmml" xref="alg1.l5.m1.1.1.3.3.1">⋅</ci><ci id="alg1.l5.m1.1.1.3.3.2.cmml" xref="alg1.l5.m1.1.1.3.3.2">𝜂</ci><ci id="alg1.l5.m1.1.1.3.3.3.cmml" xref="alg1.l5.m1.1.1.3.3.3">∇</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l5.m1.1c">w_{F}=w_{F}-\eta\cdot\nabla</annotation></semantics></math> {Update the model}

</div>
<div id="alg1.l6" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l6.1.1.1" class="ltx_text" style="font-size:80%;">6:</span></span>  <span id="alg1.l6.2" class="ltx_text ltx_font_bold">end</span> <span id="alg1.l6.3" class="ltx_text ltx_font_bold">for</span>
</div>
<div id="alg1.l7" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l7.1.1.1" class="ltx_text" style="font-size:80%;">7:</span></span>  <span id="alg1.l7.2" class="ltx_text ltx_font_bold">Mask Extraction:</span>

</div>
<div id="alg1.l8" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l8.1.1.1" class="ltx_text" style="font-size:80%;">8:</span></span>  <math id="alg1.l8.m1.1" class="ltx_Math" alttext="\Delta=w_{F}-w_{p}" display="inline"><semantics id="alg1.l8.m1.1a"><mrow id="alg1.l8.m1.1.1" xref="alg1.l8.m1.1.1.cmml"><mi mathvariant="normal" id="alg1.l8.m1.1.1.2" xref="alg1.l8.m1.1.1.2.cmml">Δ</mi><mo id="alg1.l8.m1.1.1.1" xref="alg1.l8.m1.1.1.1.cmml">=</mo><mrow id="alg1.l8.m1.1.1.3" xref="alg1.l8.m1.1.1.3.cmml"><msub id="alg1.l8.m1.1.1.3.2" xref="alg1.l8.m1.1.1.3.2.cmml"><mi id="alg1.l8.m1.1.1.3.2.2" xref="alg1.l8.m1.1.1.3.2.2.cmml">w</mi><mi id="alg1.l8.m1.1.1.3.2.3" xref="alg1.l8.m1.1.1.3.2.3.cmml">F</mi></msub><mo id="alg1.l8.m1.1.1.3.1" xref="alg1.l8.m1.1.1.3.1.cmml">−</mo><msub id="alg1.l8.m1.1.1.3.3" xref="alg1.l8.m1.1.1.3.3.cmml"><mi id="alg1.l8.m1.1.1.3.3.2" xref="alg1.l8.m1.1.1.3.3.2.cmml">w</mi><mi id="alg1.l8.m1.1.1.3.3.3" xref="alg1.l8.m1.1.1.3.3.3.cmml">p</mi></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l8.m1.1b"><apply id="alg1.l8.m1.1.1.cmml" xref="alg1.l8.m1.1.1"><eq id="alg1.l8.m1.1.1.1.cmml" xref="alg1.l8.m1.1.1.1"></eq><ci id="alg1.l8.m1.1.1.2.cmml" xref="alg1.l8.m1.1.1.2">Δ</ci><apply id="alg1.l8.m1.1.1.3.cmml" xref="alg1.l8.m1.1.1.3"><minus id="alg1.l8.m1.1.1.3.1.cmml" xref="alg1.l8.m1.1.1.3.1"></minus><apply id="alg1.l8.m1.1.1.3.2.cmml" xref="alg1.l8.m1.1.1.3.2"><csymbol cd="ambiguous" id="alg1.l8.m1.1.1.3.2.1.cmml" xref="alg1.l8.m1.1.1.3.2">subscript</csymbol><ci id="alg1.l8.m1.1.1.3.2.2.cmml" xref="alg1.l8.m1.1.1.3.2.2">𝑤</ci><ci id="alg1.l8.m1.1.1.3.2.3.cmml" xref="alg1.l8.m1.1.1.3.2.3">𝐹</ci></apply><apply id="alg1.l8.m1.1.1.3.3.cmml" xref="alg1.l8.m1.1.1.3.3"><csymbol cd="ambiguous" id="alg1.l8.m1.1.1.3.3.1.cmml" xref="alg1.l8.m1.1.1.3.3">subscript</csymbol><ci id="alg1.l8.m1.1.1.3.3.2.cmml" xref="alg1.l8.m1.1.1.3.3.2">𝑤</ci><ci id="alg1.l8.m1.1.1.3.3.3.cmml" xref="alg1.l8.m1.1.1.3.3.3">𝑝</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l8.m1.1c">\Delta=w_{F}-w_{p}</annotation></semantics></math> {Find the task vector}

</div>
<div id="alg1.l9" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l9.1.1.1" class="ltx_text" style="font-size:80%;">9:</span></span>  <math id="alg1.l9.m1.2" class="ltx_Math" alttext="m=\texttt{Sparsify}(\Delta,s)" display="inline"><semantics id="alg1.l9.m1.2a"><mrow id="alg1.l9.m1.2.3" xref="alg1.l9.m1.2.3.cmml"><mi id="alg1.l9.m1.2.3.2" xref="alg1.l9.m1.2.3.2.cmml">m</mi><mo id="alg1.l9.m1.2.3.1" xref="alg1.l9.m1.2.3.1.cmml">=</mo><mrow id="alg1.l9.m1.2.3.3" xref="alg1.l9.m1.2.3.3.cmml"><mtext class="ltx_mathvariant_monospace" id="alg1.l9.m1.2.3.3.2" xref="alg1.l9.m1.2.3.3.2a.cmml">Sparsify</mtext><mo lspace="0em" rspace="0em" id="alg1.l9.m1.2.3.3.1" xref="alg1.l9.m1.2.3.3.1.cmml">​</mo><mrow id="alg1.l9.m1.2.3.3.3.2" xref="alg1.l9.m1.2.3.3.3.1.cmml"><mo stretchy="false" id="alg1.l9.m1.2.3.3.3.2.1" xref="alg1.l9.m1.2.3.3.3.1.cmml">(</mo><mi mathvariant="normal" id="alg1.l9.m1.1.1" xref="alg1.l9.m1.1.1.cmml">Δ</mi><mo id="alg1.l9.m1.2.3.3.3.2.2" xref="alg1.l9.m1.2.3.3.3.1.cmml">,</mo><mi id="alg1.l9.m1.2.2" xref="alg1.l9.m1.2.2.cmml">s</mi><mo stretchy="false" id="alg1.l9.m1.2.3.3.3.2.3" xref="alg1.l9.m1.2.3.3.3.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l9.m1.2b"><apply id="alg1.l9.m1.2.3.cmml" xref="alg1.l9.m1.2.3"><eq id="alg1.l9.m1.2.3.1.cmml" xref="alg1.l9.m1.2.3.1"></eq><ci id="alg1.l9.m1.2.3.2.cmml" xref="alg1.l9.m1.2.3.2">𝑚</ci><apply id="alg1.l9.m1.2.3.3.cmml" xref="alg1.l9.m1.2.3.3"><times id="alg1.l9.m1.2.3.3.1.cmml" xref="alg1.l9.m1.2.3.3.1"></times><ci id="alg1.l9.m1.2.3.3.2a.cmml" xref="alg1.l9.m1.2.3.3.2"><mtext class="ltx_mathvariant_monospace" id="alg1.l9.m1.2.3.3.2.cmml" xref="alg1.l9.m1.2.3.3.2">Sparsify</mtext></ci><interval closure="open" id="alg1.l9.m1.2.3.3.3.1.cmml" xref="alg1.l9.m1.2.3.3.3.2"><ci id="alg1.l9.m1.1.1.cmml" xref="alg1.l9.m1.1.1">Δ</ci><ci id="alg1.l9.m1.2.2.cmml" xref="alg1.l9.m1.2.2">𝑠</ci></interval></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l9.m1.2c">m=\texttt{Sparsify}(\Delta,s)</annotation></semantics></math> {Create the sparsity mask by thresholding the task vector based on magnitude}

</div>
<div id="alg1.l10" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l10.1.1.1" class="ltx_text" style="font-size:80%;">10:</span></span>  <span id="alg1.l10.2" class="ltx_text ltx_font_bold">Sparse Adaptation:</span>

</div>
<div id="alg1.l11" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l11.1.1.1" class="ltx_text" style="font-size:80%;">11:</span></span>  <math id="alg1.l11.m1.1" class="ltx_Math" alttext="w\leftarrow w_{P}" display="inline"><semantics id="alg1.l11.m1.1a"><mrow id="alg1.l11.m1.1.1" xref="alg1.l11.m1.1.1.cmml"><mi id="alg1.l11.m1.1.1.2" xref="alg1.l11.m1.1.1.2.cmml">w</mi><mo stretchy="false" id="alg1.l11.m1.1.1.1" xref="alg1.l11.m1.1.1.1.cmml">←</mo><msub id="alg1.l11.m1.1.1.3" xref="alg1.l11.m1.1.1.3.cmml"><mi id="alg1.l11.m1.1.1.3.2" xref="alg1.l11.m1.1.1.3.2.cmml">w</mi><mi id="alg1.l11.m1.1.1.3.3" xref="alg1.l11.m1.1.1.3.3.cmml">P</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="alg1.l11.m1.1b"><apply id="alg1.l11.m1.1.1.cmml" xref="alg1.l11.m1.1.1"><ci id="alg1.l11.m1.1.1.1.cmml" xref="alg1.l11.m1.1.1.1">←</ci><ci id="alg1.l11.m1.1.1.2.cmml" xref="alg1.l11.m1.1.1.2">𝑤</ci><apply id="alg1.l11.m1.1.1.3.cmml" xref="alg1.l11.m1.1.1.3"><csymbol cd="ambiguous" id="alg1.l11.m1.1.1.3.1.cmml" xref="alg1.l11.m1.1.1.3">subscript</csymbol><ci id="alg1.l11.m1.1.1.3.2.cmml" xref="alg1.l11.m1.1.1.3.2">𝑤</ci><ci id="alg1.l11.m1.1.1.3.3.cmml" xref="alg1.l11.m1.1.1.3.3">𝑃</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l11.m1.1c">w\leftarrow w_{P}</annotation></semantics></math>

</div>
<div id="alg1.l12" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l12.1.1.1" class="ltx_text" style="font-size:80%;">12:</span></span>  <span id="alg1.l12.2" class="ltx_text ltx_font_bold">for</span> <math id="alg1.l12.m1.3" class="ltx_Math" alttext="t\in\{0,\dots,T\}" display="inline"><semantics id="alg1.l12.m1.3a"><mrow id="alg1.l12.m1.3.4" xref="alg1.l12.m1.3.4.cmml"><mi id="alg1.l12.m1.3.4.2" xref="alg1.l12.m1.3.4.2.cmml">t</mi><mo id="alg1.l12.m1.3.4.1" xref="alg1.l12.m1.3.4.1.cmml">∈</mo><mrow id="alg1.l12.m1.3.4.3.2" xref="alg1.l12.m1.3.4.3.1.cmml"><mo stretchy="false" id="alg1.l12.m1.3.4.3.2.1" xref="alg1.l12.m1.3.4.3.1.cmml">{</mo><mn id="alg1.l12.m1.1.1" xref="alg1.l12.m1.1.1.cmml">0</mn><mo id="alg1.l12.m1.3.4.3.2.2" xref="alg1.l12.m1.3.4.3.1.cmml">,</mo><mi mathvariant="normal" id="alg1.l12.m1.2.2" xref="alg1.l12.m1.2.2.cmml">…</mi><mo id="alg1.l12.m1.3.4.3.2.3" xref="alg1.l12.m1.3.4.3.1.cmml">,</mo><mi id="alg1.l12.m1.3.3" xref="alg1.l12.m1.3.3.cmml">T</mi><mo stretchy="false" id="alg1.l12.m1.3.4.3.2.4" xref="alg1.l12.m1.3.4.3.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l12.m1.3b"><apply id="alg1.l12.m1.3.4.cmml" xref="alg1.l12.m1.3.4"><in id="alg1.l12.m1.3.4.1.cmml" xref="alg1.l12.m1.3.4.1"></in><ci id="alg1.l12.m1.3.4.2.cmml" xref="alg1.l12.m1.3.4.2">𝑡</ci><set id="alg1.l12.m1.3.4.3.1.cmml" xref="alg1.l12.m1.3.4.3.2"><cn type="integer" id="alg1.l12.m1.1.1.cmml" xref="alg1.l12.m1.1.1">0</cn><ci id="alg1.l12.m1.2.2.cmml" xref="alg1.l12.m1.2.2">…</ci><ci id="alg1.l12.m1.3.3.cmml" xref="alg1.l12.m1.3.3">𝑇</ci></set></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l12.m1.3c">t\in\{0,\dots,T\}</annotation></semantics></math> <span id="alg1.l12.3" class="ltx_text ltx_font_bold">do</span>



</div>
<div id="alg1.l13" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l13.1.1.1" class="ltx_text" style="font-size:80%;">13:</span></span>     <math id="alg1.l13.m1.1" class="ltx_Math" alttext="\nabla=\mathbb{A}_{\mathbb{D}}(w)" display="inline"><semantics id="alg1.l13.m1.1a"><mrow id="alg1.l13.m1.1.2" xref="alg1.l13.m1.1.2.cmml"><mo id="alg1.l13.m1.1.2.2" xref="alg1.l13.m1.1.2.2.cmml">∇</mo><mo id="alg1.l13.m1.1.2.1" xref="alg1.l13.m1.1.2.1.cmml">=</mo><mrow id="alg1.l13.m1.1.2.3" xref="alg1.l13.m1.1.2.3.cmml"><msub id="alg1.l13.m1.1.2.3.2" xref="alg1.l13.m1.1.2.3.2.cmml"><mi id="alg1.l13.m1.1.2.3.2.2" xref="alg1.l13.m1.1.2.3.2.2.cmml">𝔸</mi><mi id="alg1.l13.m1.1.2.3.2.3" xref="alg1.l13.m1.1.2.3.2.3.cmml">𝔻</mi></msub><mo lspace="0em" rspace="0em" id="alg1.l13.m1.1.2.3.1" xref="alg1.l13.m1.1.2.3.1.cmml">​</mo><mrow id="alg1.l13.m1.1.2.3.3.2" xref="alg1.l13.m1.1.2.3.cmml"><mo stretchy="false" id="alg1.l13.m1.1.2.3.3.2.1" xref="alg1.l13.m1.1.2.3.cmml">(</mo><mi id="alg1.l13.m1.1.1" xref="alg1.l13.m1.1.1.cmml">w</mi><mo stretchy="false" id="alg1.l13.m1.1.2.3.3.2.2" xref="alg1.l13.m1.1.2.3.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l13.m1.1b"><apply id="alg1.l13.m1.1.2.cmml" xref="alg1.l13.m1.1.2"><eq id="alg1.l13.m1.1.2.1.cmml" xref="alg1.l13.m1.1.2.1"></eq><ci id="alg1.l13.m1.1.2.2.cmml" xref="alg1.l13.m1.1.2.2">∇</ci><apply id="alg1.l13.m1.1.2.3.cmml" xref="alg1.l13.m1.1.2.3"><times id="alg1.l13.m1.1.2.3.1.cmml" xref="alg1.l13.m1.1.2.3.1"></times><apply id="alg1.l13.m1.1.2.3.2.cmml" xref="alg1.l13.m1.1.2.3.2"><csymbol cd="ambiguous" id="alg1.l13.m1.1.2.3.2.1.cmml" xref="alg1.l13.m1.1.2.3.2">subscript</csymbol><ci id="alg1.l13.m1.1.2.3.2.2.cmml" xref="alg1.l13.m1.1.2.3.2.2">𝔸</ci><ci id="alg1.l13.m1.1.2.3.2.3.cmml" xref="alg1.l13.m1.1.2.3.2.3">𝔻</ci></apply><ci id="alg1.l13.m1.1.1.cmml" xref="alg1.l13.m1.1.1">𝑤</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l13.m1.1c">\nabla=\mathbb{A}_{\mathbb{D}}(w)</annotation></semantics></math> {Compute gradient for weights}

</div>
<div id="alg1.l14" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l14.1.1.1" class="ltx_text" style="font-size:80%;">14:</span></span>     <math id="alg1.l14.m1.1" class="ltx_Math" alttext="\hat{\nabla}=\nabla\odot m" display="inline"><semantics id="alg1.l14.m1.1a"><mrow id="alg1.l14.m1.1.1" xref="alg1.l14.m1.1.1.cmml"><mover accent="true" id="alg1.l14.m1.1.1.2" xref="alg1.l14.m1.1.1.2.cmml"><mo id="alg1.l14.m1.1.1.2.2" xref="alg1.l14.m1.1.1.2.2.cmml">∇</mo><mo id="alg1.l14.m1.1.1.2.1" xref="alg1.l14.m1.1.1.2.1.cmml">^</mo></mover><mo id="alg1.l14.m1.1.1.1" xref="alg1.l14.m1.1.1.1.cmml">=</mo><mrow id="alg1.l14.m1.1.1.3" xref="alg1.l14.m1.1.1.3.cmml"><mo id="alg1.l14.m1.1.1.3.2" xref="alg1.l14.m1.1.1.3.2.cmml">∇</mo><mo lspace="0em" rspace="0.222em" id="alg1.l14.m1.1.1.3.1" xref="alg1.l14.m1.1.1.3.1.cmml">⊙</mo><mi id="alg1.l14.m1.1.1.3.3" xref="alg1.l14.m1.1.1.3.3.cmml">m</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l14.m1.1b"><apply id="alg1.l14.m1.1.1.cmml" xref="alg1.l14.m1.1.1"><eq id="alg1.l14.m1.1.1.1.cmml" xref="alg1.l14.m1.1.1.1"></eq><apply id="alg1.l14.m1.1.1.2.cmml" xref="alg1.l14.m1.1.1.2"><ci id="alg1.l14.m1.1.1.2.1.cmml" xref="alg1.l14.m1.1.1.2.1">^</ci><ci id="alg1.l14.m1.1.1.2.2.cmml" xref="alg1.l14.m1.1.1.2.2">∇</ci></apply><apply id="alg1.l14.m1.1.1.3.cmml" xref="alg1.l14.m1.1.1.3"><csymbol cd="latexml" id="alg1.l14.m1.1.1.3.1.cmml" xref="alg1.l14.m1.1.1.3.1">direct-product</csymbol><ci id="alg1.l14.m1.1.1.3.2.cmml" xref="alg1.l14.m1.1.1.3.2">∇</ci><ci id="alg1.l14.m1.1.1.3.3.cmml" xref="alg1.l14.m1.1.1.3.3">𝑚</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l14.m1.1c">\hat{\nabla}=\nabla\odot m</annotation></semantics></math> {Apply sparse mask to gradient}

</div>
<div id="alg1.l15" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l15.1.1.1" class="ltx_text" style="font-size:80%;">15:</span></span>     <math id="alg1.l15.m1.1" class="ltx_Math" alttext="w=w-\eta\cdot\hat{\nabla}" display="inline"><semantics id="alg1.l15.m1.1a"><mrow id="alg1.l15.m1.1.1" xref="alg1.l15.m1.1.1.cmml"><mi id="alg1.l15.m1.1.1.2" xref="alg1.l15.m1.1.1.2.cmml">w</mi><mo id="alg1.l15.m1.1.1.1" xref="alg1.l15.m1.1.1.1.cmml">=</mo><mrow id="alg1.l15.m1.1.1.3" xref="alg1.l15.m1.1.1.3.cmml"><mi id="alg1.l15.m1.1.1.3.2" xref="alg1.l15.m1.1.1.3.2.cmml">w</mi><mo id="alg1.l15.m1.1.1.3.1" xref="alg1.l15.m1.1.1.3.1.cmml">−</mo><mrow id="alg1.l15.m1.1.1.3.3" xref="alg1.l15.m1.1.1.3.3.cmml"><mi id="alg1.l15.m1.1.1.3.3.2" xref="alg1.l15.m1.1.1.3.3.2.cmml">η</mi><mo lspace="0.222em" rspace="0.222em" id="alg1.l15.m1.1.1.3.3.1" xref="alg1.l15.m1.1.1.3.3.1.cmml">⋅</mo><mover accent="true" id="alg1.l15.m1.1.1.3.3.3" xref="alg1.l15.m1.1.1.3.3.3.cmml"><mo id="alg1.l15.m1.1.1.3.3.3.2" xref="alg1.l15.m1.1.1.3.3.3.2.cmml">∇</mo><mo id="alg1.l15.m1.1.1.3.3.3.1" xref="alg1.l15.m1.1.1.3.3.3.1.cmml">^</mo></mover></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l15.m1.1b"><apply id="alg1.l15.m1.1.1.cmml" xref="alg1.l15.m1.1.1"><eq id="alg1.l15.m1.1.1.1.cmml" xref="alg1.l15.m1.1.1.1"></eq><ci id="alg1.l15.m1.1.1.2.cmml" xref="alg1.l15.m1.1.1.2">𝑤</ci><apply id="alg1.l15.m1.1.1.3.cmml" xref="alg1.l15.m1.1.1.3"><minus id="alg1.l15.m1.1.1.3.1.cmml" xref="alg1.l15.m1.1.1.3.1"></minus><ci id="alg1.l15.m1.1.1.3.2.cmml" xref="alg1.l15.m1.1.1.3.2">𝑤</ci><apply id="alg1.l15.m1.1.1.3.3.cmml" xref="alg1.l15.m1.1.1.3.3"><ci id="alg1.l15.m1.1.1.3.3.1.cmml" xref="alg1.l15.m1.1.1.3.3.1">⋅</ci><ci id="alg1.l15.m1.1.1.3.3.2.cmml" xref="alg1.l15.m1.1.1.3.3.2">𝜂</ci><apply id="alg1.l15.m1.1.1.3.3.3.cmml" xref="alg1.l15.m1.1.1.3.3.3"><ci id="alg1.l15.m1.1.1.3.3.3.1.cmml" xref="alg1.l15.m1.1.1.3.3.3.1">^</ci><ci id="alg1.l15.m1.1.1.3.3.3.2.cmml" xref="alg1.l15.m1.1.1.3.3.3.2">∇</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l15.m1.1c">w=w-\eta\cdot\hat{\nabla}</annotation></semantics></math> {Update the model}

</div>
<div id="alg1.l16" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l16.1.1.1" class="ltx_text" style="font-size:80%;">16:</span></span>  <span id="alg1.l16.2" class="ltx_text ltx_font_bold">end</span> <span id="alg1.l16.3" class="ltx_text ltx_font_bold">for</span>
</div>
<div id="alg1.l17" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l17.1.1.1" class="ltx_text" style="font-size:80%;">17:</span></span>  <math id="alg1.l17.m1.1" class="ltx_Math" alttext="\hat{w}_{F}\leftarrow w" display="inline"><semantics id="alg1.l17.m1.1a"><mrow id="alg1.l17.m1.1.1" xref="alg1.l17.m1.1.1.cmml"><msub id="alg1.l17.m1.1.1.2" xref="alg1.l17.m1.1.1.2.cmml"><mover accent="true" id="alg1.l17.m1.1.1.2.2" xref="alg1.l17.m1.1.1.2.2.cmml"><mi id="alg1.l17.m1.1.1.2.2.2" xref="alg1.l17.m1.1.1.2.2.2.cmml">w</mi><mo id="alg1.l17.m1.1.1.2.2.1" xref="alg1.l17.m1.1.1.2.2.1.cmml">^</mo></mover><mi id="alg1.l17.m1.1.1.2.3" xref="alg1.l17.m1.1.1.2.3.cmml">F</mi></msub><mo stretchy="false" id="alg1.l17.m1.1.1.1" xref="alg1.l17.m1.1.1.1.cmml">←</mo><mi id="alg1.l17.m1.1.1.3" xref="alg1.l17.m1.1.1.3.cmml">w</mi></mrow><annotation-xml encoding="MathML-Content" id="alg1.l17.m1.1b"><apply id="alg1.l17.m1.1.1.cmml" xref="alg1.l17.m1.1.1"><ci id="alg1.l17.m1.1.1.1.cmml" xref="alg1.l17.m1.1.1.1">←</ci><apply id="alg1.l17.m1.1.1.2.cmml" xref="alg1.l17.m1.1.1.2"><csymbol cd="ambiguous" id="alg1.l17.m1.1.1.2.1.cmml" xref="alg1.l17.m1.1.1.2">subscript</csymbol><apply id="alg1.l17.m1.1.1.2.2.cmml" xref="alg1.l17.m1.1.1.2.2"><ci id="alg1.l17.m1.1.1.2.2.1.cmml" xref="alg1.l17.m1.1.1.2.2.1">^</ci><ci id="alg1.l17.m1.1.1.2.2.2.cmml" xref="alg1.l17.m1.1.1.2.2.2">𝑤</ci></apply><ci id="alg1.l17.m1.1.1.2.3.cmml" xref="alg1.l17.m1.1.1.2.3">𝐹</ci></apply><ci id="alg1.l17.m1.1.1.3.cmml" xref="alg1.l17.m1.1.1.3">𝑤</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l17.m1.1c">\hat{w}_{F}\leftarrow w</annotation></semantics></math>

</div>
<div id="alg1.l17a" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg1.l17a.1.1.1" class="ltx_text" style="font-size:80%;">17:</span></span>  <math id="alg1.l17a.m1.1" class="ltx_Math" alttext="\hat{w}_{F}" display="inline"><semantics id="alg1.l17a.m1.1a"><msub id="alg1.l17a.m1.1.1" xref="alg1.l17a.m1.1.1.cmml"><mover accent="true" id="alg1.l17a.m1.1.1.2" xref="alg1.l17a.m1.1.1.2.cmml"><mi id="alg1.l17a.m1.1.1.2.2" xref="alg1.l17a.m1.1.1.2.2.cmml">w</mi><mo id="alg1.l17a.m1.1.1.2.1" xref="alg1.l17a.m1.1.1.2.1.cmml">^</mo></mover><mi id="alg1.l17a.m1.1.1.3" xref="alg1.l17a.m1.1.1.3.cmml">F</mi></msub><annotation-xml encoding="MathML-Content" id="alg1.l17a.m1.1b"><apply id="alg1.l17a.m1.1.1.cmml" xref="alg1.l17a.m1.1.1"><csymbol cd="ambiguous" id="alg1.l17a.m1.1.1.1.cmml" xref="alg1.l17a.m1.1.1">subscript</csymbol><apply id="alg1.l17a.m1.1.1.2.cmml" xref="alg1.l17a.m1.1.1.2"><ci id="alg1.l17a.m1.1.1.2.1.cmml" xref="alg1.l17a.m1.1.1.2.1">^</ci><ci id="alg1.l17a.m1.1.1.2.2.cmml" xref="alg1.l17a.m1.1.1.2.2">𝑤</ci></apply><ci id="alg1.l17a.m1.1.1.3.cmml" xref="alg1.l17a.m1.1.1.3">𝐹</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l17a.m1.1c">\hat{w}_{F}</annotation></semantics></math>

</div>
</div>
</figure>
</figure>
<div id="S3.p1" class="ltx_para">
<p id="S3.p1.1" class="ltx_p">The desiderata for each multi-task adaptation paradigm motivates the design of our method. For <em id="S3.p1.1.1" class="ltx_emph ltx_font_italic">adapters</em>, we want a representation that can be easily compressed for memory efficiency. For <em id="S3.p1.1.2" class="ltx_emph ltx_font_italic">sequential training</em>, we want a representation that minimizes destructive interference between the previously learned tasks and tasks to be learned in the future.
For <em id="S3.p1.1.3" class="ltx_emph ltx_font_italic">model merging</em>, we want representations that are mutually sparse with each other in parameter space to again prevent destructive interference. We now propose LoTA, a single method that enjoys all these features. We first describe the workflow of LoTA, then revisit the problems each multi-task paradigm faces and discuss how and why LoTA successfully mitigates them.</p>
</div>
<div id="S3.p2" class="ltx_para">
<p id="S3.p2.11" class="ltx_p"><span id="S3.p2.11.1" class="ltx_text ltx_font_bold">Lottery Ticket Adaptation (LoTA).</span>
LoTA works in three phases as summarized in Figure <a href="#S3.F2" title="Figure 2 ‣ 3 Lottery Ticket Adaptation (LoTA) ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>: (1) mask calibration, (2) mask extraction, (3) sparse adaptation. In the mask calibration phase of LoTA, a base model with parameters <math id="S3.p2.1.m1.1" class="ltx_Math" alttext="w_{P}" display="inline"><semantics id="S3.p2.1.m1.1a"><msub id="S3.p2.1.m1.1.1" xref="S3.p2.1.m1.1.1.cmml"><mi id="S3.p2.1.m1.1.1.2" xref="S3.p2.1.m1.1.1.2.cmml">w</mi><mi id="S3.p2.1.m1.1.1.3" xref="S3.p2.1.m1.1.1.3.cmml">P</mi></msub><annotation-xml encoding="MathML-Content" id="S3.p2.1.m1.1b"><apply id="S3.p2.1.m1.1.1.cmml" xref="S3.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S3.p2.1.m1.1.1.1.cmml" xref="S3.p2.1.m1.1.1">subscript</csymbol><ci id="S3.p2.1.m1.1.1.2.cmml" xref="S3.p2.1.m1.1.1.2">𝑤</ci><ci id="S3.p2.1.m1.1.1.3.cmml" xref="S3.p2.1.m1.1.1.3">𝑃</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.1.m1.1c">w_{P}</annotation></semantics></math> is fine-tuned for <math id="S3.p2.2.m2.1" class="ltx_Math" alttext="\mathcal{T}" display="inline"><semantics id="S3.p2.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="S3.p2.2.m2.1.1" xref="S3.p2.2.m2.1.1.cmml">𝒯</mi><annotation-xml encoding="MathML-Content" id="S3.p2.2.m2.1b"><ci id="S3.p2.2.m2.1.1.cmml" xref="S3.p2.2.m2.1.1">𝒯</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.2.m2.1c">\mathcal{T}</annotation></semantics></math> iterations, yielding a fine-tuned model with parameters <math id="S3.p2.3.m3.1" class="ltx_Math" alttext="w_{F}" display="inline"><semantics id="S3.p2.3.m3.1a"><msub id="S3.p2.3.m3.1.1" xref="S3.p2.3.m3.1.1.cmml"><mi id="S3.p2.3.m3.1.1.2" xref="S3.p2.3.m3.1.1.2.cmml">w</mi><mi id="S3.p2.3.m3.1.1.3" xref="S3.p2.3.m3.1.1.3.cmml">F</mi></msub><annotation-xml encoding="MathML-Content" id="S3.p2.3.m3.1b"><apply id="S3.p2.3.m3.1.1.cmml" xref="S3.p2.3.m3.1.1"><csymbol cd="ambiguous" id="S3.p2.3.m3.1.1.1.cmml" xref="S3.p2.3.m3.1.1">subscript</csymbol><ci id="S3.p2.3.m3.1.1.2.cmml" xref="S3.p2.3.m3.1.1.2">𝑤</ci><ci id="S3.p2.3.m3.1.1.3.cmml" xref="S3.p2.3.m3.1.1.3">𝐹</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.3.m3.1c">w_{F}</annotation></semantics></math>. Then, in the mask extraction phase, LoTA extracts a sparsity mask <math id="S3.p2.4.m4.1" class="ltx_Math" alttext="m" display="inline"><semantics id="S3.p2.4.m4.1a"><mi id="S3.p2.4.m4.1.1" xref="S3.p2.4.m4.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="S3.p2.4.m4.1b"><ci id="S3.p2.4.m4.1.1.cmml" xref="S3.p2.4.m4.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.4.m4.1c">m</annotation></semantics></math> from the task vector <math id="S3.p2.5.m5.1" class="ltx_Math" alttext="\Delta=w_{F}-w_{P}" display="inline"><semantics id="S3.p2.5.m5.1a"><mrow id="S3.p2.5.m5.1.1" xref="S3.p2.5.m5.1.1.cmml"><mi mathvariant="normal" id="S3.p2.5.m5.1.1.2" xref="S3.p2.5.m5.1.1.2.cmml">Δ</mi><mo id="S3.p2.5.m5.1.1.1" xref="S3.p2.5.m5.1.1.1.cmml">=</mo><mrow id="S3.p2.5.m5.1.1.3" xref="S3.p2.5.m5.1.1.3.cmml"><msub id="S3.p2.5.m5.1.1.3.2" xref="S3.p2.5.m5.1.1.3.2.cmml"><mi id="S3.p2.5.m5.1.1.3.2.2" xref="S3.p2.5.m5.1.1.3.2.2.cmml">w</mi><mi id="S3.p2.5.m5.1.1.3.2.3" xref="S3.p2.5.m5.1.1.3.2.3.cmml">F</mi></msub><mo id="S3.p2.5.m5.1.1.3.1" xref="S3.p2.5.m5.1.1.3.1.cmml">−</mo><msub id="S3.p2.5.m5.1.1.3.3" xref="S3.p2.5.m5.1.1.3.3.cmml"><mi id="S3.p2.5.m5.1.1.3.3.2" xref="S3.p2.5.m5.1.1.3.3.2.cmml">w</mi><mi id="S3.p2.5.m5.1.1.3.3.3" xref="S3.p2.5.m5.1.1.3.3.3.cmml">P</mi></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.p2.5.m5.1b"><apply id="S3.p2.5.m5.1.1.cmml" xref="S3.p2.5.m5.1.1"><eq id="S3.p2.5.m5.1.1.1.cmml" xref="S3.p2.5.m5.1.1.1"></eq><ci id="S3.p2.5.m5.1.1.2.cmml" xref="S3.p2.5.m5.1.1.2">Δ</ci><apply id="S3.p2.5.m5.1.1.3.cmml" xref="S3.p2.5.m5.1.1.3"><minus id="S3.p2.5.m5.1.1.3.1.cmml" xref="S3.p2.5.m5.1.1.3.1"></minus><apply id="S3.p2.5.m5.1.1.3.2.cmml" xref="S3.p2.5.m5.1.1.3.2"><csymbol cd="ambiguous" id="S3.p2.5.m5.1.1.3.2.1.cmml" xref="S3.p2.5.m5.1.1.3.2">subscript</csymbol><ci id="S3.p2.5.m5.1.1.3.2.2.cmml" xref="S3.p2.5.m5.1.1.3.2.2">𝑤</ci><ci id="S3.p2.5.m5.1.1.3.2.3.cmml" xref="S3.p2.5.m5.1.1.3.2.3">𝐹</ci></apply><apply id="S3.p2.5.m5.1.1.3.3.cmml" xref="S3.p2.5.m5.1.1.3.3"><csymbol cd="ambiguous" id="S3.p2.5.m5.1.1.3.3.1.cmml" xref="S3.p2.5.m5.1.1.3.3">subscript</csymbol><ci id="S3.p2.5.m5.1.1.3.3.2.cmml" xref="S3.p2.5.m5.1.1.3.3.2">𝑤</ci><ci id="S3.p2.5.m5.1.1.3.3.3.cmml" xref="S3.p2.5.m5.1.1.3.3.3">𝑃</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.5.m5.1c">\Delta=w_{F}-w_{P}</annotation></semantics></math> based on the magnitude of the updates in <math id="S3.p2.6.m6.1" class="ltx_Math" alttext="\Delta" display="inline"><semantics id="S3.p2.6.m6.1a"><mi mathvariant="normal" id="S3.p2.6.m6.1.1" xref="S3.p2.6.m6.1.1.cmml">Δ</mi><annotation-xml encoding="MathML-Content" id="S3.p2.6.m6.1b"><ci id="S3.p2.6.m6.1.1.cmml" xref="S3.p2.6.m6.1.1">Δ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.6.m6.1c">\Delta</annotation></semantics></math>. <math id="S3.p2.7.m7.1" class="ltx_Math" alttext="\mathcal{T}" display="inline"><semantics id="S3.p2.7.m7.1a"><mi class="ltx_font_mathcaligraphic" id="S3.p2.7.m7.1.1" xref="S3.p2.7.m7.1.1.cmml">𝒯</mi><annotation-xml encoding="MathML-Content" id="S3.p2.7.m7.1b"><ci id="S3.p2.7.m7.1.1.cmml" xref="S3.p2.7.m7.1.1">𝒯</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.7.m7.1c">\mathcal{T}</annotation></semantics></math> could be as small as one iteration. In the sparse adaptation phase of LoTA, the model is first reset to its original state with weights <math id="S3.p2.8.m8.1" class="ltx_Math" alttext="w_{P}" display="inline"><semantics id="S3.p2.8.m8.1a"><msub id="S3.p2.8.m8.1.1" xref="S3.p2.8.m8.1.1.cmml"><mi id="S3.p2.8.m8.1.1.2" xref="S3.p2.8.m8.1.1.2.cmml">w</mi><mi id="S3.p2.8.m8.1.1.3" xref="S3.p2.8.m8.1.1.3.cmml">P</mi></msub><annotation-xml encoding="MathML-Content" id="S3.p2.8.m8.1b"><apply id="S3.p2.8.m8.1.1.cmml" xref="S3.p2.8.m8.1.1"><csymbol cd="ambiguous" id="S3.p2.8.m8.1.1.1.cmml" xref="S3.p2.8.m8.1.1">subscript</csymbol><ci id="S3.p2.8.m8.1.1.2.cmml" xref="S3.p2.8.m8.1.1.2">𝑤</ci><ci id="S3.p2.8.m8.1.1.3.cmml" xref="S3.p2.8.m8.1.1.3">𝑃</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.8.m8.1c">w_{P}</annotation></semantics></math>. Then the subnetwork <math id="S3.p2.9.m9.1" class="ltx_Math" alttext="w_{P}\odot m" display="inline"><semantics id="S3.p2.9.m9.1a"><mrow id="S3.p2.9.m9.1.1" xref="S3.p2.9.m9.1.1.cmml"><msub id="S3.p2.9.m9.1.1.2" xref="S3.p2.9.m9.1.1.2.cmml"><mi id="S3.p2.9.m9.1.1.2.2" xref="S3.p2.9.m9.1.1.2.2.cmml">w</mi><mi id="S3.p2.9.m9.1.1.2.3" xref="S3.p2.9.m9.1.1.2.3.cmml">P</mi></msub><mo lspace="0.222em" rspace="0.222em" id="S3.p2.9.m9.1.1.1" xref="S3.p2.9.m9.1.1.1.cmml">⊙</mo><mi id="S3.p2.9.m9.1.1.3" xref="S3.p2.9.m9.1.1.3.cmml">m</mi></mrow><annotation-xml encoding="MathML-Content" id="S3.p2.9.m9.1b"><apply id="S3.p2.9.m9.1.1.cmml" xref="S3.p2.9.m9.1.1"><csymbol cd="latexml" id="S3.p2.9.m9.1.1.1.cmml" xref="S3.p2.9.m9.1.1.1">direct-product</csymbol><apply id="S3.p2.9.m9.1.1.2.cmml" xref="S3.p2.9.m9.1.1.2"><csymbol cd="ambiguous" id="S3.p2.9.m9.1.1.2.1.cmml" xref="S3.p2.9.m9.1.1.2">subscript</csymbol><ci id="S3.p2.9.m9.1.1.2.2.cmml" xref="S3.p2.9.m9.1.1.2.2">𝑤</ci><ci id="S3.p2.9.m9.1.1.2.3.cmml" xref="S3.p2.9.m9.1.1.2.3">𝑃</ci></apply><ci id="S3.p2.9.m9.1.1.3.cmml" xref="S3.p2.9.m9.1.1.3">𝑚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.9.m9.1c">w_{P}\odot m</annotation></semantics></math> is fine-tuned for <math id="S3.p2.10.m10.1" class="ltx_Math" alttext="T" display="inline"><semantics id="S3.p2.10.m10.1a"><mi id="S3.p2.10.m10.1.1" xref="S3.p2.10.m10.1.1.cmml">T</mi><annotation-xml encoding="MathML-Content" id="S3.p2.10.m10.1b"><ci id="S3.p2.10.m10.1.1.cmml" xref="S3.p2.10.m10.1.1">𝑇</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.10.m10.1c">T</annotation></semantics></math> iterations, while leaving the remaining parameters <math id="S3.p2.11.m11.1" class="ltx_Math" alttext="w_{P}\odot(1-m)" display="inline"><semantics id="S3.p2.11.m11.1a"><mrow id="S3.p2.11.m11.1.1" xref="S3.p2.11.m11.1.1.cmml"><msub id="S3.p2.11.m11.1.1.3" xref="S3.p2.11.m11.1.1.3.cmml"><mi id="S3.p2.11.m11.1.1.3.2" xref="S3.p2.11.m11.1.1.3.2.cmml">w</mi><mi id="S3.p2.11.m11.1.1.3.3" xref="S3.p2.11.m11.1.1.3.3.cmml">P</mi></msub><mo lspace="0.222em" rspace="0.222em" id="S3.p2.11.m11.1.1.2" xref="S3.p2.11.m11.1.1.2.cmml">⊙</mo><mrow id="S3.p2.11.m11.1.1.1.1" xref="S3.p2.11.m11.1.1.1.1.1.cmml"><mo stretchy="false" id="S3.p2.11.m11.1.1.1.1.2" xref="S3.p2.11.m11.1.1.1.1.1.cmml">(</mo><mrow id="S3.p2.11.m11.1.1.1.1.1" xref="S3.p2.11.m11.1.1.1.1.1.cmml"><mn id="S3.p2.11.m11.1.1.1.1.1.2" xref="S3.p2.11.m11.1.1.1.1.1.2.cmml">1</mn><mo id="S3.p2.11.m11.1.1.1.1.1.1" xref="S3.p2.11.m11.1.1.1.1.1.1.cmml">−</mo><mi id="S3.p2.11.m11.1.1.1.1.1.3" xref="S3.p2.11.m11.1.1.1.1.1.3.cmml">m</mi></mrow><mo stretchy="false" id="S3.p2.11.m11.1.1.1.1.3" xref="S3.p2.11.m11.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.p2.11.m11.1b"><apply id="S3.p2.11.m11.1.1.cmml" xref="S3.p2.11.m11.1.1"><csymbol cd="latexml" id="S3.p2.11.m11.1.1.2.cmml" xref="S3.p2.11.m11.1.1.2">direct-product</csymbol><apply id="S3.p2.11.m11.1.1.3.cmml" xref="S3.p2.11.m11.1.1.3"><csymbol cd="ambiguous" id="S3.p2.11.m11.1.1.3.1.cmml" xref="S3.p2.11.m11.1.1.3">subscript</csymbol><ci id="S3.p2.11.m11.1.1.3.2.cmml" xref="S3.p2.11.m11.1.1.3.2">𝑤</ci><ci id="S3.p2.11.m11.1.1.3.3.cmml" xref="S3.p2.11.m11.1.1.3.3">𝑃</ci></apply><apply id="S3.p2.11.m11.1.1.1.1.1.cmml" xref="S3.p2.11.m11.1.1.1.1"><minus id="S3.p2.11.m11.1.1.1.1.1.1.cmml" xref="S3.p2.11.m11.1.1.1.1.1.1"></minus><cn type="integer" id="S3.p2.11.m11.1.1.1.1.1.2.cmml" xref="S3.p2.11.m11.1.1.1.1.1.2">1</cn><ci id="S3.p2.11.m11.1.1.1.1.1.3.cmml" xref="S3.p2.11.m11.1.1.1.1.1.3">𝑚</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.11.m11.1c">w_{P}\odot(1-m)</annotation></semantics></math> frozen at their initial values. We summarize the workflow of LoTA in Figure <a href="#S3.F2" title="Figure 2 ‣ 3 Lottery Ticket Adaptation (LoTA) ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> and Algorithm <a href="#alg1" title="Algorithm 1 ‣ 3 Lottery Ticket Adaptation (LoTA) ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> further.</p>
</div>
<div id="S3.p3" class="ltx_para">
<p id="S3.p3.1" class="ltx_p">By confining the adaptation updates within subnetworks (identified by <math id="S3.p3.1.m1.1" class="ltx_Math" alttext="m" display="inline"><semantics id="S3.p3.1.m1.1a"><mi id="S3.p3.1.m1.1.1" xref="S3.p3.1.m1.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="S3.p3.1.m1.1b"><ci id="S3.p3.1.m1.1.1.cmml" xref="S3.p3.1.m1.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.1.m1.1c">m</annotation></semantics></math>), <span id="S3.p3.1.1" class="ltx_text ltx_font_bold">LoTA is able to mitigate destructive interference</span>, e.g., adaptation loss during fine-tuning on future datasets or model merging, that FFT and LoRA suffer from. We discuss this in more detail under three multi-task adaptation paradigms below and provide empirical comparisons with FFT and LoRA in Section <a href="#S5" title="5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>.</p>
</div>
<figure id="S3.F2" class="ltx_figure"><img src="/html/2406.16797/assets/x2.png" id="S3.F2.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="243" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure"><span id="S3.F2.12.6.1" class="ltx_text" style="font-size:90%;">Figure 2</span>: </span><span id="S3.F2.10.5" class="ltx_text" style="font-size:90%;">Lottery Ticket Adaptation (LoTA): (1) Mask calibration via FFT for <math id="S3.F2.6.1.m1.1" class="ltx_Math" alttext="\mathcal{T}" display="inline"><semantics id="S3.F2.6.1.m1.1b"><mi class="ltx_font_mathcaligraphic" id="S3.F2.6.1.m1.1.1" xref="S3.F2.6.1.m1.1.1.cmml">𝒯</mi><annotation-xml encoding="MathML-Content" id="S3.F2.6.1.m1.1c"><ci id="S3.F2.6.1.m1.1.1.cmml" xref="S3.F2.6.1.m1.1.1">𝒯</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.F2.6.1.m1.1d">\mathcal{T}</annotation></semantics></math> iterations, (2) Extracting the sparsity mask <math id="S3.F2.7.2.m2.1" class="ltx_Math" alttext="m" display="inline"><semantics id="S3.F2.7.2.m2.1b"><mi id="S3.F2.7.2.m2.1.1" xref="S3.F2.7.2.m2.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="S3.F2.7.2.m2.1c"><ci id="S3.F2.7.2.m2.1.1.cmml" xref="S3.F2.7.2.m2.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.F2.7.2.m2.1d">m</annotation></semantics></math> from the task vector <math id="S3.F2.8.3.m3.1" class="ltx_Math" alttext="\Delta" display="inline"><semantics id="S3.F2.8.3.m3.1b"><mi mathvariant="normal" id="S3.F2.8.3.m3.1.1" xref="S3.F2.8.3.m3.1.1.cmml">Δ</mi><annotation-xml encoding="MathML-Content" id="S3.F2.8.3.m3.1c"><ci id="S3.F2.8.3.m3.1.1.cmml" xref="S3.F2.8.3.m3.1.1">Δ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.F2.8.3.m3.1d">\Delta</annotation></semantics></math>, (3) Sparse fine-tuning with sparsity mask <math id="S3.F2.9.4.m4.1" class="ltx_Math" alttext="m" display="inline"><semantics id="S3.F2.9.4.m4.1b"><mi id="S3.F2.9.4.m4.1.1" xref="S3.F2.9.4.m4.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="S3.F2.9.4.m4.1c"><ci id="S3.F2.9.4.m4.1.1.cmml" xref="S3.F2.9.4.m4.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.F2.9.4.m4.1d">m</annotation></semantics></math> for <math id="S3.F2.10.5.m5.1" class="ltx_Math" alttext="T" display="inline"><semantics id="S3.F2.10.5.m5.1b"><mi id="S3.F2.10.5.m5.1.1" xref="S3.F2.10.5.m5.1.1.cmml">T</mi><annotation-xml encoding="MathML-Content" id="S3.F2.10.5.m5.1c"><ci id="S3.F2.10.5.m5.1.1.cmml" xref="S3.F2.10.5.m5.1.1">𝑇</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.F2.10.5.m5.1d">T</annotation></semantics></math> iterations.</span></figcaption>
</figure>
<section id="S3.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">(1) Storing &amp; Loading Adapters.</h4>

<div id="S3.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="S3.SS0.SSS0.Px1.p1.1" class="ltx_p">As mentioned before, PEFT methods have emerged to reduce the memory cost of storing and loading adapters. However, the most popular and commonly used PEFT method, LoRA, restricts the adaptation updates to have a low rank, which may be insufficient for complex downstream tasks <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib34" title="" class="ltx_ref">34</a>, <a href="#bib.bib33" title="" class="ltx_ref">33</a>]</cite>. In parallel, recent work <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib42" title="" class="ltx_ref">42</a>]</cite> on compressing the delta between the fine-tuned and pre-trained model <math id="S3.SS0.SSS0.Px1.p1.1.m1.1" class="ltx_Math" alttext="\Delta=w_{F}-w_{P}" display="inline"><semantics id="S3.SS0.SSS0.Px1.p1.1.m1.1a"><mrow id="S3.SS0.SSS0.Px1.p1.1.m1.1.1" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.cmml"><mi mathvariant="normal" id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.2" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.2.cmml">Δ</mi><mo id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.1" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.1.cmml">=</mo><mrow id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.cmml"><msub id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.cmml"><mi id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.2" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.2.cmml">w</mi><mi id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.3" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.3.cmml">F</mi></msub><mo id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.1" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.1.cmml">−</mo><msub id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.cmml"><mi id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.2" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.2.cmml">w</mi><mi id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.3" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.3.cmml">P</mi></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS0.SSS0.Px1.p1.1.m1.1b"><apply id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1"><eq id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.1.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.1"></eq><ci id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.2.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.2">Δ</ci><apply id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3"><minus id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.1.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.1"></minus><apply id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2"><csymbol cd="ambiguous" id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.1.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2">subscript</csymbol><ci id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.2.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.2">𝑤</ci><ci id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.3.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.3">𝐹</ci></apply><apply id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3"><csymbol cd="ambiguous" id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.1.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3">subscript</csymbol><ci id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.2.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.2">𝑤</ci><ci id="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.3.cmml" xref="S3.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.3">𝑃</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS0.SSS0.Px1.p1.1.m1.1c">\Delta=w_{F}-w_{P}</annotation></semantics></math> suggests that FFT updates are highly compressible through a simple magnitude-based sparsification. LoTA exploits this underlying sparsity during fine-tuning and obtains better performance than LoRA, while requiring fewer parameters to be trained. As we will show, LoTA adapters can be stored efficiently because of the small number of parameters that need to be updated. LoTA also requires fewer parameters to be updated in the model when the adapter is applied to the model, so loading a LoTA adapter can be faster than loading a LoRA adapter.</p>
</div>
</section>
<section id="S3.SS0.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">(2) Sequential Training.</h4>

<div id="S3.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="S3.SS0.SSS0.Px2.p1.1" class="ltx_p">Existing fine-tuning methods, such as FFT and LoRA, are known to cause catastrophic forgetting <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib37" title="" class="ltx_ref">37</a>]</cite>. This is particularly concerning for safety alignment as any safety measure the model developers add could be erased by further fine-tuning. By restricting the task vectors to be sparse, LoTA provides robustness against catastrophic forgetting. This is because it will be less likely that an adaptation to new data requires modifying the weights needed for the previous task when fewer weights are updated. As LoTA prevents destructive interference between sequential tasks via sparse and disjoint task vectors, the same phenomenon is also helpful in adapting to new tasks. To further enhance the robustness against destructive interference in sequential training, we propose <span id="S3.SS0.SSS0.Px2.p1.1.1" class="ltx_text ltx_font_bold">Lottery Ticket Together Optimization (LoTTO)</span> which learns mutually sparse (i.e., non-overlapping) masks for sequentially learned tasks.</p>
</div>
<figure id="S3.SS0.SSS0.Px2.fig1" class="ltx_figure">
<figure id="alg2" class="ltx_float ltx_minipage ltx_align_middle ltx_framed ltx_framed_top" style="width:232.0pt;">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_float"><span id="alg2.2.1.1" class="ltx_text ltx_font_bold">Algorithm 2</span> </span> Lottery Ticket Together Optimization (LoTTO)</figcaption>
<div id="alg2.3" class="ltx_listing ltx_listing">
<div id="alg2.l0" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l0.1.1.1" class="ltx_text" style="font-size:80%;">0:</span></span>  Alignment dataset sequence <math id="alg2.l0.m1.1" class="ltx_Math" alttext="\mathbb{D}_{i=1}^{N}" display="inline"><semantics id="alg2.l0.m1.1a"><msubsup id="alg2.l0.m1.1.1" xref="alg2.l0.m1.1.1.cmml"><mi id="alg2.l0.m1.1.1.2.2" xref="alg2.l0.m1.1.1.2.2.cmml">𝔻</mi><mrow id="alg2.l0.m1.1.1.2.3" xref="alg2.l0.m1.1.1.2.3.cmml"><mi id="alg2.l0.m1.1.1.2.3.2" xref="alg2.l0.m1.1.1.2.3.2.cmml">i</mi><mo id="alg2.l0.m1.1.1.2.3.1" xref="alg2.l0.m1.1.1.2.3.1.cmml">=</mo><mn id="alg2.l0.m1.1.1.2.3.3" xref="alg2.l0.m1.1.1.2.3.3.cmml">1</mn></mrow><mi id="alg2.l0.m1.1.1.3" xref="alg2.l0.m1.1.1.3.cmml">N</mi></msubsup><annotation-xml encoding="MathML-Content" id="alg2.l0.m1.1b"><apply id="alg2.l0.m1.1.1.cmml" xref="alg2.l0.m1.1.1"><csymbol cd="ambiguous" id="alg2.l0.m1.1.1.1.cmml" xref="alg2.l0.m1.1.1">superscript</csymbol><apply id="alg2.l0.m1.1.1.2.cmml" xref="alg2.l0.m1.1.1"><csymbol cd="ambiguous" id="alg2.l0.m1.1.1.2.1.cmml" xref="alg2.l0.m1.1.1">subscript</csymbol><ci id="alg2.l0.m1.1.1.2.2.cmml" xref="alg2.l0.m1.1.1.2.2">𝔻</ci><apply id="alg2.l0.m1.1.1.2.3.cmml" xref="alg2.l0.m1.1.1.2.3"><eq id="alg2.l0.m1.1.1.2.3.1.cmml" xref="alg2.l0.m1.1.1.2.3.1"></eq><ci id="alg2.l0.m1.1.1.2.3.2.cmml" xref="alg2.l0.m1.1.1.2.3.2">𝑖</ci><cn type="integer" id="alg2.l0.m1.1.1.2.3.3.cmml" xref="alg2.l0.m1.1.1.2.3.3">1</cn></apply></apply><ci id="alg2.l0.m1.1.1.3.cmml" xref="alg2.l0.m1.1.1.3">𝑁</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l0.m1.1c">\mathbb{D}_{i=1}^{N}</annotation></semantics></math>, pre-trained weights <math id="alg2.l0.m2.1" class="ltx_Math" alttext="w_{P}" display="inline"><semantics id="alg2.l0.m2.1a"><msub id="alg2.l0.m2.1.1" xref="alg2.l0.m2.1.1.cmml"><mi id="alg2.l0.m2.1.1.2" xref="alg2.l0.m2.1.1.2.cmml">w</mi><mi id="alg2.l0.m2.1.1.3" xref="alg2.l0.m2.1.1.3.cmml">P</mi></msub><annotation-xml encoding="MathML-Content" id="alg2.l0.m2.1b"><apply id="alg2.l0.m2.1.1.cmml" xref="alg2.l0.m2.1.1"><csymbol cd="ambiguous" id="alg2.l0.m2.1.1.1.cmml" xref="alg2.l0.m2.1.1">subscript</csymbol><ci id="alg2.l0.m2.1.1.2.cmml" xref="alg2.l0.m2.1.1.2">𝑤</ci><ci id="alg2.l0.m2.1.1.3.cmml" xref="alg2.l0.m2.1.1.3">𝑃</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l0.m2.1c">w_{P}</annotation></semantics></math>, LoTA hyperparameter set <math id="alg2.l0.m3.1" class="ltx_Math" alttext="\mathbb{H}" display="inline"><semantics id="alg2.l0.m3.1a"><mi id="alg2.l0.m3.1.1" xref="alg2.l0.m3.1.1.cmml">ℍ</mi><annotation-xml encoding="MathML-Content" id="alg2.l0.m3.1b"><ci id="alg2.l0.m3.1.1.cmml" xref="alg2.l0.m3.1.1">ℍ</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l0.m3.1c">\mathbb{H}</annotation></semantics></math>.

</div>
<div id="alg2.l1" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l1.1.1.1" class="ltx_text" style="font-size:80%;">1:</span></span>  <math id="alg2.l1.m1.1" class="ltx_Math" alttext="\mathbb{C}=0" display="inline"><semantics id="alg2.l1.m1.1a"><mrow id="alg2.l1.m1.1.1" xref="alg2.l1.m1.1.1.cmml"><mi id="alg2.l1.m1.1.1.2" xref="alg2.l1.m1.1.1.2.cmml">ℂ</mi><mo id="alg2.l1.m1.1.1.1" xref="alg2.l1.m1.1.1.1.cmml">=</mo><mn id="alg2.l1.m1.1.1.3" xref="alg2.l1.m1.1.1.3.cmml">0</mn></mrow><annotation-xml encoding="MathML-Content" id="alg2.l1.m1.1b"><apply id="alg2.l1.m1.1.1.cmml" xref="alg2.l1.m1.1.1"><eq id="alg2.l1.m1.1.1.1.cmml" xref="alg2.l1.m1.1.1.1"></eq><ci id="alg2.l1.m1.1.1.2.cmml" xref="alg2.l1.m1.1.1.2">ℂ</ci><cn type="integer" id="alg2.l1.m1.1.1.3.cmml" xref="alg2.l1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l1.m1.1c">\mathbb{C}=0</annotation></semantics></math> {Initialize constraint set}

</div>
<div id="alg2.l2" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l2.1.1.1" class="ltx_text" style="font-size:80%;">2:</span></span>  <math id="alg2.l2.m1.1" class="ltx_Math" alttext="w_{i}=w_{P}" display="inline"><semantics id="alg2.l2.m1.1a"><mrow id="alg2.l2.m1.1.1" xref="alg2.l2.m1.1.1.cmml"><msub id="alg2.l2.m1.1.1.2" xref="alg2.l2.m1.1.1.2.cmml"><mi id="alg2.l2.m1.1.1.2.2" xref="alg2.l2.m1.1.1.2.2.cmml">w</mi><mi id="alg2.l2.m1.1.1.2.3" xref="alg2.l2.m1.1.1.2.3.cmml">i</mi></msub><mo id="alg2.l2.m1.1.1.1" xref="alg2.l2.m1.1.1.1.cmml">=</mo><msub id="alg2.l2.m1.1.1.3" xref="alg2.l2.m1.1.1.3.cmml"><mi id="alg2.l2.m1.1.1.3.2" xref="alg2.l2.m1.1.1.3.2.cmml">w</mi><mi id="alg2.l2.m1.1.1.3.3" xref="alg2.l2.m1.1.1.3.3.cmml">P</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="alg2.l2.m1.1b"><apply id="alg2.l2.m1.1.1.cmml" xref="alg2.l2.m1.1.1"><eq id="alg2.l2.m1.1.1.1.cmml" xref="alg2.l2.m1.1.1.1"></eq><apply id="alg2.l2.m1.1.1.2.cmml" xref="alg2.l2.m1.1.1.2"><csymbol cd="ambiguous" id="alg2.l2.m1.1.1.2.1.cmml" xref="alg2.l2.m1.1.1.2">subscript</csymbol><ci id="alg2.l2.m1.1.1.2.2.cmml" xref="alg2.l2.m1.1.1.2.2">𝑤</ci><ci id="alg2.l2.m1.1.1.2.3.cmml" xref="alg2.l2.m1.1.1.2.3">𝑖</ci></apply><apply id="alg2.l2.m1.1.1.3.cmml" xref="alg2.l2.m1.1.1.3"><csymbol cd="ambiguous" id="alg2.l2.m1.1.1.3.1.cmml" xref="alg2.l2.m1.1.1.3">subscript</csymbol><ci id="alg2.l2.m1.1.1.3.2.cmml" xref="alg2.l2.m1.1.1.3.2">𝑤</ci><ci id="alg2.l2.m1.1.1.3.3.cmml" xref="alg2.l2.m1.1.1.3.3">𝑃</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l2.m1.1c">w_{i}=w_{P}</annotation></semantics></math>

</div>
<div id="alg2.l3" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l3.1.1.1" class="ltx_text" style="font-size:80%;">3:</span></span>  <span id="alg2.l3.2" class="ltx_text ltx_font_bold">for</span> <math id="alg2.l3.m1.1" class="ltx_Math" alttext="\mathbb{D}\in\mathbb{D}_{i=1}^{N}" display="inline"><semantics id="alg2.l3.m1.1a"><mrow id="alg2.l3.m1.1.1" xref="alg2.l3.m1.1.1.cmml"><mi id="alg2.l3.m1.1.1.2" xref="alg2.l3.m1.1.1.2.cmml">𝔻</mi><mo id="alg2.l3.m1.1.1.1" xref="alg2.l3.m1.1.1.1.cmml">∈</mo><msubsup id="alg2.l3.m1.1.1.3" xref="alg2.l3.m1.1.1.3.cmml"><mi id="alg2.l3.m1.1.1.3.2.2" xref="alg2.l3.m1.1.1.3.2.2.cmml">𝔻</mi><mrow id="alg2.l3.m1.1.1.3.2.3" xref="alg2.l3.m1.1.1.3.2.3.cmml"><mi id="alg2.l3.m1.1.1.3.2.3.2" xref="alg2.l3.m1.1.1.3.2.3.2.cmml">i</mi><mo id="alg2.l3.m1.1.1.3.2.3.1" xref="alg2.l3.m1.1.1.3.2.3.1.cmml">=</mo><mn id="alg2.l3.m1.1.1.3.2.3.3" xref="alg2.l3.m1.1.1.3.2.3.3.cmml">1</mn></mrow><mi id="alg2.l3.m1.1.1.3.3" xref="alg2.l3.m1.1.1.3.3.cmml">N</mi></msubsup></mrow><annotation-xml encoding="MathML-Content" id="alg2.l3.m1.1b"><apply id="alg2.l3.m1.1.1.cmml" xref="alg2.l3.m1.1.1"><in id="alg2.l3.m1.1.1.1.cmml" xref="alg2.l3.m1.1.1.1"></in><ci id="alg2.l3.m1.1.1.2.cmml" xref="alg2.l3.m1.1.1.2">𝔻</ci><apply id="alg2.l3.m1.1.1.3.cmml" xref="alg2.l3.m1.1.1.3"><csymbol cd="ambiguous" id="alg2.l3.m1.1.1.3.1.cmml" xref="alg2.l3.m1.1.1.3">superscript</csymbol><apply id="alg2.l3.m1.1.1.3.2.cmml" xref="alg2.l3.m1.1.1.3"><csymbol cd="ambiguous" id="alg2.l3.m1.1.1.3.2.1.cmml" xref="alg2.l3.m1.1.1.3">subscript</csymbol><ci id="alg2.l3.m1.1.1.3.2.2.cmml" xref="alg2.l3.m1.1.1.3.2.2">𝔻</ci><apply id="alg2.l3.m1.1.1.3.2.3.cmml" xref="alg2.l3.m1.1.1.3.2.3"><eq id="alg2.l3.m1.1.1.3.2.3.1.cmml" xref="alg2.l3.m1.1.1.3.2.3.1"></eq><ci id="alg2.l3.m1.1.1.3.2.3.2.cmml" xref="alg2.l3.m1.1.1.3.2.3.2">𝑖</ci><cn type="integer" id="alg2.l3.m1.1.1.3.2.3.3.cmml" xref="alg2.l3.m1.1.1.3.2.3.3">1</cn></apply></apply><ci id="alg2.l3.m1.1.1.3.3.cmml" xref="alg2.l3.m1.1.1.3.3">𝑁</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l3.m1.1c">\mathbb{D}\in\mathbb{D}_{i=1}^{N}</annotation></semantics></math> <span id="alg2.l3.3" class="ltx_text ltx_font_bold">do</span>



</div>
<div id="alg2.l4" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l4.1.1.1" class="ltx_text" style="font-size:80%;">4:</span></span>     <math id="alg2.l4.m1.1" class="ltx_Math" alttext="m_{i}^{C}=C" display="inline"><semantics id="alg2.l4.m1.1a"><mrow id="alg2.l4.m1.1.1" xref="alg2.l4.m1.1.1.cmml"><msubsup id="alg2.l4.m1.1.1.2" xref="alg2.l4.m1.1.1.2.cmml"><mi id="alg2.l4.m1.1.1.2.2.2" xref="alg2.l4.m1.1.1.2.2.2.cmml">m</mi><mi id="alg2.l4.m1.1.1.2.2.3" xref="alg2.l4.m1.1.1.2.2.3.cmml">i</mi><mi id="alg2.l4.m1.1.1.2.3" xref="alg2.l4.m1.1.1.2.3.cmml">C</mi></msubsup><mo id="alg2.l4.m1.1.1.1" xref="alg2.l4.m1.1.1.1.cmml">=</mo><mi id="alg2.l4.m1.1.1.3" xref="alg2.l4.m1.1.1.3.cmml">C</mi></mrow><annotation-xml encoding="MathML-Content" id="alg2.l4.m1.1b"><apply id="alg2.l4.m1.1.1.cmml" xref="alg2.l4.m1.1.1"><eq id="alg2.l4.m1.1.1.1.cmml" xref="alg2.l4.m1.1.1.1"></eq><apply id="alg2.l4.m1.1.1.2.cmml" xref="alg2.l4.m1.1.1.2"><csymbol cd="ambiguous" id="alg2.l4.m1.1.1.2.1.cmml" xref="alg2.l4.m1.1.1.2">superscript</csymbol><apply id="alg2.l4.m1.1.1.2.2.cmml" xref="alg2.l4.m1.1.1.2"><csymbol cd="ambiguous" id="alg2.l4.m1.1.1.2.2.1.cmml" xref="alg2.l4.m1.1.1.2">subscript</csymbol><ci id="alg2.l4.m1.1.1.2.2.2.cmml" xref="alg2.l4.m1.1.1.2.2.2">𝑚</ci><ci id="alg2.l4.m1.1.1.2.2.3.cmml" xref="alg2.l4.m1.1.1.2.2.3">𝑖</ci></apply><ci id="alg2.l4.m1.1.1.2.3.cmml" xref="alg2.l4.m1.1.1.2.3">𝐶</ci></apply><ci id="alg2.l4.m1.1.1.3.cmml" xref="alg2.l4.m1.1.1.3">𝐶</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l4.m1.1c">m_{i}^{C}=C</annotation></semantics></math> {Create mask from constraint set}

</div>
<div id="alg2.l5" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l5.1.1.1" class="ltx_text" style="font-size:80%;">5:</span></span>     <math id="alg2.l5.m1.1" class="ltx_Math" alttext="w_{i}^{C}=w_{i}-\nabla\odot m_{i}^{C}" display="inline"><semantics id="alg2.l5.m1.1a"><mrow id="alg2.l5.m1.1.1" xref="alg2.l5.m1.1.1.cmml"><msubsup id="alg2.l5.m1.1.1.2" xref="alg2.l5.m1.1.1.2.cmml"><mi id="alg2.l5.m1.1.1.2.2.2" xref="alg2.l5.m1.1.1.2.2.2.cmml">w</mi><mi id="alg2.l5.m1.1.1.2.2.3" xref="alg2.l5.m1.1.1.2.2.3.cmml">i</mi><mi id="alg2.l5.m1.1.1.2.3" xref="alg2.l5.m1.1.1.2.3.cmml">C</mi></msubsup><mo id="alg2.l5.m1.1.1.1" xref="alg2.l5.m1.1.1.1.cmml">=</mo><mrow id="alg2.l5.m1.1.1.3" xref="alg2.l5.m1.1.1.3.cmml"><msub id="alg2.l5.m1.1.1.3.2" xref="alg2.l5.m1.1.1.3.2.cmml"><mi id="alg2.l5.m1.1.1.3.2.2" xref="alg2.l5.m1.1.1.3.2.2.cmml">w</mi><mi id="alg2.l5.m1.1.1.3.2.3" xref="alg2.l5.m1.1.1.3.2.3.cmml">i</mi></msub><mo id="alg2.l5.m1.1.1.3.1" xref="alg2.l5.m1.1.1.3.1.cmml">−</mo><mrow id="alg2.l5.m1.1.1.3.3" xref="alg2.l5.m1.1.1.3.3.cmml"><mo id="alg2.l5.m1.1.1.3.3.2" xref="alg2.l5.m1.1.1.3.3.2.cmml">∇</mo><mo lspace="0em" rspace="0.222em" id="alg2.l5.m1.1.1.3.3.1" xref="alg2.l5.m1.1.1.3.3.1.cmml">⊙</mo><msubsup id="alg2.l5.m1.1.1.3.3.3" xref="alg2.l5.m1.1.1.3.3.3.cmml"><mi id="alg2.l5.m1.1.1.3.3.3.2.2" xref="alg2.l5.m1.1.1.3.3.3.2.2.cmml">m</mi><mi id="alg2.l5.m1.1.1.3.3.3.2.3" xref="alg2.l5.m1.1.1.3.3.3.2.3.cmml">i</mi><mi id="alg2.l5.m1.1.1.3.3.3.3" xref="alg2.l5.m1.1.1.3.3.3.3.cmml">C</mi></msubsup></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l5.m1.1b"><apply id="alg2.l5.m1.1.1.cmml" xref="alg2.l5.m1.1.1"><eq id="alg2.l5.m1.1.1.1.cmml" xref="alg2.l5.m1.1.1.1"></eq><apply id="alg2.l5.m1.1.1.2.cmml" xref="alg2.l5.m1.1.1.2"><csymbol cd="ambiguous" id="alg2.l5.m1.1.1.2.1.cmml" xref="alg2.l5.m1.1.1.2">superscript</csymbol><apply id="alg2.l5.m1.1.1.2.2.cmml" xref="alg2.l5.m1.1.1.2"><csymbol cd="ambiguous" id="alg2.l5.m1.1.1.2.2.1.cmml" xref="alg2.l5.m1.1.1.2">subscript</csymbol><ci id="alg2.l5.m1.1.1.2.2.2.cmml" xref="alg2.l5.m1.1.1.2.2.2">𝑤</ci><ci id="alg2.l5.m1.1.1.2.2.3.cmml" xref="alg2.l5.m1.1.1.2.2.3">𝑖</ci></apply><ci id="alg2.l5.m1.1.1.2.3.cmml" xref="alg2.l5.m1.1.1.2.3">𝐶</ci></apply><apply id="alg2.l5.m1.1.1.3.cmml" xref="alg2.l5.m1.1.1.3"><minus id="alg2.l5.m1.1.1.3.1.cmml" xref="alg2.l5.m1.1.1.3.1"></minus><apply id="alg2.l5.m1.1.1.3.2.cmml" xref="alg2.l5.m1.1.1.3.2"><csymbol cd="ambiguous" id="alg2.l5.m1.1.1.3.2.1.cmml" xref="alg2.l5.m1.1.1.3.2">subscript</csymbol><ci id="alg2.l5.m1.1.1.3.2.2.cmml" xref="alg2.l5.m1.1.1.3.2.2">𝑤</ci><ci id="alg2.l5.m1.1.1.3.2.3.cmml" xref="alg2.l5.m1.1.1.3.2.3">𝑖</ci></apply><apply id="alg2.l5.m1.1.1.3.3.cmml" xref="alg2.l5.m1.1.1.3.3"><csymbol cd="latexml" id="alg2.l5.m1.1.1.3.3.1.cmml" xref="alg2.l5.m1.1.1.3.3.1">direct-product</csymbol><ci id="alg2.l5.m1.1.1.3.3.2.cmml" xref="alg2.l5.m1.1.1.3.3.2">∇</ci><apply id="alg2.l5.m1.1.1.3.3.3.cmml" xref="alg2.l5.m1.1.1.3.3.3"><csymbol cd="ambiguous" id="alg2.l5.m1.1.1.3.3.3.1.cmml" xref="alg2.l5.m1.1.1.3.3.3">superscript</csymbol><apply id="alg2.l5.m1.1.1.3.3.3.2.cmml" xref="alg2.l5.m1.1.1.3.3.3"><csymbol cd="ambiguous" id="alg2.l5.m1.1.1.3.3.3.2.1.cmml" xref="alg2.l5.m1.1.1.3.3.3">subscript</csymbol><ci id="alg2.l5.m1.1.1.3.3.3.2.2.cmml" xref="alg2.l5.m1.1.1.3.3.3.2.2">𝑚</ci><ci id="alg2.l5.m1.1.1.3.3.3.2.3.cmml" xref="alg2.l5.m1.1.1.3.3.3.2.3">𝑖</ci></apply><ci id="alg2.l5.m1.1.1.3.3.3.3.cmml" xref="alg2.l5.m1.1.1.3.3.3.3">𝐶</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l5.m1.1c">w_{i}^{C}=w_{i}-\nabla\odot m_{i}^{C}</annotation></semantics></math> {Train model with constraint mask}

</div>
<div id="alg2.l6" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l6.1.1.1" class="ltx_text" style="font-size:80%;">6:</span></span>     <math id="alg2.l6.m1.1" class="ltx_Math" alttext="m_{i}^{S}=\texttt{Sparsify}(w_{i}^{C}-w_{i})" display="inline"><semantics id="alg2.l6.m1.1a"><mrow id="alg2.l6.m1.1.1" xref="alg2.l6.m1.1.1.cmml"><msubsup id="alg2.l6.m1.1.1.3" xref="alg2.l6.m1.1.1.3.cmml"><mi id="alg2.l6.m1.1.1.3.2.2" xref="alg2.l6.m1.1.1.3.2.2.cmml">m</mi><mi id="alg2.l6.m1.1.1.3.2.3" xref="alg2.l6.m1.1.1.3.2.3.cmml">i</mi><mi id="alg2.l6.m1.1.1.3.3" xref="alg2.l6.m1.1.1.3.3.cmml">S</mi></msubsup><mo id="alg2.l6.m1.1.1.2" xref="alg2.l6.m1.1.1.2.cmml">=</mo><mrow id="alg2.l6.m1.1.1.1" xref="alg2.l6.m1.1.1.1.cmml"><mtext class="ltx_mathvariant_monospace" id="alg2.l6.m1.1.1.1.3" xref="alg2.l6.m1.1.1.1.3a.cmml">Sparsify</mtext><mo lspace="0em" rspace="0em" id="alg2.l6.m1.1.1.1.2" xref="alg2.l6.m1.1.1.1.2.cmml">​</mo><mrow id="alg2.l6.m1.1.1.1.1.1" xref="alg2.l6.m1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="alg2.l6.m1.1.1.1.1.1.2" xref="alg2.l6.m1.1.1.1.1.1.1.cmml">(</mo><mrow id="alg2.l6.m1.1.1.1.1.1.1" xref="alg2.l6.m1.1.1.1.1.1.1.cmml"><msubsup id="alg2.l6.m1.1.1.1.1.1.1.2" xref="alg2.l6.m1.1.1.1.1.1.1.2.cmml"><mi id="alg2.l6.m1.1.1.1.1.1.1.2.2.2" xref="alg2.l6.m1.1.1.1.1.1.1.2.2.2.cmml">w</mi><mi id="alg2.l6.m1.1.1.1.1.1.1.2.2.3" xref="alg2.l6.m1.1.1.1.1.1.1.2.2.3.cmml">i</mi><mi id="alg2.l6.m1.1.1.1.1.1.1.2.3" xref="alg2.l6.m1.1.1.1.1.1.1.2.3.cmml">C</mi></msubsup><mo id="alg2.l6.m1.1.1.1.1.1.1.1" xref="alg2.l6.m1.1.1.1.1.1.1.1.cmml">−</mo><msub id="alg2.l6.m1.1.1.1.1.1.1.3" xref="alg2.l6.m1.1.1.1.1.1.1.3.cmml"><mi id="alg2.l6.m1.1.1.1.1.1.1.3.2" xref="alg2.l6.m1.1.1.1.1.1.1.3.2.cmml">w</mi><mi id="alg2.l6.m1.1.1.1.1.1.1.3.3" xref="alg2.l6.m1.1.1.1.1.1.1.3.3.cmml">i</mi></msub></mrow><mo stretchy="false" id="alg2.l6.m1.1.1.1.1.1.3" xref="alg2.l6.m1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l6.m1.1b"><apply id="alg2.l6.m1.1.1.cmml" xref="alg2.l6.m1.1.1"><eq id="alg2.l6.m1.1.1.2.cmml" xref="alg2.l6.m1.1.1.2"></eq><apply id="alg2.l6.m1.1.1.3.cmml" xref="alg2.l6.m1.1.1.3"><csymbol cd="ambiguous" id="alg2.l6.m1.1.1.3.1.cmml" xref="alg2.l6.m1.1.1.3">superscript</csymbol><apply id="alg2.l6.m1.1.1.3.2.cmml" xref="alg2.l6.m1.1.1.3"><csymbol cd="ambiguous" id="alg2.l6.m1.1.1.3.2.1.cmml" xref="alg2.l6.m1.1.1.3">subscript</csymbol><ci id="alg2.l6.m1.1.1.3.2.2.cmml" xref="alg2.l6.m1.1.1.3.2.2">𝑚</ci><ci id="alg2.l6.m1.1.1.3.2.3.cmml" xref="alg2.l6.m1.1.1.3.2.3">𝑖</ci></apply><ci id="alg2.l6.m1.1.1.3.3.cmml" xref="alg2.l6.m1.1.1.3.3">𝑆</ci></apply><apply id="alg2.l6.m1.1.1.1.cmml" xref="alg2.l6.m1.1.1.1"><times id="alg2.l6.m1.1.1.1.2.cmml" xref="alg2.l6.m1.1.1.1.2"></times><ci id="alg2.l6.m1.1.1.1.3a.cmml" xref="alg2.l6.m1.1.1.1.3"><mtext class="ltx_mathvariant_monospace" id="alg2.l6.m1.1.1.1.3.cmml" xref="alg2.l6.m1.1.1.1.3">Sparsify</mtext></ci><apply id="alg2.l6.m1.1.1.1.1.1.1.cmml" xref="alg2.l6.m1.1.1.1.1.1"><minus id="alg2.l6.m1.1.1.1.1.1.1.1.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.1"></minus><apply id="alg2.l6.m1.1.1.1.1.1.1.2.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="alg2.l6.m1.1.1.1.1.1.1.2.1.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.2">superscript</csymbol><apply id="alg2.l6.m1.1.1.1.1.1.1.2.2.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="alg2.l6.m1.1.1.1.1.1.1.2.2.1.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.2">subscript</csymbol><ci id="alg2.l6.m1.1.1.1.1.1.1.2.2.2.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.2.2.2">𝑤</ci><ci id="alg2.l6.m1.1.1.1.1.1.1.2.2.3.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.2.2.3">𝑖</ci></apply><ci id="alg2.l6.m1.1.1.1.1.1.1.2.3.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.2.3">𝐶</ci></apply><apply id="alg2.l6.m1.1.1.1.1.1.1.3.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="alg2.l6.m1.1.1.1.1.1.1.3.1.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.3">subscript</csymbol><ci id="alg2.l6.m1.1.1.1.1.1.1.3.2.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.3.2">𝑤</ci><ci id="alg2.l6.m1.1.1.1.1.1.1.3.3.cmml" xref="alg2.l6.m1.1.1.1.1.1.1.3.3">𝑖</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l6.m1.1c">m_{i}^{S}=\texttt{Sparsify}(w_{i}^{C}-w_{i})</annotation></semantics></math> {Extract sparse mask from trained model}

</div>
<div id="alg2.l7" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l7.1.1.1" class="ltx_text" style="font-size:80%;">7:</span></span>     <math id="alg2.l7.m1.1" class="ltx_Math" alttext="w_{i}^{F}=w_{i}-\nabla\odot m_{i}^{S}" display="inline"><semantics id="alg2.l7.m1.1a"><mrow id="alg2.l7.m1.1.1" xref="alg2.l7.m1.1.1.cmml"><msubsup id="alg2.l7.m1.1.1.2" xref="alg2.l7.m1.1.1.2.cmml"><mi id="alg2.l7.m1.1.1.2.2.2" xref="alg2.l7.m1.1.1.2.2.2.cmml">w</mi><mi id="alg2.l7.m1.1.1.2.2.3" xref="alg2.l7.m1.1.1.2.2.3.cmml">i</mi><mi id="alg2.l7.m1.1.1.2.3" xref="alg2.l7.m1.1.1.2.3.cmml">F</mi></msubsup><mo id="alg2.l7.m1.1.1.1" xref="alg2.l7.m1.1.1.1.cmml">=</mo><mrow id="alg2.l7.m1.1.1.3" xref="alg2.l7.m1.1.1.3.cmml"><msub id="alg2.l7.m1.1.1.3.2" xref="alg2.l7.m1.1.1.3.2.cmml"><mi id="alg2.l7.m1.1.1.3.2.2" xref="alg2.l7.m1.1.1.3.2.2.cmml">w</mi><mi id="alg2.l7.m1.1.1.3.2.3" xref="alg2.l7.m1.1.1.3.2.3.cmml">i</mi></msub><mo id="alg2.l7.m1.1.1.3.1" xref="alg2.l7.m1.1.1.3.1.cmml">−</mo><mrow id="alg2.l7.m1.1.1.3.3" xref="alg2.l7.m1.1.1.3.3.cmml"><mo id="alg2.l7.m1.1.1.3.3.2" xref="alg2.l7.m1.1.1.3.3.2.cmml">∇</mo><mo lspace="0em" rspace="0.222em" id="alg2.l7.m1.1.1.3.3.1" xref="alg2.l7.m1.1.1.3.3.1.cmml">⊙</mo><msubsup id="alg2.l7.m1.1.1.3.3.3" xref="alg2.l7.m1.1.1.3.3.3.cmml"><mi id="alg2.l7.m1.1.1.3.3.3.2.2" xref="alg2.l7.m1.1.1.3.3.3.2.2.cmml">m</mi><mi id="alg2.l7.m1.1.1.3.3.3.2.3" xref="alg2.l7.m1.1.1.3.3.3.2.3.cmml">i</mi><mi id="alg2.l7.m1.1.1.3.3.3.3" xref="alg2.l7.m1.1.1.3.3.3.3.cmml">S</mi></msubsup></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l7.m1.1b"><apply id="alg2.l7.m1.1.1.cmml" xref="alg2.l7.m1.1.1"><eq id="alg2.l7.m1.1.1.1.cmml" xref="alg2.l7.m1.1.1.1"></eq><apply id="alg2.l7.m1.1.1.2.cmml" xref="alg2.l7.m1.1.1.2"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.2.1.cmml" xref="alg2.l7.m1.1.1.2">superscript</csymbol><apply id="alg2.l7.m1.1.1.2.2.cmml" xref="alg2.l7.m1.1.1.2"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.2.2.1.cmml" xref="alg2.l7.m1.1.1.2">subscript</csymbol><ci id="alg2.l7.m1.1.1.2.2.2.cmml" xref="alg2.l7.m1.1.1.2.2.2">𝑤</ci><ci id="alg2.l7.m1.1.1.2.2.3.cmml" xref="alg2.l7.m1.1.1.2.2.3">𝑖</ci></apply><ci id="alg2.l7.m1.1.1.2.3.cmml" xref="alg2.l7.m1.1.1.2.3">𝐹</ci></apply><apply id="alg2.l7.m1.1.1.3.cmml" xref="alg2.l7.m1.1.1.3"><minus id="alg2.l7.m1.1.1.3.1.cmml" xref="alg2.l7.m1.1.1.3.1"></minus><apply id="alg2.l7.m1.1.1.3.2.cmml" xref="alg2.l7.m1.1.1.3.2"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.3.2.1.cmml" xref="alg2.l7.m1.1.1.3.2">subscript</csymbol><ci id="alg2.l7.m1.1.1.3.2.2.cmml" xref="alg2.l7.m1.1.1.3.2.2">𝑤</ci><ci id="alg2.l7.m1.1.1.3.2.3.cmml" xref="alg2.l7.m1.1.1.3.2.3">𝑖</ci></apply><apply id="alg2.l7.m1.1.1.3.3.cmml" xref="alg2.l7.m1.1.1.3.3"><csymbol cd="latexml" id="alg2.l7.m1.1.1.3.3.1.cmml" xref="alg2.l7.m1.1.1.3.3.1">direct-product</csymbol><ci id="alg2.l7.m1.1.1.3.3.2.cmml" xref="alg2.l7.m1.1.1.3.3.2">∇</ci><apply id="alg2.l7.m1.1.1.3.3.3.cmml" xref="alg2.l7.m1.1.1.3.3.3"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.3.3.3.1.cmml" xref="alg2.l7.m1.1.1.3.3.3">superscript</csymbol><apply id="alg2.l7.m1.1.1.3.3.3.2.cmml" xref="alg2.l7.m1.1.1.3.3.3"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.3.3.3.2.1.cmml" xref="alg2.l7.m1.1.1.3.3.3">subscript</csymbol><ci id="alg2.l7.m1.1.1.3.3.3.2.2.cmml" xref="alg2.l7.m1.1.1.3.3.3.2.2">𝑚</ci><ci id="alg2.l7.m1.1.1.3.3.3.2.3.cmml" xref="alg2.l7.m1.1.1.3.3.3.2.3">𝑖</ci></apply><ci id="alg2.l7.m1.1.1.3.3.3.3.cmml" xref="alg2.l7.m1.1.1.3.3.3.3">𝑆</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l7.m1.1c">w_{i}^{F}=w_{i}-\nabla\odot m_{i}^{S}</annotation></semantics></math> {Train model with sparse mask}

</div>
<div id="alg2.l8" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l8.1.1.1" class="ltx_text" style="font-size:80%;">8:</span></span>     <math id="alg2.l8.m1.1" class="ltx_Math" alttext="\mathbb{C}=\mathbb{C}\cup m_{i}^{S}" display="inline"><semantics id="alg2.l8.m1.1a"><mrow id="alg2.l8.m1.1.1" xref="alg2.l8.m1.1.1.cmml"><mi id="alg2.l8.m1.1.1.2" xref="alg2.l8.m1.1.1.2.cmml">ℂ</mi><mo id="alg2.l8.m1.1.1.1" xref="alg2.l8.m1.1.1.1.cmml">=</mo><mrow id="alg2.l8.m1.1.1.3" xref="alg2.l8.m1.1.1.3.cmml"><mi id="alg2.l8.m1.1.1.3.2" xref="alg2.l8.m1.1.1.3.2.cmml">ℂ</mi><mo id="alg2.l8.m1.1.1.3.1" xref="alg2.l8.m1.1.1.3.1.cmml">∪</mo><msubsup id="alg2.l8.m1.1.1.3.3" xref="alg2.l8.m1.1.1.3.3.cmml"><mi id="alg2.l8.m1.1.1.3.3.2.2" xref="alg2.l8.m1.1.1.3.3.2.2.cmml">m</mi><mi id="alg2.l8.m1.1.1.3.3.2.3" xref="alg2.l8.m1.1.1.3.3.2.3.cmml">i</mi><mi id="alg2.l8.m1.1.1.3.3.3" xref="alg2.l8.m1.1.1.3.3.3.cmml">S</mi></msubsup></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l8.m1.1b"><apply id="alg2.l8.m1.1.1.cmml" xref="alg2.l8.m1.1.1"><eq id="alg2.l8.m1.1.1.1.cmml" xref="alg2.l8.m1.1.1.1"></eq><ci id="alg2.l8.m1.1.1.2.cmml" xref="alg2.l8.m1.1.1.2">ℂ</ci><apply id="alg2.l8.m1.1.1.3.cmml" xref="alg2.l8.m1.1.1.3"><union id="alg2.l8.m1.1.1.3.1.cmml" xref="alg2.l8.m1.1.1.3.1"></union><ci id="alg2.l8.m1.1.1.3.2.cmml" xref="alg2.l8.m1.1.1.3.2">ℂ</ci><apply id="alg2.l8.m1.1.1.3.3.cmml" xref="alg2.l8.m1.1.1.3.3"><csymbol cd="ambiguous" id="alg2.l8.m1.1.1.3.3.1.cmml" xref="alg2.l8.m1.1.1.3.3">superscript</csymbol><apply id="alg2.l8.m1.1.1.3.3.2.cmml" xref="alg2.l8.m1.1.1.3.3"><csymbol cd="ambiguous" id="alg2.l8.m1.1.1.3.3.2.1.cmml" xref="alg2.l8.m1.1.1.3.3">subscript</csymbol><ci id="alg2.l8.m1.1.1.3.3.2.2.cmml" xref="alg2.l8.m1.1.1.3.3.2.2">𝑚</ci><ci id="alg2.l8.m1.1.1.3.3.2.3.cmml" xref="alg2.l8.m1.1.1.3.3.2.3">𝑖</ci></apply><ci id="alg2.l8.m1.1.1.3.3.3.cmml" xref="alg2.l8.m1.1.1.3.3.3">𝑆</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l8.m1.1c">\mathbb{C}=\mathbb{C}\cup m_{i}^{S}</annotation></semantics></math> {Update constraints}

</div>
<div id="alg2.l9" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l9.1.1.1" class="ltx_text" style="font-size:80%;">9:</span></span>  <span id="alg2.l9.2" class="ltx_text ltx_font_bold">end</span> <span id="alg2.l9.3" class="ltx_text ltx_font_bold">for</span>
</div>
<div id="alg2.l9a" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline"><span id="alg2.l9a.1.1.1" class="ltx_text" style="font-size:80%;">9:</span></span>  <math id="alg2.l9a.m1.1" class="ltx_Math" alttext="\hat{w}_{N}^{F}" display="inline"><semantics id="alg2.l9a.m1.1a"><msubsup id="alg2.l9a.m1.1.1" xref="alg2.l9a.m1.1.1.cmml"><mover accent="true" id="alg2.l9a.m1.1.1.2.2" xref="alg2.l9a.m1.1.1.2.2.cmml"><mi id="alg2.l9a.m1.1.1.2.2.2" xref="alg2.l9a.m1.1.1.2.2.2.cmml">w</mi><mo id="alg2.l9a.m1.1.1.2.2.1" xref="alg2.l9a.m1.1.1.2.2.1.cmml">^</mo></mover><mi id="alg2.l9a.m1.1.1.2.3" xref="alg2.l9a.m1.1.1.2.3.cmml">N</mi><mi id="alg2.l9a.m1.1.1.3" xref="alg2.l9a.m1.1.1.3.cmml">F</mi></msubsup><annotation-xml encoding="MathML-Content" id="alg2.l9a.m1.1b"><apply id="alg2.l9a.m1.1.1.cmml" xref="alg2.l9a.m1.1.1"><csymbol cd="ambiguous" id="alg2.l9a.m1.1.1.1.cmml" xref="alg2.l9a.m1.1.1">superscript</csymbol><apply id="alg2.l9a.m1.1.1.2.cmml" xref="alg2.l9a.m1.1.1"><csymbol cd="ambiguous" id="alg2.l9a.m1.1.1.2.1.cmml" xref="alg2.l9a.m1.1.1">subscript</csymbol><apply id="alg2.l9a.m1.1.1.2.2.cmml" xref="alg2.l9a.m1.1.1.2.2"><ci id="alg2.l9a.m1.1.1.2.2.1.cmml" xref="alg2.l9a.m1.1.1.2.2.1">^</ci><ci id="alg2.l9a.m1.1.1.2.2.2.cmml" xref="alg2.l9a.m1.1.1.2.2.2">𝑤</ci></apply><ci id="alg2.l9a.m1.1.1.2.3.cmml" xref="alg2.l9a.m1.1.1.2.3">𝑁</ci></apply><ci id="alg2.l9a.m1.1.1.3.cmml" xref="alg2.l9a.m1.1.1.3">𝐹</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l9a.m1.1c">\hat{w}_{N}^{F}</annotation></semantics></math>

</div>
</div>
</figure>
</figure>
</section>
<section id="S3.SS0.SSS0.Px3" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Further Enhancing Sequential Training via Lottery Ticket Together Optimization (LoTTO).</h4>

<div id="S3.SS0.SSS0.Px3.p1" class="ltx_para">
<p id="S3.SS0.SSS0.Px3.p1.2" class="ltx_p">Without loss of generality, suppose we have two tasks, Task A and Task B, and that we have already learned Task A with LoTA. LoTTO calibrates a sparsity mask for Task B by first training a model where the only weights that can be updated are those that are <em id="S3.SS0.SSS0.Px3.p1.2.1" class="ltx_emph ltx_font_italic">not</em> updated when running LoTA on Task A, and then using a sparse set of those weights to train the final model. LoTTO enables learning new tasks without forgetting old ones, by restricting weight adaptations to not interfere with the weights that were important for previous adaptations. This procedure can be applied inductively to enable sequential adaptation to multiple tasks, so that a model developer seeking to adapt a model adapted with LoTA (potentially on several tasks), just needs to ensure that they do not update the task vector with respect to the base model. We envision that LoTTO will be applied to capabilities one at a time; first the model learns math, then coding, then reasoning, then planning, etc. If <math id="S3.SS0.SSS0.Px3.p1.1.m1.1" class="ltx_Math" alttext="1\%" display="inline"><semantics id="S3.SS0.SSS0.Px3.p1.1.m1.1a"><mrow id="S3.SS0.SSS0.Px3.p1.1.m1.1.1" xref="S3.SS0.SSS0.Px3.p1.1.m1.1.1.cmml"><mn id="S3.SS0.SSS0.Px3.p1.1.m1.1.1.2" xref="S3.SS0.SSS0.Px3.p1.1.m1.1.1.2.cmml">1</mn><mo id="S3.SS0.SSS0.Px3.p1.1.m1.1.1.1" xref="S3.SS0.SSS0.Px3.p1.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.SS0.SSS0.Px3.p1.1.m1.1b"><apply id="S3.SS0.SSS0.Px3.p1.1.m1.1.1.cmml" xref="S3.SS0.SSS0.Px3.p1.1.m1.1.1"><csymbol cd="latexml" id="S3.SS0.SSS0.Px3.p1.1.m1.1.1.1.cmml" xref="S3.SS0.SSS0.Px3.p1.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S3.SS0.SSS0.Px3.p1.1.m1.1.1.2.cmml" xref="S3.SS0.SSS0.Px3.p1.1.m1.1.1.2">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS0.SSS0.Px3.p1.1.m1.1c">1\%</annotation></semantics></math> of the weights are updated for each capability, then LoTTO can fit at most <math id="S3.SS0.SSS0.Px3.p1.2.m2.1" class="ltx_Math" alttext="100" display="inline"><semantics id="S3.SS0.SSS0.Px3.p1.2.m2.1a"><mn id="S3.SS0.SSS0.Px3.p1.2.m2.1.1" xref="S3.SS0.SSS0.Px3.p1.2.m2.1.1.cmml">100</mn><annotation-xml encoding="MathML-Content" id="S3.SS0.SSS0.Px3.p1.2.m2.1b"><cn type="integer" id="S3.SS0.SSS0.Px3.p1.2.m2.1.1.cmml" xref="S3.SS0.SSS0.Px3.p1.2.m2.1.1">100</cn></annotation-xml><annotation encoding="application/x-tex" id="S3.SS0.SSS0.Px3.p1.2.m2.1c">100</annotation></semantics></math> capabilities. We believe this is sufficient for nearly all practical applications, but when the subnetworks for different tasks overlap, we do not expect that LoTTO will result in any worse forgetting than LoTA, which we will show itself has better robustness to forgetting than FFT.</p>
</div>
</section>
<section id="S3.SS0.SSS0.Px4" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">(3) Model Merging.</h4>

<div id="S3.SS0.SSS0.Px4.p1" class="ltx_para">
<p id="S3.SS0.SSS0.Px4.p1.1" class="ltx_p">Existing model merging methods typically aim to merge task vectors of relatively similar language datasets <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib17" title="" class="ltx_ref">17</a>, <a href="#bib.bib18" title="" class="ltx_ref">18</a>, <a href="#bib.bib19" title="" class="ltx_ref">19</a>]</cite>, and they do so after a post hoc sparsification <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib20" title="" class="ltx_ref">20</a>, <a href="#bib.bib38" title="" class="ltx_ref">38</a>, <a href="#bib.bib39" title="" class="ltx_ref">39</a>]</cite> of the task vectors. The post hoc sparsification aids in ensuring the task vectors are disjoint – hence can prevent destructive interference – but, in return, degrades the performance of each individual task. LoTA, on the other hand, enforces sparsity during fine-tuning and directly trains <em id="S3.SS0.SSS0.Px4.p1.1.1" class="ltx_emph ltx_font_italic">sparse task vectors</em>, obviating the need for post hoc sparsification.</p>
</div>
</section>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>Experimental Setup</h2>

<div id="S4.p1" class="ltx_para">
<p id="S4.p1.1" class="ltx_p">In this section, we provide details of the experimental setup, including the baselines, model, dataset, and metric selection. We present the results later in  <a href="#S5" title="5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">5</span></a>. We share our <a target="_blank" href="https://github.com/kiddyboots216/lottery-ticket-adaptation" title="" class="ltx_ref ltx_href">open-source code</a>. We are limited to an academic computing budget, and all results are conducted with a single A100 GPU. We typically do <math id="S4.p1.1.m1.1" class="ltx_Math" alttext="1-3" display="inline"><semantics id="S4.p1.1.m1.1a"><mrow id="S4.p1.1.m1.1.1" xref="S4.p1.1.m1.1.1.cmml"><mn id="S4.p1.1.m1.1.1.2" xref="S4.p1.1.m1.1.1.2.cmml">1</mn><mo id="S4.p1.1.m1.1.1.1" xref="S4.p1.1.m1.1.1.1.cmml">−</mo><mn id="S4.p1.1.m1.1.1.3" xref="S4.p1.1.m1.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.p1.1.m1.1b"><apply id="S4.p1.1.m1.1.1.cmml" xref="S4.p1.1.m1.1.1"><minus id="S4.p1.1.m1.1.1.1.cmml" xref="S4.p1.1.m1.1.1.1"></minus><cn type="integer" id="S4.p1.1.m1.1.1.2.cmml" xref="S4.p1.1.m1.1.1.2">1</cn><cn type="integer" id="S4.p1.1.m1.1.1.3.cmml" xref="S4.p1.1.m1.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p1.1.m1.1c">1-3</annotation></semantics></math> epochs of training for each dataset as this is a standard choice in LLM fine-tuning and indicate it specifically for single-epoch fine-tuning. We use the RMSProp optimizer with default hyperparameters.</p>
</div>
<div id="S4.p2" class="ltx_para">
<p id="S4.p2.2" class="ltx_p"><span id="S4.p2.2.1" class="ltx_text ltx_font_bold">Baselines &amp; Hyperparameters.</span>
Across all three multi-task adaptation paradigms, we compare LoTA against FFT and LoRA. We extensively tune the hyperparameters for FFT and LoRA to ensure that we are comparing against strong baselines. We do not tune hyperparameters for LoTA and directly transfer the hyperparameters from FFT. We fix the sparsity ratio hyperparameter in LoTA to <math id="S4.p2.1.m1.1" class="ltx_Math" alttext="90\%" display="inline"><semantics id="S4.p2.1.m1.1a"><mrow id="S4.p2.1.m1.1.1" xref="S4.p2.1.m1.1.1.cmml"><mn id="S4.p2.1.m1.1.1.2" xref="S4.p2.1.m1.1.1.2.cmml">90</mn><mo id="S4.p2.1.m1.1.1.1" xref="S4.p2.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S4.p2.1.m1.1b"><apply id="S4.p2.1.m1.1.1.cmml" xref="S4.p2.1.m1.1.1"><csymbol cd="latexml" id="S4.p2.1.m1.1.1.1.cmml" xref="S4.p2.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S4.p2.1.m1.1.1.2.cmml" xref="S4.p2.1.m1.1.1.2">90</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p2.1.m1.1c">90\%</annotation></semantics></math>, so that the number of parameters updated roughly matches that of the best-performing LoRA rank of <math id="S4.p2.2.m2.1" class="ltx_Math" alttext="256" display="inline"><semantics id="S4.p2.2.m2.1a"><mn id="S4.p2.2.m2.1.1" xref="S4.p2.2.m2.1.1.cmml">256</mn><annotation-xml encoding="MathML-Content" id="S4.p2.2.m2.1b"><cn type="integer" id="S4.p2.2.m2.1.1.cmml" xref="S4.p2.2.m2.1.1">256</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.p2.2.m2.1c">256</annotation></semantics></math>. We provide an ablation study with higher sparsity levels in Section <a href="#S5.SS1" title="5.1 LoTA Design Choices ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5.1</span></a>. We report all ranges for hyperparameters in  <a href="#A1" title="Appendix A Additional Experimental Details ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Appendix</span> <span class="ltx_text ltx_ref_tag">A</span></a>.</p>
</div>
<div id="S4.p3" class="ltx_para">
<p id="S4.p3.1" class="ltx_p"><span id="S4.p3.1.1" class="ltx_text ltx_font_bold">Models.</span>
We use the best performing open-weights model families, Mistral <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib43" title="" class="ltx_ref">43</a>]</cite> and Llama 3 <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib44" title="" class="ltx_ref">44</a>, <a href="#bib.bib45" title="" class="ltx_ref">45</a>]</cite>, specifically Mistral-7B and Llama-3-8B – the largest models we can adapt with FFT on a single GPU.</p>
</div>
<div id="S4.p4" class="ltx_para">
<p id="S4.p4.1" class="ltx_p"><span id="S4.p4.1.1" class="ltx_text ltx_font_bold">Tasks</span>
We consider six capabilities: instruction following, safety, math, coding, summarization, and reasoning.
We now briefly discuss each capability, the datasets we use to fine-tune and evaluate the presented methods, and the motivation behind the choices.</p>
</div>
<div id="S4.p5" class="ltx_para">
<p id="S4.p5.1" class="ltx_p"><span id="S4.p5.1.1" class="ltx_text ltx_font_bold">Instruction Following.</span>
The most widely-used instruction-tuned LLMs are the “Instruct” or “chat” versions of base models, such as Llama-3-8B-Instruct <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib44" title="" class="ltx_ref">44</a>]</cite>. This is because the process of tuning models on human instructions aligns models to human preferences across a range of tasks <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib5" title="" class="ltx_ref">5</a>]</cite>. For this, we adapt models to data from UltraFeedback <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib46" title="" class="ltx_ref">46</a>]</cite>, which contains a mixture of datasets covering truthfulness, honesty, and helpfulness in addition to instruction-following. We measure the instruction following ability by length-controlled AlpacaEval2 Win Rate <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib47" title="" class="ltx_ref">47</a>]</cite>, which we refer to as “winrate”. A high winrate means that GPT-4 <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib3" title="" class="ltx_ref">3</a>]</cite> prefers the responses of our model on a set of representative prompts over its own responses. Winrate is the metric most closely correlated with human rating preference <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib16" title="" class="ltx_ref">16</a>]</cite>.
Another common benchmark for “chat” models is MT-Bench <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib48" title="" class="ltx_ref">48</a>]</cite>, but there is a significant degree of data contamination between MT-Bench and other task-specific training datasets <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib49" title="" class="ltx_ref">49</a>]</cite>–hence, we do not evaluate on MT-Bench.</p>
</div>
<div id="S4.p6" class="ltx_para">
<p id="S4.p6.1" class="ltx_p"><span id="S4.p6.1.1" class="ltx_text ltx_font_bold">Reasoning.</span>
We train on the standard set of <math id="S4.p6.1.m1.1" class="ltx_Math" alttext="8" display="inline"><semantics id="S4.p6.1.m1.1a"><mn id="S4.p6.1.m1.1.1" xref="S4.p6.1.m1.1.1.cmml">8</mn><annotation-xml encoding="MathML-Content" id="S4.p6.1.m1.1b"><cn type="integer" id="S4.p6.1.m1.1.1.cmml" xref="S4.p6.1.m1.1.1">8</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.p6.1.m1.1c">8</annotation></semantics></math> commonsense reasoning tasks <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib50" title="" class="ltx_ref">50</a>, <a href="#bib.bib51" title="" class="ltx_ref">51</a>, <a href="#bib.bib52" title="" class="ltx_ref">52</a>, <a href="#bib.bib53" title="" class="ltx_ref">53</a>, <a href="#bib.bib54" title="" class="ltx_ref">54</a>, <a href="#bib.bib55" title="" class="ltx_ref">55</a>, <a href="#bib.bib56" title="" class="ltx_ref">56</a>]</cite> (Boolq, PIQA, SocialIQA, Hellaswag, Winograde, ARC-easy, ARC-challenge, OpenBookQA) and report the exact-match accuracy on the test set. As a representative task, we use ARC-easy.</p>
</div>
<div id="S4.p7" class="ltx_para">
<p id="S4.p7.1" class="ltx_p"><span id="S4.p7.1.1" class="ltx_text ltx_font_bold">Math.</span>
We use the set of <math id="S4.p7.1.m1.1" class="ltx_Math" alttext="9" display="inline"><semantics id="S4.p7.1.m1.1a"><mn id="S4.p7.1.m1.1.1" xref="S4.p7.1.m1.1.1.cmml">9</mn><annotation-xml encoding="MathML-Content" id="S4.p7.1.m1.1b"><cn type="integer" id="S4.p7.1.m1.1.1.cmml" xref="S4.p7.1.m1.1.1">9</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.p7.1.m1.1c">9</annotation></semantics></math> math instruction datasets from <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib57" title="" class="ltx_ref">57</a>]</cite> for fine-tuning and report performance on the test set of GSM8k <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib58" title="" class="ltx_ref">58</a>]</cite>. When only considering a single task, we choose GSM8k as the representative task as it is commonly used as a single training and test task by other papers.</p>
</div>
<div id="S4.p8" class="ltx_para">
<p id="S4.p8.1" class="ltx_p"><span id="S4.p8.1.1" class="ltx_text ltx_font_bold">Code Generation.</span>
We use data that instructs the model to write SQL queries given some context <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib59" title="" class="ltx_ref">59</a>]</cite>(SQL-create-context) and report the ROUGE-1 F1 score <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib60" title="" class="ltx_ref">60</a>]</cite> on the test set.</p>
</div>
<div id="S4.p9" class="ltx_para">
<p id="S4.p9.1" class="ltx_p"><span id="S4.p9.1.1" class="ltx_text ltx_font_bold">Summarization.</span>
We use data from Samsum <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib61" title="" class="ltx_ref">61</a>]</cite> and report the ROUGE-1 F1 score on the test set.</p>
</div>
<div id="S4.p10" class="ltx_para">
<p id="S4.p10.3" class="ltx_p"><span id="S4.p10.3.1" class="ltx_text ltx_font_bold">Safety.</span>
We define safety as a latent capability generated by instruction tuning.
A recent concern in AI policy is that, while frontier models such as GPT-3.5/GPT-4 are aligned, they can also be fine-tuned and this presents an opportunity to misalign them. Recently, <cite class="ltx_cite ltx_citemacro_citet">Qi et al. [<a href="#bib.bib62" title="" class="ltx_ref">62</a>]</cite> show that by fine-tuning GPT-3.5 on just <math id="S4.p10.1.m1.1" class="ltx_Math" alttext="100" display="inline"><semantics id="S4.p10.1.m1.1a"><mn id="S4.p10.1.m1.1.1" xref="S4.p10.1.m1.1.1.cmml">100</mn><annotation-xml encoding="MathML-Content" id="S4.p10.1.m1.1b"><cn type="integer" id="S4.p10.1.m1.1.1.cmml" xref="S4.p10.1.m1.1.1">100</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.p10.1.m1.1c">100</annotation></semantics></math> harmful examples for a few epochs, they can ask it to answer harmful queries that it ordinarily would refuse, and <cite class="ltx_cite ltx_citemacro_citet">Zhan et al. [<a href="#bib.bib63" title="" class="ltx_ref">63</a>]</cite> show the same for GPT-4.  <cite class="ltx_cite ltx_citemacro_citet">Lermen et al. [<a href="#bib.bib37" title="" class="ltx_ref">37</a>]</cite> show that this can be done with LoRA rather than the fine-tuning method OpenAI are using in their fine-tuning API (presumably FFT). This is more than an academic concern; SB-1047 <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib64" title="" class="ltx_ref">64</a>]</cite>, recently passed in California, requires model developers providing access to frontier models to implement best effort mitigations to prevent users from misaligning those models. We evaluate the safety of our models on HEx-Phi <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib62" title="" class="ltx_ref">62</a>]</cite>, a dataset of 330 questions spanning multiple categories such as malware, fraud, etc. The safety score (higher is better) is the percentage of queries from the test set where the model refuses to respond. Aligned models such as Llama-3-Instruct will score <math id="S4.p10.2.m2.1" class="ltx_Math" alttext="100\%" display="inline"><semantics id="S4.p10.2.m2.1a"><mrow id="S4.p10.2.m2.1.1" xref="S4.p10.2.m2.1.1.cmml"><mn id="S4.p10.2.m2.1.1.2" xref="S4.p10.2.m2.1.1.2.cmml">100</mn><mo id="S4.p10.2.m2.1.1.1" xref="S4.p10.2.m2.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S4.p10.2.m2.1b"><apply id="S4.p10.2.m2.1.1.cmml" xref="S4.p10.2.m2.1.1"><csymbol cd="latexml" id="S4.p10.2.m2.1.1.1.cmml" xref="S4.p10.2.m2.1.1.1">percent</csymbol><cn type="integer" id="S4.p10.2.m2.1.1.2.cmml" xref="S4.p10.2.m2.1.1.2">100</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p10.2.m2.1c">100\%</annotation></semantics></math> on this task, but because we are doing the alignment ourselves starting from a base model, our baseline Instruct model only gets <math id="S4.p10.3.m3.1" class="ltx_Math" alttext="93\%" display="inline"><semantics id="S4.p10.3.m3.1a"><mrow id="S4.p10.3.m3.1.1" xref="S4.p10.3.m3.1.1.cmml"><mn id="S4.p10.3.m3.1.1.2" xref="S4.p10.3.m3.1.1.2.cmml">93</mn><mo id="S4.p10.3.m3.1.1.1" xref="S4.p10.3.m3.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S4.p10.3.m3.1b"><apply id="S4.p10.3.m3.1.1.cmml" xref="S4.p10.3.m3.1.1"><csymbol cd="latexml" id="S4.p10.3.m3.1.1.1.cmml" xref="S4.p10.3.m3.1.1.1">percent</csymbol><cn type="integer" id="S4.p10.3.m3.1.1.2.cmml" xref="S4.p10.3.m3.1.1.2">93</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p10.3.m3.1c">93\%</annotation></semantics></math> on this task. Given that we are primarily interested in measuring the <em id="S4.p10.3.2" class="ltx_emph ltx_font_italic">forgetting</em> of safety alignment, we do not see this as a major limitation.</p>
</div>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Experimental Results</h2>

<section id="S5.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.1 </span>LoTA Design Choices</h3>

<div id="S5.SS1.p1" class="ltx_para">
<p id="S5.SS1.p1.1" class="ltx_p">Before comparing LoTA to other baselines, we first evaluate the impact of varying the two hyperparameters in our method: the sparsity level and how many iterations we calibrate the mask for.</p>
</div>
<div id="S5.SS1.p2" class="ltx_para">
<p id="S5.SS1.p2.6" class="ltx_p"><span id="S5.SS1.p2.6.1" class="ltx_text ltx_font_bold">Sparsity.</span> We vary the sparsity parameter in LoTA in <a href="#S5.T1" title="In 5.1 LoTA Design Choices ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">1</span></a>. Multiple sparsity thresholds work well; a small amount of sparsity (i.e., <math id="S5.SS1.p2.1.m1.1" class="ltx_Math" alttext="10\%" display="inline"><semantics id="S5.SS1.p2.1.m1.1a"><mrow id="S5.SS1.p2.1.m1.1.1" xref="S5.SS1.p2.1.m1.1.1.cmml"><mn id="S5.SS1.p2.1.m1.1.1.2" xref="S5.SS1.p2.1.m1.1.1.2.cmml">10</mn><mo id="S5.SS1.p2.1.m1.1.1.1" xref="S5.SS1.p2.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.1.m1.1b"><apply id="S5.SS1.p2.1.m1.1.1.cmml" xref="S5.SS1.p2.1.m1.1.1"><csymbol cd="latexml" id="S5.SS1.p2.1.m1.1.1.1.cmml" xref="S5.SS1.p2.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.SS1.p2.1.m1.1.1.2.cmml" xref="S5.SS1.p2.1.m1.1.1.2">10</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.1.m1.1c">10\%</annotation></semantics></math>) seems to have a negative impact on the model, but this is within the error bars and may just be variance. Attempting to achieve <math id="S5.SS1.p2.2.m2.1" class="ltx_Math" alttext="99\%" display="inline"><semantics id="S5.SS1.p2.2.m2.1a"><mrow id="S5.SS1.p2.2.m2.1.1" xref="S5.SS1.p2.2.m2.1.1.cmml"><mn id="S5.SS1.p2.2.m2.1.1.2" xref="S5.SS1.p2.2.m2.1.1.2.cmml">99</mn><mo id="S5.SS1.p2.2.m2.1.1.1" xref="S5.SS1.p2.2.m2.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.2.m2.1b"><apply id="S5.SS1.p2.2.m2.1.1.cmml" xref="S5.SS1.p2.2.m2.1.1"><csymbol cd="latexml" id="S5.SS1.p2.2.m2.1.1.1.cmml" xref="S5.SS1.p2.2.m2.1.1.1">percent</csymbol><cn type="integer" id="S5.SS1.p2.2.m2.1.1.2.cmml" xref="S5.SS1.p2.2.m2.1.1.2">99</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.2.m2.1c">99\%</annotation></semantics></math> sparsity from the task vector of an FFT model does not yield good results. Instead, we can first obtain a mask with <math id="S5.SS1.p2.3.m3.1" class="ltx_Math" alttext="90\%" display="inline"><semantics id="S5.SS1.p2.3.m3.1a"><mrow id="S5.SS1.p2.3.m3.1.1" xref="S5.SS1.p2.3.m3.1.1.cmml"><mn id="S5.SS1.p2.3.m3.1.1.2" xref="S5.SS1.p2.3.m3.1.1.2.cmml">90</mn><mo id="S5.SS1.p2.3.m3.1.1.1" xref="S5.SS1.p2.3.m3.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.3.m3.1b"><apply id="S5.SS1.p2.3.m3.1.1.cmml" xref="S5.SS1.p2.3.m3.1.1"><csymbol cd="latexml" id="S5.SS1.p2.3.m3.1.1.1.cmml" xref="S5.SS1.p2.3.m3.1.1.1">percent</csymbol><cn type="integer" id="S5.SS1.p2.3.m3.1.1.2.cmml" xref="S5.SS1.p2.3.m3.1.1.2">90</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.3.m3.1c">90\%</annotation></semantics></math> sparsity, then train a LoTA model with this mask, and then use the task vector from the <math id="S5.SS1.p2.4.m4.1" class="ltx_Math" alttext="90\%" display="inline"><semantics id="S5.SS1.p2.4.m4.1a"><mrow id="S5.SS1.p2.4.m4.1.1" xref="S5.SS1.p2.4.m4.1.1.cmml"><mn id="S5.SS1.p2.4.m4.1.1.2" xref="S5.SS1.p2.4.m4.1.1.2.cmml">90</mn><mo id="S5.SS1.p2.4.m4.1.1.1" xref="S5.SS1.p2.4.m4.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.4.m4.1b"><apply id="S5.SS1.p2.4.m4.1.1.cmml" xref="S5.SS1.p2.4.m4.1.1"><csymbol cd="latexml" id="S5.SS1.p2.4.m4.1.1.1.cmml" xref="S5.SS1.p2.4.m4.1.1.1">percent</csymbol><cn type="integer" id="S5.SS1.p2.4.m4.1.1.2.cmml" xref="S5.SS1.p2.4.m4.1.1.2">90</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.4.m4.1c">90\%</annotation></semantics></math>-sparse LoTA model to calibrate the mask for our <math id="S5.SS1.p2.5.m5.1" class="ltx_Math" alttext="99\%" display="inline"><semantics id="S5.SS1.p2.5.m5.1a"><mrow id="S5.SS1.p2.5.m5.1.1" xref="S5.SS1.p2.5.m5.1.1.cmml"><mn id="S5.SS1.p2.5.m5.1.1.2" xref="S5.SS1.p2.5.m5.1.1.2.cmml">99</mn><mo id="S5.SS1.p2.5.m5.1.1.1" xref="S5.SS1.p2.5.m5.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.5.m5.1b"><apply id="S5.SS1.p2.5.m5.1.1.cmml" xref="S5.SS1.p2.5.m5.1.1"><csymbol cd="latexml" id="S5.SS1.p2.5.m5.1.1.1.cmml" xref="S5.SS1.p2.5.m5.1.1.1">percent</csymbol><cn type="integer" id="S5.SS1.p2.5.m5.1.1.2.cmml" xref="S5.SS1.p2.5.m5.1.1.2">99</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.5.m5.1c">99\%</annotation></semantics></math>-sparsity model, which we denote as <math id="S5.SS1.p2.6.m6.1" class="ltx_Math" alttext="99\%^{*}" display="inline"><semantics id="S5.SS1.p2.6.m6.1a"><msup id="S5.SS1.p2.6.m6.1.1" xref="S5.SS1.p2.6.m6.1.1.cmml"><mrow id="S5.SS1.p2.6.m6.1.1.2" xref="S5.SS1.p2.6.m6.1.1.2.cmml"><mn id="S5.SS1.p2.6.m6.1.1.2.2" xref="S5.SS1.p2.6.m6.1.1.2.2.cmml">99</mn><mo id="S5.SS1.p2.6.m6.1.1.2.1" xref="S5.SS1.p2.6.m6.1.1.2.1.cmml">%</mo></mrow><mo id="S5.SS1.p2.6.m6.1.1.3" xref="S5.SS1.p2.6.m6.1.1.3.cmml">∗</mo></msup><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.6.m6.1b"><apply id="S5.SS1.p2.6.m6.1.1.cmml" xref="S5.SS1.p2.6.m6.1.1"><csymbol cd="ambiguous" id="S5.SS1.p2.6.m6.1.1.1.cmml" xref="S5.SS1.p2.6.m6.1.1">superscript</csymbol><apply id="S5.SS1.p2.6.m6.1.1.2.cmml" xref="S5.SS1.p2.6.m6.1.1.2"><csymbol cd="latexml" id="S5.SS1.p2.6.m6.1.1.2.1.cmml" xref="S5.SS1.p2.6.m6.1.1.2.1">percent</csymbol><cn type="integer" id="S5.SS1.p2.6.m6.1.1.2.2.cmml" xref="S5.SS1.p2.6.m6.1.1.2.2">99</cn></apply><times id="S5.SS1.p2.6.m6.1.1.3.cmml" xref="S5.SS1.p2.6.m6.1.1.3"></times></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.6.m6.1c">99\%^{*}</annotation></semantics></math>.
Composing multiple steps of calibration trades off performance in the high-compression setting with compute overhead.</p>
</div>
<figure id="S5.T1" class="ltx_table">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table"><span id="S5.T1.20.2.1" class="ltx_text" style="font-size:90%;">Table 1</span>: </span><span id="S5.T1.2.1" class="ltx_text" style="font-size:90%;">The impact of varying the sparsity ratio on the performance of LoTA. <math id="S5.T1.2.1.m1.1" class="ltx_Math" alttext="99\%^{*}" display="inline"><semantics id="S5.T1.2.1.m1.1b"><msup id="S5.T1.2.1.m1.1.1" xref="S5.T1.2.1.m1.1.1.cmml"><mrow id="S5.T1.2.1.m1.1.1.2" xref="S5.T1.2.1.m1.1.1.2.cmml"><mn id="S5.T1.2.1.m1.1.1.2.2" xref="S5.T1.2.1.m1.1.1.2.2.cmml">99</mn><mo id="S5.T1.2.1.m1.1.1.2.1" xref="S5.T1.2.1.m1.1.1.2.1.cmml">%</mo></mrow><mo id="S5.T1.2.1.m1.1.1.3" xref="S5.T1.2.1.m1.1.1.3.cmml">∗</mo></msup><annotation-xml encoding="MathML-Content" id="S5.T1.2.1.m1.1c"><apply id="S5.T1.2.1.m1.1.1.cmml" xref="S5.T1.2.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T1.2.1.m1.1.1.1.cmml" xref="S5.T1.2.1.m1.1.1">superscript</csymbol><apply id="S5.T1.2.1.m1.1.1.2.cmml" xref="S5.T1.2.1.m1.1.1.2"><csymbol cd="latexml" id="S5.T1.2.1.m1.1.1.2.1.cmml" xref="S5.T1.2.1.m1.1.1.2.1">percent</csymbol><cn type="integer" id="S5.T1.2.1.m1.1.1.2.2.cmml" xref="S5.T1.2.1.m1.1.1.2.2">99</cn></apply><times id="S5.T1.2.1.m1.1.1.3.cmml" xref="S5.T1.2.1.m1.1.1.3"></times></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.2.1.m1.1d">99\%^{*}</annotation></semantics></math> denotes the model calibrated with iterative LoTA.</span></figcaption>
<div id="S5.T1.18" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:433.6pt;height:40.9pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(25.8pt,-2.4pt) scale(1.13481687869252,1.13481687869252) ;">
<table id="S5.T1.18.16" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S5.T1.10.8.8" class="ltx_tr">
<th id="S5.T1.10.8.8.9" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_tt">Sparsity</th>
<th id="S5.T1.3.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt"><math id="S5.T1.3.1.1.1.m1.1" class="ltx_Math" alttext="0\%" display="inline"><semantics id="S5.T1.3.1.1.1.m1.1a"><mrow id="S5.T1.3.1.1.1.m1.1.1" xref="S5.T1.3.1.1.1.m1.1.1.cmml"><mn id="S5.T1.3.1.1.1.m1.1.1.2" xref="S5.T1.3.1.1.1.m1.1.1.2.cmml">0</mn><mo id="S5.T1.3.1.1.1.m1.1.1.1" xref="S5.T1.3.1.1.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T1.3.1.1.1.m1.1b"><apply id="S5.T1.3.1.1.1.m1.1.1.cmml" xref="S5.T1.3.1.1.1.m1.1.1"><csymbol cd="latexml" id="S5.T1.3.1.1.1.m1.1.1.1.cmml" xref="S5.T1.3.1.1.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T1.3.1.1.1.m1.1.1.2.cmml" xref="S5.T1.3.1.1.1.m1.1.1.2">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.3.1.1.1.m1.1c">0\%</annotation></semantics></math></th>
<td id="S5.T1.4.2.2.2" class="ltx_td ltx_align_center ltx_border_tt"><math id="S5.T1.4.2.2.2.m1.1" class="ltx_Math" alttext="10\%" display="inline"><semantics id="S5.T1.4.2.2.2.m1.1a"><mrow id="S5.T1.4.2.2.2.m1.1.1" xref="S5.T1.4.2.2.2.m1.1.1.cmml"><mn id="S5.T1.4.2.2.2.m1.1.1.2" xref="S5.T1.4.2.2.2.m1.1.1.2.cmml">10</mn><mo id="S5.T1.4.2.2.2.m1.1.1.1" xref="S5.T1.4.2.2.2.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T1.4.2.2.2.m1.1b"><apply id="S5.T1.4.2.2.2.m1.1.1.cmml" xref="S5.T1.4.2.2.2.m1.1.1"><csymbol cd="latexml" id="S5.T1.4.2.2.2.m1.1.1.1.cmml" xref="S5.T1.4.2.2.2.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T1.4.2.2.2.m1.1.1.2.cmml" xref="S5.T1.4.2.2.2.m1.1.1.2">10</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.4.2.2.2.m1.1c">10\%</annotation></semantics></math></td>
<td id="S5.T1.5.3.3.3" class="ltx_td ltx_align_center ltx_border_tt"><math id="S5.T1.5.3.3.3.m1.1" class="ltx_Math" alttext="25\%" display="inline"><semantics id="S5.T1.5.3.3.3.m1.1a"><mrow id="S5.T1.5.3.3.3.m1.1.1" xref="S5.T1.5.3.3.3.m1.1.1.cmml"><mn id="S5.T1.5.3.3.3.m1.1.1.2" xref="S5.T1.5.3.3.3.m1.1.1.2.cmml">25</mn><mo id="S5.T1.5.3.3.3.m1.1.1.1" xref="S5.T1.5.3.3.3.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T1.5.3.3.3.m1.1b"><apply id="S5.T1.5.3.3.3.m1.1.1.cmml" xref="S5.T1.5.3.3.3.m1.1.1"><csymbol cd="latexml" id="S5.T1.5.3.3.3.m1.1.1.1.cmml" xref="S5.T1.5.3.3.3.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T1.5.3.3.3.m1.1.1.2.cmml" xref="S5.T1.5.3.3.3.m1.1.1.2">25</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.5.3.3.3.m1.1c">25\%</annotation></semantics></math></td>
<td id="S5.T1.6.4.4.4" class="ltx_td ltx_align_center ltx_border_tt"><math id="S5.T1.6.4.4.4.m1.1" class="ltx_Math" alttext="50\%" display="inline"><semantics id="S5.T1.6.4.4.4.m1.1a"><mrow id="S5.T1.6.4.4.4.m1.1.1" xref="S5.T1.6.4.4.4.m1.1.1.cmml"><mn id="S5.T1.6.4.4.4.m1.1.1.2" xref="S5.T1.6.4.4.4.m1.1.1.2.cmml">50</mn><mo id="S5.T1.6.4.4.4.m1.1.1.1" xref="S5.T1.6.4.4.4.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T1.6.4.4.4.m1.1b"><apply id="S5.T1.6.4.4.4.m1.1.1.cmml" xref="S5.T1.6.4.4.4.m1.1.1"><csymbol cd="latexml" id="S5.T1.6.4.4.4.m1.1.1.1.cmml" xref="S5.T1.6.4.4.4.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T1.6.4.4.4.m1.1.1.2.cmml" xref="S5.T1.6.4.4.4.m1.1.1.2">50</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.6.4.4.4.m1.1c">50\%</annotation></semantics></math></td>
<td id="S5.T1.7.5.5.5" class="ltx_td ltx_align_center ltx_border_tt"><math id="S5.T1.7.5.5.5.m1.1" class="ltx_Math" alttext="75\%" display="inline"><semantics id="S5.T1.7.5.5.5.m1.1a"><mrow id="S5.T1.7.5.5.5.m1.1.1" xref="S5.T1.7.5.5.5.m1.1.1.cmml"><mn id="S5.T1.7.5.5.5.m1.1.1.2" xref="S5.T1.7.5.5.5.m1.1.1.2.cmml">75</mn><mo id="S5.T1.7.5.5.5.m1.1.1.1" xref="S5.T1.7.5.5.5.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T1.7.5.5.5.m1.1b"><apply id="S5.T1.7.5.5.5.m1.1.1.cmml" xref="S5.T1.7.5.5.5.m1.1.1"><csymbol cd="latexml" id="S5.T1.7.5.5.5.m1.1.1.1.cmml" xref="S5.T1.7.5.5.5.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T1.7.5.5.5.m1.1.1.2.cmml" xref="S5.T1.7.5.5.5.m1.1.1.2">75</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.7.5.5.5.m1.1c">75\%</annotation></semantics></math></td>
<td id="S5.T1.8.6.6.6" class="ltx_td ltx_align_center ltx_border_tt"><math id="S5.T1.8.6.6.6.m1.1" class="ltx_Math" alttext="90\%" display="inline"><semantics id="S5.T1.8.6.6.6.m1.1a"><mrow id="S5.T1.8.6.6.6.m1.1.1" xref="S5.T1.8.6.6.6.m1.1.1.cmml"><mn id="S5.T1.8.6.6.6.m1.1.1.2" xref="S5.T1.8.6.6.6.m1.1.1.2.cmml">90</mn><mo id="S5.T1.8.6.6.6.m1.1.1.1" xref="S5.T1.8.6.6.6.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T1.8.6.6.6.m1.1b"><apply id="S5.T1.8.6.6.6.m1.1.1.cmml" xref="S5.T1.8.6.6.6.m1.1.1"><csymbol cd="latexml" id="S5.T1.8.6.6.6.m1.1.1.1.cmml" xref="S5.T1.8.6.6.6.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T1.8.6.6.6.m1.1.1.2.cmml" xref="S5.T1.8.6.6.6.m1.1.1.2">90</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.8.6.6.6.m1.1c">90\%</annotation></semantics></math></td>
<td id="S5.T1.9.7.7.7" class="ltx_td ltx_align_center ltx_border_tt"><math id="S5.T1.9.7.7.7.m1.1" class="ltx_Math" alttext="99\%" display="inline"><semantics id="S5.T1.9.7.7.7.m1.1a"><mrow id="S5.T1.9.7.7.7.m1.1.1" xref="S5.T1.9.7.7.7.m1.1.1.cmml"><mn id="S5.T1.9.7.7.7.m1.1.1.2" xref="S5.T1.9.7.7.7.m1.1.1.2.cmml">99</mn><mo id="S5.T1.9.7.7.7.m1.1.1.1" xref="S5.T1.9.7.7.7.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T1.9.7.7.7.m1.1b"><apply id="S5.T1.9.7.7.7.m1.1.1.cmml" xref="S5.T1.9.7.7.7.m1.1.1"><csymbol cd="latexml" id="S5.T1.9.7.7.7.m1.1.1.1.cmml" xref="S5.T1.9.7.7.7.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T1.9.7.7.7.m1.1.1.2.cmml" xref="S5.T1.9.7.7.7.m1.1.1.2">99</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.9.7.7.7.m1.1c">99\%</annotation></semantics></math></td>
<td id="S5.T1.10.8.8.8" class="ltx_td ltx_align_center ltx_border_tt"><math id="S5.T1.10.8.8.8.m1.1" class="ltx_Math" alttext="99\%^{*}" display="inline"><semantics id="S5.T1.10.8.8.8.m1.1a"><msup id="S5.T1.10.8.8.8.m1.1.1" xref="S5.T1.10.8.8.8.m1.1.1.cmml"><mrow id="S5.T1.10.8.8.8.m1.1.1.2" xref="S5.T1.10.8.8.8.m1.1.1.2.cmml"><mn id="S5.T1.10.8.8.8.m1.1.1.2.2" xref="S5.T1.10.8.8.8.m1.1.1.2.2.cmml">99</mn><mo id="S5.T1.10.8.8.8.m1.1.1.2.1" xref="S5.T1.10.8.8.8.m1.1.1.2.1.cmml">%</mo></mrow><mo id="S5.T1.10.8.8.8.m1.1.1.3" xref="S5.T1.10.8.8.8.m1.1.1.3.cmml">∗</mo></msup><annotation-xml encoding="MathML-Content" id="S5.T1.10.8.8.8.m1.1b"><apply id="S5.T1.10.8.8.8.m1.1.1.cmml" xref="S5.T1.10.8.8.8.m1.1.1"><csymbol cd="ambiguous" id="S5.T1.10.8.8.8.m1.1.1.1.cmml" xref="S5.T1.10.8.8.8.m1.1.1">superscript</csymbol><apply id="S5.T1.10.8.8.8.m1.1.1.2.cmml" xref="S5.T1.10.8.8.8.m1.1.1.2"><csymbol cd="latexml" id="S5.T1.10.8.8.8.m1.1.1.2.1.cmml" xref="S5.T1.10.8.8.8.m1.1.1.2.1">percent</csymbol><cn type="integer" id="S5.T1.10.8.8.8.m1.1.1.2.2.cmml" xref="S5.T1.10.8.8.8.m1.1.1.2.2">99</cn></apply><times id="S5.T1.10.8.8.8.m1.1.1.3.cmml" xref="S5.T1.10.8.8.8.m1.1.1.3"></times></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.10.8.8.8.m1.1c">99\%^{*}</annotation></semantics></math></td>
</tr>
<tr id="S5.T1.18.16.16" class="ltx_tr">
<th id="S5.T1.18.16.16.9" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_bb ltx_border_t">Performance</th>
<th id="S5.T1.11.9.9.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb ltx_border_t"><math id="S5.T1.11.9.9.1.m1.1" class="ltx_Math" alttext="19.0_{1.0}" display="inline"><semantics id="S5.T1.11.9.9.1.m1.1a"><msub id="S5.T1.11.9.9.1.m1.1.1" xref="S5.T1.11.9.9.1.m1.1.1.cmml"><mn id="S5.T1.11.9.9.1.m1.1.1.2" xref="S5.T1.11.9.9.1.m1.1.1.2.cmml">19.0</mn><mn id="S5.T1.11.9.9.1.m1.1.1.3" xref="S5.T1.11.9.9.1.m1.1.1.3.cmml">1.0</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T1.11.9.9.1.m1.1b"><apply id="S5.T1.11.9.9.1.m1.1.1.cmml" xref="S5.T1.11.9.9.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T1.11.9.9.1.m1.1.1.1.cmml" xref="S5.T1.11.9.9.1.m1.1.1">subscript</csymbol><cn type="float" id="S5.T1.11.9.9.1.m1.1.1.2.cmml" xref="S5.T1.11.9.9.1.m1.1.1.2">19.0</cn><cn type="float" id="S5.T1.11.9.9.1.m1.1.1.3.cmml" xref="S5.T1.11.9.9.1.m1.1.1.3">1.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.11.9.9.1.m1.1c">19.0_{1.0}</annotation></semantics></math></th>
<td id="S5.T1.12.10.10.2" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><math id="S5.T1.12.10.10.2.m1.1" class="ltx_Math" alttext="18.2_{1.0}" display="inline"><semantics id="S5.T1.12.10.10.2.m1.1a"><msub id="S5.T1.12.10.10.2.m1.1.1" xref="S5.T1.12.10.10.2.m1.1.1.cmml"><mn id="S5.T1.12.10.10.2.m1.1.1.2" xref="S5.T1.12.10.10.2.m1.1.1.2.cmml">18.2</mn><mn id="S5.T1.12.10.10.2.m1.1.1.3" xref="S5.T1.12.10.10.2.m1.1.1.3.cmml">1.0</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T1.12.10.10.2.m1.1b"><apply id="S5.T1.12.10.10.2.m1.1.1.cmml" xref="S5.T1.12.10.10.2.m1.1.1"><csymbol cd="ambiguous" id="S5.T1.12.10.10.2.m1.1.1.1.cmml" xref="S5.T1.12.10.10.2.m1.1.1">subscript</csymbol><cn type="float" id="S5.T1.12.10.10.2.m1.1.1.2.cmml" xref="S5.T1.12.10.10.2.m1.1.1.2">18.2</cn><cn type="float" id="S5.T1.12.10.10.2.m1.1.1.3.cmml" xref="S5.T1.12.10.10.2.m1.1.1.3">1.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.12.10.10.2.m1.1c">18.2_{1.0}</annotation></semantics></math></td>
<td id="S5.T1.13.11.11.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><math id="S5.T1.13.11.11.3.m1.1" class="ltx_Math" alttext="18.5_{1.0}" display="inline"><semantics id="S5.T1.13.11.11.3.m1.1a"><msub id="S5.T1.13.11.11.3.m1.1.1" xref="S5.T1.13.11.11.3.m1.1.1.cmml"><mn id="S5.T1.13.11.11.3.m1.1.1.2" xref="S5.T1.13.11.11.3.m1.1.1.2.cmml">18.5</mn><mn id="S5.T1.13.11.11.3.m1.1.1.3" xref="S5.T1.13.11.11.3.m1.1.1.3.cmml">1.0</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T1.13.11.11.3.m1.1b"><apply id="S5.T1.13.11.11.3.m1.1.1.cmml" xref="S5.T1.13.11.11.3.m1.1.1"><csymbol cd="ambiguous" id="S5.T1.13.11.11.3.m1.1.1.1.cmml" xref="S5.T1.13.11.11.3.m1.1.1">subscript</csymbol><cn type="float" id="S5.T1.13.11.11.3.m1.1.1.2.cmml" xref="S5.T1.13.11.11.3.m1.1.1.2">18.5</cn><cn type="float" id="S5.T1.13.11.11.3.m1.1.1.3.cmml" xref="S5.T1.13.11.11.3.m1.1.1.3">1.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.13.11.11.3.m1.1c">18.5_{1.0}</annotation></semantics></math></td>
<td id="S5.T1.14.12.12.4" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><math id="S5.T1.14.12.12.4.m1.1" class="ltx_Math" alttext="18.2_{1.0}" display="inline"><semantics id="S5.T1.14.12.12.4.m1.1a"><msub id="S5.T1.14.12.12.4.m1.1.1" xref="S5.T1.14.12.12.4.m1.1.1.cmml"><mn id="S5.T1.14.12.12.4.m1.1.1.2" xref="S5.T1.14.12.12.4.m1.1.1.2.cmml">18.2</mn><mn id="S5.T1.14.12.12.4.m1.1.1.3" xref="S5.T1.14.12.12.4.m1.1.1.3.cmml">1.0</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T1.14.12.12.4.m1.1b"><apply id="S5.T1.14.12.12.4.m1.1.1.cmml" xref="S5.T1.14.12.12.4.m1.1.1"><csymbol cd="ambiguous" id="S5.T1.14.12.12.4.m1.1.1.1.cmml" xref="S5.T1.14.12.12.4.m1.1.1">subscript</csymbol><cn type="float" id="S5.T1.14.12.12.4.m1.1.1.2.cmml" xref="S5.T1.14.12.12.4.m1.1.1.2">18.2</cn><cn type="float" id="S5.T1.14.12.12.4.m1.1.1.3.cmml" xref="S5.T1.14.12.12.4.m1.1.1.3">1.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.14.12.12.4.m1.1c">18.2_{1.0}</annotation></semantics></math></td>
<td id="S5.T1.15.13.13.5" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><math id="S5.T1.15.13.13.5.m1.1" class="ltx_Math" alttext="19.5_{0.9}" display="inline"><semantics id="S5.T1.15.13.13.5.m1.1a"><msub id="S5.T1.15.13.13.5.m1.1.1" xref="S5.T1.15.13.13.5.m1.1.1.cmml"><mn id="S5.T1.15.13.13.5.m1.1.1.2" xref="S5.T1.15.13.13.5.m1.1.1.2.cmml">19.5</mn><mn id="S5.T1.15.13.13.5.m1.1.1.3" xref="S5.T1.15.13.13.5.m1.1.1.3.cmml">0.9</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T1.15.13.13.5.m1.1b"><apply id="S5.T1.15.13.13.5.m1.1.1.cmml" xref="S5.T1.15.13.13.5.m1.1.1"><csymbol cd="ambiguous" id="S5.T1.15.13.13.5.m1.1.1.1.cmml" xref="S5.T1.15.13.13.5.m1.1.1">subscript</csymbol><cn type="float" id="S5.T1.15.13.13.5.m1.1.1.2.cmml" xref="S5.T1.15.13.13.5.m1.1.1.2">19.5</cn><cn type="float" id="S5.T1.15.13.13.5.m1.1.1.3.cmml" xref="S5.T1.15.13.13.5.m1.1.1.3">0.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.15.13.13.5.m1.1c">19.5_{0.9}</annotation></semantics></math></td>
<td id="S5.T1.16.14.14.6" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><math id="S5.T1.16.14.14.6.m1.1" class="ltx_Math" alttext="19.0_{1.0}" display="inline"><semantics id="S5.T1.16.14.14.6.m1.1a"><msub id="S5.T1.16.14.14.6.m1.1.1" xref="S5.T1.16.14.14.6.m1.1.1.cmml"><mn id="S5.T1.16.14.14.6.m1.1.1.2" xref="S5.T1.16.14.14.6.m1.1.1.2.cmml">19.0</mn><mn id="S5.T1.16.14.14.6.m1.1.1.3" xref="S5.T1.16.14.14.6.m1.1.1.3.cmml">1.0</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T1.16.14.14.6.m1.1b"><apply id="S5.T1.16.14.14.6.m1.1.1.cmml" xref="S5.T1.16.14.14.6.m1.1.1"><csymbol cd="ambiguous" id="S5.T1.16.14.14.6.m1.1.1.1.cmml" xref="S5.T1.16.14.14.6.m1.1.1">subscript</csymbol><cn type="float" id="S5.T1.16.14.14.6.m1.1.1.2.cmml" xref="S5.T1.16.14.14.6.m1.1.1.2">19.0</cn><cn type="float" id="S5.T1.16.14.14.6.m1.1.1.3.cmml" xref="S5.T1.16.14.14.6.m1.1.1.3">1.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.16.14.14.6.m1.1c">19.0_{1.0}</annotation></semantics></math></td>
<td id="S5.T1.17.15.15.7" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><math id="S5.T1.17.15.15.7.m1.1" class="ltx_Math" alttext="13.1_{0.8}" display="inline"><semantics id="S5.T1.17.15.15.7.m1.1a"><msub id="S5.T1.17.15.15.7.m1.1.1" xref="S5.T1.17.15.15.7.m1.1.1.cmml"><mn id="S5.T1.17.15.15.7.m1.1.1.2" xref="S5.T1.17.15.15.7.m1.1.1.2.cmml">13.1</mn><mn id="S5.T1.17.15.15.7.m1.1.1.3" xref="S5.T1.17.15.15.7.m1.1.1.3.cmml">0.8</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T1.17.15.15.7.m1.1b"><apply id="S5.T1.17.15.15.7.m1.1.1.cmml" xref="S5.T1.17.15.15.7.m1.1.1"><csymbol cd="ambiguous" id="S5.T1.17.15.15.7.m1.1.1.1.cmml" xref="S5.T1.17.15.15.7.m1.1.1">subscript</csymbol><cn type="float" id="S5.T1.17.15.15.7.m1.1.1.2.cmml" xref="S5.T1.17.15.15.7.m1.1.1.2">13.1</cn><cn type="float" id="S5.T1.17.15.15.7.m1.1.1.3.cmml" xref="S5.T1.17.15.15.7.m1.1.1.3">0.8</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.17.15.15.7.m1.1c">13.1_{0.8}</annotation></semantics></math></td>
<td id="S5.T1.18.16.16.8" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><math id="S5.T1.18.16.16.8.m1.1" class="ltx_Math" alttext="17.4_{0.9}" display="inline"><semantics id="S5.T1.18.16.16.8.m1.1a"><msub id="S5.T1.18.16.16.8.m1.1.1" xref="S5.T1.18.16.16.8.m1.1.1.cmml"><mn id="S5.T1.18.16.16.8.m1.1.1.2" xref="S5.T1.18.16.16.8.m1.1.1.2.cmml">17.4</mn><mn id="S5.T1.18.16.16.8.m1.1.1.3" xref="S5.T1.18.16.16.8.m1.1.1.3.cmml">0.9</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T1.18.16.16.8.m1.1b"><apply id="S5.T1.18.16.16.8.m1.1.1.cmml" xref="S5.T1.18.16.16.8.m1.1.1"><csymbol cd="ambiguous" id="S5.T1.18.16.16.8.m1.1.1.1.cmml" xref="S5.T1.18.16.16.8.m1.1.1">subscript</csymbol><cn type="float" id="S5.T1.18.16.16.8.m1.1.1.2.cmml" xref="S5.T1.18.16.16.8.m1.1.1.2">17.4</cn><cn type="float" id="S5.T1.18.16.16.8.m1.1.1.3.cmml" xref="S5.T1.18.16.16.8.m1.1.1.3">0.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T1.18.16.16.8.m1.1c">17.4_{0.9}</annotation></semantics></math></td>
</tr>
</tbody>
</table>
</span></div>
</figure>
<div id="S5.SS1.p3" class="ltx_para">
<p id="S5.SS1.p3.1" class="ltx_p"><span id="S5.SS1.p3.1.1" class="ltx_text ltx_font_bold">Compute Overhead.</span> Arguably, the main reason why PEFT methods such as LoRA are commonly used is because they reduce the memory consumption in the backward pass, thus enabling the use of a larger pass. <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib33" title="" class="ltx_ref">33</a>]</cite> recently critically analyzed this and found that to fit tasks such as math and code generation, LoRA needs to use high ranks, which in turn reduces the speedup to at most <math id="S5.SS1.p3.1.m1.1" class="ltx_Math" alttext="15\%" display="inline"><semantics id="S5.SS1.p3.1.m1.1a"><mrow id="S5.SS1.p3.1.m1.1.1" xref="S5.SS1.p3.1.m1.1.1.cmml"><mn id="S5.SS1.p3.1.m1.1.1.2" xref="S5.SS1.p3.1.m1.1.1.2.cmml">15</mn><mo id="S5.SS1.p3.1.m1.1.1.1" xref="S5.SS1.p3.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p3.1.m1.1b"><apply id="S5.SS1.p3.1.m1.1.1.cmml" xref="S5.SS1.p3.1.m1.1.1"><csymbol cd="latexml" id="S5.SS1.p3.1.m1.1.1.1.cmml" xref="S5.SS1.p3.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.SS1.p3.1.m1.1.1.2.cmml" xref="S5.SS1.p3.1.m1.1.1.2">15</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p3.1.m1.1c">15\%</annotation></semantics></math>. We nevertheless acknowledge that any and all PEFT methods will be faster than LoTA during <em id="S5.SS1.p3.1.2" class="ltx_emph ltx_font_italic">training</em> because LoTA requires an initial pass over the dataset to calibrate the sparsity mask.</p>
</div>
<figure id="S5.T2" class="ltx_table">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table"><span id="S5.T2.8.1.1" class="ltx_text" style="font-size:90%;">Table 2</span>: </span><span id="S5.T2.9.2" class="ltx_text" style="font-size:90%;">LoTA performance degrades gracefully as a smaller fraction of the data is used on the most challenging task (instruction following). 0 data usage corresponds to just creating a random mask.</span></figcaption>
<table id="S5.T2.6" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S5.T2.2.2" class="ltx_tr">
<th id="S5.T2.2.2.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">Data Used</th>
<th id="S5.T2.1.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><math id="S5.T2.1.1.1.m1.1" class="ltx_Math" alttext="10\%" display="inline"><semantics id="S5.T2.1.1.1.m1.1a"><mrow id="S5.T2.1.1.1.m1.1.1" xref="S5.T2.1.1.1.m1.1.1.cmml"><mn id="S5.T2.1.1.1.m1.1.1.2" xref="S5.T2.1.1.1.m1.1.1.2.cmml">10</mn><mo id="S5.T2.1.1.1.m1.1.1.1" xref="S5.T2.1.1.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T2.1.1.1.m1.1b"><apply id="S5.T2.1.1.1.m1.1.1.cmml" xref="S5.T2.1.1.1.m1.1.1"><csymbol cd="latexml" id="S5.T2.1.1.1.m1.1.1.1.cmml" xref="S5.T2.1.1.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T2.1.1.1.m1.1.1.2.cmml" xref="S5.T2.1.1.1.m1.1.1.2">10</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.1.1.1.m1.1c">10\%</annotation></semantics></math></th>
<th id="S5.T2.2.2.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><math id="S5.T2.2.2.2.m1.1" class="ltx_Math" alttext="1\%" display="inline"><semantics id="S5.T2.2.2.2.m1.1a"><mrow id="S5.T2.2.2.2.m1.1.1" xref="S5.T2.2.2.2.m1.1.1.cmml"><mn id="S5.T2.2.2.2.m1.1.1.2" xref="S5.T2.2.2.2.m1.1.1.2.cmml">1</mn><mo id="S5.T2.2.2.2.m1.1.1.1" xref="S5.T2.2.2.2.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T2.2.2.2.m1.1b"><apply id="S5.T2.2.2.2.m1.1.1.cmml" xref="S5.T2.2.2.2.m1.1.1"><csymbol cd="latexml" id="S5.T2.2.2.2.m1.1.1.1.cmml" xref="S5.T2.2.2.2.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T2.2.2.2.m1.1.1.2.cmml" xref="S5.T2.2.2.2.m1.1.1.2">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.2.2.2.m1.1c">1\%</annotation></semantics></math></th>
<th id="S5.T2.2.2.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">0</th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S5.T2.6.6" class="ltx_tr">
<td id="S5.T2.3.3.1" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">Performance Drop (from <math id="S5.T2.3.3.1.m1.1" class="ltx_Math" alttext="100\%" display="inline"><semantics id="S5.T2.3.3.1.m1.1a"><mrow id="S5.T2.3.3.1.m1.1.1" xref="S5.T2.3.3.1.m1.1.1.cmml"><mn id="S5.T2.3.3.1.m1.1.1.2" xref="S5.T2.3.3.1.m1.1.1.2.cmml">100</mn><mo id="S5.T2.3.3.1.m1.1.1.1" xref="S5.T2.3.3.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T2.3.3.1.m1.1b"><apply id="S5.T2.3.3.1.m1.1.1.cmml" xref="S5.T2.3.3.1.m1.1.1"><csymbol cd="latexml" id="S5.T2.3.3.1.m1.1.1.1.cmml" xref="S5.T2.3.3.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T2.3.3.1.m1.1.1.2.cmml" xref="S5.T2.3.3.1.m1.1.1.2">100</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.3.3.1.m1.1c">100\%</annotation></semantics></math>)</td>
<td id="S5.T2.4.4.2" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><math id="S5.T2.4.4.2.m1.1" class="ltx_Math" alttext="4.2\%" display="inline"><semantics id="S5.T2.4.4.2.m1.1a"><mrow id="S5.T2.4.4.2.m1.1.1" xref="S5.T2.4.4.2.m1.1.1.cmml"><mn id="S5.T2.4.4.2.m1.1.1.2" xref="S5.T2.4.4.2.m1.1.1.2.cmml">4.2</mn><mo id="S5.T2.4.4.2.m1.1.1.1" xref="S5.T2.4.4.2.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T2.4.4.2.m1.1b"><apply id="S5.T2.4.4.2.m1.1.1.cmml" xref="S5.T2.4.4.2.m1.1.1"><csymbol cd="latexml" id="S5.T2.4.4.2.m1.1.1.1.cmml" xref="S5.T2.4.4.2.m1.1.1.1">percent</csymbol><cn type="float" id="S5.T2.4.4.2.m1.1.1.2.cmml" xref="S5.T2.4.4.2.m1.1.1.2">4.2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.4.4.2.m1.1c">4.2\%</annotation></semantics></math></td>
<td id="S5.T2.5.5.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><math id="S5.T2.5.5.3.m1.1" class="ltx_Math" alttext="10\%" display="inline"><semantics id="S5.T2.5.5.3.m1.1a"><mrow id="S5.T2.5.5.3.m1.1.1" xref="S5.T2.5.5.3.m1.1.1.cmml"><mn id="S5.T2.5.5.3.m1.1.1.2" xref="S5.T2.5.5.3.m1.1.1.2.cmml">10</mn><mo id="S5.T2.5.5.3.m1.1.1.1" xref="S5.T2.5.5.3.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T2.5.5.3.m1.1b"><apply id="S5.T2.5.5.3.m1.1.1.cmml" xref="S5.T2.5.5.3.m1.1.1"><csymbol cd="latexml" id="S5.T2.5.5.3.m1.1.1.1.cmml" xref="S5.T2.5.5.3.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T2.5.5.3.m1.1.1.2.cmml" xref="S5.T2.5.5.3.m1.1.1.2">10</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.5.5.3.m1.1c">10\%</annotation></semantics></math></td>
<td id="S5.T2.6.6.4" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><math id="S5.T2.6.6.4.m1.1" class="ltx_Math" alttext="20\%" display="inline"><semantics id="S5.T2.6.6.4.m1.1a"><mrow id="S5.T2.6.6.4.m1.1.1" xref="S5.T2.6.6.4.m1.1.1.cmml"><mn id="S5.T2.6.6.4.m1.1.1.2" xref="S5.T2.6.6.4.m1.1.1.2.cmml">20</mn><mo id="S5.T2.6.6.4.m1.1.1.1" xref="S5.T2.6.6.4.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.T2.6.6.4.m1.1b"><apply id="S5.T2.6.6.4.m1.1.1.cmml" xref="S5.T2.6.6.4.m1.1.1"><csymbol cd="latexml" id="S5.T2.6.6.4.m1.1.1.1.cmml" xref="S5.T2.6.6.4.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.T2.6.6.4.m1.1.1.2.cmml" xref="S5.T2.6.6.4.m1.1.1.2">20</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.6.6.4.m1.1c">20\%</annotation></semantics></math></td>
</tr>
</tbody>
</table>
</figure>
<div id="S5.SS1.p4" class="ltx_para">
<p id="S5.SS1.p4.3" class="ltx_p">In <a href="#S5.T2" title="In 5.1 LoTA Design Choices ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">2</span></a>, we now consider how the performance drops if we calibrate the mask for a fraction of the overall dataset, including the baseline where we use a random mask which corresponds to <math id="S5.SS1.p4.1.m1.1" class="ltx_Math" alttext="0\%" display="inline"><semantics id="S5.SS1.p4.1.m1.1a"><mrow id="S5.SS1.p4.1.m1.1.1" xref="S5.SS1.p4.1.m1.1.1.cmml"><mn id="S5.SS1.p4.1.m1.1.1.2" xref="S5.SS1.p4.1.m1.1.1.2.cmml">0</mn><mo id="S5.SS1.p4.1.m1.1.1.1" xref="S5.SS1.p4.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p4.1.m1.1b"><apply id="S5.SS1.p4.1.m1.1.1.cmml" xref="S5.SS1.p4.1.m1.1.1"><csymbol cd="latexml" id="S5.SS1.p4.1.m1.1.1.1.cmml" xref="S5.SS1.p4.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.SS1.p4.1.m1.1.1.2.cmml" xref="S5.SS1.p4.1.m1.1.1.2">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p4.1.m1.1c">0\%</annotation></semantics></math> data used. LoTA can be efficiently calibrated, and even training on a small fraction of the dataset can provide <math id="S5.SS1.p4.2.m2.1" class="ltx_Math" alttext="90\%" display="inline"><semantics id="S5.SS1.p4.2.m2.1a"><mrow id="S5.SS1.p4.2.m2.1.1" xref="S5.SS1.p4.2.m2.1.1.cmml"><mn id="S5.SS1.p4.2.m2.1.1.2" xref="S5.SS1.p4.2.m2.1.1.2.cmml">90</mn><mo id="S5.SS1.p4.2.m2.1.1.1" xref="S5.SS1.p4.2.m2.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p4.2.m2.1b"><apply id="S5.SS1.p4.2.m2.1.1.cmml" xref="S5.SS1.p4.2.m2.1.1"><csymbol cd="latexml" id="S5.SS1.p4.2.m2.1.1.1.cmml" xref="S5.SS1.p4.2.m2.1.1.1">percent</csymbol><cn type="integer" id="S5.SS1.p4.2.m2.1.1.2.cmml" xref="S5.SS1.p4.2.m2.1.1.2">90</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p4.2.m2.1c">90\%</annotation></semantics></math> of the performance of the fully calibrated mask.
If we take just a single iteration to calibrate the mask for LoTA, we can obtain <math id="S5.SS1.p4.3.m3.1" class="ltx_Math" alttext="56.1\%" display="inline"><semantics id="S5.SS1.p4.3.m3.1a"><mrow id="S5.SS1.p4.3.m3.1.1" xref="S5.SS1.p4.3.m3.1.1.cmml"><mn id="S5.SS1.p4.3.m3.1.1.2" xref="S5.SS1.p4.3.m3.1.1.2.cmml">56.1</mn><mo id="S5.SS1.p4.3.m3.1.1.1" xref="S5.SS1.p4.3.m3.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p4.3.m3.1b"><apply id="S5.SS1.p4.3.m3.1.1.cmml" xref="S5.SS1.p4.3.m3.1.1"><csymbol cd="latexml" id="S5.SS1.p4.3.m3.1.1.1.cmml" xref="S5.SS1.p4.3.m3.1.1.1">percent</csymbol><cn type="float" id="S5.SS1.p4.3.m3.1.1.2.cmml" xref="S5.SS1.p4.3.m3.1.1.2">56.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p4.3.m3.1c">56.1\%</annotation></semantics></math> on GSM8k, higher than any result LoRA can obtain.
Furthermore, instruction tuning datasets are generally quite small (we completed all experiments in a few hours on just a single GPU). Given that the mask can be efficiently calibrated, transferred from other datasets, or in the worst case, calibrated on the entire dataset in a few hours, we do not anticipate that the compute overhead of LoTA will be a major limitation of the method.</p>
</div>
<div id="S5.SS1.p5" class="ltx_para">
<p id="S5.SS1.p5.1" class="ltx_p"><span id="S5.SS1.p5.1.1" class="ltx_text ltx_font_bold">Memory Overhead.</span> LoTA has a minor overhead when compared to FFT, because a bit-mask must be loaded for each parameter group. We provide an efficient implementation for this in our open-source code. repository. We can finetune Llama-3-8B with LoTA on a single 80GB A100, so we do not anticipate this minor memory overhead being a major limitation of our method.</p>
</div>
<div id="S5.SS1.p6" class="ltx_para">
<p id="S5.SS1.p6.3" class="ltx_p">We now evaluate LoTA and the baseline methods, FFT and LoRA, across all three paradigms of multi-task adaptation. In all experiments, we use LoTA with <math id="S5.SS1.p6.1.m1.1" class="ltx_Math" alttext="90\%" display="inline"><semantics id="S5.SS1.p6.1.m1.1a"><mrow id="S5.SS1.p6.1.m1.1.1" xref="S5.SS1.p6.1.m1.1.1.cmml"><mn id="S5.SS1.p6.1.m1.1.1.2" xref="S5.SS1.p6.1.m1.1.1.2.cmml">90</mn><mo id="S5.SS1.p6.1.m1.1.1.1" xref="S5.SS1.p6.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p6.1.m1.1b"><apply id="S5.SS1.p6.1.m1.1.1.cmml" xref="S5.SS1.p6.1.m1.1.1"><csymbol cd="latexml" id="S5.SS1.p6.1.m1.1.1.1.cmml" xref="S5.SS1.p6.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.SS1.p6.1.m1.1.1.2.cmml" xref="S5.SS1.p6.1.m1.1.1.2">90</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p6.1.m1.1c">90\%</annotation></semantics></math> sparsity where the mask is calibrated by training for <em id="S5.SS1.p6.3.1" class="ltx_emph ltx_font_italic">a single epoch</em> (<math id="S5.SS1.p6.2.m2.1" class="ltx_Math" alttext="\mathcal{T}" display="inline"><semantics id="S5.SS1.p6.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="S5.SS1.p6.2.m2.1.1" xref="S5.SS1.p6.2.m2.1.1.cmml">𝒯</mi><annotation-xml encoding="MathML-Content" id="S5.SS1.p6.2.m2.1b"><ci id="S5.SS1.p6.2.m2.1.1.cmml" xref="S5.SS1.p6.2.m2.1.1">𝒯</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p6.2.m2.1c">\mathcal{T}</annotation></semantics></math> is one epoch) on the adaptation dataset. We choose <math id="S5.SS1.p6.3.m3.1" class="ltx_Math" alttext="90\%" display="inline"><semantics id="S5.SS1.p6.3.m3.1a"><mrow id="S5.SS1.p6.3.m3.1.1" xref="S5.SS1.p6.3.m3.1.1.cmml"><mn id="S5.SS1.p6.3.m3.1.1.2" xref="S5.SS1.p6.3.m3.1.1.2.cmml">90</mn><mo id="S5.SS1.p6.3.m3.1.1.1" xref="S5.SS1.p6.3.m3.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p6.3.m3.1b"><apply id="S5.SS1.p6.3.m3.1.1.cmml" xref="S5.SS1.p6.3.m3.1.1"><csymbol cd="latexml" id="S5.SS1.p6.3.m3.1.1.1.cmml" xref="S5.SS1.p6.3.m3.1.1.1">percent</csymbol><cn type="integer" id="S5.SS1.p6.3.m3.1.1.2.cmml" xref="S5.SS1.p6.3.m3.1.1.2">90</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p6.3.m3.1c">90\%</annotation></semantics></math> sparsity because this is a nontrivial sparsity level that we find achieves good performance across the range of tasks we consider. We ablate the level of sparsity and amount of calibration data in <a href="#S5.SS1" title="5.1 LoTA Design Choices ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">5.1</span></a>.</p>
</div>
</section>
<section id="S5.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.2 </span>Adapting to a Single Task</h3>

<div id="S5.SS2.p1" class="ltx_para">
<p id="S5.SS2.p1.1" class="ltx_p">We first consider the simplest setting, in which we fit an adapter to each dataset of interest starting from a pre-trained base model, i.e., one adapter per task. In <a href="#S5.T3" title="In 5.2 Adapting to a Single Task ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">3</span></a>, we find that LoTA outperforms LoRA and performs similarly to FFT. Although LoRA is able to achieve similar performance to LoTA on the easier tasks, such as SQL, Samsum, there is a clear gap in performance on the more challenging tasks, such as Instruction Following and GSM8k. LoTA consistently recovers the performance of FFT.</p>
</div>
<figure id="S5.T3" class="ltx_table">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table"><span id="S5.T3.33.1.1" class="ltx_text" style="font-size:90%;">Table 3</span>: </span><span id="S5.T3.34.2" class="ltx_text" style="font-size:90%;">Performance comparison of Full Finetuning (FFT), LoRA and LoTA on single-task datasets for 3 epochs. We report the winrate on instruction following, the accuracy of exact match on the reasoning and math tasks, and the ROUGE-1 score on the SQL generation and summarization tasks. LoTA outperforms LoRA on the challenging tasks of instruction following, reasoning, and math, obtaining comparable performance to FFT. <span id="S5.T3.34.2.1" class="ltx_text ltx_font_bold">bold</span>: best method, <span id="S5.T3.34.2.2" class="ltx_text ltx_framed ltx_framed_underline">underline</span>: second best method.
</span></figcaption>
<div id="S5.T3.29" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:433.6pt;height:124.9pt;vertical-align:-1.0pt;"><span class="ltx_transformed_inner" style="transform:translate(-3.7pt,1.0pt) scale(0.983443293214372,0.983443293214372) ;">
<table id="S5.T3.29.29" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S5.T3.29.29.30.1" class="ltx_tr">
<th id="S5.T3.29.29.30.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_th_row ltx_border_tt">Model</th>
<th id="S5.T3.29.29.30.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_th_row ltx_border_r ltx_border_tt">Method</th>
<th id="S5.T3.29.29.30.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">Instruction Following</th>
<th id="S5.T3.29.29.30.1.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">Reasoning</th>
<th id="S5.T3.29.29.30.1.5" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">GSM8k</th>
<th id="S5.T3.29.29.30.1.6" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">SQL</th>
<th id="S5.T3.29.29.30.1.7" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">Summarization</th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S5.T3.5.5.5" class="ltx_tr">
<th id="S5.T3.5.5.5.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t" rowspan="3"><span id="S5.T3.5.5.5.6.1" class="ltx_text">Mistral</span></th>
<th id="S5.T3.5.5.5.7" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_r ltx_border_t">FFT</th>
<td id="S5.T3.1.1.1.1" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.1.1.1.1.m1.1" class="ltx_Math" alttext="\underline{19.0_{0.98}}" display="inline"><semantics id="S5.T3.1.1.1.1.m1.1a"><munder accentunder="true" id="S5.T3.1.1.1.1.m1.1.1" xref="S5.T3.1.1.1.1.m1.1.1.cmml"><msub id="S5.T3.1.1.1.1.m1.1.1.2" xref="S5.T3.1.1.1.1.m1.1.1.2.cmml"><mn id="S5.T3.1.1.1.1.m1.1.1.2.2" xref="S5.T3.1.1.1.1.m1.1.1.2.2.cmml">19.0</mn><mn id="S5.T3.1.1.1.1.m1.1.1.2.3" xref="S5.T3.1.1.1.1.m1.1.1.2.3.cmml">0.98</mn></msub><mo id="S5.T3.1.1.1.1.m1.1.1.1" xref="S5.T3.1.1.1.1.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.1.1.1.1.m1.1b"><apply id="S5.T3.1.1.1.1.m1.1.1.cmml" xref="S5.T3.1.1.1.1.m1.1.1"><ci id="S5.T3.1.1.1.1.m1.1.1.1.cmml" xref="S5.T3.1.1.1.1.m1.1.1.1">¯</ci><apply id="S5.T3.1.1.1.1.m1.1.1.2.cmml" xref="S5.T3.1.1.1.1.m1.1.1.2"><csymbol cd="ambiguous" id="S5.T3.1.1.1.1.m1.1.1.2.1.cmml" xref="S5.T3.1.1.1.1.m1.1.1.2">subscript</csymbol><cn type="float" id="S5.T3.1.1.1.1.m1.1.1.2.2.cmml" xref="S5.T3.1.1.1.1.m1.1.1.2.2">19.0</cn><cn type="float" id="S5.T3.1.1.1.1.m1.1.1.2.3.cmml" xref="S5.T3.1.1.1.1.m1.1.1.2.3">0.98</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.1.1.1.1.m1.1c">\underline{19.0_{0.98}}</annotation></semantics></math></td>
<td id="S5.T3.2.2.2.2" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.2.2.2.2.m1.1" class="ltx_Math" alttext="83.5" display="inline"><semantics id="S5.T3.2.2.2.2.m1.1a"><mn id="S5.T3.2.2.2.2.m1.1.1" xref="S5.T3.2.2.2.2.m1.1.1.cmml">83.5</mn><annotation-xml encoding="MathML-Content" id="S5.T3.2.2.2.2.m1.1b"><cn type="float" id="S5.T3.2.2.2.2.m1.1.1.cmml" xref="S5.T3.2.2.2.2.m1.1.1">83.5</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.2.2.2.2.m1.1c">83.5</annotation></semantics></math></td>
<td id="S5.T3.3.3.3.3" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.3.3.3.3.m1.1" class="ltx_Math" alttext="\mathbf{59.8_{1.0}}" display="inline"><semantics id="S5.T3.3.3.3.3.m1.1a"><msub id="S5.T3.3.3.3.3.m1.1.1" xref="S5.T3.3.3.3.3.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.3.3.3.3.m1.1.1.2" xref="S5.T3.3.3.3.3.m1.1.1.2.cmml">59.8</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.3.3.3.3.m1.1.1.3" xref="S5.T3.3.3.3.3.m1.1.1.3.cmml">1.0</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.3.3.3.3.m1.1b"><apply id="S5.T3.3.3.3.3.m1.1.1.cmml" xref="S5.T3.3.3.3.3.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.3.3.3.3.m1.1.1.1.cmml" xref="S5.T3.3.3.3.3.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.3.3.3.3.m1.1.1.2.cmml" xref="S5.T3.3.3.3.3.m1.1.1.2">59.8</cn><cn type="float" id="S5.T3.3.3.3.3.m1.1.1.3.cmml" xref="S5.T3.3.3.3.3.m1.1.1.3">1.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.3.3.3.3.m1.1c">\mathbf{59.8_{1.0}}</annotation></semantics></math></td>
<td id="S5.T3.4.4.4.4" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.4.4.4.4.m1.1" class="ltx_Math" alttext="98.9_{0.1}" display="inline"><semantics id="S5.T3.4.4.4.4.m1.1a"><msub id="S5.T3.4.4.4.4.m1.1.1" xref="S5.T3.4.4.4.4.m1.1.1.cmml"><mn id="S5.T3.4.4.4.4.m1.1.1.2" xref="S5.T3.4.4.4.4.m1.1.1.2.cmml">98.9</mn><mn id="S5.T3.4.4.4.4.m1.1.1.3" xref="S5.T3.4.4.4.4.m1.1.1.3.cmml">0.1</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.4.4.4.4.m1.1b"><apply id="S5.T3.4.4.4.4.m1.1.1.cmml" xref="S5.T3.4.4.4.4.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.4.4.4.4.m1.1.1.1.cmml" xref="S5.T3.4.4.4.4.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.4.4.4.4.m1.1.1.2.cmml" xref="S5.T3.4.4.4.4.m1.1.1.2">98.9</cn><cn type="float" id="S5.T3.4.4.4.4.m1.1.1.3.cmml" xref="S5.T3.4.4.4.4.m1.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.4.4.4.4.m1.1c">98.9_{0.1}</annotation></semantics></math></td>
<td id="S5.T3.5.5.5.5" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.5.5.5.5.m1.1" class="ltx_Math" alttext="52.0_{0.2}" display="inline"><semantics id="S5.T3.5.5.5.5.m1.1a"><msub id="S5.T3.5.5.5.5.m1.1.1" xref="S5.T3.5.5.5.5.m1.1.1.cmml"><mn id="S5.T3.5.5.5.5.m1.1.1.2" xref="S5.T3.5.5.5.5.m1.1.1.2.cmml">52.0</mn><mn id="S5.T3.5.5.5.5.m1.1.1.3" xref="S5.T3.5.5.5.5.m1.1.1.3.cmml">0.2</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.5.5.5.5.m1.1b"><apply id="S5.T3.5.5.5.5.m1.1.1.cmml" xref="S5.T3.5.5.5.5.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.5.5.5.5.m1.1.1.1.cmml" xref="S5.T3.5.5.5.5.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.5.5.5.5.m1.1.1.2.cmml" xref="S5.T3.5.5.5.5.m1.1.1.2">52.0</cn><cn type="float" id="S5.T3.5.5.5.5.m1.1.1.3.cmml" xref="S5.T3.5.5.5.5.m1.1.1.3">0.2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.5.5.5.5.m1.1c">52.0_{0.2}</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.10.10.10" class="ltx_tr">
<th id="S5.T3.10.10.10.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_r">LoRA</th>
<td id="S5.T3.6.6.6.1" class="ltx_td ltx_align_center"><math id="S5.T3.6.6.6.1.m1.1" class="ltx_Math" alttext="15.3_{0.8}" display="inline"><semantics id="S5.T3.6.6.6.1.m1.1a"><msub id="S5.T3.6.6.6.1.m1.1.1" xref="S5.T3.6.6.6.1.m1.1.1.cmml"><mn id="S5.T3.6.6.6.1.m1.1.1.2" xref="S5.T3.6.6.6.1.m1.1.1.2.cmml">15.3</mn><mn id="S5.T3.6.6.6.1.m1.1.1.3" xref="S5.T3.6.6.6.1.m1.1.1.3.cmml">0.8</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.6.6.6.1.m1.1b"><apply id="S5.T3.6.6.6.1.m1.1.1.cmml" xref="S5.T3.6.6.6.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.6.6.6.1.m1.1.1.1.cmml" xref="S5.T3.6.6.6.1.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.6.6.6.1.m1.1.1.2.cmml" xref="S5.T3.6.6.6.1.m1.1.1.2">15.3</cn><cn type="float" id="S5.T3.6.6.6.1.m1.1.1.3.cmml" xref="S5.T3.6.6.6.1.m1.1.1.3">0.8</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.6.6.6.1.m1.1c">15.3_{0.8}</annotation></semantics></math></td>
<td id="S5.T3.7.7.7.2" class="ltx_td ltx_align_center"><span id="S5.T3.7.7.7.2.1" class="ltx_text ltx_framed ltx_framed_underline"><math id="S5.T3.7.7.7.2.1.m1.1" class="ltx_Math" alttext="86.8" display="inline"><semantics id="S5.T3.7.7.7.2.1.m1.1a"><mn id="S5.T3.7.7.7.2.1.m1.1.1" xref="S5.T3.7.7.7.2.1.m1.1.1.cmml">86.8</mn><annotation-xml encoding="MathML-Content" id="S5.T3.7.7.7.2.1.m1.1b"><cn type="float" id="S5.T3.7.7.7.2.1.m1.1.1.cmml" xref="S5.T3.7.7.7.2.1.m1.1.1">86.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.7.7.7.2.1.m1.1c">86.8</annotation></semantics></math></span></td>
<td id="S5.T3.8.8.8.3" class="ltx_td ltx_align_center"><math id="S5.T3.8.8.8.3.m1.1" class="ltx_Math" alttext="57.0_{1.1}" display="inline"><semantics id="S5.T3.8.8.8.3.m1.1a"><msub id="S5.T3.8.8.8.3.m1.1.1" xref="S5.T3.8.8.8.3.m1.1.1.cmml"><mn id="S5.T3.8.8.8.3.m1.1.1.2" xref="S5.T3.8.8.8.3.m1.1.1.2.cmml">57.0</mn><mn id="S5.T3.8.8.8.3.m1.1.1.3" xref="S5.T3.8.8.8.3.m1.1.1.3.cmml">1.1</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.8.8.8.3.m1.1b"><apply id="S5.T3.8.8.8.3.m1.1.1.cmml" xref="S5.T3.8.8.8.3.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.8.8.8.3.m1.1.1.1.cmml" xref="S5.T3.8.8.8.3.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.8.8.8.3.m1.1.1.2.cmml" xref="S5.T3.8.8.8.3.m1.1.1.2">57.0</cn><cn type="float" id="S5.T3.8.8.8.3.m1.1.1.3.cmml" xref="S5.T3.8.8.8.3.m1.1.1.3">1.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.8.8.8.3.m1.1c">57.0_{1.1}</annotation></semantics></math></td>
<td id="S5.T3.9.9.9.4" class="ltx_td ltx_align_center"><math id="S5.T3.9.9.9.4.m1.1" class="ltx_Math" alttext="98.9_{0.1}" display="inline"><semantics id="S5.T3.9.9.9.4.m1.1a"><msub id="S5.T3.9.9.9.4.m1.1.1" xref="S5.T3.9.9.9.4.m1.1.1.cmml"><mn id="S5.T3.9.9.9.4.m1.1.1.2" xref="S5.T3.9.9.9.4.m1.1.1.2.cmml">98.9</mn><mn id="S5.T3.9.9.9.4.m1.1.1.3" xref="S5.T3.9.9.9.4.m1.1.1.3.cmml">0.1</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.9.9.9.4.m1.1b"><apply id="S5.T3.9.9.9.4.m1.1.1.cmml" xref="S5.T3.9.9.9.4.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.9.9.9.4.m1.1.1.1.cmml" xref="S5.T3.9.9.9.4.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.9.9.9.4.m1.1.1.2.cmml" xref="S5.T3.9.9.9.4.m1.1.1.2">98.9</cn><cn type="float" id="S5.T3.9.9.9.4.m1.1.1.3.cmml" xref="S5.T3.9.9.9.4.m1.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.9.9.9.4.m1.1c">98.9_{0.1}</annotation></semantics></math></td>
<td id="S5.T3.10.10.10.5" class="ltx_td ltx_align_center"><math id="S5.T3.10.10.10.5.m1.1" class="ltx_Math" alttext="\underline{52.9_{0.3}}" display="inline"><semantics id="S5.T3.10.10.10.5.m1.1a"><munder accentunder="true" id="S5.T3.10.10.10.5.m1.1.1" xref="S5.T3.10.10.10.5.m1.1.1.cmml"><msub id="S5.T3.10.10.10.5.m1.1.1.2" xref="S5.T3.10.10.10.5.m1.1.1.2.cmml"><mn id="S5.T3.10.10.10.5.m1.1.1.2.2" xref="S5.T3.10.10.10.5.m1.1.1.2.2.cmml">52.9</mn><mn id="S5.T3.10.10.10.5.m1.1.1.2.3" xref="S5.T3.10.10.10.5.m1.1.1.2.3.cmml">0.3</mn></msub><mo id="S5.T3.10.10.10.5.m1.1.1.1" xref="S5.T3.10.10.10.5.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.10.10.10.5.m1.1b"><apply id="S5.T3.10.10.10.5.m1.1.1.cmml" xref="S5.T3.10.10.10.5.m1.1.1"><ci id="S5.T3.10.10.10.5.m1.1.1.1.cmml" xref="S5.T3.10.10.10.5.m1.1.1.1">¯</ci><apply id="S5.T3.10.10.10.5.m1.1.1.2.cmml" xref="S5.T3.10.10.10.5.m1.1.1.2"><csymbol cd="ambiguous" id="S5.T3.10.10.10.5.m1.1.1.2.1.cmml" xref="S5.T3.10.10.10.5.m1.1.1.2">subscript</csymbol><cn type="float" id="S5.T3.10.10.10.5.m1.1.1.2.2.cmml" xref="S5.T3.10.10.10.5.m1.1.1.2.2">52.9</cn><cn type="float" id="S5.T3.10.10.10.5.m1.1.1.2.3.cmml" xref="S5.T3.10.10.10.5.m1.1.1.2.3">0.3</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.10.10.10.5.m1.1c">\underline{52.9_{0.3}}</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.15.15.15" class="ltx_tr">
<th id="S5.T3.15.15.15.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_r">LoTA (ours)</th>
<td id="S5.T3.11.11.11.1" class="ltx_td ltx_align_center"><math id="S5.T3.11.11.11.1.m1.1" class="ltx_Math" alttext="\mathbf{19.0_{0.7}}" display="inline"><semantics id="S5.T3.11.11.11.1.m1.1a"><msub id="S5.T3.11.11.11.1.m1.1.1" xref="S5.T3.11.11.11.1.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.11.11.11.1.m1.1.1.2" xref="S5.T3.11.11.11.1.m1.1.1.2.cmml">19.0</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.11.11.11.1.m1.1.1.3" xref="S5.T3.11.11.11.1.m1.1.1.3.cmml">0.7</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.11.11.11.1.m1.1b"><apply id="S5.T3.11.11.11.1.m1.1.1.cmml" xref="S5.T3.11.11.11.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.11.11.11.1.m1.1.1.1.cmml" xref="S5.T3.11.11.11.1.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.11.11.11.1.m1.1.1.2.cmml" xref="S5.T3.11.11.11.1.m1.1.1.2">19.0</cn><cn type="float" id="S5.T3.11.11.11.1.m1.1.1.3.cmml" xref="S5.T3.11.11.11.1.m1.1.1.3">0.7</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.11.11.11.1.m1.1c">\mathbf{19.0_{0.7}}</annotation></semantics></math></td>
<td id="S5.T3.12.12.12.2" class="ltx_td ltx_align_center"><math id="S5.T3.12.12.12.2.m1.1" class="ltx_Math" alttext="\mathbf{87.5}" display="inline"><semantics id="S5.T3.12.12.12.2.m1.1a"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.12.12.12.2.m1.1.1" xref="S5.T3.12.12.12.2.m1.1.1.cmml">87.5</mn><annotation-xml encoding="MathML-Content" id="S5.T3.12.12.12.2.m1.1b"><cn type="float" id="S5.T3.12.12.12.2.m1.1.1.cmml" xref="S5.T3.12.12.12.2.m1.1.1">87.5</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.12.12.12.2.m1.1c">\mathbf{87.5}</annotation></semantics></math></td>
<td id="S5.T3.13.13.13.3" class="ltx_td ltx_align_center"><math id="S5.T3.13.13.13.3.m1.1" class="ltx_Math" alttext="\underline{59.3_{1.1}}" display="inline"><semantics id="S5.T3.13.13.13.3.m1.1a"><munder accentunder="true" id="S5.T3.13.13.13.3.m1.1.1" xref="S5.T3.13.13.13.3.m1.1.1.cmml"><msub id="S5.T3.13.13.13.3.m1.1.1.2" xref="S5.T3.13.13.13.3.m1.1.1.2.cmml"><mn id="S5.T3.13.13.13.3.m1.1.1.2.2" xref="S5.T3.13.13.13.3.m1.1.1.2.2.cmml">59.3</mn><mn id="S5.T3.13.13.13.3.m1.1.1.2.3" xref="S5.T3.13.13.13.3.m1.1.1.2.3.cmml">1.1</mn></msub><mo id="S5.T3.13.13.13.3.m1.1.1.1" xref="S5.T3.13.13.13.3.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.13.13.13.3.m1.1b"><apply id="S5.T3.13.13.13.3.m1.1.1.cmml" xref="S5.T3.13.13.13.3.m1.1.1"><ci id="S5.T3.13.13.13.3.m1.1.1.1.cmml" xref="S5.T3.13.13.13.3.m1.1.1.1">¯</ci><apply id="S5.T3.13.13.13.3.m1.1.1.2.cmml" xref="S5.T3.13.13.13.3.m1.1.1.2"><csymbol cd="ambiguous" id="S5.T3.13.13.13.3.m1.1.1.2.1.cmml" xref="S5.T3.13.13.13.3.m1.1.1.2">subscript</csymbol><cn type="float" id="S5.T3.13.13.13.3.m1.1.1.2.2.cmml" xref="S5.T3.13.13.13.3.m1.1.1.2.2">59.3</cn><cn type="float" id="S5.T3.13.13.13.3.m1.1.1.2.3.cmml" xref="S5.T3.13.13.13.3.m1.1.1.2.3">1.1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.13.13.13.3.m1.1c">\underline{59.3_{1.1}}</annotation></semantics></math></td>
<td id="S5.T3.14.14.14.4" class="ltx_td ltx_align_center"><math id="S5.T3.14.14.14.4.m1.1" class="ltx_Math" alttext="\mathbf{98.9_{0.1}}" display="inline"><semantics id="S5.T3.14.14.14.4.m1.1a"><msub id="S5.T3.14.14.14.4.m1.1.1" xref="S5.T3.14.14.14.4.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.14.14.14.4.m1.1.1.2" xref="S5.T3.14.14.14.4.m1.1.1.2.cmml">98.9</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.14.14.14.4.m1.1.1.3" xref="S5.T3.14.14.14.4.m1.1.1.3.cmml">0.1</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.14.14.14.4.m1.1b"><apply id="S5.T3.14.14.14.4.m1.1.1.cmml" xref="S5.T3.14.14.14.4.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.14.14.14.4.m1.1.1.1.cmml" xref="S5.T3.14.14.14.4.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.14.14.14.4.m1.1.1.2.cmml" xref="S5.T3.14.14.14.4.m1.1.1.2">98.9</cn><cn type="float" id="S5.T3.14.14.14.4.m1.1.1.3.cmml" xref="S5.T3.14.14.14.4.m1.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.14.14.14.4.m1.1c">\mathbf{98.9_{0.1}}</annotation></semantics></math></td>
<td id="S5.T3.15.15.15.5" class="ltx_td ltx_align_center"><math id="S5.T3.15.15.15.5.m1.1" class="ltx_Math" alttext="\mathbf{52.9_{0.2}}" display="inline"><semantics id="S5.T3.15.15.15.5.m1.1a"><msub id="S5.T3.15.15.15.5.m1.1.1" xref="S5.T3.15.15.15.5.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.15.15.15.5.m1.1.1.2" xref="S5.T3.15.15.15.5.m1.1.1.2.cmml">52.9</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.15.15.15.5.m1.1.1.3" xref="S5.T3.15.15.15.5.m1.1.1.3.cmml">0.2</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.15.15.15.5.m1.1b"><apply id="S5.T3.15.15.15.5.m1.1.1.cmml" xref="S5.T3.15.15.15.5.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.15.15.15.5.m1.1.1.1.cmml" xref="S5.T3.15.15.15.5.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.15.15.15.5.m1.1.1.2.cmml" xref="S5.T3.15.15.15.5.m1.1.1.2">52.9</cn><cn type="float" id="S5.T3.15.15.15.5.m1.1.1.3.cmml" xref="S5.T3.15.15.15.5.m1.1.1.3">0.2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.15.15.15.5.m1.1c">\mathbf{52.9_{0.2}}</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.19.19.19" class="ltx_tr">
<th id="S5.T3.19.19.19.5" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_bb ltx_border_t" rowspan="3"><span id="S5.T3.19.19.19.5.1" class="ltx_text">Llama 3</span></th>
<th id="S5.T3.19.19.19.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_r ltx_border_t">FFT</th>
<td id="S5.T3.16.16.16.1" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.16.16.16.1.m1.1" class="ltx_Math" alttext="\underline{17.61_{0.8}}" display="inline"><semantics id="S5.T3.16.16.16.1.m1.1a"><munder accentunder="true" id="S5.T3.16.16.16.1.m1.1.1" xref="S5.T3.16.16.16.1.m1.1.1.cmml"><msub id="S5.T3.16.16.16.1.m1.1.1.2" xref="S5.T3.16.16.16.1.m1.1.1.2.cmml"><mn id="S5.T3.16.16.16.1.m1.1.1.2.2" xref="S5.T3.16.16.16.1.m1.1.1.2.2.cmml">17.61</mn><mn id="S5.T3.16.16.16.1.m1.1.1.2.3" xref="S5.T3.16.16.16.1.m1.1.1.2.3.cmml">0.8</mn></msub><mo id="S5.T3.16.16.16.1.m1.1.1.1" xref="S5.T3.16.16.16.1.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.16.16.16.1.m1.1b"><apply id="S5.T3.16.16.16.1.m1.1.1.cmml" xref="S5.T3.16.16.16.1.m1.1.1"><ci id="S5.T3.16.16.16.1.m1.1.1.1.cmml" xref="S5.T3.16.16.16.1.m1.1.1.1">¯</ci><apply id="S5.T3.16.16.16.1.m1.1.1.2.cmml" xref="S5.T3.16.16.16.1.m1.1.1.2"><csymbol cd="ambiguous" id="S5.T3.16.16.16.1.m1.1.1.2.1.cmml" xref="S5.T3.16.16.16.1.m1.1.1.2">subscript</csymbol><cn type="float" id="S5.T3.16.16.16.1.m1.1.1.2.2.cmml" xref="S5.T3.16.16.16.1.m1.1.1.2.2">17.61</cn><cn type="float" id="S5.T3.16.16.16.1.m1.1.1.2.3.cmml" xref="S5.T3.16.16.16.1.m1.1.1.2.3">0.8</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.16.16.16.1.m1.1c">\underline{17.61_{0.8}}</annotation></semantics></math></td>
<td id="S5.T3.19.19.19.7" class="ltx_td ltx_align_center ltx_border_t"><span id="S5.T3.19.19.19.7.1" class="ltx_text ltx_font_bold">84.8</span></td>
<td id="S5.T3.17.17.17.2" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.17.17.17.2.m1.1" class="ltx_Math" alttext="\mathbf{63.4_{0.1}}" display="inline"><semantics id="S5.T3.17.17.17.2.m1.1a"><msub id="S5.T3.17.17.17.2.m1.1.1" xref="S5.T3.17.17.17.2.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.17.17.17.2.m1.1.1.2" xref="S5.T3.17.17.17.2.m1.1.1.2.cmml">63.4</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.17.17.17.2.m1.1.1.3" xref="S5.T3.17.17.17.2.m1.1.1.3.cmml">0.1</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.17.17.17.2.m1.1b"><apply id="S5.T3.17.17.17.2.m1.1.1.cmml" xref="S5.T3.17.17.17.2.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.17.17.17.2.m1.1.1.1.cmml" xref="S5.T3.17.17.17.2.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.17.17.17.2.m1.1.1.2.cmml" xref="S5.T3.17.17.17.2.m1.1.1.2">63.4</cn><cn type="float" id="S5.T3.17.17.17.2.m1.1.1.3.cmml" xref="S5.T3.17.17.17.2.m1.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.17.17.17.2.m1.1c">\mathbf{63.4_{0.1}}</annotation></semantics></math></td>
<td id="S5.T3.18.18.18.3" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.18.18.18.3.m1.1" class="ltx_Math" alttext="\mathbf{99.4_{0.1}}" display="inline"><semantics id="S5.T3.18.18.18.3.m1.1a"><msub id="S5.T3.18.18.18.3.m1.1.1" xref="S5.T3.18.18.18.3.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.18.18.18.3.m1.1.1.2" xref="S5.T3.18.18.18.3.m1.1.1.2.cmml">99.4</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.18.18.18.3.m1.1.1.3" xref="S5.T3.18.18.18.3.m1.1.1.3.cmml">0.1</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.18.18.18.3.m1.1b"><apply id="S5.T3.18.18.18.3.m1.1.1.cmml" xref="S5.T3.18.18.18.3.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.18.18.18.3.m1.1.1.1.cmml" xref="S5.T3.18.18.18.3.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.18.18.18.3.m1.1.1.2.cmml" xref="S5.T3.18.18.18.3.m1.1.1.2">99.4</cn><cn type="float" id="S5.T3.18.18.18.3.m1.1.1.3.cmml" xref="S5.T3.18.18.18.3.m1.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.18.18.18.3.m1.1c">\mathbf{99.4_{0.1}}</annotation></semantics></math></td>
<td id="S5.T3.19.19.19.4" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.19.19.19.4.m1.1" class="ltx_Math" alttext="\mathbf{53.6_{1.9}}" display="inline"><semantics id="S5.T3.19.19.19.4.m1.1a"><msub id="S5.T3.19.19.19.4.m1.1.1" xref="S5.T3.19.19.19.4.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.19.19.19.4.m1.1.1.2" xref="S5.T3.19.19.19.4.m1.1.1.2.cmml">53.6</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.19.19.19.4.m1.1.1.3" xref="S5.T3.19.19.19.4.m1.1.1.3.cmml">1.9</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.19.19.19.4.m1.1b"><apply id="S5.T3.19.19.19.4.m1.1.1.cmml" xref="S5.T3.19.19.19.4.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.19.19.19.4.m1.1.1.1.cmml" xref="S5.T3.19.19.19.4.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.19.19.19.4.m1.1.1.2.cmml" xref="S5.T3.19.19.19.4.m1.1.1.2">53.6</cn><cn type="float" id="S5.T3.19.19.19.4.m1.1.1.3.cmml" xref="S5.T3.19.19.19.4.m1.1.1.3">1.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.19.19.19.4.m1.1c">\mathbf{53.6_{1.9}}</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.24.24.24" class="ltx_tr">
<th id="S5.T3.24.24.24.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_r">LoRA</th>
<td id="S5.T3.20.20.20.1" class="ltx_td ltx_align_center"><math id="S5.T3.20.20.20.1.m1.1" class="ltx_Math" alttext="14.2_{0.8}" display="inline"><semantics id="S5.T3.20.20.20.1.m1.1a"><msub id="S5.T3.20.20.20.1.m1.1.1" xref="S5.T3.20.20.20.1.m1.1.1.cmml"><mn id="S5.T3.20.20.20.1.m1.1.1.2" xref="S5.T3.20.20.20.1.m1.1.1.2.cmml">14.2</mn><mn id="S5.T3.20.20.20.1.m1.1.1.3" xref="S5.T3.20.20.20.1.m1.1.1.3.cmml">0.8</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.20.20.20.1.m1.1b"><apply id="S5.T3.20.20.20.1.m1.1.1.cmml" xref="S5.T3.20.20.20.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.20.20.20.1.m1.1.1.1.cmml" xref="S5.T3.20.20.20.1.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.20.20.20.1.m1.1.1.2.cmml" xref="S5.T3.20.20.20.1.m1.1.1.2">14.2</cn><cn type="float" id="S5.T3.20.20.20.1.m1.1.1.3.cmml" xref="S5.T3.20.20.20.1.m1.1.1.3">0.8</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.20.20.20.1.m1.1c">14.2_{0.8}</annotation></semantics></math></td>
<td id="S5.T3.21.21.21.2" class="ltx_td ltx_align_center"><math id="S5.T3.21.21.21.2.m1.1" class="ltx_Math" alttext="84.1" display="inline"><semantics id="S5.T3.21.21.21.2.m1.1a"><mn id="S5.T3.21.21.21.2.m1.1.1" xref="S5.T3.21.21.21.2.m1.1.1.cmml">84.1</mn><annotation-xml encoding="MathML-Content" id="S5.T3.21.21.21.2.m1.1b"><cn type="float" id="S5.T3.21.21.21.2.m1.1.1.cmml" xref="S5.T3.21.21.21.2.m1.1.1">84.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.21.21.21.2.m1.1c">84.1</annotation></semantics></math></td>
<td id="S5.T3.22.22.22.3" class="ltx_td ltx_align_center"><math id="S5.T3.22.22.22.3.m1.1" class="ltx_Math" alttext="62.3_{0.4}" display="inline"><semantics id="S5.T3.22.22.22.3.m1.1a"><msub id="S5.T3.22.22.22.3.m1.1.1" xref="S5.T3.22.22.22.3.m1.1.1.cmml"><mn id="S5.T3.22.22.22.3.m1.1.1.2" xref="S5.T3.22.22.22.3.m1.1.1.2.cmml">62.3</mn><mn id="S5.T3.22.22.22.3.m1.1.1.3" xref="S5.T3.22.22.22.3.m1.1.1.3.cmml">0.4</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.22.22.22.3.m1.1b"><apply id="S5.T3.22.22.22.3.m1.1.1.cmml" xref="S5.T3.22.22.22.3.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.22.22.22.3.m1.1.1.1.cmml" xref="S5.T3.22.22.22.3.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.22.22.22.3.m1.1.1.2.cmml" xref="S5.T3.22.22.22.3.m1.1.1.2">62.3</cn><cn type="float" id="S5.T3.22.22.22.3.m1.1.1.3.cmml" xref="S5.T3.22.22.22.3.m1.1.1.3">0.4</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.22.22.22.3.m1.1c">62.3_{0.4}</annotation></semantics></math></td>
<td id="S5.T3.23.23.23.4" class="ltx_td ltx_align_center"><math id="S5.T3.23.23.23.4.m1.1" class="ltx_Math" alttext="98.7_{0.1}" display="inline"><semantics id="S5.T3.23.23.23.4.m1.1a"><msub id="S5.T3.23.23.23.4.m1.1.1" xref="S5.T3.23.23.23.4.m1.1.1.cmml"><mn id="S5.T3.23.23.23.4.m1.1.1.2" xref="S5.T3.23.23.23.4.m1.1.1.2.cmml">98.7</mn><mn id="S5.T3.23.23.23.4.m1.1.1.3" xref="S5.T3.23.23.23.4.m1.1.1.3.cmml">0.1</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.23.23.23.4.m1.1b"><apply id="S5.T3.23.23.23.4.m1.1.1.cmml" xref="S5.T3.23.23.23.4.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.23.23.23.4.m1.1.1.1.cmml" xref="S5.T3.23.23.23.4.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.23.23.23.4.m1.1.1.2.cmml" xref="S5.T3.23.23.23.4.m1.1.1.2">98.7</cn><cn type="float" id="S5.T3.23.23.23.4.m1.1.1.3.cmml" xref="S5.T3.23.23.23.4.m1.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.23.23.23.4.m1.1c">98.7_{0.1}</annotation></semantics></math></td>
<td id="S5.T3.24.24.24.5" class="ltx_td ltx_align_center"><math id="S5.T3.24.24.24.5.m1.1" class="ltx_Math" alttext="\underline{52.3_{0.2}}" display="inline"><semantics id="S5.T3.24.24.24.5.m1.1a"><munder accentunder="true" id="S5.T3.24.24.24.5.m1.1.1" xref="S5.T3.24.24.24.5.m1.1.1.cmml"><msub id="S5.T3.24.24.24.5.m1.1.1.2" xref="S5.T3.24.24.24.5.m1.1.1.2.cmml"><mn id="S5.T3.24.24.24.5.m1.1.1.2.2" xref="S5.T3.24.24.24.5.m1.1.1.2.2.cmml">52.3</mn><mn id="S5.T3.24.24.24.5.m1.1.1.2.3" xref="S5.T3.24.24.24.5.m1.1.1.2.3.cmml">0.2</mn></msub><mo id="S5.T3.24.24.24.5.m1.1.1.1" xref="S5.T3.24.24.24.5.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.24.24.24.5.m1.1b"><apply id="S5.T3.24.24.24.5.m1.1.1.cmml" xref="S5.T3.24.24.24.5.m1.1.1"><ci id="S5.T3.24.24.24.5.m1.1.1.1.cmml" xref="S5.T3.24.24.24.5.m1.1.1.1">¯</ci><apply id="S5.T3.24.24.24.5.m1.1.1.2.cmml" xref="S5.T3.24.24.24.5.m1.1.1.2"><csymbol cd="ambiguous" id="S5.T3.24.24.24.5.m1.1.1.2.1.cmml" xref="S5.T3.24.24.24.5.m1.1.1.2">subscript</csymbol><cn type="float" id="S5.T3.24.24.24.5.m1.1.1.2.2.cmml" xref="S5.T3.24.24.24.5.m1.1.1.2.2">52.3</cn><cn type="float" id="S5.T3.24.24.24.5.m1.1.1.2.3.cmml" xref="S5.T3.24.24.24.5.m1.1.1.2.3">0.2</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.24.24.24.5.m1.1c">\underline{52.3_{0.2}}</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.29.29.29" class="ltx_tr">
<th id="S5.T3.29.29.29.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_bb ltx_border_r">LoTA (ours)</th>
<td id="S5.T3.25.25.25.1" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.25.25.25.1.m1.1" class="ltx_Math" alttext="\mathbf{18.0_{0.7}}" display="inline"><semantics id="S5.T3.25.25.25.1.m1.1a"><msub id="S5.T3.25.25.25.1.m1.1.1" xref="S5.T3.25.25.25.1.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.25.25.25.1.m1.1.1.2" xref="S5.T3.25.25.25.1.m1.1.1.2.cmml">18.0</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.25.25.25.1.m1.1.1.3" xref="S5.T3.25.25.25.1.m1.1.1.3.cmml">0.7</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T3.25.25.25.1.m1.1b"><apply id="S5.T3.25.25.25.1.m1.1.1.cmml" xref="S5.T3.25.25.25.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.25.25.25.1.m1.1.1.1.cmml" xref="S5.T3.25.25.25.1.m1.1.1">subscript</csymbol><cn type="float" id="S5.T3.25.25.25.1.m1.1.1.2.cmml" xref="S5.T3.25.25.25.1.m1.1.1.2">18.0</cn><cn type="float" id="S5.T3.25.25.25.1.m1.1.1.3.cmml" xref="S5.T3.25.25.25.1.m1.1.1.3">0.7</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.25.25.25.1.m1.1c">\mathbf{18.0_{0.7}}</annotation></semantics></math></td>
<td id="S5.T3.26.26.26.2" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.26.26.26.2.m1.1" class="ltx_Math" alttext="\underline{84.4}" display="inline"><semantics id="S5.T3.26.26.26.2.m1.1a"><munder accentunder="true" id="S5.T3.26.26.26.2.m1.1.1" xref="S5.T3.26.26.26.2.m1.1.1.cmml"><mn id="S5.T3.26.26.26.2.m1.1.1.2" xref="S5.T3.26.26.26.2.m1.1.1.2.cmml">84.4</mn><mo id="S5.T3.26.26.26.2.m1.1.1.1" xref="S5.T3.26.26.26.2.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.26.26.26.2.m1.1b"><apply id="S5.T3.26.26.26.2.m1.1.1.cmml" xref="S5.T3.26.26.26.2.m1.1.1"><ci id="S5.T3.26.26.26.2.m1.1.1.1.cmml" xref="S5.T3.26.26.26.2.m1.1.1.1">¯</ci><cn type="float" id="S5.T3.26.26.26.2.m1.1.1.2.cmml" xref="S5.T3.26.26.26.2.m1.1.1.2">84.4</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.26.26.26.2.m1.1c">\underline{84.4}</annotation></semantics></math></td>
<td id="S5.T3.27.27.27.3" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.27.27.27.3.m1.1" class="ltx_Math" alttext="\underline{63.2_{0.7}}" display="inline"><semantics id="S5.T3.27.27.27.3.m1.1a"><munder accentunder="true" id="S5.T3.27.27.27.3.m1.1.1" xref="S5.T3.27.27.27.3.m1.1.1.cmml"><msub id="S5.T3.27.27.27.3.m1.1.1.2" xref="S5.T3.27.27.27.3.m1.1.1.2.cmml"><mn id="S5.T3.27.27.27.3.m1.1.1.2.2" xref="S5.T3.27.27.27.3.m1.1.1.2.2.cmml">63.2</mn><mn id="S5.T3.27.27.27.3.m1.1.1.2.3" xref="S5.T3.27.27.27.3.m1.1.1.2.3.cmml">0.7</mn></msub><mo id="S5.T3.27.27.27.3.m1.1.1.1" xref="S5.T3.27.27.27.3.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.27.27.27.3.m1.1b"><apply id="S5.T3.27.27.27.3.m1.1.1.cmml" xref="S5.T3.27.27.27.3.m1.1.1"><ci id="S5.T3.27.27.27.3.m1.1.1.1.cmml" xref="S5.T3.27.27.27.3.m1.1.1.1">¯</ci><apply id="S5.T3.27.27.27.3.m1.1.1.2.cmml" xref="S5.T3.27.27.27.3.m1.1.1.2"><csymbol cd="ambiguous" id="S5.T3.27.27.27.3.m1.1.1.2.1.cmml" xref="S5.T3.27.27.27.3.m1.1.1.2">subscript</csymbol><cn type="float" id="S5.T3.27.27.27.3.m1.1.1.2.2.cmml" xref="S5.T3.27.27.27.3.m1.1.1.2.2">63.2</cn><cn type="float" id="S5.T3.27.27.27.3.m1.1.1.2.3.cmml" xref="S5.T3.27.27.27.3.m1.1.1.2.3">0.7</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.27.27.27.3.m1.1c">\underline{63.2_{0.7}}</annotation></semantics></math></td>
<td id="S5.T3.28.28.28.4" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.28.28.28.4.m1.1" class="ltx_Math" alttext="\underline{99.0_{0.1}}" display="inline"><semantics id="S5.T3.28.28.28.4.m1.1a"><munder accentunder="true" id="S5.T3.28.28.28.4.m1.1.1" xref="S5.T3.28.28.28.4.m1.1.1.cmml"><msub id="S5.T3.28.28.28.4.m1.1.1.2" xref="S5.T3.28.28.28.4.m1.1.1.2.cmml"><mn id="S5.T3.28.28.28.4.m1.1.1.2.2" xref="S5.T3.28.28.28.4.m1.1.1.2.2.cmml">99.0</mn><mn id="S5.T3.28.28.28.4.m1.1.1.2.3" xref="S5.T3.28.28.28.4.m1.1.1.2.3.cmml">0.1</mn></msub><mo id="S5.T3.28.28.28.4.m1.1.1.1" xref="S5.T3.28.28.28.4.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.28.28.28.4.m1.1b"><apply id="S5.T3.28.28.28.4.m1.1.1.cmml" xref="S5.T3.28.28.28.4.m1.1.1"><ci id="S5.T3.28.28.28.4.m1.1.1.1.cmml" xref="S5.T3.28.28.28.4.m1.1.1.1">¯</ci><apply id="S5.T3.28.28.28.4.m1.1.1.2.cmml" xref="S5.T3.28.28.28.4.m1.1.1.2"><csymbol cd="ambiguous" id="S5.T3.28.28.28.4.m1.1.1.2.1.cmml" xref="S5.T3.28.28.28.4.m1.1.1.2">subscript</csymbol><cn type="float" id="S5.T3.28.28.28.4.m1.1.1.2.2.cmml" xref="S5.T3.28.28.28.4.m1.1.1.2.2">99.0</cn><cn type="float" id="S5.T3.28.28.28.4.m1.1.1.2.3.cmml" xref="S5.T3.28.28.28.4.m1.1.1.2.3">0.1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.28.28.28.4.m1.1c">\underline{99.0_{0.1}}</annotation></semantics></math></td>
<td id="S5.T3.29.29.29.5" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.29.29.29.5.m1.1" class="ltx_Math" alttext="\underline{52.3_{0.3}}" display="inline"><semantics id="S5.T3.29.29.29.5.m1.1a"><munder accentunder="true" id="S5.T3.29.29.29.5.m1.1.1" xref="S5.T3.29.29.29.5.m1.1.1.cmml"><msub id="S5.T3.29.29.29.5.m1.1.1.2" xref="S5.T3.29.29.29.5.m1.1.1.2.cmml"><mn id="S5.T3.29.29.29.5.m1.1.1.2.2" xref="S5.T3.29.29.29.5.m1.1.1.2.2.cmml">52.3</mn><mn id="S5.T3.29.29.29.5.m1.1.1.2.3" xref="S5.T3.29.29.29.5.m1.1.1.2.3.cmml">0.3</mn></msub><mo id="S5.T3.29.29.29.5.m1.1.1.1" xref="S5.T3.29.29.29.5.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.29.29.29.5.m1.1b"><apply id="S5.T3.29.29.29.5.m1.1.1.cmml" xref="S5.T3.29.29.29.5.m1.1.1"><ci id="S5.T3.29.29.29.5.m1.1.1.1.cmml" xref="S5.T3.29.29.29.5.m1.1.1.1">¯</ci><apply id="S5.T3.29.29.29.5.m1.1.1.2.cmml" xref="S5.T3.29.29.29.5.m1.1.1.2"><csymbol cd="ambiguous" id="S5.T3.29.29.29.5.m1.1.1.2.1.cmml" xref="S5.T3.29.29.29.5.m1.1.1.2">subscript</csymbol><cn type="float" id="S5.T3.29.29.29.5.m1.1.1.2.2.cmml" xref="S5.T3.29.29.29.5.m1.1.1.2.2">52.3</cn><cn type="float" id="S5.T3.29.29.29.5.m1.1.1.2.3.cmml" xref="S5.T3.29.29.29.5.m1.1.1.2.3">0.3</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.29.29.29.5.m1.1c">\underline{52.3_{0.3}}</annotation></semantics></math></td>
</tr>
</tbody>
</table>
</span></div>
</figure>
<div id="S5.SS2.p2" class="ltx_para">
<p id="S5.SS2.p2.2" class="ltx_p"><span id="S5.SS2.p2.2.1" class="ltx_text ltx_font_bold">High-Rank Adaptation Beats Low-Rank Adaptation.</span> LoRA has a heavy regularizing effect on training. On the commonsense reasoning task, because the base model can get nontrivial zero-shot performance, LoRA actually improves the performance over that of FFT for Mistral. Note here that we do a grid search over learning rate and rank for LoRA, and the best performance is at <math id="S5.SS2.p2.1.m1.2" class="ltx_Math" alttext="r=64,\eta=1e-4" display="inline"><semantics id="S5.SS2.p2.1.m1.2a"><mrow id="S5.SS2.p2.1.m1.2.2.2" xref="S5.SS2.p2.1.m1.2.2.3.cmml"><mrow id="S5.SS2.p2.1.m1.1.1.1.1" xref="S5.SS2.p2.1.m1.1.1.1.1.cmml"><mi id="S5.SS2.p2.1.m1.1.1.1.1.2" xref="S5.SS2.p2.1.m1.1.1.1.1.2.cmml">r</mi><mo id="S5.SS2.p2.1.m1.1.1.1.1.1" xref="S5.SS2.p2.1.m1.1.1.1.1.1.cmml">=</mo><mn id="S5.SS2.p2.1.m1.1.1.1.1.3" xref="S5.SS2.p2.1.m1.1.1.1.1.3.cmml">64</mn></mrow><mo id="S5.SS2.p2.1.m1.2.2.2.3" xref="S5.SS2.p2.1.m1.2.2.3a.cmml">,</mo><mrow id="S5.SS2.p2.1.m1.2.2.2.2" xref="S5.SS2.p2.1.m1.2.2.2.2.cmml"><mi id="S5.SS2.p2.1.m1.2.2.2.2.2" xref="S5.SS2.p2.1.m1.2.2.2.2.2.cmml">η</mi><mo id="S5.SS2.p2.1.m1.2.2.2.2.1" xref="S5.SS2.p2.1.m1.2.2.2.2.1.cmml">=</mo><mrow id="S5.SS2.p2.1.m1.2.2.2.2.3" xref="S5.SS2.p2.1.m1.2.2.2.2.3.cmml"><mrow id="S5.SS2.p2.1.m1.2.2.2.2.3.2" xref="S5.SS2.p2.1.m1.2.2.2.2.3.2.cmml"><mn id="S5.SS2.p2.1.m1.2.2.2.2.3.2.2" xref="S5.SS2.p2.1.m1.2.2.2.2.3.2.2.cmml">1</mn><mo lspace="0em" rspace="0em" id="S5.SS2.p2.1.m1.2.2.2.2.3.2.1" xref="S5.SS2.p2.1.m1.2.2.2.2.3.2.1.cmml">​</mo><mi id="S5.SS2.p2.1.m1.2.2.2.2.3.2.3" xref="S5.SS2.p2.1.m1.2.2.2.2.3.2.3.cmml">e</mi></mrow><mo id="S5.SS2.p2.1.m1.2.2.2.2.3.1" xref="S5.SS2.p2.1.m1.2.2.2.2.3.1.cmml">−</mo><mn id="S5.SS2.p2.1.m1.2.2.2.2.3.3" xref="S5.SS2.p2.1.m1.2.2.2.2.3.3.cmml">4</mn></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.p2.1.m1.2b"><apply id="S5.SS2.p2.1.m1.2.2.3.cmml" xref="S5.SS2.p2.1.m1.2.2.2"><csymbol cd="ambiguous" id="S5.SS2.p2.1.m1.2.2.3a.cmml" xref="S5.SS2.p2.1.m1.2.2.2.3">formulae-sequence</csymbol><apply id="S5.SS2.p2.1.m1.1.1.1.1.cmml" xref="S5.SS2.p2.1.m1.1.1.1.1"><eq id="S5.SS2.p2.1.m1.1.1.1.1.1.cmml" xref="S5.SS2.p2.1.m1.1.1.1.1.1"></eq><ci id="S5.SS2.p2.1.m1.1.1.1.1.2.cmml" xref="S5.SS2.p2.1.m1.1.1.1.1.2">𝑟</ci><cn type="integer" id="S5.SS2.p2.1.m1.1.1.1.1.3.cmml" xref="S5.SS2.p2.1.m1.1.1.1.1.3">64</cn></apply><apply id="S5.SS2.p2.1.m1.2.2.2.2.cmml" xref="S5.SS2.p2.1.m1.2.2.2.2"><eq id="S5.SS2.p2.1.m1.2.2.2.2.1.cmml" xref="S5.SS2.p2.1.m1.2.2.2.2.1"></eq><ci id="S5.SS2.p2.1.m1.2.2.2.2.2.cmml" xref="S5.SS2.p2.1.m1.2.2.2.2.2">𝜂</ci><apply id="S5.SS2.p2.1.m1.2.2.2.2.3.cmml" xref="S5.SS2.p2.1.m1.2.2.2.2.3"><minus id="S5.SS2.p2.1.m1.2.2.2.2.3.1.cmml" xref="S5.SS2.p2.1.m1.2.2.2.2.3.1"></minus><apply id="S5.SS2.p2.1.m1.2.2.2.2.3.2.cmml" xref="S5.SS2.p2.1.m1.2.2.2.2.3.2"><times id="S5.SS2.p2.1.m1.2.2.2.2.3.2.1.cmml" xref="S5.SS2.p2.1.m1.2.2.2.2.3.2.1"></times><cn type="integer" id="S5.SS2.p2.1.m1.2.2.2.2.3.2.2.cmml" xref="S5.SS2.p2.1.m1.2.2.2.2.3.2.2">1</cn><ci id="S5.SS2.p2.1.m1.2.2.2.2.3.2.3.cmml" xref="S5.SS2.p2.1.m1.2.2.2.2.3.2.3">𝑒</ci></apply><cn type="integer" id="S5.SS2.p2.1.m1.2.2.2.2.3.3.cmml" xref="S5.SS2.p2.1.m1.2.2.2.2.3.3">4</cn></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.p2.1.m1.2c">r=64,\eta=1e-4</annotation></semantics></math>. However, sparsity also has a regularizing effect, and LoTA is even more successful on the reasoning task when using the same learning rate as we searched for the base model (<math id="S5.SS2.p2.2.m2.1" class="ltx_Math" alttext="1e-6" display="inline"><semantics id="S5.SS2.p2.2.m2.1a"><mrow id="S5.SS2.p2.2.m2.1.1" xref="S5.SS2.p2.2.m2.1.1.cmml"><mrow id="S5.SS2.p2.2.m2.1.1.2" xref="S5.SS2.p2.2.m2.1.1.2.cmml"><mn id="S5.SS2.p2.2.m2.1.1.2.2" xref="S5.SS2.p2.2.m2.1.1.2.2.cmml">1</mn><mo lspace="0em" rspace="0em" id="S5.SS2.p2.2.m2.1.1.2.1" xref="S5.SS2.p2.2.m2.1.1.2.1.cmml">​</mo><mi id="S5.SS2.p2.2.m2.1.1.2.3" xref="S5.SS2.p2.2.m2.1.1.2.3.cmml">e</mi></mrow><mo id="S5.SS2.p2.2.m2.1.1.1" xref="S5.SS2.p2.2.m2.1.1.1.cmml">−</mo><mn id="S5.SS2.p2.2.m2.1.1.3" xref="S5.SS2.p2.2.m2.1.1.3.cmml">6</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.p2.2.m2.1b"><apply id="S5.SS2.p2.2.m2.1.1.cmml" xref="S5.SS2.p2.2.m2.1.1"><minus id="S5.SS2.p2.2.m2.1.1.1.cmml" xref="S5.SS2.p2.2.m2.1.1.1"></minus><apply id="S5.SS2.p2.2.m2.1.1.2.cmml" xref="S5.SS2.p2.2.m2.1.1.2"><times id="S5.SS2.p2.2.m2.1.1.2.1.cmml" xref="S5.SS2.p2.2.m2.1.1.2.1"></times><cn type="integer" id="S5.SS2.p2.2.m2.1.1.2.2.cmml" xref="S5.SS2.p2.2.m2.1.1.2.2">1</cn><ci id="S5.SS2.p2.2.m2.1.1.2.3.cmml" xref="S5.SS2.p2.2.m2.1.1.2.3">𝑒</ci></apply><cn type="integer" id="S5.SS2.p2.2.m2.1.1.3.cmml" xref="S5.SS2.p2.2.m2.1.1.3">6</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.p2.2.m2.1c">1e-6</annotation></semantics></math>). On GSM8k, the regularization hurts performance significantly; <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib34" title="" class="ltx_ref">34</a>]</cite> report a similar-sized gap between LoRA and FFT on this dataset when training LLama-2-7B. If LoRA is underfitting the data, it may be better suited for settings where we only train for a single epoch, or on smaller datasets. In <a href="#S5.T4" title="In 5.2 Adapting to a Single Task ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">4</span></a>, we make a side-by-side comparison of LoRA and LoTA when training for a single epoch and find that LoTA outperforms LoRA significantly across all tasks; in fact, the difference is even more pronounced after a single epoch.</p>
</div>
<figure id="S5.T4" class="ltx_table">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table"><span id="S5.T4.11.1.1" class="ltx_text" style="font-size:90%;">Table 4</span>: </span><span id="S5.T4.12.2" class="ltx_text" style="font-size:90%;">Performance comparison on single-task datasets for 1 epoch. <span id="S5.T4.12.2.1" class="ltx_text ltx_font_bold">bold</span>: best method</span></figcaption>
<table id="S5.T4.8" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S5.T4.8.9.1" class="ltx_tr">
<th id="S5.T4.8.9.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_th_row ltx_border_tt">Method</th>
<th id="S5.T4.8.9.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_th_row ltx_border_r ltx_border_tt">Model</th>
<th id="S5.T4.8.9.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">Arc</th>
<th id="S5.T4.8.9.1.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">GSM8k</th>
<th id="S5.T4.8.9.1.5" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">SQL</th>
<th id="S5.T4.8.9.1.6" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">Summarization</th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S5.T4.4.4" class="ltx_tr">
<th id="S5.T4.4.4.5" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t">LoRA</th>
<th id="S5.T4.4.4.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_r ltx_border_t">Mistral</th>
<td id="S5.T4.1.1.1" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.1.1.1.m1.1" class="ltx_Math" alttext="70.4_{0.6}" display="inline"><semantics id="S5.T4.1.1.1.m1.1a"><msub id="S5.T4.1.1.1.m1.1.1" xref="S5.T4.1.1.1.m1.1.1.cmml"><mn id="S5.T4.1.1.1.m1.1.1.2" xref="S5.T4.1.1.1.m1.1.1.2.cmml">70.4</mn><mn id="S5.T4.1.1.1.m1.1.1.3" xref="S5.T4.1.1.1.m1.1.1.3.cmml">0.6</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T4.1.1.1.m1.1b"><apply id="S5.T4.1.1.1.m1.1.1.cmml" xref="S5.T4.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.1.1.1.m1.1.1.1.cmml" xref="S5.T4.1.1.1.m1.1.1">subscript</csymbol><cn type="float" id="S5.T4.1.1.1.m1.1.1.2.cmml" xref="S5.T4.1.1.1.m1.1.1.2">70.4</cn><cn type="float" id="S5.T4.1.1.1.m1.1.1.3.cmml" xref="S5.T4.1.1.1.m1.1.1.3">0.6</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.1.1.1.m1.1c">70.4_{0.6}</annotation></semantics></math></td>
<td id="S5.T4.2.2.2" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.2.2.2.m1.1" class="ltx_Math" alttext="46.3_{1.1}" display="inline"><semantics id="S5.T4.2.2.2.m1.1a"><msub id="S5.T4.2.2.2.m1.1.1" xref="S5.T4.2.2.2.m1.1.1.cmml"><mn id="S5.T4.2.2.2.m1.1.1.2" xref="S5.T4.2.2.2.m1.1.1.2.cmml">46.3</mn><mn id="S5.T4.2.2.2.m1.1.1.3" xref="S5.T4.2.2.2.m1.1.1.3.cmml">1.1</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T4.2.2.2.m1.1b"><apply id="S5.T4.2.2.2.m1.1.1.cmml" xref="S5.T4.2.2.2.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.2.2.2.m1.1.1.1.cmml" xref="S5.T4.2.2.2.m1.1.1">subscript</csymbol><cn type="float" id="S5.T4.2.2.2.m1.1.1.2.cmml" xref="S5.T4.2.2.2.m1.1.1.2">46.3</cn><cn type="float" id="S5.T4.2.2.2.m1.1.1.3.cmml" xref="S5.T4.2.2.2.m1.1.1.3">1.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.2.2.2.m1.1c">46.3_{1.1}</annotation></semantics></math></td>
<td id="S5.T4.3.3.3" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.3.3.3.m1.1" class="ltx_Math" alttext="98.6_{0.1}" display="inline"><semantics id="S5.T4.3.3.3.m1.1a"><msub id="S5.T4.3.3.3.m1.1.1" xref="S5.T4.3.3.3.m1.1.1.cmml"><mn id="S5.T4.3.3.3.m1.1.1.2" xref="S5.T4.3.3.3.m1.1.1.2.cmml">98.6</mn><mn id="S5.T4.3.3.3.m1.1.1.3" xref="S5.T4.3.3.3.m1.1.1.3.cmml">0.1</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T4.3.3.3.m1.1b"><apply id="S5.T4.3.3.3.m1.1.1.cmml" xref="S5.T4.3.3.3.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.3.3.3.m1.1.1.1.cmml" xref="S5.T4.3.3.3.m1.1.1">subscript</csymbol><cn type="float" id="S5.T4.3.3.3.m1.1.1.2.cmml" xref="S5.T4.3.3.3.m1.1.1.2">98.6</cn><cn type="float" id="S5.T4.3.3.3.m1.1.1.3.cmml" xref="S5.T4.3.3.3.m1.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.3.3.3.m1.1c">98.6_{0.1}</annotation></semantics></math></td>
<td id="S5.T4.4.4.4" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.4.4.4.m1.1" class="ltx_Math" alttext="51.8_{0.2}" display="inline"><semantics id="S5.T4.4.4.4.m1.1a"><msub id="S5.T4.4.4.4.m1.1.1" xref="S5.T4.4.4.4.m1.1.1.cmml"><mn id="S5.T4.4.4.4.m1.1.1.2" xref="S5.T4.4.4.4.m1.1.1.2.cmml">51.8</mn><mn id="S5.T4.4.4.4.m1.1.1.3" xref="S5.T4.4.4.4.m1.1.1.3.cmml">0.2</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T4.4.4.4.m1.1b"><apply id="S5.T4.4.4.4.m1.1.1.cmml" xref="S5.T4.4.4.4.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.4.4.4.m1.1.1.1.cmml" xref="S5.T4.4.4.4.m1.1.1">subscript</csymbol><cn type="float" id="S5.T4.4.4.4.m1.1.1.2.cmml" xref="S5.T4.4.4.4.m1.1.1.2">51.8</cn><cn type="float" id="S5.T4.4.4.4.m1.1.1.3.cmml" xref="S5.T4.4.4.4.m1.1.1.3">0.2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.4.4.4.m1.1c">51.8_{0.2}</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.8.8" class="ltx_tr">
<th id="S5.T4.8.8.5" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_bb">LoTA (ours)</th>
<th id="S5.T4.8.8.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_bb ltx_border_r">Mistral</th>
<td id="S5.T4.5.5.1" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T4.5.5.1.m1.1" class="ltx_Math" alttext="\mathbf{73.8_{0.7}}" display="inline"><semantics id="S5.T4.5.5.1.m1.1a"><msub id="S5.T4.5.5.1.m1.1.1" xref="S5.T4.5.5.1.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.5.5.1.m1.1.1.2" xref="S5.T4.5.5.1.m1.1.1.2.cmml">73.8</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.5.5.1.m1.1.1.3" xref="S5.T4.5.5.1.m1.1.1.3.cmml">0.7</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T4.5.5.1.m1.1b"><apply id="S5.T4.5.5.1.m1.1.1.cmml" xref="S5.T4.5.5.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.5.5.1.m1.1.1.1.cmml" xref="S5.T4.5.5.1.m1.1.1">subscript</csymbol><cn type="float" id="S5.T4.5.5.1.m1.1.1.2.cmml" xref="S5.T4.5.5.1.m1.1.1.2">73.8</cn><cn type="float" id="S5.T4.5.5.1.m1.1.1.3.cmml" xref="S5.T4.5.5.1.m1.1.1.3">0.7</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.5.5.1.m1.1c">\mathbf{73.8_{0.7}}</annotation></semantics></math></td>
<td id="S5.T4.6.6.2" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T4.6.6.2.m1.1" class="ltx_Math" alttext="\mathbf{53.5_{1.0}}" display="inline"><semantics id="S5.T4.6.6.2.m1.1a"><msub id="S5.T4.6.6.2.m1.1.1" xref="S5.T4.6.6.2.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.6.6.2.m1.1.1.2" xref="S5.T4.6.6.2.m1.1.1.2.cmml">53.5</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.6.6.2.m1.1.1.3" xref="S5.T4.6.6.2.m1.1.1.3.cmml">1.0</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T4.6.6.2.m1.1b"><apply id="S5.T4.6.6.2.m1.1.1.cmml" xref="S5.T4.6.6.2.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.6.6.2.m1.1.1.1.cmml" xref="S5.T4.6.6.2.m1.1.1">subscript</csymbol><cn type="float" id="S5.T4.6.6.2.m1.1.1.2.cmml" xref="S5.T4.6.6.2.m1.1.1.2">53.5</cn><cn type="float" id="S5.T4.6.6.2.m1.1.1.3.cmml" xref="S5.T4.6.6.2.m1.1.1.3">1.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.6.6.2.m1.1c">\mathbf{53.5_{1.0}}</annotation></semantics></math></td>
<td id="S5.T4.7.7.3" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T4.7.7.3.m1.1" class="ltx_Math" alttext="\mathbf{99.3_{0.1}}" display="inline"><semantics id="S5.T4.7.7.3.m1.1a"><msub id="S5.T4.7.7.3.m1.1.1" xref="S5.T4.7.7.3.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.7.7.3.m1.1.1.2" xref="S5.T4.7.7.3.m1.1.1.2.cmml">99.3</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.7.7.3.m1.1.1.3" xref="S5.T4.7.7.3.m1.1.1.3.cmml">0.1</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T4.7.7.3.m1.1b"><apply id="S5.T4.7.7.3.m1.1.1.cmml" xref="S5.T4.7.7.3.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.7.7.3.m1.1.1.1.cmml" xref="S5.T4.7.7.3.m1.1.1">subscript</csymbol><cn type="float" id="S5.T4.7.7.3.m1.1.1.2.cmml" xref="S5.T4.7.7.3.m1.1.1.2">99.3</cn><cn type="float" id="S5.T4.7.7.3.m1.1.1.3.cmml" xref="S5.T4.7.7.3.m1.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.7.7.3.m1.1c">\mathbf{99.3_{0.1}}</annotation></semantics></math></td>
<td id="S5.T4.8.8.4" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T4.8.8.4.m1.1" class="ltx_Math" alttext="\mathbf{54.3_{2.5}}" display="inline"><semantics id="S5.T4.8.8.4.m1.1a"><msub id="S5.T4.8.8.4.m1.1.1" xref="S5.T4.8.8.4.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.8.8.4.m1.1.1.2" xref="S5.T4.8.8.4.m1.1.1.2.cmml">54.3</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.8.8.4.m1.1.1.3" xref="S5.T4.8.8.4.m1.1.1.3.cmml">2.5</mn></msub><annotation-xml encoding="MathML-Content" id="S5.T4.8.8.4.m1.1b"><apply id="S5.T4.8.8.4.m1.1.1.cmml" xref="S5.T4.8.8.4.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.8.8.4.m1.1.1.1.cmml" xref="S5.T4.8.8.4.m1.1.1">subscript</csymbol><cn type="float" id="S5.T4.8.8.4.m1.1.1.2.cmml" xref="S5.T4.8.8.4.m1.1.1.2">54.3</cn><cn type="float" id="S5.T4.8.8.4.m1.1.1.3.cmml" xref="S5.T4.8.8.4.m1.1.1.3">2.5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.8.8.4.m1.1c">\mathbf{54.3_{2.5}}</annotation></semantics></math></td>
</tr>
</tbody>
</table>
</figure>
<section id="S5.SS2.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Storing LoTA Adapters Efficiently.</h4>

<div id="S5.SS2.SSS0.Px1.p1" class="ltx_para">
<p id="S5.SS2.SSS0.Px1.p1.14" class="ltx_p">Although FFT generally performs better than PEFT methods, it is typically infeasible to store a full copy of the model weights for each task. Practitioners, therefore, consider a tradeoff between memory and adaptation performance when loading task-specific adapters. We now discuss the memory consumption of LoTA and LoRA.
Storing the sparse task vector from LoTA requires <math id="S5.SS2.SSS0.Px1.p1.1.m1.1" class="ltx_Math" alttext="40" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.1.m1.1a"><mn id="S5.SS2.SSS0.Px1.p1.1.m1.1.1" xref="S5.SS2.SSS0.Px1.p1.1.m1.1.1.cmml">40</mn><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.1.m1.1b"><cn type="integer" id="S5.SS2.SSS0.Px1.p1.1.m1.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.1.m1.1.1">40</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.1.m1.1c">40</annotation></semantics></math> bits per parameter; <math id="S5.SS2.SSS0.Px1.p1.2.m2.1" class="ltx_Math" alttext="32" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.2.m2.1a"><mn id="S5.SS2.SSS0.Px1.p1.2.m2.1.1" xref="S5.SS2.SSS0.Px1.p1.2.m2.1.1.cmml">32</mn><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.2.m2.1b"><cn type="integer" id="S5.SS2.SSS0.Px1.p1.2.m2.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.2.m2.1.1">32</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.2.m2.1c">32</annotation></semantics></math> from the parameter, and <math id="S5.SS2.SSS0.Px1.p1.3.m3.1" class="ltx_Math" alttext="8" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.3.m3.1a"><mn id="S5.SS2.SSS0.Px1.p1.3.m3.1.1" xref="S5.SS2.SSS0.Px1.p1.3.m3.1.1.cmml">8</mn><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.3.m3.1b"><cn type="integer" id="S5.SS2.SSS0.Px1.p1.3.m3.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.3.m3.1.1">8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.3.m3.1c">8</annotation></semantics></math> for the delta encoding of the sparse indices. If we use <math id="S5.SS2.SSS0.Px1.p1.4.m4.1" class="ltx_Math" alttext="90\%" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.4.m4.1a"><mrow id="S5.SS2.SSS0.Px1.p1.4.m4.1.1" xref="S5.SS2.SSS0.Px1.p1.4.m4.1.1.cmml"><mn id="S5.SS2.SSS0.Px1.p1.4.m4.1.1.2" xref="S5.SS2.SSS0.Px1.p1.4.m4.1.1.2.cmml">90</mn><mo id="S5.SS2.SSS0.Px1.p1.4.m4.1.1.1" xref="S5.SS2.SSS0.Px1.p1.4.m4.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.4.m4.1b"><apply id="S5.SS2.SSS0.Px1.p1.4.m4.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.4.m4.1.1"><csymbol cd="latexml" id="S5.SS2.SSS0.Px1.p1.4.m4.1.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.4.m4.1.1.1">percent</csymbol><cn type="integer" id="S5.SS2.SSS0.Px1.p1.4.m4.1.1.2.cmml" xref="S5.SS2.SSS0.Px1.p1.4.m4.1.1.2">90</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.4.m4.1c">90\%</annotation></semantics></math>-sparse LoTA, our task vector is compressed <math id="S5.SS2.SSS0.Px1.p1.5.m5.1" class="ltx_math_unparsed" alttext="8\times" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.5.m5.1a"><mrow id="S5.SS2.SSS0.Px1.p1.5.m5.1b"><mn id="S5.SS2.SSS0.Px1.p1.5.m5.1.1">8</mn><mo lspace="0.222em" id="S5.SS2.SSS0.Px1.p1.5.m5.1.2">×</mo></mrow><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.5.m5.1c">8\times</annotation></semantics></math>; if we use <math id="S5.SS2.SSS0.Px1.p1.6.m6.1" class="ltx_Math" alttext="99\%" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.6.m6.1a"><mrow id="S5.SS2.SSS0.Px1.p1.6.m6.1.1" xref="S5.SS2.SSS0.Px1.p1.6.m6.1.1.cmml"><mn id="S5.SS2.SSS0.Px1.p1.6.m6.1.1.2" xref="S5.SS2.SSS0.Px1.p1.6.m6.1.1.2.cmml">99</mn><mo id="S5.SS2.SSS0.Px1.p1.6.m6.1.1.1" xref="S5.SS2.SSS0.Px1.p1.6.m6.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.6.m6.1b"><apply id="S5.SS2.SSS0.Px1.p1.6.m6.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.6.m6.1.1"><csymbol cd="latexml" id="S5.SS2.SSS0.Px1.p1.6.m6.1.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.6.m6.1.1.1">percent</csymbol><cn type="integer" id="S5.SS2.SSS0.Px1.p1.6.m6.1.1.2.cmml" xref="S5.SS2.SSS0.Px1.p1.6.m6.1.1.2">99</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.6.m6.1c">99\%</annotation></semantics></math>-sparse LoTA, our task vector is compressed <math id="S5.SS2.SSS0.Px1.p1.7.m7.1" class="ltx_math_unparsed" alttext="80\times" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.7.m7.1a"><mrow id="S5.SS2.SSS0.Px1.p1.7.m7.1b"><mn id="S5.SS2.SSS0.Px1.p1.7.m7.1.1">80</mn><mo lspace="0.222em" id="S5.SS2.SSS0.Px1.p1.7.m7.1.2">×</mo></mrow><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.7.m7.1c">80\times</annotation></semantics></math>. The memory-utility tradeoff between LoTA and LoRA can be quantified in terms of each method’s performance at a given level of compression, which translates into the sparsity level for LoTA and the rank <math id="S5.SS2.SSS0.Px1.p1.8.m8.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.8.m8.1a"><mi id="S5.SS2.SSS0.Px1.p1.8.m8.1.1" xref="S5.SS2.SSS0.Px1.p1.8.m8.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.8.m8.1b"><ci id="S5.SS2.SSS0.Px1.p1.8.m8.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.8.m8.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.8.m8.1c">r</annotation></semantics></math> for LoRA. Comparing the compression-utility tradeoff between LoRA and LoTA is challenging because the size of the saved LoRA adapter is determined by the rank parameter <math id="S5.SS2.SSS0.Px1.p1.9.m9.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.9.m9.1a"><mi id="S5.SS2.SSS0.Px1.p1.9.m9.1.1" xref="S5.SS2.SSS0.Px1.p1.9.m9.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.9.m9.1b"><ci id="S5.SS2.SSS0.Px1.p1.9.m9.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.9.m9.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.9.m9.1c">r</annotation></semantics></math>, and more is not always better (which is why we have to tune <math id="S5.SS2.SSS0.Px1.p1.10.m10.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.10.m10.1a"><mi id="S5.SS2.SSS0.Px1.p1.10.m10.1.1" xref="S5.SS2.SSS0.Px1.p1.10.m10.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.10.m10.1b"><ci id="S5.SS2.SSS0.Px1.p1.10.m10.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.10.m10.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.10.m10.1c">r</annotation></semantics></math> for every task for LoRA).
As a single point of comparison, we can look at the performance on instruction following, where <math id="S5.SS2.SSS0.Px1.p1.11.m11.1" class="ltx_Math" alttext="99\%" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.11.m11.1a"><mrow id="S5.SS2.SSS0.Px1.p1.11.m11.1.1" xref="S5.SS2.SSS0.Px1.p1.11.m11.1.1.cmml"><mn id="S5.SS2.SSS0.Px1.p1.11.m11.1.1.2" xref="S5.SS2.SSS0.Px1.p1.11.m11.1.1.2.cmml">99</mn><mo id="S5.SS2.SSS0.Px1.p1.11.m11.1.1.1" xref="S5.SS2.SSS0.Px1.p1.11.m11.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.11.m11.1b"><apply id="S5.SS2.SSS0.Px1.p1.11.m11.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.11.m11.1.1"><csymbol cd="latexml" id="S5.SS2.SSS0.Px1.p1.11.m11.1.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.11.m11.1.1.1">percent</csymbol><cn type="integer" id="S5.SS2.SSS0.Px1.p1.11.m11.1.1.2.cmml" xref="S5.SS2.SSS0.Px1.p1.11.m11.1.1.2">99</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.11.m11.1c">99\%</annotation></semantics></math>-sparse LoTA (<math id="S5.SS2.SSS0.Px1.p1.12.m12.1" class="ltx_Math" alttext="17.4" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.12.m12.1a"><mn id="S5.SS2.SSS0.Px1.p1.12.m12.1.1" xref="S5.SS2.SSS0.Px1.p1.12.m12.1.1.cmml">17.4</mn><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.12.m12.1b"><cn type="float" id="S5.SS2.SSS0.Px1.p1.12.m12.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.12.m12.1.1">17.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.12.m12.1c">17.4</annotation></semantics></math>) outperforms LoRA for any rank (best performance of <math id="S5.SS2.SSS0.Px1.p1.13.m13.1" class="ltx_Math" alttext="15.3" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.13.m13.1a"><mn id="S5.SS2.SSS0.Px1.p1.13.m13.1.1" xref="S5.SS2.SSS0.Px1.p1.13.m13.1.1.cmml">15.3</mn><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.13.m13.1b"><cn type="float" id="S5.SS2.SSS0.Px1.p1.13.m13.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.13.m13.1.1">15.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.13.m13.1c">15.3</annotation></semantics></math> achieved at <math id="S5.SS2.SSS0.Px1.p1.14.m14.1" class="ltx_Math" alttext="r=64" display="inline"><semantics id="S5.SS2.SSS0.Px1.p1.14.m14.1a"><mrow id="S5.SS2.SSS0.Px1.p1.14.m14.1.1" xref="S5.SS2.SSS0.Px1.p1.14.m14.1.1.cmml"><mi id="S5.SS2.SSS0.Px1.p1.14.m14.1.1.2" xref="S5.SS2.SSS0.Px1.p1.14.m14.1.1.2.cmml">r</mi><mo id="S5.SS2.SSS0.Px1.p1.14.m14.1.1.1" xref="S5.SS2.SSS0.Px1.p1.14.m14.1.1.1.cmml">=</mo><mn id="S5.SS2.SSS0.Px1.p1.14.m14.1.1.3" xref="S5.SS2.SSS0.Px1.p1.14.m14.1.1.3.cmml">64</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.SSS0.Px1.p1.14.m14.1b"><apply id="S5.SS2.SSS0.Px1.p1.14.m14.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.14.m14.1.1"><eq id="S5.SS2.SSS0.Px1.p1.14.m14.1.1.1.cmml" xref="S5.SS2.SSS0.Px1.p1.14.m14.1.1.1"></eq><ci id="S5.SS2.SSS0.Px1.p1.14.m14.1.1.2.cmml" xref="S5.SS2.SSS0.Px1.p1.14.m14.1.1.2">𝑟</ci><cn type="integer" id="S5.SS2.SSS0.Px1.p1.14.m14.1.1.3.cmml" xref="S5.SS2.SSS0.Px1.p1.14.m14.1.1.3">64</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.SSS0.Px1.p1.14.m14.1c">r=64</annotation></semantics></math>, increasing rank reduces performance), and they both achieve a similar compression factor.</p>
</div>
</section>
</section>
<section id="S5.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.3 </span>Sequential Training</h3>

<div id="S5.SS3.p1" class="ltx_para">
<p id="S5.SS3.p1.1" class="ltx_p">During sequential training, a model is first adapted to one capability (Task A) and then to another capability (Task B) (such as when creating a specialized model for math) or a set of capabilities (the most common setting for open-sourced fine-tunes of frontier models). The main challenges we seek to mitigate are (1) catastrophic forgetting of Task A and (2) the inability to adapt to Task B.
We set instruction following as Task A in all settings because this maps to the enterprise adaptation setting, where user queries need to be answered with a combination of instruction following and domain-specific knowledge. In particular, OpenAI offers a fine-tuning API for their aligned models (GPT-3.5 and GPT-4) <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib3" title="" class="ltx_ref">3</a>]</cite>.</p>
</div>
<figure id="S5.T5" class="ltx_table">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table"><span id="S5.T5.19.1.1" class="ltx_text" style="font-size:90%;">Table 5</span>: </span><span id="S5.T5.20.2" class="ltx_text" style="font-size:90%;">Sequential learning first on Task A (Instruction Following) then on Task B (varied). Fine-tuning on Tasks A and B is performed using both FFT and LoTA. The utility of each task is computed after fine-tuning on Task B is completed. Note that there is no utility when we fine-tune on harmful data to evaluate the catastrophic forgetting of Safety, and the baseline is the safety score of the Instruct model. FT=Fine-tuning. We reproduce the baseline of doing FFT on each method independently as reported in <a href="#S5.T3" title="In 5.2 Adapting to a Single Task ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">3</span></a> for convenience; note that on the reasoning task LoTA outperforms FFT. <span id="S5.T5.20.2.1" class="ltx_text ltx_font_bold">bold</span>: best method, <span id="S5.T5.20.2.2" class="ltx_text ltx_framed ltx_framed_underline">underline</span> second-best method; we do not report second-best when only two methods are presented. All results are with Mistral. We reuse the same mask for LoTTO calibrated on GSM8k for MathInstruct, Reasoning, GSM8k+Arc+SQL and Safety.</span></figcaption>
<div id="S5.T5.15" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:433.6pt;height:320pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(-51.5pt,38.0pt) scale(0.808069790073809,0.808069790073809) ;">
<table id="S5.T5.15.15" class="ltx_tabular ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S5.T5.15.15.16.1" class="ltx_tr">
<td id="S5.T5.15.15.16.1.1" class="ltx_td ltx_align_center ltx_border_tt">Task B</td>
<td id="S5.T5.15.15.16.1.2" class="ltx_td ltx_align_center ltx_border_tt">Method on Task A</td>
<td id="S5.T5.15.15.16.1.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_tt">Method on Task B</td>
<td id="S5.T5.15.15.16.1.4" class="ltx_td ltx_align_center ltx_border_tt">Utility of Task A (Drop)</td>
<td id="S5.T5.15.15.16.1.5" class="ltx_td ltx_align_center ltx_border_tt">Utility of Task B (Drop)</td>
</tr>
<tr id="S5.T5.15.15.17.2" class="ltx_tr">
<td id="S5.T5.15.15.17.2.1" class="ltx_td ltx_align_center ltx_border_t">Instruction Following</td>
<td id="S5.T5.15.15.17.2.2" class="ltx_td ltx_align_center ltx_border_t">Baseline</td>
<td id="S5.T5.15.15.17.2.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">-</td>
<td id="S5.T5.15.15.17.2.4" class="ltx_td ltx_align_center ltx_border_t">19.0 (-)</td>
<td id="S5.T5.15.15.17.2.5" class="ltx_td ltx_align_center ltx_border_t">-</td>
</tr>
<tr id="S5.T5.15.15.18.3" class="ltx_tr">
<td id="S5.T5.15.15.18.3.1" class="ltx_td ltx_align_center ltx_border_t" rowspan="7"><span id="S5.T5.15.15.18.3.1.1" class="ltx_text">GSM8k</span></td>
<td id="S5.T5.15.15.18.3.2" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T5.15.15.18.3.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">Baseline</td>
<td id="S5.T5.15.15.18.3.4" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T5.15.15.18.3.5" class="ltx_td ltx_align_center ltx_border_t">59.8 (-)</td>
</tr>
<tr id="S5.T5.15.15.19.4" class="ltx_tr">
<td id="S5.T5.15.15.19.4.1" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T5.15.15.19.4.2" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T5.15.15.19.4.3" class="ltx_td ltx_align_center">15.2 (3.8)</td>
<td id="S5.T5.15.15.19.4.4" class="ltx_td ltx_align_center">58.3 (1.5)</td>
</tr>
<tr id="S5.T5.15.15.20.5" class="ltx_tr">
<td id="S5.T5.15.15.20.5.1" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T5.15.15.20.5.2" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T5.15.15.20.5.3" class="ltx_td ltx_align_center"><span id="S5.T5.15.15.20.5.3.1" class="ltx_text ltx_framed ltx_framed_underline">17.7 (1.3)</span></td>
<td id="S5.T5.15.15.20.5.4" class="ltx_td ltx_align_center"><span id="S5.T5.15.15.20.5.4.1" class="ltx_text ltx_framed ltx_framed_underline">58.7 (1.1)</span></td>
</tr>
<tr id="S5.T5.15.15.21.6" class="ltx_tr">
<td id="S5.T5.15.15.21.6.1" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T5.15.15.21.6.2" class="ltx_td ltx_align_center ltx_border_r">LoTA (ours)</td>
<td id="S5.T5.15.15.21.6.3" class="ltx_td ltx_align_center">15.9 (3.1)</td>
<td id="S5.T5.15.15.21.6.4" class="ltx_td ltx_align_center">54.2 (5.6)</td>
</tr>
<tr id="S5.T5.15.15.22.7" class="ltx_tr">
<td id="S5.T5.15.15.22.7.1" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T5.15.15.22.7.2" class="ltx_td ltx_align_center ltx_border_r">LoTTO (ours)</td>
<td id="S5.T5.15.15.22.7.3" class="ltx_td ltx_align_center"><span id="S5.T5.15.15.22.7.3.1" class="ltx_text ltx_font_bold">17.8 (1.2)</span></td>
<td id="S5.T5.15.15.22.7.4" class="ltx_td ltx_align_center"><span id="S5.T5.15.15.22.7.4.1" class="ltx_text ltx_font_bold">59.1 (0.7)</span></td>
</tr>
<tr id="S5.T5.15.15.23.8" class="ltx_tr">
<td id="S5.T5.15.15.23.8.1" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T5.15.15.23.8.2" class="ltx_td ltx_align_center ltx_border_r">LoRA</td>
<td id="S5.T5.15.15.23.8.3" class="ltx_td ltx_align_center">14.1 (4.2)</td>
<td id="S5.T5.15.15.23.8.4" class="ltx_td ltx_align_center">55.5 (4.9)</td>
</tr>
<tr id="S5.T5.15.15.24.9" class="ltx_tr">
<td id="S5.T5.15.15.24.9.1" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T5.15.15.24.9.2" class="ltx_td ltx_align_center ltx_border_r">FFT (Mixed)</td>
<td id="S5.T5.15.15.24.9.3" class="ltx_td ltx_align_center">16.3 (2.7)</td>
<td id="S5.T5.15.15.24.9.4" class="ltx_td ltx_align_center">55.5 (4.3)</td>
</tr>
<tr id="S5.T5.15.15.25.10" class="ltx_tr">
<td id="S5.T5.15.15.25.10.1" class="ltx_td ltx_align_center ltx_border_t" rowspan="2"><span id="S5.T5.15.15.25.10.1.1" class="ltx_text">MathInstruct</span></td>
<td id="S5.T5.15.15.25.10.2" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T5.15.15.25.10.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">Baseline</td>
<td id="S5.T5.15.15.25.10.4" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T5.15.15.25.10.5" class="ltx_td ltx_align_center ltx_border_t">56.7 (-)</td>
</tr>
<tr id="S5.T5.2.2.2" class="ltx_tr">
<td id="S5.T5.2.2.2.3" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T5.2.2.2.4" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T5.1.1.1.1" class="ltx_td ltx_align_center"><math id="S5.T5.1.1.1.1.m1.1" class="ltx_Math" alttext="14.2_{0.8}~{}(4.8)" display="inline"><semantics id="S5.T5.1.1.1.1.m1.1a"><mrow id="S5.T5.1.1.1.1.m1.1.2" xref="S5.T5.1.1.1.1.m1.1.2.cmml"><msub id="S5.T5.1.1.1.1.m1.1.2.2" xref="S5.T5.1.1.1.1.m1.1.2.2.cmml"><mn id="S5.T5.1.1.1.1.m1.1.2.2.2" xref="S5.T5.1.1.1.1.m1.1.2.2.2.cmml">14.2</mn><mn id="S5.T5.1.1.1.1.m1.1.2.2.3" xref="S5.T5.1.1.1.1.m1.1.2.2.3.cmml">0.8</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.1.1.1.1.m1.1.2.1" xref="S5.T5.1.1.1.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T5.1.1.1.1.m1.1.2.3.2" xref="S5.T5.1.1.1.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T5.1.1.1.1.m1.1.2.3.2.1" xref="S5.T5.1.1.1.1.m1.1.2.cmml">(</mo><mn id="S5.T5.1.1.1.1.m1.1.1" xref="S5.T5.1.1.1.1.m1.1.1.cmml">4.8</mn><mo stretchy="false" id="S5.T5.1.1.1.1.m1.1.2.3.2.2" xref="S5.T5.1.1.1.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.1.1.1.1.m1.1b"><apply id="S5.T5.1.1.1.1.m1.1.2.cmml" xref="S5.T5.1.1.1.1.m1.1.2"><times id="S5.T5.1.1.1.1.m1.1.2.1.cmml" xref="S5.T5.1.1.1.1.m1.1.2.1"></times><apply id="S5.T5.1.1.1.1.m1.1.2.2.cmml" xref="S5.T5.1.1.1.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T5.1.1.1.1.m1.1.2.2.1.cmml" xref="S5.T5.1.1.1.1.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T5.1.1.1.1.m1.1.2.2.2.cmml" xref="S5.T5.1.1.1.1.m1.1.2.2.2">14.2</cn><cn type="float" id="S5.T5.1.1.1.1.m1.1.2.2.3.cmml" xref="S5.T5.1.1.1.1.m1.1.2.2.3">0.8</cn></apply><cn type="float" id="S5.T5.1.1.1.1.m1.1.1.cmml" xref="S5.T5.1.1.1.1.m1.1.1">4.8</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.1.1.1.1.m1.1c">14.2_{0.8}~{}(4.8)</annotation></semantics></math></td>
<td id="S5.T5.2.2.2.2" class="ltx_td ltx_align_center"><math id="S5.T5.2.2.2.2.m1.1" class="ltx_Math" alttext="51.3_{0.2}~{}(5.4)" display="inline"><semantics id="S5.T5.2.2.2.2.m1.1a"><mrow id="S5.T5.2.2.2.2.m1.1.2" xref="S5.T5.2.2.2.2.m1.1.2.cmml"><msub id="S5.T5.2.2.2.2.m1.1.2.2" xref="S5.T5.2.2.2.2.m1.1.2.2.cmml"><mn id="S5.T5.2.2.2.2.m1.1.2.2.2" xref="S5.T5.2.2.2.2.m1.1.2.2.2.cmml">51.3</mn><mn id="S5.T5.2.2.2.2.m1.1.2.2.3" xref="S5.T5.2.2.2.2.m1.1.2.2.3.cmml">0.2</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.2.2.2.2.m1.1.2.1" xref="S5.T5.2.2.2.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T5.2.2.2.2.m1.1.2.3.2" xref="S5.T5.2.2.2.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T5.2.2.2.2.m1.1.2.3.2.1" xref="S5.T5.2.2.2.2.m1.1.2.cmml">(</mo><mn id="S5.T5.2.2.2.2.m1.1.1" xref="S5.T5.2.2.2.2.m1.1.1.cmml">5.4</mn><mo stretchy="false" id="S5.T5.2.2.2.2.m1.1.2.3.2.2" xref="S5.T5.2.2.2.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.2.2.2.2.m1.1b"><apply id="S5.T5.2.2.2.2.m1.1.2.cmml" xref="S5.T5.2.2.2.2.m1.1.2"><times id="S5.T5.2.2.2.2.m1.1.2.1.cmml" xref="S5.T5.2.2.2.2.m1.1.2.1"></times><apply id="S5.T5.2.2.2.2.m1.1.2.2.cmml" xref="S5.T5.2.2.2.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T5.2.2.2.2.m1.1.2.2.1.cmml" xref="S5.T5.2.2.2.2.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T5.2.2.2.2.m1.1.2.2.2.cmml" xref="S5.T5.2.2.2.2.m1.1.2.2.2">51.3</cn><cn type="float" id="S5.T5.2.2.2.2.m1.1.2.2.3.cmml" xref="S5.T5.2.2.2.2.m1.1.2.2.3">0.2</cn></apply><cn type="float" id="S5.T5.2.2.2.2.m1.1.1.cmml" xref="S5.T5.2.2.2.2.m1.1.1">5.4</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.2.2.2.2.m1.1c">51.3_{0.2}~{}(5.4)</annotation></semantics></math></td>
</tr>
<tr id="S5.T5.4.4.4" class="ltx_tr">
<td id="S5.T5.4.4.4.3" class="ltx_td"></td>
<td id="S5.T5.4.4.4.4" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T5.4.4.4.5" class="ltx_td ltx_align_center ltx_border_r">LoTA (ours)</td>
<td id="S5.T5.3.3.3.1" class="ltx_td ltx_align_center"><math id="S5.T5.3.3.3.1.m1.1" class="ltx_Math" alttext="\textbf{16.0}_{0.7}~{}(-3.0)" display="inline"><semantics id="S5.T5.3.3.3.1.m1.1a"><mrow id="S5.T5.3.3.3.1.m1.1.1" xref="S5.T5.3.3.3.1.m1.1.1.cmml"><msub id="S5.T5.3.3.3.1.m1.1.1.3" xref="S5.T5.3.3.3.1.m1.1.1.3.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T5.3.3.3.1.m1.1.1.3.2" xref="S5.T5.3.3.3.1.m1.1.1.3.2a.cmml">16.0</mtext><mn id="S5.T5.3.3.3.1.m1.1.1.3.3" xref="S5.T5.3.3.3.1.m1.1.1.3.3.cmml">0.7</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.3.3.3.1.m1.1.1.2" xref="S5.T5.3.3.3.1.m1.1.1.2.cmml">​</mo><mrow id="S5.T5.3.3.3.1.m1.1.1.1.1" xref="S5.T5.3.3.3.1.m1.1.1.1.1.1.cmml"><mo stretchy="false" id="S5.T5.3.3.3.1.m1.1.1.1.1.2" xref="S5.T5.3.3.3.1.m1.1.1.1.1.1.cmml">(</mo><mrow id="S5.T5.3.3.3.1.m1.1.1.1.1.1" xref="S5.T5.3.3.3.1.m1.1.1.1.1.1.cmml"><mo id="S5.T5.3.3.3.1.m1.1.1.1.1.1a" xref="S5.T5.3.3.3.1.m1.1.1.1.1.1.cmml">−</mo><mn id="S5.T5.3.3.3.1.m1.1.1.1.1.1.2" xref="S5.T5.3.3.3.1.m1.1.1.1.1.1.2.cmml">3.0</mn></mrow><mo stretchy="false" id="S5.T5.3.3.3.1.m1.1.1.1.1.3" xref="S5.T5.3.3.3.1.m1.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.3.3.3.1.m1.1b"><apply id="S5.T5.3.3.3.1.m1.1.1.cmml" xref="S5.T5.3.3.3.1.m1.1.1"><times id="S5.T5.3.3.3.1.m1.1.1.2.cmml" xref="S5.T5.3.3.3.1.m1.1.1.2"></times><apply id="S5.T5.3.3.3.1.m1.1.1.3.cmml" xref="S5.T5.3.3.3.1.m1.1.1.3"><csymbol cd="ambiguous" id="S5.T5.3.3.3.1.m1.1.1.3.1.cmml" xref="S5.T5.3.3.3.1.m1.1.1.3">subscript</csymbol><ci id="S5.T5.3.3.3.1.m1.1.1.3.2a.cmml" xref="S5.T5.3.3.3.1.m1.1.1.3.2"><mtext class="ltx_mathvariant_bold" id="S5.T5.3.3.3.1.m1.1.1.3.2.cmml" xref="S5.T5.3.3.3.1.m1.1.1.3.2">16.0</mtext></ci><cn type="float" id="S5.T5.3.3.3.1.m1.1.1.3.3.cmml" xref="S5.T5.3.3.3.1.m1.1.1.3.3">0.7</cn></apply><apply id="S5.T5.3.3.3.1.m1.1.1.1.1.1.cmml" xref="S5.T5.3.3.3.1.m1.1.1.1.1"><minus id="S5.T5.3.3.3.1.m1.1.1.1.1.1.1.cmml" xref="S5.T5.3.3.3.1.m1.1.1.1.1"></minus><cn type="float" id="S5.T5.3.3.3.1.m1.1.1.1.1.1.2.cmml" xref="S5.T5.3.3.3.1.m1.1.1.1.1.1.2">3.0</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.3.3.3.1.m1.1c">\textbf{16.0}_{0.7}~{}(-3.0)</annotation></semantics></math></td>
<td id="S5.T5.4.4.4.2" class="ltx_td ltx_align_center"><math id="S5.T5.4.4.4.2.m1.1" class="ltx_Math" alttext="\textbf{55.5}_{0.1}~{}(1.2)" display="inline"><semantics id="S5.T5.4.4.4.2.m1.1a"><mrow id="S5.T5.4.4.4.2.m1.1.2" xref="S5.T5.4.4.4.2.m1.1.2.cmml"><msub id="S5.T5.4.4.4.2.m1.1.2.2" xref="S5.T5.4.4.4.2.m1.1.2.2.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T5.4.4.4.2.m1.1.2.2.2" xref="S5.T5.4.4.4.2.m1.1.2.2.2a.cmml">55.5</mtext><mn id="S5.T5.4.4.4.2.m1.1.2.2.3" xref="S5.T5.4.4.4.2.m1.1.2.2.3.cmml">0.1</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.4.4.4.2.m1.1.2.1" xref="S5.T5.4.4.4.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T5.4.4.4.2.m1.1.2.3.2" xref="S5.T5.4.4.4.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T5.4.4.4.2.m1.1.2.3.2.1" xref="S5.T5.4.4.4.2.m1.1.2.cmml">(</mo><mn id="S5.T5.4.4.4.2.m1.1.1" xref="S5.T5.4.4.4.2.m1.1.1.cmml">1.2</mn><mo stretchy="false" id="S5.T5.4.4.4.2.m1.1.2.3.2.2" xref="S5.T5.4.4.4.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.4.4.4.2.m1.1b"><apply id="S5.T5.4.4.4.2.m1.1.2.cmml" xref="S5.T5.4.4.4.2.m1.1.2"><times id="S5.T5.4.4.4.2.m1.1.2.1.cmml" xref="S5.T5.4.4.4.2.m1.1.2.1"></times><apply id="S5.T5.4.4.4.2.m1.1.2.2.cmml" xref="S5.T5.4.4.4.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T5.4.4.4.2.m1.1.2.2.1.cmml" xref="S5.T5.4.4.4.2.m1.1.2.2">subscript</csymbol><ci id="S5.T5.4.4.4.2.m1.1.2.2.2a.cmml" xref="S5.T5.4.4.4.2.m1.1.2.2.2"><mtext class="ltx_mathvariant_bold" id="S5.T5.4.4.4.2.m1.1.2.2.2.cmml" xref="S5.T5.4.4.4.2.m1.1.2.2.2">55.5</mtext></ci><cn type="float" id="S5.T5.4.4.4.2.m1.1.2.2.3.cmml" xref="S5.T5.4.4.4.2.m1.1.2.2.3">0.1</cn></apply><cn type="float" id="S5.T5.4.4.4.2.m1.1.1.cmml" xref="S5.T5.4.4.4.2.m1.1.1">1.2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.4.4.4.2.m1.1c">\textbf{55.5}_{0.1}~{}(1.2)</annotation></semantics></math></td>
</tr>
<tr id="S5.T5.15.15.26.11" class="ltx_tr">
<td id="S5.T5.15.15.26.11.1" class="ltx_td ltx_align_center ltx_border_t" rowspan="3"><span id="S5.T5.15.15.26.11.1.1" class="ltx_text">Reasoning</span></td>
<td id="S5.T5.15.15.26.11.2" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T5.15.15.26.11.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">Baseline</td>
<td id="S5.T5.15.15.26.11.4" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T5.15.15.26.11.5" class="ltx_td ltx_align_center ltx_border_t">83.5 (-)</td>
</tr>
<tr id="S5.T5.5.5.5" class="ltx_tr">
<td id="S5.T5.5.5.5.2" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T5.5.5.5.3" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T5.5.5.5.1" class="ltx_td ltx_align_center"><math id="S5.T5.5.5.5.1.m1.1" class="ltx_Math" alttext="0.2_{0.1}~{}(18.8)" display="inline"><semantics id="S5.T5.5.5.5.1.m1.1a"><mrow id="S5.T5.5.5.5.1.m1.1.2" xref="S5.T5.5.5.5.1.m1.1.2.cmml"><msub id="S5.T5.5.5.5.1.m1.1.2.2" xref="S5.T5.5.5.5.1.m1.1.2.2.cmml"><mn id="S5.T5.5.5.5.1.m1.1.2.2.2" xref="S5.T5.5.5.5.1.m1.1.2.2.2.cmml">0.2</mn><mn id="S5.T5.5.5.5.1.m1.1.2.2.3" xref="S5.T5.5.5.5.1.m1.1.2.2.3.cmml">0.1</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.5.5.5.1.m1.1.2.1" xref="S5.T5.5.5.5.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T5.5.5.5.1.m1.1.2.3.2" xref="S5.T5.5.5.5.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T5.5.5.5.1.m1.1.2.3.2.1" xref="S5.T5.5.5.5.1.m1.1.2.cmml">(</mo><mn id="S5.T5.5.5.5.1.m1.1.1" xref="S5.T5.5.5.5.1.m1.1.1.cmml">18.8</mn><mo stretchy="false" id="S5.T5.5.5.5.1.m1.1.2.3.2.2" xref="S5.T5.5.5.5.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.5.5.5.1.m1.1b"><apply id="S5.T5.5.5.5.1.m1.1.2.cmml" xref="S5.T5.5.5.5.1.m1.1.2"><times id="S5.T5.5.5.5.1.m1.1.2.1.cmml" xref="S5.T5.5.5.5.1.m1.1.2.1"></times><apply id="S5.T5.5.5.5.1.m1.1.2.2.cmml" xref="S5.T5.5.5.5.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T5.5.5.5.1.m1.1.2.2.1.cmml" xref="S5.T5.5.5.5.1.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T5.5.5.5.1.m1.1.2.2.2.cmml" xref="S5.T5.5.5.5.1.m1.1.2.2.2">0.2</cn><cn type="float" id="S5.T5.5.5.5.1.m1.1.2.2.3.cmml" xref="S5.T5.5.5.5.1.m1.1.2.2.3">0.1</cn></apply><cn type="float" id="S5.T5.5.5.5.1.m1.1.1.cmml" xref="S5.T5.5.5.5.1.m1.1.1">18.8</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.5.5.5.1.m1.1c">0.2_{0.1}~{}(18.8)</annotation></semantics></math></td>
<td id="S5.T5.5.5.5.4" class="ltx_td ltx_align_center">82.3 (1.2)</td>
</tr>
<tr id="S5.T5.6.6.6" class="ltx_tr">
<td id="S5.T5.6.6.6.2" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T5.6.6.6.3" class="ltx_td ltx_align_center ltx_border_r">LoTTO (ours)</td>
<td id="S5.T5.6.6.6.1" class="ltx_td ltx_align_center"><math id="S5.T5.6.6.6.1.m1.1" class="ltx_Math" alttext="\textbf{16.5}_{0.9}~{}(2.5)" display="inline"><semantics id="S5.T5.6.6.6.1.m1.1a"><mrow id="S5.T5.6.6.6.1.m1.1.2" xref="S5.T5.6.6.6.1.m1.1.2.cmml"><msub id="S5.T5.6.6.6.1.m1.1.2.2" xref="S5.T5.6.6.6.1.m1.1.2.2.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T5.6.6.6.1.m1.1.2.2.2" xref="S5.T5.6.6.6.1.m1.1.2.2.2a.cmml">16.5</mtext><mn id="S5.T5.6.6.6.1.m1.1.2.2.3" xref="S5.T5.6.6.6.1.m1.1.2.2.3.cmml">0.9</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.6.6.6.1.m1.1.2.1" xref="S5.T5.6.6.6.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T5.6.6.6.1.m1.1.2.3.2" xref="S5.T5.6.6.6.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T5.6.6.6.1.m1.1.2.3.2.1" xref="S5.T5.6.6.6.1.m1.1.2.cmml">(</mo><mn id="S5.T5.6.6.6.1.m1.1.1" xref="S5.T5.6.6.6.1.m1.1.1.cmml">2.5</mn><mo stretchy="false" id="S5.T5.6.6.6.1.m1.1.2.3.2.2" xref="S5.T5.6.6.6.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.6.6.6.1.m1.1b"><apply id="S5.T5.6.6.6.1.m1.1.2.cmml" xref="S5.T5.6.6.6.1.m1.1.2"><times id="S5.T5.6.6.6.1.m1.1.2.1.cmml" xref="S5.T5.6.6.6.1.m1.1.2.1"></times><apply id="S5.T5.6.6.6.1.m1.1.2.2.cmml" xref="S5.T5.6.6.6.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T5.6.6.6.1.m1.1.2.2.1.cmml" xref="S5.T5.6.6.6.1.m1.1.2.2">subscript</csymbol><ci id="S5.T5.6.6.6.1.m1.1.2.2.2a.cmml" xref="S5.T5.6.6.6.1.m1.1.2.2.2"><mtext class="ltx_mathvariant_bold" id="S5.T5.6.6.6.1.m1.1.2.2.2.cmml" xref="S5.T5.6.6.6.1.m1.1.2.2.2">16.5</mtext></ci><cn type="float" id="S5.T5.6.6.6.1.m1.1.2.2.3.cmml" xref="S5.T5.6.6.6.1.m1.1.2.2.3">0.9</cn></apply><cn type="float" id="S5.T5.6.6.6.1.m1.1.1.cmml" xref="S5.T5.6.6.6.1.m1.1.1">2.5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.6.6.6.1.m1.1c">\textbf{16.5}_{0.9}~{}(2.5)</annotation></semantics></math></td>
<td id="S5.T5.6.6.6.4" class="ltx_td ltx_align_center">
<span id="S5.T5.6.6.6.4.1" class="ltx_text ltx_font_bold">83.7</span> (-)</td>
</tr>
<tr id="S5.T5.7.7.7" class="ltx_tr">
<td id="S5.T5.7.7.7.2" class="ltx_td ltx_align_center ltx_border_t" rowspan="4"><span id="S5.T5.7.7.7.2.1" class="ltx_text">GSM8k+Arc+SQL</span></td>
<td id="S5.T5.7.7.7.3" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T5.7.7.7.4" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">Baseline</td>
<td id="S5.T5.7.7.7.5" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T5.7.7.7.1" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T5.7.7.7.1.m1.1" class="ltx_Math" alttext="77.0" display="inline"><semantics id="S5.T5.7.7.7.1.m1.1a"><mn id="S5.T5.7.7.7.1.m1.1.1" xref="S5.T5.7.7.7.1.m1.1.1.cmml">77.0</mn><annotation-xml encoding="MathML-Content" id="S5.T5.7.7.7.1.m1.1b"><cn type="float" id="S5.T5.7.7.7.1.m1.1.1.cmml" xref="S5.T5.7.7.7.1.m1.1.1">77.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.7.7.7.1.m1.1c">77.0</annotation></semantics></math></td>
</tr>
<tr id="S5.T5.9.9.9" class="ltx_tr">
<td id="S5.T5.9.9.9.3" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T5.9.9.9.4" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T5.8.8.8.1" class="ltx_td ltx_align_center"><math id="S5.T5.8.8.8.1.m1.1" class="ltx_Math" alttext="0.5_{0.2}~{}(18.6)" display="inline"><semantics id="S5.T5.8.8.8.1.m1.1a"><mrow id="S5.T5.8.8.8.1.m1.1.2" xref="S5.T5.8.8.8.1.m1.1.2.cmml"><msub id="S5.T5.8.8.8.1.m1.1.2.2" xref="S5.T5.8.8.8.1.m1.1.2.2.cmml"><mn id="S5.T5.8.8.8.1.m1.1.2.2.2" xref="S5.T5.8.8.8.1.m1.1.2.2.2.cmml">0.5</mn><mn id="S5.T5.8.8.8.1.m1.1.2.2.3" xref="S5.T5.8.8.8.1.m1.1.2.2.3.cmml">0.2</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.8.8.8.1.m1.1.2.1" xref="S5.T5.8.8.8.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T5.8.8.8.1.m1.1.2.3.2" xref="S5.T5.8.8.8.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T5.8.8.8.1.m1.1.2.3.2.1" xref="S5.T5.8.8.8.1.m1.1.2.cmml">(</mo><mn id="S5.T5.8.8.8.1.m1.1.1" xref="S5.T5.8.8.8.1.m1.1.1.cmml">18.6</mn><mo stretchy="false" id="S5.T5.8.8.8.1.m1.1.2.3.2.2" xref="S5.T5.8.8.8.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.8.8.8.1.m1.1b"><apply id="S5.T5.8.8.8.1.m1.1.2.cmml" xref="S5.T5.8.8.8.1.m1.1.2"><times id="S5.T5.8.8.8.1.m1.1.2.1.cmml" xref="S5.T5.8.8.8.1.m1.1.2.1"></times><apply id="S5.T5.8.8.8.1.m1.1.2.2.cmml" xref="S5.T5.8.8.8.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T5.8.8.8.1.m1.1.2.2.1.cmml" xref="S5.T5.8.8.8.1.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T5.8.8.8.1.m1.1.2.2.2.cmml" xref="S5.T5.8.8.8.1.m1.1.2.2.2">0.5</cn><cn type="float" id="S5.T5.8.8.8.1.m1.1.2.2.3.cmml" xref="S5.T5.8.8.8.1.m1.1.2.2.3">0.2</cn></apply><cn type="float" id="S5.T5.8.8.8.1.m1.1.1.cmml" xref="S5.T5.8.8.8.1.m1.1.1">18.6</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.8.8.8.1.m1.1c">0.5_{0.2}~{}(18.6)</annotation></semantics></math></td>
<td id="S5.T5.9.9.9.2" class="ltx_td ltx_align_center"><math id="S5.T5.9.9.9.2.m1.1" class="ltx_Math" alttext="\underline{75.0~{}(2.0)}" display="inline"><semantics id="S5.T5.9.9.9.2.m1.1a"><munder accentunder="true" id="S5.T5.9.9.9.2.m1.1.1" xref="S5.T5.9.9.9.2.m1.1.1.cmml"><mrow id="S5.T5.9.9.9.2.m1.1.1.1" xref="S5.T5.9.9.9.2.m1.1.1.1.cmml"><mn id="S5.T5.9.9.9.2.m1.1.1.1.3" xref="S5.T5.9.9.9.2.m1.1.1.1.3.cmml">75.0</mn><mo lspace="0.330em" rspace="0em" id="S5.T5.9.9.9.2.m1.1.1.1.2" xref="S5.T5.9.9.9.2.m1.1.1.1.2.cmml">​</mo><mrow id="S5.T5.9.9.9.2.m1.1.1.1.4.2" xref="S5.T5.9.9.9.2.m1.1.1.1.cmml"><mo stretchy="false" id="S5.T5.9.9.9.2.m1.1.1.1.4.2.1" xref="S5.T5.9.9.9.2.m1.1.1.1.cmml">(</mo><mn id="S5.T5.9.9.9.2.m1.1.1.1.1" xref="S5.T5.9.9.9.2.m1.1.1.1.1.cmml">2.0</mn><mo stretchy="false" id="S5.T5.9.9.9.2.m1.1.1.1.4.2.2" xref="S5.T5.9.9.9.2.m1.1.1.1.cmml">)</mo></mrow></mrow><mo id="S5.T5.9.9.9.2.m1.1.1.2" xref="S5.T5.9.9.9.2.m1.1.1.2.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T5.9.9.9.2.m1.1b"><apply id="S5.T5.9.9.9.2.m1.1.1.cmml" xref="S5.T5.9.9.9.2.m1.1.1"><ci id="S5.T5.9.9.9.2.m1.1.1.2.cmml" xref="S5.T5.9.9.9.2.m1.1.1.2">¯</ci><apply id="S5.T5.9.9.9.2.m1.1.1.1.cmml" xref="S5.T5.9.9.9.2.m1.1.1.1"><times id="S5.T5.9.9.9.2.m1.1.1.1.2.cmml" xref="S5.T5.9.9.9.2.m1.1.1.1.2"></times><cn type="float" id="S5.T5.9.9.9.2.m1.1.1.1.3.cmml" xref="S5.T5.9.9.9.2.m1.1.1.1.3">75.0</cn><cn type="float" id="S5.T5.9.9.9.2.m1.1.1.1.1.cmml" xref="S5.T5.9.9.9.2.m1.1.1.1.1">2.0</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.9.9.9.2.m1.1c">\underline{75.0~{}(2.0)}</annotation></semantics></math></td>
</tr>
<tr id="S5.T5.11.11.11" class="ltx_tr">
<td id="S5.T5.11.11.11.3" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T5.11.11.11.4" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T5.10.10.10.1" class="ltx_td ltx_align_center"><math id="S5.T5.10.10.10.1.m1.1" class="ltx_Math" alttext="\underline{11.5_{0.7}~{}(7.5)}" display="inline"><semantics id="S5.T5.10.10.10.1.m1.1a"><munder accentunder="true" id="S5.T5.10.10.10.1.m1.1.1" xref="S5.T5.10.10.10.1.m1.1.1.cmml"><mrow id="S5.T5.10.10.10.1.m1.1.1.1" xref="S5.T5.10.10.10.1.m1.1.1.1.cmml"><msub id="S5.T5.10.10.10.1.m1.1.1.1.3" xref="S5.T5.10.10.10.1.m1.1.1.1.3.cmml"><mn id="S5.T5.10.10.10.1.m1.1.1.1.3.2" xref="S5.T5.10.10.10.1.m1.1.1.1.3.2.cmml">11.5</mn><mn id="S5.T5.10.10.10.1.m1.1.1.1.3.3" xref="S5.T5.10.10.10.1.m1.1.1.1.3.3.cmml">0.7</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.10.10.10.1.m1.1.1.1.2" xref="S5.T5.10.10.10.1.m1.1.1.1.2.cmml">​</mo><mrow id="S5.T5.10.10.10.1.m1.1.1.1.4.2" xref="S5.T5.10.10.10.1.m1.1.1.1.cmml"><mo stretchy="false" id="S5.T5.10.10.10.1.m1.1.1.1.4.2.1" xref="S5.T5.10.10.10.1.m1.1.1.1.cmml">(</mo><mn id="S5.T5.10.10.10.1.m1.1.1.1.1" xref="S5.T5.10.10.10.1.m1.1.1.1.1.cmml">7.5</mn><mo stretchy="false" id="S5.T5.10.10.10.1.m1.1.1.1.4.2.2" xref="S5.T5.10.10.10.1.m1.1.1.1.cmml">)</mo></mrow></mrow><mo id="S5.T5.10.10.10.1.m1.1.1.2" xref="S5.T5.10.10.10.1.m1.1.1.2.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T5.10.10.10.1.m1.1b"><apply id="S5.T5.10.10.10.1.m1.1.1.cmml" xref="S5.T5.10.10.10.1.m1.1.1"><ci id="S5.T5.10.10.10.1.m1.1.1.2.cmml" xref="S5.T5.10.10.10.1.m1.1.1.2">¯</ci><apply id="S5.T5.10.10.10.1.m1.1.1.1.cmml" xref="S5.T5.10.10.10.1.m1.1.1.1"><times id="S5.T5.10.10.10.1.m1.1.1.1.2.cmml" xref="S5.T5.10.10.10.1.m1.1.1.1.2"></times><apply id="S5.T5.10.10.10.1.m1.1.1.1.3.cmml" xref="S5.T5.10.10.10.1.m1.1.1.1.3"><csymbol cd="ambiguous" id="S5.T5.10.10.10.1.m1.1.1.1.3.1.cmml" xref="S5.T5.10.10.10.1.m1.1.1.1.3">subscript</csymbol><cn type="float" id="S5.T5.10.10.10.1.m1.1.1.1.3.2.cmml" xref="S5.T5.10.10.10.1.m1.1.1.1.3.2">11.5</cn><cn type="float" id="S5.T5.10.10.10.1.m1.1.1.1.3.3.cmml" xref="S5.T5.10.10.10.1.m1.1.1.1.3.3">0.7</cn></apply><cn type="float" id="S5.T5.10.10.10.1.m1.1.1.1.1.cmml" xref="S5.T5.10.10.10.1.m1.1.1.1.1">7.5</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.10.10.10.1.m1.1c">\underline{11.5_{0.7}~{}(7.5)}</annotation></semantics></math></td>
<td id="S5.T5.11.11.11.2" class="ltx_td ltx_align_center"><span id="S5.T5.11.11.11.2.1" class="ltx_text ltx_markedasmath ltx_font_bold">75.4 (1.6)</span></td>
</tr>
<tr id="S5.T5.13.13.13" class="ltx_tr">
<td id="S5.T5.13.13.13.3" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T5.13.13.13.4" class="ltx_td ltx_align_center ltx_border_r">LoTTO (ours)</td>
<td id="S5.T5.12.12.12.1" class="ltx_td ltx_align_center"><math id="S5.T5.12.12.12.1.m1.1" class="ltx_Math" alttext="\textbf{15.9}_{0.9}~{}(3.1)" display="inline"><semantics id="S5.T5.12.12.12.1.m1.1a"><mrow id="S5.T5.12.12.12.1.m1.1.2" xref="S5.T5.12.12.12.1.m1.1.2.cmml"><msub id="S5.T5.12.12.12.1.m1.1.2.2" xref="S5.T5.12.12.12.1.m1.1.2.2.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T5.12.12.12.1.m1.1.2.2.2" xref="S5.T5.12.12.12.1.m1.1.2.2.2a.cmml">15.9</mtext><mn id="S5.T5.12.12.12.1.m1.1.2.2.3" xref="S5.T5.12.12.12.1.m1.1.2.2.3.cmml">0.9</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.12.12.12.1.m1.1.2.1" xref="S5.T5.12.12.12.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T5.12.12.12.1.m1.1.2.3.2" xref="S5.T5.12.12.12.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T5.12.12.12.1.m1.1.2.3.2.1" xref="S5.T5.12.12.12.1.m1.1.2.cmml">(</mo><mn id="S5.T5.12.12.12.1.m1.1.1" xref="S5.T5.12.12.12.1.m1.1.1.cmml">3.1</mn><mo stretchy="false" id="S5.T5.12.12.12.1.m1.1.2.3.2.2" xref="S5.T5.12.12.12.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.12.12.12.1.m1.1b"><apply id="S5.T5.12.12.12.1.m1.1.2.cmml" xref="S5.T5.12.12.12.1.m1.1.2"><times id="S5.T5.12.12.12.1.m1.1.2.1.cmml" xref="S5.T5.12.12.12.1.m1.1.2.1"></times><apply id="S5.T5.12.12.12.1.m1.1.2.2.cmml" xref="S5.T5.12.12.12.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T5.12.12.12.1.m1.1.2.2.1.cmml" xref="S5.T5.12.12.12.1.m1.1.2.2">subscript</csymbol><ci id="S5.T5.12.12.12.1.m1.1.2.2.2a.cmml" xref="S5.T5.12.12.12.1.m1.1.2.2.2"><mtext class="ltx_mathvariant_bold" id="S5.T5.12.12.12.1.m1.1.2.2.2.cmml" xref="S5.T5.12.12.12.1.m1.1.2.2.2">15.9</mtext></ci><cn type="float" id="S5.T5.12.12.12.1.m1.1.2.2.3.cmml" xref="S5.T5.12.12.12.1.m1.1.2.2.3">0.9</cn></apply><cn type="float" id="S5.T5.12.12.12.1.m1.1.1.cmml" xref="S5.T5.12.12.12.1.m1.1.1">3.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.12.12.12.1.m1.1c">\textbf{15.9}_{0.9}~{}(3.1)</annotation></semantics></math></td>
<td id="S5.T5.13.13.13.2" class="ltx_td ltx_align_center"><math id="S5.T5.13.13.13.2.m1.1" class="ltx_Math" alttext="73.8~{}(3.2)" display="inline"><semantics id="S5.T5.13.13.13.2.m1.1a"><mrow id="S5.T5.13.13.13.2.m1.1.2" xref="S5.T5.13.13.13.2.m1.1.2.cmml"><mn id="S5.T5.13.13.13.2.m1.1.2.2" xref="S5.T5.13.13.13.2.m1.1.2.2.cmml">73.8</mn><mo lspace="0.330em" rspace="0em" id="S5.T5.13.13.13.2.m1.1.2.1" xref="S5.T5.13.13.13.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T5.13.13.13.2.m1.1.2.3.2" xref="S5.T5.13.13.13.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T5.13.13.13.2.m1.1.2.3.2.1" xref="S5.T5.13.13.13.2.m1.1.2.cmml">(</mo><mn id="S5.T5.13.13.13.2.m1.1.1" xref="S5.T5.13.13.13.2.m1.1.1.cmml">3.2</mn><mo stretchy="false" id="S5.T5.13.13.13.2.m1.1.2.3.2.2" xref="S5.T5.13.13.13.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.13.13.13.2.m1.1b"><apply id="S5.T5.13.13.13.2.m1.1.2.cmml" xref="S5.T5.13.13.13.2.m1.1.2"><times id="S5.T5.13.13.13.2.m1.1.2.1.cmml" xref="S5.T5.13.13.13.2.m1.1.2.1"></times><cn type="float" id="S5.T5.13.13.13.2.m1.1.2.2.cmml" xref="S5.T5.13.13.13.2.m1.1.2.2">73.8</cn><cn type="float" id="S5.T5.13.13.13.2.m1.1.1.cmml" xref="S5.T5.13.13.13.2.m1.1.1">3.2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.13.13.13.2.m1.1c">73.8~{}(3.2)</annotation></semantics></math></td>
</tr>
<tr id="S5.T5.15.15.27.12" class="ltx_tr">
<td id="S5.T5.15.15.27.12.1" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t" rowspan="3"><span id="S5.T5.15.15.27.12.1.1" class="ltx_text">Safety</span></td>
<td id="S5.T5.15.15.27.12.2" class="ltx_td ltx_align_center ltx_border_t">Baseline</td>
<td id="S5.T5.15.15.27.12.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">-</td>
<td id="S5.T5.15.15.27.12.4" class="ltx_td ltx_align_center ltx_border_t">93.1 (-)</td>
<td id="S5.T5.15.15.27.12.5" class="ltx_td ltx_align_center ltx_border_t">-</td>
</tr>
<tr id="S5.T5.14.14.14" class="ltx_tr">
<td id="S5.T5.14.14.14.2" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T5.14.14.14.3" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T5.14.14.14.1" class="ltx_td ltx_align_center"><math id="S5.T5.14.14.14.1.m1.1" class="ltx_Math" alttext="19.1_{3.5}~{}(73.9)" display="inline"><semantics id="S5.T5.14.14.14.1.m1.1a"><mrow id="S5.T5.14.14.14.1.m1.1.2" xref="S5.T5.14.14.14.1.m1.1.2.cmml"><msub id="S5.T5.14.14.14.1.m1.1.2.2" xref="S5.T5.14.14.14.1.m1.1.2.2.cmml"><mn id="S5.T5.14.14.14.1.m1.1.2.2.2" xref="S5.T5.14.14.14.1.m1.1.2.2.2.cmml">19.1</mn><mn id="S5.T5.14.14.14.1.m1.1.2.2.3" xref="S5.T5.14.14.14.1.m1.1.2.2.3.cmml">3.5</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.14.14.14.1.m1.1.2.1" xref="S5.T5.14.14.14.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T5.14.14.14.1.m1.1.2.3.2" xref="S5.T5.14.14.14.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T5.14.14.14.1.m1.1.2.3.2.1" xref="S5.T5.14.14.14.1.m1.1.2.cmml">(</mo><mn id="S5.T5.14.14.14.1.m1.1.1" xref="S5.T5.14.14.14.1.m1.1.1.cmml">73.9</mn><mo stretchy="false" id="S5.T5.14.14.14.1.m1.1.2.3.2.2" xref="S5.T5.14.14.14.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.14.14.14.1.m1.1b"><apply id="S5.T5.14.14.14.1.m1.1.2.cmml" xref="S5.T5.14.14.14.1.m1.1.2"><times id="S5.T5.14.14.14.1.m1.1.2.1.cmml" xref="S5.T5.14.14.14.1.m1.1.2.1"></times><apply id="S5.T5.14.14.14.1.m1.1.2.2.cmml" xref="S5.T5.14.14.14.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T5.14.14.14.1.m1.1.2.2.1.cmml" xref="S5.T5.14.14.14.1.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T5.14.14.14.1.m1.1.2.2.2.cmml" xref="S5.T5.14.14.14.1.m1.1.2.2.2">19.1</cn><cn type="float" id="S5.T5.14.14.14.1.m1.1.2.2.3.cmml" xref="S5.T5.14.14.14.1.m1.1.2.2.3">3.5</cn></apply><cn type="float" id="S5.T5.14.14.14.1.m1.1.1.cmml" xref="S5.T5.14.14.14.1.m1.1.1">73.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.14.14.14.1.m1.1c">19.1_{3.5}~{}(73.9)</annotation></semantics></math></td>
<td id="S5.T5.14.14.14.4" class="ltx_td ltx_align_center">-</td>
</tr>
<tr id="S5.T5.15.15.15" class="ltx_tr">
<td id="S5.T5.15.15.15.2" class="ltx_td ltx_align_center ltx_border_bb">LoTA (ours)</td>
<td id="S5.T5.15.15.15.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_r">LoTTO (ours)</td>
<td id="S5.T5.15.15.15.1" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T5.15.15.15.1.m1.1" class="ltx_Math" alttext="\textbf{63.4}_{2.2}~{}(29.7)" display="inline"><semantics id="S5.T5.15.15.15.1.m1.1a"><mrow id="S5.T5.15.15.15.1.m1.1.2" xref="S5.T5.15.15.15.1.m1.1.2.cmml"><msub id="S5.T5.15.15.15.1.m1.1.2.2" xref="S5.T5.15.15.15.1.m1.1.2.2.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T5.15.15.15.1.m1.1.2.2.2" xref="S5.T5.15.15.15.1.m1.1.2.2.2a.cmml">63.4</mtext><mn id="S5.T5.15.15.15.1.m1.1.2.2.3" xref="S5.T5.15.15.15.1.m1.1.2.2.3.cmml">2.2</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T5.15.15.15.1.m1.1.2.1" xref="S5.T5.15.15.15.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T5.15.15.15.1.m1.1.2.3.2" xref="S5.T5.15.15.15.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T5.15.15.15.1.m1.1.2.3.2.1" xref="S5.T5.15.15.15.1.m1.1.2.cmml">(</mo><mn id="S5.T5.15.15.15.1.m1.1.1" xref="S5.T5.15.15.15.1.m1.1.1.cmml">29.7</mn><mo stretchy="false" id="S5.T5.15.15.15.1.m1.1.2.3.2.2" xref="S5.T5.15.15.15.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T5.15.15.15.1.m1.1b"><apply id="S5.T5.15.15.15.1.m1.1.2.cmml" xref="S5.T5.15.15.15.1.m1.1.2"><times id="S5.T5.15.15.15.1.m1.1.2.1.cmml" xref="S5.T5.15.15.15.1.m1.1.2.1"></times><apply id="S5.T5.15.15.15.1.m1.1.2.2.cmml" xref="S5.T5.15.15.15.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T5.15.15.15.1.m1.1.2.2.1.cmml" xref="S5.T5.15.15.15.1.m1.1.2.2">subscript</csymbol><ci id="S5.T5.15.15.15.1.m1.1.2.2.2a.cmml" xref="S5.T5.15.15.15.1.m1.1.2.2.2"><mtext class="ltx_mathvariant_bold" id="S5.T5.15.15.15.1.m1.1.2.2.2.cmml" xref="S5.T5.15.15.15.1.m1.1.2.2.2">63.4</mtext></ci><cn type="float" id="S5.T5.15.15.15.1.m1.1.2.2.3.cmml" xref="S5.T5.15.15.15.1.m1.1.2.2.3">2.2</cn></apply><cn type="float" id="S5.T5.15.15.15.1.m1.1.1.cmml" xref="S5.T5.15.15.15.1.m1.1.1">29.7</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.15.15.15.1.m1.1c">\textbf{63.4}_{2.2}~{}(29.7)</annotation></semantics></math></td>
<td id="S5.T5.15.15.15.4" class="ltx_td ltx_align_center ltx_border_bb">-</td>
</tr>
</tbody>
</table>
</span></div>
</figure>
<div id="S5.SS3.p2" class="ltx_para">
<p id="S5.SS3.p2.1" class="ltx_p"><span id="S5.SS3.p2.1.1" class="ltx_text ltx_font_bold">Mitigating Catastrophic Forgetting While Enabling Adaptation to Downstream Datasets.</span>
In <a href="#S5.T5" title="In 5.3 Sequential Training ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">5</span></a> we consider a range of method combinations for a simplified setting where we seek to adapt an Instruct model to Math data without catastrophic forgetting. We will go row-by-row through the table and analyze each set of results. Even when training on just a single, relatively small dataset, FFT in both phases suffers a significant drop in winrate. An easy way to mitigate this is to simply train the initial Instruct model with LoTA. Following this, FFT on GSM8k does not significantly reduce winrate. However, it does present a potentially unwelcome tradeoff in task accuracy. For this, we turn to LoTTO, which achieves the best performance across both tasks.</p>
</div>
<div id="S5.SS3.p3" class="ltx_para">
<p id="S5.SS3.p3.1" class="ltx_p"><span id="S5.SS3.p3.1.1" class="ltx_text ltx_font_bold">Does LoRA Really Forget Less, or Does It Just Learn Less?</span>
Recent work evaluates the performance of pre-trained models before and after fine-tuning on domain-specific tasks with LoRA <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib33" title="" class="ltx_ref">33</a>, <a href="#bib.bib65" title="" class="ltx_ref">65</a>]</cite> and concludes that LoRA is less prone to catastrophic forgetting than FFT. In <a href="#S5.T5" title="In 5.3 Sequential Training ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">5</span></a> we find that adapting the FFT Instruct model with LoRA does not lead to less degradation in winrate than adapting the FFT Instruct model with FFT, but it does lead to worse performance on the downstream task. Doing LoRA and then LoRA, or LoRA and then FFT, also does not match LoTA’s ability to prevent forgetting while enabling adaptation to downstream data.</p>
</div>
<div id="S5.SS3.p4" class="ltx_para">
<p id="S5.SS3.p4.1" class="ltx_p"><span id="S5.SS3.p4.1.1" class="ltx_text ltx_font_bold">How Much Does Data Reuse Help?</span>
The simplest and arguably most performant method from prior work that we found for mitigating catastrophic forgetting is simply to mix in some in-distribution data from Task A. This is the line marked with “FFT (Mixed)”, and it does mitigate forgetting on Task A, but at the cost of performance on Task B. We ablate the amount of data from Task A to be used between <math id="S5.SS3.p4.1.m1.1" class="ltx_Math" alttext="1-100\%" display="inline"><semantics id="S5.SS3.p4.1.m1.1a"><mrow id="S5.SS3.p4.1.m1.1.1" xref="S5.SS3.p4.1.m1.1.1.cmml"><mn id="S5.SS3.p4.1.m1.1.1.2" xref="S5.SS3.p4.1.m1.1.1.2.cmml">1</mn><mo id="S5.SS3.p4.1.m1.1.1.1" xref="S5.SS3.p4.1.m1.1.1.1.cmml">−</mo><mrow id="S5.SS3.p4.1.m1.1.1.3" xref="S5.SS3.p4.1.m1.1.1.3.cmml"><mn id="S5.SS3.p4.1.m1.1.1.3.2" xref="S5.SS3.p4.1.m1.1.1.3.2.cmml">100</mn><mo id="S5.SS3.p4.1.m1.1.1.3.1" xref="S5.SS3.p4.1.m1.1.1.3.1.cmml">%</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.SS3.p4.1.m1.1b"><apply id="S5.SS3.p4.1.m1.1.1.cmml" xref="S5.SS3.p4.1.m1.1.1"><minus id="S5.SS3.p4.1.m1.1.1.1.cmml" xref="S5.SS3.p4.1.m1.1.1.1"></minus><cn type="integer" id="S5.SS3.p4.1.m1.1.1.2.cmml" xref="S5.SS3.p4.1.m1.1.1.2">1</cn><apply id="S5.SS3.p4.1.m1.1.1.3.cmml" xref="S5.SS3.p4.1.m1.1.1.3"><csymbol cd="latexml" id="S5.SS3.p4.1.m1.1.1.3.1.cmml" xref="S5.SS3.p4.1.m1.1.1.3.1">percent</csymbol><cn type="integer" id="S5.SS3.p4.1.m1.1.1.3.2.cmml" xref="S5.SS3.p4.1.m1.1.1.3.2">100</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p4.1.m1.1c">1-100\%</annotation></semantics></math> of the dataset size of the data from Task B, and this is the best result in terms of mitigating forgetting on Task A. As we mix in less and less data from Task A, this method approaches just doing FFT sequentially in performance, so we omit those results for brevity.</p>
</div>
<div id="S5.SS3.p5" class="ltx_para">
<p id="S5.SS3.p5.1" class="ltx_p"><span id="S5.SS3.p5.1.1" class="ltx_text ltx_font_bold">Adapting Sequentially to Multiple Tasks with LoTTO.</span> We now consider the more challenging setting when we need to adapt to <em id="S5.SS3.p5.1.2" class="ltx_emph ltx_font_italic">multiple</em> tasks without catastrophic forgetting while still obtaining good performance on those tasks.
In <a href="#S5.T5" title="In 5.3 Sequential Training ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">5</span></a> we adapt an Instruct model to a mix of reasoning, math, and SQL data and find a surprising result; the FFT Instruct model collapses almost completely to a winrate of less than half a percent. As we showed in <a href="#S5.T5" title="In 5.3 Sequential Training ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">5</span></a> this can be mitigated by mixing in more instruction following data, albeit at a cost. The LoTA Instruct model still degrades in performance (<math id="S5.SS3.p5.1.m1.1" class="ltx_Math" alttext="19\rightarrow 11" display="inline"><semantics id="S5.SS3.p5.1.m1.1a"><mrow id="S5.SS3.p5.1.m1.1.1" xref="S5.SS3.p5.1.m1.1.1.cmml"><mn id="S5.SS3.p5.1.m1.1.1.2" xref="S5.SS3.p5.1.m1.1.1.2.cmml">19</mn><mo stretchy="false" id="S5.SS3.p5.1.m1.1.1.1" xref="S5.SS3.p5.1.m1.1.1.1.cmml">→</mo><mn id="S5.SS3.p5.1.m1.1.1.3" xref="S5.SS3.p5.1.m1.1.1.3.cmml">11</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.SS3.p5.1.m1.1b"><apply id="S5.SS3.p5.1.m1.1.1.cmml" xref="S5.SS3.p5.1.m1.1.1"><ci id="S5.SS3.p5.1.m1.1.1.1.cmml" xref="S5.SS3.p5.1.m1.1.1.1">→</ci><cn type="integer" id="S5.SS3.p5.1.m1.1.1.2.cmml" xref="S5.SS3.p5.1.m1.1.1.2">19</cn><cn type="integer" id="S5.SS3.p5.1.m1.1.1.3.cmml" xref="S5.SS3.p5.1.m1.1.1.3">11</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p5.1.m1.1c">19\rightarrow 11</annotation></semantics></math>) but nowhere near as much. In the line marked “LoTTO”, we instead adapt the LoTA Instruct model to the downstream tasks using the mask calibrated from GSM8k with LoTTO. Applying LoTTO to make the downstream adaptation be mutually sparse with the initial LoTA Instruct model increases performance significantly on instruction following and math. Performance on Arc and SQL suffers somewhat because our mask does not consider those tasks, but this means that our mask is much cheaper to calibrate and is, in fact, generalizable not only across tasks within a domain (as shown in <a href="#S5.T5" title="In 5.3 Sequential Training ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">5</span></a>) but also <em id="S5.SS3.p5.1.3" class="ltx_emph ltx_font_italic">across</em> domains.</p>
</div>
<div id="S5.SS3.p6" class="ltx_para">
<p id="S5.SS3.p6.2" class="ltx_p"><span id="S5.SS3.p6.2.1" class="ltx_text ltx_font_bold">Mitigating Catastrophic Forgetting of Safety Alignment.</span>
In the “Safety” row of <a href="#S5.T5" title="In 5.3 Sequential Training ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">5</span></a>, we consider fine-tuning the Mistral Instruct model we trained ourselves on the <math id="S5.SS3.p6.1.m1.1" class="ltx_Math" alttext="100" display="inline"><semantics id="S5.SS3.p6.1.m1.1a"><mn id="S5.SS3.p6.1.m1.1.1" xref="S5.SS3.p6.1.m1.1.1.cmml">100</mn><annotation-xml encoding="MathML-Content" id="S5.SS3.p6.1.m1.1b"><cn type="integer" id="S5.SS3.p6.1.m1.1.1.cmml" xref="S5.SS3.p6.1.m1.1.1">100</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p6.1.m1.1c">100</annotation></semantics></math> harmful instructions from <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib62" title="" class="ltx_ref">62</a>]</cite>. The baseline model gets a score of <math id="S5.SS3.p6.2.m2.1" class="ltx_Math" alttext="93\%" display="inline"><semantics id="S5.SS3.p6.2.m2.1a"><mrow id="S5.SS3.p6.2.m2.1.1" xref="S5.SS3.p6.2.m2.1.1.cmml"><mn id="S5.SS3.p6.2.m2.1.1.2" xref="S5.SS3.p6.2.m2.1.1.2.cmml">93</mn><mo id="S5.SS3.p6.2.m2.1.1.1" xref="S5.SS3.p6.2.m2.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS3.p6.2.m2.1b"><apply id="S5.SS3.p6.2.m2.1.1.cmml" xref="S5.SS3.p6.2.m2.1.1"><csymbol cd="latexml" id="S5.SS3.p6.2.m2.1.1.1.cmml" xref="S5.SS3.p6.2.m2.1.1.1">percent</csymbol><cn type="integer" id="S5.SS3.p6.2.m2.1.1.2.cmml" xref="S5.SS3.p6.2.m2.1.1.2">93</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p6.2.m2.1c">93\%</annotation></semantics></math> and training with FFT quickly degrades safety. Adapting the LoTA Instruct with LoTTO (again, with the LoTTO mask calibrated from GSM8k) mitigates this safety drop significantly, even though our LoTTO mask was calibrated on an extremely different dataset. Therefore, a potential mitigation would be for an entity providing a fine-tuning API such as OpenAI to do the safety training with LoTA, calibrate the LoTTO mask on a utility dataset, and then do fine-tuning on their client’s dataset with LoTTO. We do not intend to present our method as an active defense against fine-tuning attacks; although finetuning with LoTTO improves safety, given sufficient data and access to the model weights, any attacker can of course undo safety tuning entirely. However, catastrophic forgetting of safety alignment is an important problem with real-world applications, and we find it compelling that our method can mitigate this.</p>
</div>
</section>
<section id="S5.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.4 </span>LoTA for Model Merging</h3>

<div id="S5.SS4.p1" class="ltx_para">
<p id="S5.SS4.p1.1" class="ltx_p">We now consider the setting of model merging, where we train models on disjoint datasets fully in parallel and then merge together the task vectors with the goal of producing a model with good performance on multiple tasks. Prior work in model merging mostly considers merging similar datasets, such as the commonsense reasoning datasets, but it is relatively easy to merge models when the datasets are similar and becomes increasingly hard as the datasets become more heterogeneous due to the gradient mismatch <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib66" title="" class="ltx_ref">66</a>]</cite>. We consider merging models trained on heterogeneous datasets.</p>
</div>
<div id="S5.SS4.p2" class="ltx_para">
<p id="S5.SS4.p2.1" class="ltx_p"><span id="S5.SS4.p2.1.1" class="ltx_text ltx_font_bold">Merging Models. </span>
We use TIES-Merging <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib20" title="" class="ltx_ref">20</a>]</cite> to merge models trained on heterogeneous tasks. TIES performs post-hoc sparsification on each task vector and requires a 2-D hyperparameter search for this quantity, which we perform for the merge of FFT models. Naturally, we could optimize the performance of Task A by fully sparsifying Task B, and vice versa; we report the result that achieves good performance on Task B while maintaining some performance on Task A, and report the full range of hyperparameters in <a href="#A1" title="Appendix A Additional Experimental Details ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Appendix</span> <span class="ltx_text ltx_ref_tag">A</span></a>. LoTA is inherently sparse, so when we merge a LoTA model with an FFT model we do not need to perform hyperparameter search on the LoTA model, and we use the same level of sparsity for the FFT model that we obtained when merging together two FFT models. When merging two LoTA models together, no hyperparameter search is required at all as both models are inherently sparse. We could in theory sparsify the LoTA models beyond their existing levels with post-hoc sparsification, but we do not tune this hyperparameter.</p>
</div>
<div id="S5.SS4.p3" class="ltx_para">
<p id="S5.SS4.p3.1" class="ltx_p"><span id="S5.SS4.p3.1.1" class="ltx_text ltx_font_bold">Challenge of Overlapping Sparsity in Model Merging.</span>
In the sequential training paradigm, we exploited the fact that masks for different tasks have a significant overlap in order to generalize our LoTTO mask calibrated on GSM8k to provide robustness to forgetting across a range of other tasks. However, this same phenomenon of overlapping sparsity presents a challenge in the model merging setting.
The challenge is that because the merging is parallel, we cannot use LoTTO to calibrate the masks to be disjointly sparse as we did in the sequential training setting. One alternative is to enforce random disjoint masks for each task, corresponding to “<math id="S5.SS4.p3.1.m1.1" class="ltx_Math" alttext="0\%" display="inline"><semantics id="S5.SS4.p3.1.m1.1a"><mrow id="S5.SS4.p3.1.m1.1.1" xref="S5.SS4.p3.1.m1.1.1.cmml"><mn id="S5.SS4.p3.1.m1.1.1.2" xref="S5.SS4.p3.1.m1.1.1.2.cmml">0</mn><mo id="S5.SS4.p3.1.m1.1.1.1" xref="S5.SS4.p3.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS4.p3.1.m1.1b"><apply id="S5.SS4.p3.1.m1.1.1.cmml" xref="S5.SS4.p3.1.m1.1.1"><csymbol cd="latexml" id="S5.SS4.p3.1.m1.1.1.1.cmml" xref="S5.SS4.p3.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S5.SS4.p3.1.m1.1.1.2.cmml" xref="S5.SS4.p3.1.m1.1.1.2">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p3.1.m1.1c">0\%</annotation></semantics></math> Data Used” in <a href="#S5.T2" title="In 5.1 LoTA Design Choices ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">2</span></a>.</p>
</div>
<figure id="S5.T6" class="ltx_table">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table"><span id="S5.T6.28.1.1" class="ltx_text" style="font-size:90%;">Table 6</span>: </span><span id="S5.T6.29.2" class="ltx_text" style="font-size:90%;">Model merging using TIES of Task A, Instruction Following, and Task B, (varied). Fine-tuning on Tasks A and B is performed using both full fine-tuning (FFT) and LoTA. The utility of each task is computed after merging the two task vectors. <span id="S5.T6.29.2.1" class="ltx_text ltx_font_bold">bold</span>: best result, <span id="S5.T6.29.2.2" class="ltx_text ltx_framed ltx_framed_underline">underline</span>: second best result. We reproduce the baseline for each task on FFT from <a href="#S5.T3" title="In 5.2 Adapting to a Single Task ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">3</span></a> for convenience; note again that for some tasks the LoTA baseline outperforms the FFT baseline. All results are with Mistral.</span></figcaption>
<div id="S5.T6.24" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:433.6pt;height:248.1pt;vertical-align:-0.8pt;"><span class="ltx_transformed_inner" style="transform:translate(-51.5pt,29.4pt) scale(0.808069790073809,0.808069790073809) ;">
<table id="S5.T6.24.24" class="ltx_tabular ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S5.T6.24.24.25.1" class="ltx_tr">
<td id="S5.T6.24.24.25.1.1" class="ltx_td ltx_align_center ltx_border_tt">Task B</td>
<td id="S5.T6.24.24.25.1.2" class="ltx_td ltx_align_center ltx_border_tt">Method on Task A</td>
<td id="S5.T6.24.24.25.1.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_tt">Method on Task B</td>
<td id="S5.T6.24.24.25.1.4" class="ltx_td ltx_align_center ltx_border_tt">Utility of Task A (Drop)</td>
<td id="S5.T6.24.24.25.1.5" class="ltx_td ltx_align_center ltx_border_tt">Utility of Task B (Drop)</td>
</tr>
<tr id="S5.T6.24.24.26.2" class="ltx_tr">
<td id="S5.T6.24.24.26.2.1" class="ltx_td ltx_align_center ltx_border_t">Instruction Following</td>
<td id="S5.T6.24.24.26.2.2" class="ltx_td ltx_align_center ltx_border_t">Baseline</td>
<td id="S5.T6.24.24.26.2.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">-</td>
<td id="S5.T6.24.24.26.2.4" class="ltx_td ltx_align_center ltx_border_t">19.0 (-)</td>
<td id="S5.T6.24.24.26.2.5" class="ltx_td ltx_align_center ltx_border_t">-</td>
</tr>
<tr id="S5.T6.24.24.27.3" class="ltx_tr">
<td id="S5.T6.24.24.27.3.1" class="ltx_td ltx_align_center ltx_border_t" rowspan="4"><span id="S5.T6.24.24.27.3.1.1" class="ltx_text">GSM8k</span></td>
<td id="S5.T6.24.24.27.3.2" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T6.24.24.27.3.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">Baseline</td>
<td id="S5.T6.24.24.27.3.4" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T6.24.24.27.3.5" class="ltx_td ltx_align_center ltx_border_t">59.8 (-)</td>
</tr>
<tr id="S5.T6.2.2.2" class="ltx_tr">
<td id="S5.T6.2.2.2.3" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T6.2.2.2.4" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T6.1.1.1.1" class="ltx_td ltx_align_center"><math id="S5.T6.1.1.1.1.m1.1" class="ltx_Math" alttext="6.9_{0.5}~{}(12.1)" display="inline"><semantics id="S5.T6.1.1.1.1.m1.1a"><mrow id="S5.T6.1.1.1.1.m1.1.2" xref="S5.T6.1.1.1.1.m1.1.2.cmml"><msub id="S5.T6.1.1.1.1.m1.1.2.2" xref="S5.T6.1.1.1.1.m1.1.2.2.cmml"><mn id="S5.T6.1.1.1.1.m1.1.2.2.2" xref="S5.T6.1.1.1.1.m1.1.2.2.2.cmml">6.9</mn><mn id="S5.T6.1.1.1.1.m1.1.2.2.3" xref="S5.T6.1.1.1.1.m1.1.2.2.3.cmml">0.5</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.1.1.1.1.m1.1.2.1" xref="S5.T6.1.1.1.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.1.1.1.1.m1.1.2.3.2" xref="S5.T6.1.1.1.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.1.1.1.1.m1.1.2.3.2.1" xref="S5.T6.1.1.1.1.m1.1.2.cmml">(</mo><mn id="S5.T6.1.1.1.1.m1.1.1" xref="S5.T6.1.1.1.1.m1.1.1.cmml">12.1</mn><mo stretchy="false" id="S5.T6.1.1.1.1.m1.1.2.3.2.2" xref="S5.T6.1.1.1.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.1.1.1.1.m1.1b"><apply id="S5.T6.1.1.1.1.m1.1.2.cmml" xref="S5.T6.1.1.1.1.m1.1.2"><times id="S5.T6.1.1.1.1.m1.1.2.1.cmml" xref="S5.T6.1.1.1.1.m1.1.2.1"></times><apply id="S5.T6.1.1.1.1.m1.1.2.2.cmml" xref="S5.T6.1.1.1.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.1.1.1.1.m1.1.2.2.1.cmml" xref="S5.T6.1.1.1.1.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.1.1.1.1.m1.1.2.2.2.cmml" xref="S5.T6.1.1.1.1.m1.1.2.2.2">6.9</cn><cn type="float" id="S5.T6.1.1.1.1.m1.1.2.2.3.cmml" xref="S5.T6.1.1.1.1.m1.1.2.2.3">0.5</cn></apply><cn type="float" id="S5.T6.1.1.1.1.m1.1.1.cmml" xref="S5.T6.1.1.1.1.m1.1.1">12.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.1.1.1.1.m1.1c">6.9_{0.5}~{}(12.1)</annotation></semantics></math></td>
<td id="S5.T6.2.2.2.2" class="ltx_td ltx_align_center"><math id="S5.T6.2.2.2.2.m1.1" class="ltx_Math" alttext="59.1_{0.1}~{}(0.7)" display="inline"><semantics id="S5.T6.2.2.2.2.m1.1a"><mrow id="S5.T6.2.2.2.2.m1.1.2" xref="S5.T6.2.2.2.2.m1.1.2.cmml"><msub id="S5.T6.2.2.2.2.m1.1.2.2" xref="S5.T6.2.2.2.2.m1.1.2.2.cmml"><mn id="S5.T6.2.2.2.2.m1.1.2.2.2" xref="S5.T6.2.2.2.2.m1.1.2.2.2.cmml">59.1</mn><mn id="S5.T6.2.2.2.2.m1.1.2.2.3" xref="S5.T6.2.2.2.2.m1.1.2.2.3.cmml">0.1</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.2.2.2.2.m1.1.2.1" xref="S5.T6.2.2.2.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.2.2.2.2.m1.1.2.3.2" xref="S5.T6.2.2.2.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.2.2.2.2.m1.1.2.3.2.1" xref="S5.T6.2.2.2.2.m1.1.2.cmml">(</mo><mn id="S5.T6.2.2.2.2.m1.1.1" xref="S5.T6.2.2.2.2.m1.1.1.cmml">0.7</mn><mo stretchy="false" id="S5.T6.2.2.2.2.m1.1.2.3.2.2" xref="S5.T6.2.2.2.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.2.2.2.2.m1.1b"><apply id="S5.T6.2.2.2.2.m1.1.2.cmml" xref="S5.T6.2.2.2.2.m1.1.2"><times id="S5.T6.2.2.2.2.m1.1.2.1.cmml" xref="S5.T6.2.2.2.2.m1.1.2.1"></times><apply id="S5.T6.2.2.2.2.m1.1.2.2.cmml" xref="S5.T6.2.2.2.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.2.2.2.2.m1.1.2.2.1.cmml" xref="S5.T6.2.2.2.2.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.2.2.2.2.m1.1.2.2.2.cmml" xref="S5.T6.2.2.2.2.m1.1.2.2.2">59.1</cn><cn type="float" id="S5.T6.2.2.2.2.m1.1.2.2.3.cmml" xref="S5.T6.2.2.2.2.m1.1.2.2.3">0.1</cn></apply><cn type="float" id="S5.T6.2.2.2.2.m1.1.1.cmml" xref="S5.T6.2.2.2.2.m1.1.1">0.7</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.2.2.2.2.m1.1c">59.1_{0.1}~{}(0.7)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.4.4.4" class="ltx_tr">
<td id="S5.T6.4.4.4.3" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T6.4.4.4.4" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T6.3.3.3.1" class="ltx_td ltx_align_center"><math id="S5.T6.3.3.3.1.m1.1" class="ltx_Math" alttext="\textbf{16.1}_{0.8}~{}(2.9)" display="inline"><semantics id="S5.T6.3.3.3.1.m1.1a"><mrow id="S5.T6.3.3.3.1.m1.1.2" xref="S5.T6.3.3.3.1.m1.1.2.cmml"><msub id="S5.T6.3.3.3.1.m1.1.2.2" xref="S5.T6.3.3.3.1.m1.1.2.2.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T6.3.3.3.1.m1.1.2.2.2" xref="S5.T6.3.3.3.1.m1.1.2.2.2a.cmml">16.1</mtext><mn id="S5.T6.3.3.3.1.m1.1.2.2.3" xref="S5.T6.3.3.3.1.m1.1.2.2.3.cmml">0.8</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.3.3.3.1.m1.1.2.1" xref="S5.T6.3.3.3.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.3.3.3.1.m1.1.2.3.2" xref="S5.T6.3.3.3.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.3.3.3.1.m1.1.2.3.2.1" xref="S5.T6.3.3.3.1.m1.1.2.cmml">(</mo><mn id="S5.T6.3.3.3.1.m1.1.1" xref="S5.T6.3.3.3.1.m1.1.1.cmml">2.9</mn><mo stretchy="false" id="S5.T6.3.3.3.1.m1.1.2.3.2.2" xref="S5.T6.3.3.3.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.3.3.3.1.m1.1b"><apply id="S5.T6.3.3.3.1.m1.1.2.cmml" xref="S5.T6.3.3.3.1.m1.1.2"><times id="S5.T6.3.3.3.1.m1.1.2.1.cmml" xref="S5.T6.3.3.3.1.m1.1.2.1"></times><apply id="S5.T6.3.3.3.1.m1.1.2.2.cmml" xref="S5.T6.3.3.3.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.3.3.3.1.m1.1.2.2.1.cmml" xref="S5.T6.3.3.3.1.m1.1.2.2">subscript</csymbol><ci id="S5.T6.3.3.3.1.m1.1.2.2.2a.cmml" xref="S5.T6.3.3.3.1.m1.1.2.2.2"><mtext class="ltx_mathvariant_bold" id="S5.T6.3.3.3.1.m1.1.2.2.2.cmml" xref="S5.T6.3.3.3.1.m1.1.2.2.2">16.1</mtext></ci><cn type="float" id="S5.T6.3.3.3.1.m1.1.2.2.3.cmml" xref="S5.T6.3.3.3.1.m1.1.2.2.3">0.8</cn></apply><cn type="float" id="S5.T6.3.3.3.1.m1.1.1.cmml" xref="S5.T6.3.3.3.1.m1.1.1">2.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.3.3.3.1.m1.1c">\textbf{16.1}_{0.8}~{}(2.9)</annotation></semantics></math></td>
<td id="S5.T6.4.4.4.2" class="ltx_td ltx_align_center"><math id="S5.T6.4.4.4.2.m1.1" class="ltx_Math" alttext="\textbf{60.5}_{1.1}~{}(+0.7)" display="inline"><semantics id="S5.T6.4.4.4.2.m1.1a"><mrow id="S5.T6.4.4.4.2.m1.1.1" xref="S5.T6.4.4.4.2.m1.1.1.cmml"><msub id="S5.T6.4.4.4.2.m1.1.1.3" xref="S5.T6.4.4.4.2.m1.1.1.3.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T6.4.4.4.2.m1.1.1.3.2" xref="S5.T6.4.4.4.2.m1.1.1.3.2a.cmml">60.5</mtext><mn id="S5.T6.4.4.4.2.m1.1.1.3.3" xref="S5.T6.4.4.4.2.m1.1.1.3.3.cmml">1.1</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.4.4.4.2.m1.1.1.2" xref="S5.T6.4.4.4.2.m1.1.1.2.cmml">​</mo><mrow id="S5.T6.4.4.4.2.m1.1.1.1.1" xref="S5.T6.4.4.4.2.m1.1.1.1.1.1.cmml"><mo stretchy="false" id="S5.T6.4.4.4.2.m1.1.1.1.1.2" xref="S5.T6.4.4.4.2.m1.1.1.1.1.1.cmml">(</mo><mrow id="S5.T6.4.4.4.2.m1.1.1.1.1.1" xref="S5.T6.4.4.4.2.m1.1.1.1.1.1.cmml"><mo id="S5.T6.4.4.4.2.m1.1.1.1.1.1a" xref="S5.T6.4.4.4.2.m1.1.1.1.1.1.cmml">+</mo><mn id="S5.T6.4.4.4.2.m1.1.1.1.1.1.2" xref="S5.T6.4.4.4.2.m1.1.1.1.1.1.2.cmml">0.7</mn></mrow><mo stretchy="false" id="S5.T6.4.4.4.2.m1.1.1.1.1.3" xref="S5.T6.4.4.4.2.m1.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.4.4.4.2.m1.1b"><apply id="S5.T6.4.4.4.2.m1.1.1.cmml" xref="S5.T6.4.4.4.2.m1.1.1"><times id="S5.T6.4.4.4.2.m1.1.1.2.cmml" xref="S5.T6.4.4.4.2.m1.1.1.2"></times><apply id="S5.T6.4.4.4.2.m1.1.1.3.cmml" xref="S5.T6.4.4.4.2.m1.1.1.3"><csymbol cd="ambiguous" id="S5.T6.4.4.4.2.m1.1.1.3.1.cmml" xref="S5.T6.4.4.4.2.m1.1.1.3">subscript</csymbol><ci id="S5.T6.4.4.4.2.m1.1.1.3.2a.cmml" xref="S5.T6.4.4.4.2.m1.1.1.3.2"><mtext class="ltx_mathvariant_bold" id="S5.T6.4.4.4.2.m1.1.1.3.2.cmml" xref="S5.T6.4.4.4.2.m1.1.1.3.2">60.5</mtext></ci><cn type="float" id="S5.T6.4.4.4.2.m1.1.1.3.3.cmml" xref="S5.T6.4.4.4.2.m1.1.1.3.3">1.1</cn></apply><apply id="S5.T6.4.4.4.2.m1.1.1.1.1.1.cmml" xref="S5.T6.4.4.4.2.m1.1.1.1.1"><plus id="S5.T6.4.4.4.2.m1.1.1.1.1.1.1.cmml" xref="S5.T6.4.4.4.2.m1.1.1.1.1"></plus><cn type="float" id="S5.T6.4.4.4.2.m1.1.1.1.1.1.2.cmml" xref="S5.T6.4.4.4.2.m1.1.1.1.1.1.2">0.7</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.4.4.4.2.m1.1c">\textbf{60.5}_{1.1}~{}(+0.7)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.6.6.6" class="ltx_tr">
<td id="S5.T6.6.6.6.3" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T6.6.6.6.4" class="ltx_td ltx_align_center ltx_border_r">LoTA (ours)</td>
<td id="S5.T6.5.5.5.1" class="ltx_td ltx_align_center"><math id="S5.T6.5.5.5.1.m1.1" class="ltx_Math" alttext="14.0_{0.8}~{}(5.0)" display="inline"><semantics id="S5.T6.5.5.5.1.m1.1a"><mrow id="S5.T6.5.5.5.1.m1.1.2" xref="S5.T6.5.5.5.1.m1.1.2.cmml"><msub id="S5.T6.5.5.5.1.m1.1.2.2" xref="S5.T6.5.5.5.1.m1.1.2.2.cmml"><mn id="S5.T6.5.5.5.1.m1.1.2.2.2" xref="S5.T6.5.5.5.1.m1.1.2.2.2.cmml">14.0</mn><mn id="S5.T6.5.5.5.1.m1.1.2.2.3" xref="S5.T6.5.5.5.1.m1.1.2.2.3.cmml">0.8</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.5.5.5.1.m1.1.2.1" xref="S5.T6.5.5.5.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.5.5.5.1.m1.1.2.3.2" xref="S5.T6.5.5.5.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.5.5.5.1.m1.1.2.3.2.1" xref="S5.T6.5.5.5.1.m1.1.2.cmml">(</mo><mn id="S5.T6.5.5.5.1.m1.1.1" xref="S5.T6.5.5.5.1.m1.1.1.cmml">5.0</mn><mo stretchy="false" id="S5.T6.5.5.5.1.m1.1.2.3.2.2" xref="S5.T6.5.5.5.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.5.5.5.1.m1.1b"><apply id="S5.T6.5.5.5.1.m1.1.2.cmml" xref="S5.T6.5.5.5.1.m1.1.2"><times id="S5.T6.5.5.5.1.m1.1.2.1.cmml" xref="S5.T6.5.5.5.1.m1.1.2.1"></times><apply id="S5.T6.5.5.5.1.m1.1.2.2.cmml" xref="S5.T6.5.5.5.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.5.5.5.1.m1.1.2.2.1.cmml" xref="S5.T6.5.5.5.1.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.5.5.5.1.m1.1.2.2.2.cmml" xref="S5.T6.5.5.5.1.m1.1.2.2.2">14.0</cn><cn type="float" id="S5.T6.5.5.5.1.m1.1.2.2.3.cmml" xref="S5.T6.5.5.5.1.m1.1.2.2.3">0.8</cn></apply><cn type="float" id="S5.T6.5.5.5.1.m1.1.1.cmml" xref="S5.T6.5.5.5.1.m1.1.1">5.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.5.5.5.1.m1.1c">14.0_{0.8}~{}(5.0)</annotation></semantics></math></td>
<td id="S5.T6.6.6.6.2" class="ltx_td ltx_align_center"><math id="S5.T6.6.6.6.2.m1.1" class="ltx_Math" alttext="\underline{59.3}_{1.0}~{}(0.5)" display="inline"><semantics id="S5.T6.6.6.6.2.m1.1a"><mrow id="S5.T6.6.6.6.2.m1.1.2" xref="S5.T6.6.6.6.2.m1.1.2.cmml"><msub id="S5.T6.6.6.6.2.m1.1.2.2" xref="S5.T6.6.6.6.2.m1.1.2.2.cmml"><munder accentunder="true" id="S5.T6.6.6.6.2.m1.1.2.2.2" xref="S5.T6.6.6.6.2.m1.1.2.2.2.cmml"><mn id="S5.T6.6.6.6.2.m1.1.2.2.2.2" xref="S5.T6.6.6.6.2.m1.1.2.2.2.2.cmml">59.3</mn><mo id="S5.T6.6.6.6.2.m1.1.2.2.2.1" xref="S5.T6.6.6.6.2.m1.1.2.2.2.1.cmml">¯</mo></munder><mn id="S5.T6.6.6.6.2.m1.1.2.2.3" xref="S5.T6.6.6.6.2.m1.1.2.2.3.cmml">1.0</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.6.6.6.2.m1.1.2.1" xref="S5.T6.6.6.6.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.6.6.6.2.m1.1.2.3.2" xref="S5.T6.6.6.6.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.6.6.6.2.m1.1.2.3.2.1" xref="S5.T6.6.6.6.2.m1.1.2.cmml">(</mo><mn id="S5.T6.6.6.6.2.m1.1.1" xref="S5.T6.6.6.6.2.m1.1.1.cmml">0.5</mn><mo stretchy="false" id="S5.T6.6.6.6.2.m1.1.2.3.2.2" xref="S5.T6.6.6.6.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.6.6.6.2.m1.1b"><apply id="S5.T6.6.6.6.2.m1.1.2.cmml" xref="S5.T6.6.6.6.2.m1.1.2"><times id="S5.T6.6.6.6.2.m1.1.2.1.cmml" xref="S5.T6.6.6.6.2.m1.1.2.1"></times><apply id="S5.T6.6.6.6.2.m1.1.2.2.cmml" xref="S5.T6.6.6.6.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.6.6.6.2.m1.1.2.2.1.cmml" xref="S5.T6.6.6.6.2.m1.1.2.2">subscript</csymbol><apply id="S5.T6.6.6.6.2.m1.1.2.2.2.cmml" xref="S5.T6.6.6.6.2.m1.1.2.2.2"><ci id="S5.T6.6.6.6.2.m1.1.2.2.2.1.cmml" xref="S5.T6.6.6.6.2.m1.1.2.2.2.1">¯</ci><cn type="float" id="S5.T6.6.6.6.2.m1.1.2.2.2.2.cmml" xref="S5.T6.6.6.6.2.m1.1.2.2.2.2">59.3</cn></apply><cn type="float" id="S5.T6.6.6.6.2.m1.1.2.2.3.cmml" xref="S5.T6.6.6.6.2.m1.1.2.2.3">1.0</cn></apply><cn type="float" id="S5.T6.6.6.6.2.m1.1.1.cmml" xref="S5.T6.6.6.6.2.m1.1.1">0.5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.6.6.6.2.m1.1c">\underline{59.3}_{1.0}~{}(0.5)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.8.8.8" class="ltx_tr">
<td id="S5.T6.8.8.8.3" class="ltx_td"></td>
<td id="S5.T6.8.8.8.4" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T6.8.8.8.5" class="ltx_td ltx_align_center ltx_border_r">LoTA (ours)</td>
<td id="S5.T6.7.7.7.1" class="ltx_td ltx_align_center"><math id="S5.T6.7.7.7.1.m1.1" class="ltx_Math" alttext="\underline{15.1}_{0.8}~{}(3.9)" display="inline"><semantics id="S5.T6.7.7.7.1.m1.1a"><mrow id="S5.T6.7.7.7.1.m1.1.2" xref="S5.T6.7.7.7.1.m1.1.2.cmml"><msub id="S5.T6.7.7.7.1.m1.1.2.2" xref="S5.T6.7.7.7.1.m1.1.2.2.cmml"><munder accentunder="true" id="S5.T6.7.7.7.1.m1.1.2.2.2" xref="S5.T6.7.7.7.1.m1.1.2.2.2.cmml"><mn id="S5.T6.7.7.7.1.m1.1.2.2.2.2" xref="S5.T6.7.7.7.1.m1.1.2.2.2.2.cmml">15.1</mn><mo id="S5.T6.7.7.7.1.m1.1.2.2.2.1" xref="S5.T6.7.7.7.1.m1.1.2.2.2.1.cmml">¯</mo></munder><mn id="S5.T6.7.7.7.1.m1.1.2.2.3" xref="S5.T6.7.7.7.1.m1.1.2.2.3.cmml">0.8</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.7.7.7.1.m1.1.2.1" xref="S5.T6.7.7.7.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.7.7.7.1.m1.1.2.3.2" xref="S5.T6.7.7.7.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.7.7.7.1.m1.1.2.3.2.1" xref="S5.T6.7.7.7.1.m1.1.2.cmml">(</mo><mn id="S5.T6.7.7.7.1.m1.1.1" xref="S5.T6.7.7.7.1.m1.1.1.cmml">3.9</mn><mo stretchy="false" id="S5.T6.7.7.7.1.m1.1.2.3.2.2" xref="S5.T6.7.7.7.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.7.7.7.1.m1.1b"><apply id="S5.T6.7.7.7.1.m1.1.2.cmml" xref="S5.T6.7.7.7.1.m1.1.2"><times id="S5.T6.7.7.7.1.m1.1.2.1.cmml" xref="S5.T6.7.7.7.1.m1.1.2.1"></times><apply id="S5.T6.7.7.7.1.m1.1.2.2.cmml" xref="S5.T6.7.7.7.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.7.7.7.1.m1.1.2.2.1.cmml" xref="S5.T6.7.7.7.1.m1.1.2.2">subscript</csymbol><apply id="S5.T6.7.7.7.1.m1.1.2.2.2.cmml" xref="S5.T6.7.7.7.1.m1.1.2.2.2"><ci id="S5.T6.7.7.7.1.m1.1.2.2.2.1.cmml" xref="S5.T6.7.7.7.1.m1.1.2.2.2.1">¯</ci><cn type="float" id="S5.T6.7.7.7.1.m1.1.2.2.2.2.cmml" xref="S5.T6.7.7.7.1.m1.1.2.2.2.2">15.1</cn></apply><cn type="float" id="S5.T6.7.7.7.1.m1.1.2.2.3.cmml" xref="S5.T6.7.7.7.1.m1.1.2.2.3">0.8</cn></apply><cn type="float" id="S5.T6.7.7.7.1.m1.1.1.cmml" xref="S5.T6.7.7.7.1.m1.1.1">3.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.7.7.7.1.m1.1c">\underline{15.1}_{0.8}~{}(3.9)</annotation></semantics></math></td>
<td id="S5.T6.8.8.8.2" class="ltx_td ltx_align_center"><math id="S5.T6.8.8.8.2.m1.1" class="ltx_Math" alttext="58.4_{1.1}~{}(1.5)" display="inline"><semantics id="S5.T6.8.8.8.2.m1.1a"><mrow id="S5.T6.8.8.8.2.m1.1.2" xref="S5.T6.8.8.8.2.m1.1.2.cmml"><msub id="S5.T6.8.8.8.2.m1.1.2.2" xref="S5.T6.8.8.8.2.m1.1.2.2.cmml"><mn id="S5.T6.8.8.8.2.m1.1.2.2.2" xref="S5.T6.8.8.8.2.m1.1.2.2.2.cmml">58.4</mn><mn id="S5.T6.8.8.8.2.m1.1.2.2.3" xref="S5.T6.8.8.8.2.m1.1.2.2.3.cmml">1.1</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.8.8.8.2.m1.1.2.1" xref="S5.T6.8.8.8.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.8.8.8.2.m1.1.2.3.2" xref="S5.T6.8.8.8.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.8.8.8.2.m1.1.2.3.2.1" xref="S5.T6.8.8.8.2.m1.1.2.cmml">(</mo><mn id="S5.T6.8.8.8.2.m1.1.1" xref="S5.T6.8.8.8.2.m1.1.1.cmml">1.5</mn><mo stretchy="false" id="S5.T6.8.8.8.2.m1.1.2.3.2.2" xref="S5.T6.8.8.8.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.8.8.8.2.m1.1b"><apply id="S5.T6.8.8.8.2.m1.1.2.cmml" xref="S5.T6.8.8.8.2.m1.1.2"><times id="S5.T6.8.8.8.2.m1.1.2.1.cmml" xref="S5.T6.8.8.8.2.m1.1.2.1"></times><apply id="S5.T6.8.8.8.2.m1.1.2.2.cmml" xref="S5.T6.8.8.8.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.8.8.8.2.m1.1.2.2.1.cmml" xref="S5.T6.8.8.8.2.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.8.8.8.2.m1.1.2.2.2.cmml" xref="S5.T6.8.8.8.2.m1.1.2.2.2">58.4</cn><cn type="float" id="S5.T6.8.8.8.2.m1.1.2.2.3.cmml" xref="S5.T6.8.8.8.2.m1.1.2.2.3">1.1</cn></apply><cn type="float" id="S5.T6.8.8.8.2.m1.1.1.cmml" xref="S5.T6.8.8.8.2.m1.1.1">1.5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.8.8.8.2.m1.1c">58.4_{1.1}~{}(1.5)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.24.24.28.4" class="ltx_tr">
<td id="S5.T6.24.24.28.4.1" class="ltx_td ltx_align_center ltx_border_t" rowspan="5"><span id="S5.T6.24.24.28.4.1.1" class="ltx_text">Samsum</span></td>
<td id="S5.T6.24.24.28.4.2" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T6.24.24.28.4.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">Baseline</td>
<td id="S5.T6.24.24.28.4.4" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T6.24.24.28.4.5" class="ltx_td ltx_align_center ltx_border_t">52.0 (-)</td>
</tr>
<tr id="S5.T6.10.10.10" class="ltx_tr">
<td id="S5.T6.10.10.10.3" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T6.10.10.10.4" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T6.9.9.9.1" class="ltx_td ltx_align_center"><math id="S5.T6.9.9.9.1.m1.1" class="ltx_Math" alttext="8.1_{0.7}~{}(10.9)" display="inline"><semantics id="S5.T6.9.9.9.1.m1.1a"><mrow id="S5.T6.9.9.9.1.m1.1.2" xref="S5.T6.9.9.9.1.m1.1.2.cmml"><msub id="S5.T6.9.9.9.1.m1.1.2.2" xref="S5.T6.9.9.9.1.m1.1.2.2.cmml"><mn id="S5.T6.9.9.9.1.m1.1.2.2.2" xref="S5.T6.9.9.9.1.m1.1.2.2.2.cmml">8.1</mn><mn id="S5.T6.9.9.9.1.m1.1.2.2.3" xref="S5.T6.9.9.9.1.m1.1.2.2.3.cmml">0.7</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.9.9.9.1.m1.1.2.1" xref="S5.T6.9.9.9.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.9.9.9.1.m1.1.2.3.2" xref="S5.T6.9.9.9.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.9.9.9.1.m1.1.2.3.2.1" xref="S5.T6.9.9.9.1.m1.1.2.cmml">(</mo><mn id="S5.T6.9.9.9.1.m1.1.1" xref="S5.T6.9.9.9.1.m1.1.1.cmml">10.9</mn><mo stretchy="false" id="S5.T6.9.9.9.1.m1.1.2.3.2.2" xref="S5.T6.9.9.9.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.9.9.9.1.m1.1b"><apply id="S5.T6.9.9.9.1.m1.1.2.cmml" xref="S5.T6.9.9.9.1.m1.1.2"><times id="S5.T6.9.9.9.1.m1.1.2.1.cmml" xref="S5.T6.9.9.9.1.m1.1.2.1"></times><apply id="S5.T6.9.9.9.1.m1.1.2.2.cmml" xref="S5.T6.9.9.9.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.9.9.9.1.m1.1.2.2.1.cmml" xref="S5.T6.9.9.9.1.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.9.9.9.1.m1.1.2.2.2.cmml" xref="S5.T6.9.9.9.1.m1.1.2.2.2">8.1</cn><cn type="float" id="S5.T6.9.9.9.1.m1.1.2.2.3.cmml" xref="S5.T6.9.9.9.1.m1.1.2.2.3">0.7</cn></apply><cn type="float" id="S5.T6.9.9.9.1.m1.1.1.cmml" xref="S5.T6.9.9.9.1.m1.1.1">10.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.9.9.9.1.m1.1c">8.1_{0.7}~{}(10.9)</annotation></semantics></math></td>
<td id="S5.T6.10.10.10.2" class="ltx_td ltx_align_center"><math id="S5.T6.10.10.10.2.m1.1" class="ltx_Math" alttext="47.0_{0.0}~{}(5.0)" display="inline"><semantics id="S5.T6.10.10.10.2.m1.1a"><mrow id="S5.T6.10.10.10.2.m1.1.2" xref="S5.T6.10.10.10.2.m1.1.2.cmml"><msub id="S5.T6.10.10.10.2.m1.1.2.2" xref="S5.T6.10.10.10.2.m1.1.2.2.cmml"><mn id="S5.T6.10.10.10.2.m1.1.2.2.2" xref="S5.T6.10.10.10.2.m1.1.2.2.2.cmml">47.0</mn><mn id="S5.T6.10.10.10.2.m1.1.2.2.3" xref="S5.T6.10.10.10.2.m1.1.2.2.3.cmml">0.0</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.10.10.10.2.m1.1.2.1" xref="S5.T6.10.10.10.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.10.10.10.2.m1.1.2.3.2" xref="S5.T6.10.10.10.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.10.10.10.2.m1.1.2.3.2.1" xref="S5.T6.10.10.10.2.m1.1.2.cmml">(</mo><mn id="S5.T6.10.10.10.2.m1.1.1" xref="S5.T6.10.10.10.2.m1.1.1.cmml">5.0</mn><mo stretchy="false" id="S5.T6.10.10.10.2.m1.1.2.3.2.2" xref="S5.T6.10.10.10.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.10.10.10.2.m1.1b"><apply id="S5.T6.10.10.10.2.m1.1.2.cmml" xref="S5.T6.10.10.10.2.m1.1.2"><times id="S5.T6.10.10.10.2.m1.1.2.1.cmml" xref="S5.T6.10.10.10.2.m1.1.2.1"></times><apply id="S5.T6.10.10.10.2.m1.1.2.2.cmml" xref="S5.T6.10.10.10.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.10.10.10.2.m1.1.2.2.1.cmml" xref="S5.T6.10.10.10.2.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.10.10.10.2.m1.1.2.2.2.cmml" xref="S5.T6.10.10.10.2.m1.1.2.2.2">47.0</cn><cn type="float" id="S5.T6.10.10.10.2.m1.1.2.2.3.cmml" xref="S5.T6.10.10.10.2.m1.1.2.2.3">0.0</cn></apply><cn type="float" id="S5.T6.10.10.10.2.m1.1.1.cmml" xref="S5.T6.10.10.10.2.m1.1.1">5.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.10.10.10.2.m1.1c">47.0_{0.0}~{}(5.0)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.12.12.12" class="ltx_tr">
<td id="S5.T6.12.12.12.3" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T6.12.12.12.4" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T6.11.11.11.1" class="ltx_td ltx_align_center"><math id="S5.T6.11.11.11.1.m1.1" class="ltx_Math" alttext="\underline{13.8}_{0.8}~{}(5.2)" display="inline"><semantics id="S5.T6.11.11.11.1.m1.1a"><mrow id="S5.T6.11.11.11.1.m1.1.2" xref="S5.T6.11.11.11.1.m1.1.2.cmml"><msub id="S5.T6.11.11.11.1.m1.1.2.2" xref="S5.T6.11.11.11.1.m1.1.2.2.cmml"><munder accentunder="true" id="S5.T6.11.11.11.1.m1.1.2.2.2" xref="S5.T6.11.11.11.1.m1.1.2.2.2.cmml"><mn id="S5.T6.11.11.11.1.m1.1.2.2.2.2" xref="S5.T6.11.11.11.1.m1.1.2.2.2.2.cmml">13.8</mn><mo id="S5.T6.11.11.11.1.m1.1.2.2.2.1" xref="S5.T6.11.11.11.1.m1.1.2.2.2.1.cmml">¯</mo></munder><mn id="S5.T6.11.11.11.1.m1.1.2.2.3" xref="S5.T6.11.11.11.1.m1.1.2.2.3.cmml">0.8</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.11.11.11.1.m1.1.2.1" xref="S5.T6.11.11.11.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.11.11.11.1.m1.1.2.3.2" xref="S5.T6.11.11.11.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.11.11.11.1.m1.1.2.3.2.1" xref="S5.T6.11.11.11.1.m1.1.2.cmml">(</mo><mn id="S5.T6.11.11.11.1.m1.1.1" xref="S5.T6.11.11.11.1.m1.1.1.cmml">5.2</mn><mo stretchy="false" id="S5.T6.11.11.11.1.m1.1.2.3.2.2" xref="S5.T6.11.11.11.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.11.11.11.1.m1.1b"><apply id="S5.T6.11.11.11.1.m1.1.2.cmml" xref="S5.T6.11.11.11.1.m1.1.2"><times id="S5.T6.11.11.11.1.m1.1.2.1.cmml" xref="S5.T6.11.11.11.1.m1.1.2.1"></times><apply id="S5.T6.11.11.11.1.m1.1.2.2.cmml" xref="S5.T6.11.11.11.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.11.11.11.1.m1.1.2.2.1.cmml" xref="S5.T6.11.11.11.1.m1.1.2.2">subscript</csymbol><apply id="S5.T6.11.11.11.1.m1.1.2.2.2.cmml" xref="S5.T6.11.11.11.1.m1.1.2.2.2"><ci id="S5.T6.11.11.11.1.m1.1.2.2.2.1.cmml" xref="S5.T6.11.11.11.1.m1.1.2.2.2.1">¯</ci><cn type="float" id="S5.T6.11.11.11.1.m1.1.2.2.2.2.cmml" xref="S5.T6.11.11.11.1.m1.1.2.2.2.2">13.8</cn></apply><cn type="float" id="S5.T6.11.11.11.1.m1.1.2.2.3.cmml" xref="S5.T6.11.11.11.1.m1.1.2.2.3">0.8</cn></apply><cn type="float" id="S5.T6.11.11.11.1.m1.1.1.cmml" xref="S5.T6.11.11.11.1.m1.1.1">5.2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.11.11.11.1.m1.1c">\underline{13.8}_{0.8}~{}(5.2)</annotation></semantics></math></td>
<td id="S5.T6.12.12.12.2" class="ltx_td ltx_align_center"><math id="S5.T6.12.12.12.2.m1.1" class="ltx_Math" alttext="51.8_{0.2}~{}(0.2)" display="inline"><semantics id="S5.T6.12.12.12.2.m1.1a"><mrow id="S5.T6.12.12.12.2.m1.1.2" xref="S5.T6.12.12.12.2.m1.1.2.cmml"><msub id="S5.T6.12.12.12.2.m1.1.2.2" xref="S5.T6.12.12.12.2.m1.1.2.2.cmml"><mn id="S5.T6.12.12.12.2.m1.1.2.2.2" xref="S5.T6.12.12.12.2.m1.1.2.2.2.cmml">51.8</mn><mn id="S5.T6.12.12.12.2.m1.1.2.2.3" xref="S5.T6.12.12.12.2.m1.1.2.2.3.cmml">0.2</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.12.12.12.2.m1.1.2.1" xref="S5.T6.12.12.12.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.12.12.12.2.m1.1.2.3.2" xref="S5.T6.12.12.12.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.12.12.12.2.m1.1.2.3.2.1" xref="S5.T6.12.12.12.2.m1.1.2.cmml">(</mo><mn id="S5.T6.12.12.12.2.m1.1.1" xref="S5.T6.12.12.12.2.m1.1.1.cmml">0.2</mn><mo stretchy="false" id="S5.T6.12.12.12.2.m1.1.2.3.2.2" xref="S5.T6.12.12.12.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.12.12.12.2.m1.1b"><apply id="S5.T6.12.12.12.2.m1.1.2.cmml" xref="S5.T6.12.12.12.2.m1.1.2"><times id="S5.T6.12.12.12.2.m1.1.2.1.cmml" xref="S5.T6.12.12.12.2.m1.1.2.1"></times><apply id="S5.T6.12.12.12.2.m1.1.2.2.cmml" xref="S5.T6.12.12.12.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.12.12.12.2.m1.1.2.2.1.cmml" xref="S5.T6.12.12.12.2.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.12.12.12.2.m1.1.2.2.2.cmml" xref="S5.T6.12.12.12.2.m1.1.2.2.2">51.8</cn><cn type="float" id="S5.T6.12.12.12.2.m1.1.2.2.3.cmml" xref="S5.T6.12.12.12.2.m1.1.2.2.3">0.2</cn></apply><cn type="float" id="S5.T6.12.12.12.2.m1.1.1.cmml" xref="S5.T6.12.12.12.2.m1.1.1">0.2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.12.12.12.2.m1.1c">51.8_{0.2}~{}(0.2)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.14.14.14" class="ltx_tr">
<td id="S5.T6.14.14.14.3" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T6.14.14.14.4" class="ltx_td ltx_align_center ltx_border_r">LoTA (ours)</td>
<td id="S5.T6.13.13.13.1" class="ltx_td ltx_align_center"><math id="S5.T6.13.13.13.1.m1.1" class="ltx_Math" alttext="10.4_{0.7}~{}(8.6)" display="inline"><semantics id="S5.T6.13.13.13.1.m1.1a"><mrow id="S5.T6.13.13.13.1.m1.1.2" xref="S5.T6.13.13.13.1.m1.1.2.cmml"><msub id="S5.T6.13.13.13.1.m1.1.2.2" xref="S5.T6.13.13.13.1.m1.1.2.2.cmml"><mn id="S5.T6.13.13.13.1.m1.1.2.2.2" xref="S5.T6.13.13.13.1.m1.1.2.2.2.cmml">10.4</mn><mn id="S5.T6.13.13.13.1.m1.1.2.2.3" xref="S5.T6.13.13.13.1.m1.1.2.2.3.cmml">0.7</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.13.13.13.1.m1.1.2.1" xref="S5.T6.13.13.13.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.13.13.13.1.m1.1.2.3.2" xref="S5.T6.13.13.13.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.13.13.13.1.m1.1.2.3.2.1" xref="S5.T6.13.13.13.1.m1.1.2.cmml">(</mo><mn id="S5.T6.13.13.13.1.m1.1.1" xref="S5.T6.13.13.13.1.m1.1.1.cmml">8.6</mn><mo stretchy="false" id="S5.T6.13.13.13.1.m1.1.2.3.2.2" xref="S5.T6.13.13.13.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.13.13.13.1.m1.1b"><apply id="S5.T6.13.13.13.1.m1.1.2.cmml" xref="S5.T6.13.13.13.1.m1.1.2"><times id="S5.T6.13.13.13.1.m1.1.2.1.cmml" xref="S5.T6.13.13.13.1.m1.1.2.1"></times><apply id="S5.T6.13.13.13.1.m1.1.2.2.cmml" xref="S5.T6.13.13.13.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.13.13.13.1.m1.1.2.2.1.cmml" xref="S5.T6.13.13.13.1.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.13.13.13.1.m1.1.2.2.2.cmml" xref="S5.T6.13.13.13.1.m1.1.2.2.2">10.4</cn><cn type="float" id="S5.T6.13.13.13.1.m1.1.2.2.3.cmml" xref="S5.T6.13.13.13.1.m1.1.2.2.3">0.7</cn></apply><cn type="float" id="S5.T6.13.13.13.1.m1.1.1.cmml" xref="S5.T6.13.13.13.1.m1.1.1">8.6</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.13.13.13.1.m1.1c">10.4_{0.7}~{}(8.6)</annotation></semantics></math></td>
<td id="S5.T6.14.14.14.2" class="ltx_td ltx_align_center"><math id="S5.T6.14.14.14.2.m1.1" class="ltx_Math" alttext="\textbf{53.3}_{0.1}~{}(+1.3)" display="inline"><semantics id="S5.T6.14.14.14.2.m1.1a"><mrow id="S5.T6.14.14.14.2.m1.1.1" xref="S5.T6.14.14.14.2.m1.1.1.cmml"><msub id="S5.T6.14.14.14.2.m1.1.1.3" xref="S5.T6.14.14.14.2.m1.1.1.3.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T6.14.14.14.2.m1.1.1.3.2" xref="S5.T6.14.14.14.2.m1.1.1.3.2a.cmml">53.3</mtext><mn id="S5.T6.14.14.14.2.m1.1.1.3.3" xref="S5.T6.14.14.14.2.m1.1.1.3.3.cmml">0.1</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.14.14.14.2.m1.1.1.2" xref="S5.T6.14.14.14.2.m1.1.1.2.cmml">​</mo><mrow id="S5.T6.14.14.14.2.m1.1.1.1.1" xref="S5.T6.14.14.14.2.m1.1.1.1.1.1.cmml"><mo stretchy="false" id="S5.T6.14.14.14.2.m1.1.1.1.1.2" xref="S5.T6.14.14.14.2.m1.1.1.1.1.1.cmml">(</mo><mrow id="S5.T6.14.14.14.2.m1.1.1.1.1.1" xref="S5.T6.14.14.14.2.m1.1.1.1.1.1.cmml"><mo id="S5.T6.14.14.14.2.m1.1.1.1.1.1a" xref="S5.T6.14.14.14.2.m1.1.1.1.1.1.cmml">+</mo><mn id="S5.T6.14.14.14.2.m1.1.1.1.1.1.2" xref="S5.T6.14.14.14.2.m1.1.1.1.1.1.2.cmml">1.3</mn></mrow><mo stretchy="false" id="S5.T6.14.14.14.2.m1.1.1.1.1.3" xref="S5.T6.14.14.14.2.m1.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.14.14.14.2.m1.1b"><apply id="S5.T6.14.14.14.2.m1.1.1.cmml" xref="S5.T6.14.14.14.2.m1.1.1"><times id="S5.T6.14.14.14.2.m1.1.1.2.cmml" xref="S5.T6.14.14.14.2.m1.1.1.2"></times><apply id="S5.T6.14.14.14.2.m1.1.1.3.cmml" xref="S5.T6.14.14.14.2.m1.1.1.3"><csymbol cd="ambiguous" id="S5.T6.14.14.14.2.m1.1.1.3.1.cmml" xref="S5.T6.14.14.14.2.m1.1.1.3">subscript</csymbol><ci id="S5.T6.14.14.14.2.m1.1.1.3.2a.cmml" xref="S5.T6.14.14.14.2.m1.1.1.3.2"><mtext class="ltx_mathvariant_bold" id="S5.T6.14.14.14.2.m1.1.1.3.2.cmml" xref="S5.T6.14.14.14.2.m1.1.1.3.2">53.3</mtext></ci><cn type="float" id="S5.T6.14.14.14.2.m1.1.1.3.3.cmml" xref="S5.T6.14.14.14.2.m1.1.1.3.3">0.1</cn></apply><apply id="S5.T6.14.14.14.2.m1.1.1.1.1.1.cmml" xref="S5.T6.14.14.14.2.m1.1.1.1.1"><plus id="S5.T6.14.14.14.2.m1.1.1.1.1.1.1.cmml" xref="S5.T6.14.14.14.2.m1.1.1.1.1"></plus><cn type="float" id="S5.T6.14.14.14.2.m1.1.1.1.1.1.2.cmml" xref="S5.T6.14.14.14.2.m1.1.1.1.1.1.2">1.3</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.14.14.14.2.m1.1c">\textbf{53.3}_{0.1}~{}(+1.3)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.16.16.16" class="ltx_tr">
<td id="S5.T6.16.16.16.3" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T6.16.16.16.4" class="ltx_td ltx_align_center ltx_border_r">LoTA (ours)</td>
<td id="S5.T6.15.15.15.1" class="ltx_td ltx_align_center"><math id="S5.T6.15.15.15.1.m1.1" class="ltx_Math" alttext="\textbf{15.4}_{0.9}~{}(3.6)" display="inline"><semantics id="S5.T6.15.15.15.1.m1.1a"><mrow id="S5.T6.15.15.15.1.m1.1.2" xref="S5.T6.15.15.15.1.m1.1.2.cmml"><msub id="S5.T6.15.15.15.1.m1.1.2.2" xref="S5.T6.15.15.15.1.m1.1.2.2.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T6.15.15.15.1.m1.1.2.2.2" xref="S5.T6.15.15.15.1.m1.1.2.2.2a.cmml">15.4</mtext><mn id="S5.T6.15.15.15.1.m1.1.2.2.3" xref="S5.T6.15.15.15.1.m1.1.2.2.3.cmml">0.9</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.15.15.15.1.m1.1.2.1" xref="S5.T6.15.15.15.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.15.15.15.1.m1.1.2.3.2" xref="S5.T6.15.15.15.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.15.15.15.1.m1.1.2.3.2.1" xref="S5.T6.15.15.15.1.m1.1.2.cmml">(</mo><mn id="S5.T6.15.15.15.1.m1.1.1" xref="S5.T6.15.15.15.1.m1.1.1.cmml">3.6</mn><mo stretchy="false" id="S5.T6.15.15.15.1.m1.1.2.3.2.2" xref="S5.T6.15.15.15.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.15.15.15.1.m1.1b"><apply id="S5.T6.15.15.15.1.m1.1.2.cmml" xref="S5.T6.15.15.15.1.m1.1.2"><times id="S5.T6.15.15.15.1.m1.1.2.1.cmml" xref="S5.T6.15.15.15.1.m1.1.2.1"></times><apply id="S5.T6.15.15.15.1.m1.1.2.2.cmml" xref="S5.T6.15.15.15.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.15.15.15.1.m1.1.2.2.1.cmml" xref="S5.T6.15.15.15.1.m1.1.2.2">subscript</csymbol><ci id="S5.T6.15.15.15.1.m1.1.2.2.2a.cmml" xref="S5.T6.15.15.15.1.m1.1.2.2.2"><mtext class="ltx_mathvariant_bold" id="S5.T6.15.15.15.1.m1.1.2.2.2.cmml" xref="S5.T6.15.15.15.1.m1.1.2.2.2">15.4</mtext></ci><cn type="float" id="S5.T6.15.15.15.1.m1.1.2.2.3.cmml" xref="S5.T6.15.15.15.1.m1.1.2.2.3">0.9</cn></apply><cn type="float" id="S5.T6.15.15.15.1.m1.1.1.cmml" xref="S5.T6.15.15.15.1.m1.1.1">3.6</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.15.15.15.1.m1.1c">\textbf{15.4}_{0.9}~{}(3.6)</annotation></semantics></math></td>
<td id="S5.T6.16.16.16.2" class="ltx_td ltx_align_center"><math id="S5.T6.16.16.16.2.m1.1" class="ltx_Math" alttext="\underline{52.7}_{0.1}~{}(0.1)" display="inline"><semantics id="S5.T6.16.16.16.2.m1.1a"><mrow id="S5.T6.16.16.16.2.m1.1.2" xref="S5.T6.16.16.16.2.m1.1.2.cmml"><msub id="S5.T6.16.16.16.2.m1.1.2.2" xref="S5.T6.16.16.16.2.m1.1.2.2.cmml"><munder accentunder="true" id="S5.T6.16.16.16.2.m1.1.2.2.2" xref="S5.T6.16.16.16.2.m1.1.2.2.2.cmml"><mn id="S5.T6.16.16.16.2.m1.1.2.2.2.2" xref="S5.T6.16.16.16.2.m1.1.2.2.2.2.cmml">52.7</mn><mo id="S5.T6.16.16.16.2.m1.1.2.2.2.1" xref="S5.T6.16.16.16.2.m1.1.2.2.2.1.cmml">¯</mo></munder><mn id="S5.T6.16.16.16.2.m1.1.2.2.3" xref="S5.T6.16.16.16.2.m1.1.2.2.3.cmml">0.1</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.16.16.16.2.m1.1.2.1" xref="S5.T6.16.16.16.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.16.16.16.2.m1.1.2.3.2" xref="S5.T6.16.16.16.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.16.16.16.2.m1.1.2.3.2.1" xref="S5.T6.16.16.16.2.m1.1.2.cmml">(</mo><mn id="S5.T6.16.16.16.2.m1.1.1" xref="S5.T6.16.16.16.2.m1.1.1.cmml">0.1</mn><mo stretchy="false" id="S5.T6.16.16.16.2.m1.1.2.3.2.2" xref="S5.T6.16.16.16.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.16.16.16.2.m1.1b"><apply id="S5.T6.16.16.16.2.m1.1.2.cmml" xref="S5.T6.16.16.16.2.m1.1.2"><times id="S5.T6.16.16.16.2.m1.1.2.1.cmml" xref="S5.T6.16.16.16.2.m1.1.2.1"></times><apply id="S5.T6.16.16.16.2.m1.1.2.2.cmml" xref="S5.T6.16.16.16.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.16.16.16.2.m1.1.2.2.1.cmml" xref="S5.T6.16.16.16.2.m1.1.2.2">subscript</csymbol><apply id="S5.T6.16.16.16.2.m1.1.2.2.2.cmml" xref="S5.T6.16.16.16.2.m1.1.2.2.2"><ci id="S5.T6.16.16.16.2.m1.1.2.2.2.1.cmml" xref="S5.T6.16.16.16.2.m1.1.2.2.2.1">¯</ci><cn type="float" id="S5.T6.16.16.16.2.m1.1.2.2.2.2.cmml" xref="S5.T6.16.16.16.2.m1.1.2.2.2.2">52.7</cn></apply><cn type="float" id="S5.T6.16.16.16.2.m1.1.2.2.3.cmml" xref="S5.T6.16.16.16.2.m1.1.2.2.3">0.1</cn></apply><cn type="float" id="S5.T6.16.16.16.2.m1.1.1.cmml" xref="S5.T6.16.16.16.2.m1.1.1">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.16.16.16.2.m1.1c">\underline{52.7}_{0.1}~{}(0.1)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.24.24.29.5" class="ltx_tr">
<td id="S5.T6.24.24.29.5.1" class="ltx_td ltx_align_center ltx_border_t" rowspan="4"><span id="S5.T6.24.24.29.5.1.1" class="ltx_text">GSM8k+Samsum</span></td>
<td id="S5.T6.24.24.29.5.2" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T6.24.24.29.5.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">Baseline</td>
<td id="S5.T6.24.24.29.5.4" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="S5.T6.24.24.29.5.5" class="ltx_td ltx_align_center ltx_border_t">55.9 (-)</td>
</tr>
<tr id="S5.T6.18.18.18" class="ltx_tr">
<td id="S5.T6.18.18.18.3" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T6.18.18.18.4" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T6.17.17.17.1" class="ltx_td ltx_align_center"><math id="S5.T6.17.17.17.1.m1.1" class="ltx_Math" alttext="7.6_{0.6}~{}(11.4)" display="inline"><semantics id="S5.T6.17.17.17.1.m1.1a"><mrow id="S5.T6.17.17.17.1.m1.1.2" xref="S5.T6.17.17.17.1.m1.1.2.cmml"><msub id="S5.T6.17.17.17.1.m1.1.2.2" xref="S5.T6.17.17.17.1.m1.1.2.2.cmml"><mn id="S5.T6.17.17.17.1.m1.1.2.2.2" xref="S5.T6.17.17.17.1.m1.1.2.2.2.cmml">7.6</mn><mn id="S5.T6.17.17.17.1.m1.1.2.2.3" xref="S5.T6.17.17.17.1.m1.1.2.2.3.cmml">0.6</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.17.17.17.1.m1.1.2.1" xref="S5.T6.17.17.17.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.17.17.17.1.m1.1.2.3.2" xref="S5.T6.17.17.17.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.17.17.17.1.m1.1.2.3.2.1" xref="S5.T6.17.17.17.1.m1.1.2.cmml">(</mo><mn id="S5.T6.17.17.17.1.m1.1.1" xref="S5.T6.17.17.17.1.m1.1.1.cmml">11.4</mn><mo stretchy="false" id="S5.T6.17.17.17.1.m1.1.2.3.2.2" xref="S5.T6.17.17.17.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.17.17.17.1.m1.1b"><apply id="S5.T6.17.17.17.1.m1.1.2.cmml" xref="S5.T6.17.17.17.1.m1.1.2"><times id="S5.T6.17.17.17.1.m1.1.2.1.cmml" xref="S5.T6.17.17.17.1.m1.1.2.1"></times><apply id="S5.T6.17.17.17.1.m1.1.2.2.cmml" xref="S5.T6.17.17.17.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.17.17.17.1.m1.1.2.2.1.cmml" xref="S5.T6.17.17.17.1.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.17.17.17.1.m1.1.2.2.2.cmml" xref="S5.T6.17.17.17.1.m1.1.2.2.2">7.6</cn><cn type="float" id="S5.T6.17.17.17.1.m1.1.2.2.3.cmml" xref="S5.T6.17.17.17.1.m1.1.2.2.3">0.6</cn></apply><cn type="float" id="S5.T6.17.17.17.1.m1.1.1.cmml" xref="S5.T6.17.17.17.1.m1.1.1">11.4</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.17.17.17.1.m1.1c">7.6_{0.6}~{}(11.4)</annotation></semantics></math></td>
<td id="S5.T6.18.18.18.2" class="ltx_td ltx_align_center"><math id="S5.T6.18.18.18.2.m1.1" class="ltx_Math" alttext="49.7_{1.1}~{}(6.2)" display="inline"><semantics id="S5.T6.18.18.18.2.m1.1a"><mrow id="S5.T6.18.18.18.2.m1.1.2" xref="S5.T6.18.18.18.2.m1.1.2.cmml"><msub id="S5.T6.18.18.18.2.m1.1.2.2" xref="S5.T6.18.18.18.2.m1.1.2.2.cmml"><mn id="S5.T6.18.18.18.2.m1.1.2.2.2" xref="S5.T6.18.18.18.2.m1.1.2.2.2.cmml">49.7</mn><mn id="S5.T6.18.18.18.2.m1.1.2.2.3" xref="S5.T6.18.18.18.2.m1.1.2.2.3.cmml">1.1</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.18.18.18.2.m1.1.2.1" xref="S5.T6.18.18.18.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.18.18.18.2.m1.1.2.3.2" xref="S5.T6.18.18.18.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.18.18.18.2.m1.1.2.3.2.1" xref="S5.T6.18.18.18.2.m1.1.2.cmml">(</mo><mn id="S5.T6.18.18.18.2.m1.1.1" xref="S5.T6.18.18.18.2.m1.1.1.cmml">6.2</mn><mo stretchy="false" id="S5.T6.18.18.18.2.m1.1.2.3.2.2" xref="S5.T6.18.18.18.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.18.18.18.2.m1.1b"><apply id="S5.T6.18.18.18.2.m1.1.2.cmml" xref="S5.T6.18.18.18.2.m1.1.2"><times id="S5.T6.18.18.18.2.m1.1.2.1.cmml" xref="S5.T6.18.18.18.2.m1.1.2.1"></times><apply id="S5.T6.18.18.18.2.m1.1.2.2.cmml" xref="S5.T6.18.18.18.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.18.18.18.2.m1.1.2.2.1.cmml" xref="S5.T6.18.18.18.2.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.18.18.18.2.m1.1.2.2.2.cmml" xref="S5.T6.18.18.18.2.m1.1.2.2.2">49.7</cn><cn type="float" id="S5.T6.18.18.18.2.m1.1.2.2.3.cmml" xref="S5.T6.18.18.18.2.m1.1.2.2.3">1.1</cn></apply><cn type="float" id="S5.T6.18.18.18.2.m1.1.1.cmml" xref="S5.T6.18.18.18.2.m1.1.1">6.2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.18.18.18.2.m1.1c">49.7_{1.1}~{}(6.2)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.20.20.20" class="ltx_tr">
<td id="S5.T6.20.20.20.3" class="ltx_td ltx_align_center">LoTA (ours)</td>
<td id="S5.T6.20.20.20.4" class="ltx_td ltx_align_center ltx_border_r">FFT</td>
<td id="S5.T6.19.19.19.1" class="ltx_td ltx_align_center"><math id="S5.T6.19.19.19.1.m1.1" class="ltx_Math" alttext="8.9_{0.6}~{}(10.1)" display="inline"><semantics id="S5.T6.19.19.19.1.m1.1a"><mrow id="S5.T6.19.19.19.1.m1.1.2" xref="S5.T6.19.19.19.1.m1.1.2.cmml"><msub id="S5.T6.19.19.19.1.m1.1.2.2" xref="S5.T6.19.19.19.1.m1.1.2.2.cmml"><mn id="S5.T6.19.19.19.1.m1.1.2.2.2" xref="S5.T6.19.19.19.1.m1.1.2.2.2.cmml">8.9</mn><mn id="S5.T6.19.19.19.1.m1.1.2.2.3" xref="S5.T6.19.19.19.1.m1.1.2.2.3.cmml">0.6</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.19.19.19.1.m1.1.2.1" xref="S5.T6.19.19.19.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.19.19.19.1.m1.1.2.3.2" xref="S5.T6.19.19.19.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.19.19.19.1.m1.1.2.3.2.1" xref="S5.T6.19.19.19.1.m1.1.2.cmml">(</mo><mn id="S5.T6.19.19.19.1.m1.1.1" xref="S5.T6.19.19.19.1.m1.1.1.cmml">10.1</mn><mo stretchy="false" id="S5.T6.19.19.19.1.m1.1.2.3.2.2" xref="S5.T6.19.19.19.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.19.19.19.1.m1.1b"><apply id="S5.T6.19.19.19.1.m1.1.2.cmml" xref="S5.T6.19.19.19.1.m1.1.2"><times id="S5.T6.19.19.19.1.m1.1.2.1.cmml" xref="S5.T6.19.19.19.1.m1.1.2.1"></times><apply id="S5.T6.19.19.19.1.m1.1.2.2.cmml" xref="S5.T6.19.19.19.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.19.19.19.1.m1.1.2.2.1.cmml" xref="S5.T6.19.19.19.1.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.19.19.19.1.m1.1.2.2.2.cmml" xref="S5.T6.19.19.19.1.m1.1.2.2.2">8.9</cn><cn type="float" id="S5.T6.19.19.19.1.m1.1.2.2.3.cmml" xref="S5.T6.19.19.19.1.m1.1.2.2.3">0.6</cn></apply><cn type="float" id="S5.T6.19.19.19.1.m1.1.1.cmml" xref="S5.T6.19.19.19.1.m1.1.1">10.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.19.19.19.1.m1.1c">8.9_{0.6}~{}(10.1)</annotation></semantics></math></td>
<td id="S5.T6.20.20.20.2" class="ltx_td ltx_align_center"><math id="S5.T6.20.20.20.2.m1.1" class="ltx_Math" alttext="\underline{50.7}_{1.1}~{}(5.2)" display="inline"><semantics id="S5.T6.20.20.20.2.m1.1a"><mrow id="S5.T6.20.20.20.2.m1.1.2" xref="S5.T6.20.20.20.2.m1.1.2.cmml"><msub id="S5.T6.20.20.20.2.m1.1.2.2" xref="S5.T6.20.20.20.2.m1.1.2.2.cmml"><munder accentunder="true" id="S5.T6.20.20.20.2.m1.1.2.2.2" xref="S5.T6.20.20.20.2.m1.1.2.2.2.cmml"><mn id="S5.T6.20.20.20.2.m1.1.2.2.2.2" xref="S5.T6.20.20.20.2.m1.1.2.2.2.2.cmml">50.7</mn><mo id="S5.T6.20.20.20.2.m1.1.2.2.2.1" xref="S5.T6.20.20.20.2.m1.1.2.2.2.1.cmml">¯</mo></munder><mn id="S5.T6.20.20.20.2.m1.1.2.2.3" xref="S5.T6.20.20.20.2.m1.1.2.2.3.cmml">1.1</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.20.20.20.2.m1.1.2.1" xref="S5.T6.20.20.20.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.20.20.20.2.m1.1.2.3.2" xref="S5.T6.20.20.20.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.20.20.20.2.m1.1.2.3.2.1" xref="S5.T6.20.20.20.2.m1.1.2.cmml">(</mo><mn id="S5.T6.20.20.20.2.m1.1.1" xref="S5.T6.20.20.20.2.m1.1.1.cmml">5.2</mn><mo stretchy="false" id="S5.T6.20.20.20.2.m1.1.2.3.2.2" xref="S5.T6.20.20.20.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.20.20.20.2.m1.1b"><apply id="S5.T6.20.20.20.2.m1.1.2.cmml" xref="S5.T6.20.20.20.2.m1.1.2"><times id="S5.T6.20.20.20.2.m1.1.2.1.cmml" xref="S5.T6.20.20.20.2.m1.1.2.1"></times><apply id="S5.T6.20.20.20.2.m1.1.2.2.cmml" xref="S5.T6.20.20.20.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.20.20.20.2.m1.1.2.2.1.cmml" xref="S5.T6.20.20.20.2.m1.1.2.2">subscript</csymbol><apply id="S5.T6.20.20.20.2.m1.1.2.2.2.cmml" xref="S5.T6.20.20.20.2.m1.1.2.2.2"><ci id="S5.T6.20.20.20.2.m1.1.2.2.2.1.cmml" xref="S5.T6.20.20.20.2.m1.1.2.2.2.1">¯</ci><cn type="float" id="S5.T6.20.20.20.2.m1.1.2.2.2.2.cmml" xref="S5.T6.20.20.20.2.m1.1.2.2.2.2">50.7</cn></apply><cn type="float" id="S5.T6.20.20.20.2.m1.1.2.2.3.cmml" xref="S5.T6.20.20.20.2.m1.1.2.2.3">1.1</cn></apply><cn type="float" id="S5.T6.20.20.20.2.m1.1.1.cmml" xref="S5.T6.20.20.20.2.m1.1.1">5.2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.20.20.20.2.m1.1c">\underline{50.7}_{1.1}~{}(5.2)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.22.22.22" class="ltx_tr">
<td id="S5.T6.22.22.22.3" class="ltx_td ltx_align_center">FFT</td>
<td id="S5.T6.22.22.22.4" class="ltx_td ltx_align_center ltx_border_r">LoTA (ours)</td>
<td id="S5.T6.21.21.21.1" class="ltx_td ltx_align_center"><math id="S5.T6.21.21.21.1.m1.1" class="ltx_Math" alttext="\underline{10.7}_{0.7}~{}(8.3)" display="inline"><semantics id="S5.T6.21.21.21.1.m1.1a"><mrow id="S5.T6.21.21.21.1.m1.1.2" xref="S5.T6.21.21.21.1.m1.1.2.cmml"><msub id="S5.T6.21.21.21.1.m1.1.2.2" xref="S5.T6.21.21.21.1.m1.1.2.2.cmml"><munder accentunder="true" id="S5.T6.21.21.21.1.m1.1.2.2.2" xref="S5.T6.21.21.21.1.m1.1.2.2.2.cmml"><mn id="S5.T6.21.21.21.1.m1.1.2.2.2.2" xref="S5.T6.21.21.21.1.m1.1.2.2.2.2.cmml">10.7</mn><mo id="S5.T6.21.21.21.1.m1.1.2.2.2.1" xref="S5.T6.21.21.21.1.m1.1.2.2.2.1.cmml">¯</mo></munder><mn id="S5.T6.21.21.21.1.m1.1.2.2.3" xref="S5.T6.21.21.21.1.m1.1.2.2.3.cmml">0.7</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.21.21.21.1.m1.1.2.1" xref="S5.T6.21.21.21.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.21.21.21.1.m1.1.2.3.2" xref="S5.T6.21.21.21.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.21.21.21.1.m1.1.2.3.2.1" xref="S5.T6.21.21.21.1.m1.1.2.cmml">(</mo><mn id="S5.T6.21.21.21.1.m1.1.1" xref="S5.T6.21.21.21.1.m1.1.1.cmml">8.3</mn><mo stretchy="false" id="S5.T6.21.21.21.1.m1.1.2.3.2.2" xref="S5.T6.21.21.21.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.21.21.21.1.m1.1b"><apply id="S5.T6.21.21.21.1.m1.1.2.cmml" xref="S5.T6.21.21.21.1.m1.1.2"><times id="S5.T6.21.21.21.1.m1.1.2.1.cmml" xref="S5.T6.21.21.21.1.m1.1.2.1"></times><apply id="S5.T6.21.21.21.1.m1.1.2.2.cmml" xref="S5.T6.21.21.21.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.21.21.21.1.m1.1.2.2.1.cmml" xref="S5.T6.21.21.21.1.m1.1.2.2">subscript</csymbol><apply id="S5.T6.21.21.21.1.m1.1.2.2.2.cmml" xref="S5.T6.21.21.21.1.m1.1.2.2.2"><ci id="S5.T6.21.21.21.1.m1.1.2.2.2.1.cmml" xref="S5.T6.21.21.21.1.m1.1.2.2.2.1">¯</ci><cn type="float" id="S5.T6.21.21.21.1.m1.1.2.2.2.2.cmml" xref="S5.T6.21.21.21.1.m1.1.2.2.2.2">10.7</cn></apply><cn type="float" id="S5.T6.21.21.21.1.m1.1.2.2.3.cmml" xref="S5.T6.21.21.21.1.m1.1.2.2.3">0.7</cn></apply><cn type="float" id="S5.T6.21.21.21.1.m1.1.1.cmml" xref="S5.T6.21.21.21.1.m1.1.1">8.3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.21.21.21.1.m1.1c">\underline{10.7}_{0.7}~{}(8.3)</annotation></semantics></math></td>
<td id="S5.T6.22.22.22.2" class="ltx_td ltx_align_center"><math id="S5.T6.22.22.22.2.m1.1" class="ltx_Math" alttext="\textbf{55.0}_{1.1}~{}(0.9)" display="inline"><semantics id="S5.T6.22.22.22.2.m1.1a"><mrow id="S5.T6.22.22.22.2.m1.1.2" xref="S5.T6.22.22.22.2.m1.1.2.cmml"><msub id="S5.T6.22.22.22.2.m1.1.2.2" xref="S5.T6.22.22.22.2.m1.1.2.2.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T6.22.22.22.2.m1.1.2.2.2" xref="S5.T6.22.22.22.2.m1.1.2.2.2a.cmml">55.0</mtext><mn id="S5.T6.22.22.22.2.m1.1.2.2.3" xref="S5.T6.22.22.22.2.m1.1.2.2.3.cmml">1.1</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.22.22.22.2.m1.1.2.1" xref="S5.T6.22.22.22.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.22.22.22.2.m1.1.2.3.2" xref="S5.T6.22.22.22.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.22.22.22.2.m1.1.2.3.2.1" xref="S5.T6.22.22.22.2.m1.1.2.cmml">(</mo><mn id="S5.T6.22.22.22.2.m1.1.1" xref="S5.T6.22.22.22.2.m1.1.1.cmml">0.9</mn><mo stretchy="false" id="S5.T6.22.22.22.2.m1.1.2.3.2.2" xref="S5.T6.22.22.22.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.22.22.22.2.m1.1b"><apply id="S5.T6.22.22.22.2.m1.1.2.cmml" xref="S5.T6.22.22.22.2.m1.1.2"><times id="S5.T6.22.22.22.2.m1.1.2.1.cmml" xref="S5.T6.22.22.22.2.m1.1.2.1"></times><apply id="S5.T6.22.22.22.2.m1.1.2.2.cmml" xref="S5.T6.22.22.22.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.22.22.22.2.m1.1.2.2.1.cmml" xref="S5.T6.22.22.22.2.m1.1.2.2">subscript</csymbol><ci id="S5.T6.22.22.22.2.m1.1.2.2.2a.cmml" xref="S5.T6.22.22.22.2.m1.1.2.2.2"><mtext class="ltx_mathvariant_bold" id="S5.T6.22.22.22.2.m1.1.2.2.2.cmml" xref="S5.T6.22.22.22.2.m1.1.2.2.2">55.0</mtext></ci><cn type="float" id="S5.T6.22.22.22.2.m1.1.2.2.3.cmml" xref="S5.T6.22.22.22.2.m1.1.2.2.3">1.1</cn></apply><cn type="float" id="S5.T6.22.22.22.2.m1.1.1.cmml" xref="S5.T6.22.22.22.2.m1.1.1">0.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.22.22.22.2.m1.1c">\textbf{55.0}_{1.1}~{}(0.9)</annotation></semantics></math></td>
</tr>
<tr id="S5.T6.24.24.24" class="ltx_tr">
<td id="S5.T6.24.24.24.3" class="ltx_td ltx_border_bb"></td>
<td id="S5.T6.24.24.24.4" class="ltx_td ltx_align_center ltx_border_bb">LoTA (ours)</td>
<td id="S5.T6.24.24.24.5" class="ltx_td ltx_align_center ltx_border_bb ltx_border_r">LoTA (ours)</td>
<td id="S5.T6.23.23.23.1" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T6.23.23.23.1.m1.1" class="ltx_Math" alttext="\textbf{13.1}_{0.8}~{}(5.9)" display="inline"><semantics id="S5.T6.23.23.23.1.m1.1a"><mrow id="S5.T6.23.23.23.1.m1.1.2" xref="S5.T6.23.23.23.1.m1.1.2.cmml"><msub id="S5.T6.23.23.23.1.m1.1.2.2" xref="S5.T6.23.23.23.1.m1.1.2.2.cmml"><mtext class="ltx_mathvariant_bold" id="S5.T6.23.23.23.1.m1.1.2.2.2" xref="S5.T6.23.23.23.1.m1.1.2.2.2a.cmml">13.1</mtext><mn id="S5.T6.23.23.23.1.m1.1.2.2.3" xref="S5.T6.23.23.23.1.m1.1.2.2.3.cmml">0.8</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.23.23.23.1.m1.1.2.1" xref="S5.T6.23.23.23.1.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.23.23.23.1.m1.1.2.3.2" xref="S5.T6.23.23.23.1.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.23.23.23.1.m1.1.2.3.2.1" xref="S5.T6.23.23.23.1.m1.1.2.cmml">(</mo><mn id="S5.T6.23.23.23.1.m1.1.1" xref="S5.T6.23.23.23.1.m1.1.1.cmml">5.9</mn><mo stretchy="false" id="S5.T6.23.23.23.1.m1.1.2.3.2.2" xref="S5.T6.23.23.23.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.23.23.23.1.m1.1b"><apply id="S5.T6.23.23.23.1.m1.1.2.cmml" xref="S5.T6.23.23.23.1.m1.1.2"><times id="S5.T6.23.23.23.1.m1.1.2.1.cmml" xref="S5.T6.23.23.23.1.m1.1.2.1"></times><apply id="S5.T6.23.23.23.1.m1.1.2.2.cmml" xref="S5.T6.23.23.23.1.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.23.23.23.1.m1.1.2.2.1.cmml" xref="S5.T6.23.23.23.1.m1.1.2.2">subscript</csymbol><ci id="S5.T6.23.23.23.1.m1.1.2.2.2a.cmml" xref="S5.T6.23.23.23.1.m1.1.2.2.2"><mtext class="ltx_mathvariant_bold" id="S5.T6.23.23.23.1.m1.1.2.2.2.cmml" xref="S5.T6.23.23.23.1.m1.1.2.2.2">13.1</mtext></ci><cn type="float" id="S5.T6.23.23.23.1.m1.1.2.2.3.cmml" xref="S5.T6.23.23.23.1.m1.1.2.2.3">0.8</cn></apply><cn type="float" id="S5.T6.23.23.23.1.m1.1.1.cmml" xref="S5.T6.23.23.23.1.m1.1.1">5.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.23.23.23.1.m1.1c">\textbf{13.1}_{0.8}~{}(5.9)</annotation></semantics></math></td>
<td id="S5.T6.24.24.24.2" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T6.24.24.24.2.m1.1" class="ltx_Math" alttext="46.6_{1.0}~{}(9.3)" display="inline"><semantics id="S5.T6.24.24.24.2.m1.1a"><mrow id="S5.T6.24.24.24.2.m1.1.2" xref="S5.T6.24.24.24.2.m1.1.2.cmml"><msub id="S5.T6.24.24.24.2.m1.1.2.2" xref="S5.T6.24.24.24.2.m1.1.2.2.cmml"><mn id="S5.T6.24.24.24.2.m1.1.2.2.2" xref="S5.T6.24.24.24.2.m1.1.2.2.2.cmml">46.6</mn><mn id="S5.T6.24.24.24.2.m1.1.2.2.3" xref="S5.T6.24.24.24.2.m1.1.2.2.3.cmml">1.0</mn></msub><mo lspace="0.330em" rspace="0em" id="S5.T6.24.24.24.2.m1.1.2.1" xref="S5.T6.24.24.24.2.m1.1.2.1.cmml">​</mo><mrow id="S5.T6.24.24.24.2.m1.1.2.3.2" xref="S5.T6.24.24.24.2.m1.1.2.cmml"><mo stretchy="false" id="S5.T6.24.24.24.2.m1.1.2.3.2.1" xref="S5.T6.24.24.24.2.m1.1.2.cmml">(</mo><mn id="S5.T6.24.24.24.2.m1.1.1" xref="S5.T6.24.24.24.2.m1.1.1.cmml">9.3</mn><mo stretchy="false" id="S5.T6.24.24.24.2.m1.1.2.3.2.2" xref="S5.T6.24.24.24.2.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.T6.24.24.24.2.m1.1b"><apply id="S5.T6.24.24.24.2.m1.1.2.cmml" xref="S5.T6.24.24.24.2.m1.1.2"><times id="S5.T6.24.24.24.2.m1.1.2.1.cmml" xref="S5.T6.24.24.24.2.m1.1.2.1"></times><apply id="S5.T6.24.24.24.2.m1.1.2.2.cmml" xref="S5.T6.24.24.24.2.m1.1.2.2"><csymbol cd="ambiguous" id="S5.T6.24.24.24.2.m1.1.2.2.1.cmml" xref="S5.T6.24.24.24.2.m1.1.2.2">subscript</csymbol><cn type="float" id="S5.T6.24.24.24.2.m1.1.2.2.2.cmml" xref="S5.T6.24.24.24.2.m1.1.2.2.2">46.6</cn><cn type="float" id="S5.T6.24.24.24.2.m1.1.2.2.3.cmml" xref="S5.T6.24.24.24.2.m1.1.2.2.3">1.0</cn></apply><cn type="float" id="S5.T6.24.24.24.2.m1.1.1.cmml" xref="S5.T6.24.24.24.2.m1.1.1">9.3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T6.24.24.24.2.m1.1c">46.6_{1.0}~{}(9.3)</annotation></semantics></math></td>
</tr>
</tbody>
</table>
</span></div>
</figure>
<div id="S5.SS4.p4" class="ltx_para">
<p id="S5.SS4.p4.1" class="ltx_p">In <a href="#S5.T6" title="In 5.4 LoTA for Model Merging ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">6</span></a> we merge together models trained on GSM8k and Samsum with the Mistral model that we trained for instruction following. We consider the full combination of merging FFT and LoTA models. The merge of two FFT models performs poorly in all settings on both tasks, indicating that post-hoc sparsification does not perform well for heterogeneous tasks, which is in line with recent model merging theory <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib66" title="" class="ltx_ref">66</a>]</cite>. The merges that contain LoTA models have better performance across all tasks, but there is no combination that is pareto-optimal across all tasks.</p>
</div>
</section>
</section>
<section id="S6" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">6 </span>Related Work</h2>

<section id="S6.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Model Pruning &amp; Quantization.</h4>

<div id="S6.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px1.p1.1" class="ltx_p">Model pruning and quantization have been receiving increased attention for efficient storage and/or inference of large models. While most of the existing methods prune <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib67" title="" class="ltx_ref">67</a>, <a href="#bib.bib68" title="" class="ltx_ref">68</a>, <a href="#bib.bib69" title="" class="ltx_ref">69</a>]</cite> or quantize <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib70" title="" class="ltx_ref">70</a>, <a href="#bib.bib71" title="" class="ltx_ref">71</a>, <a href="#bib.bib72" title="" class="ltx_ref">72</a>, <a href="#bib.bib73" title="" class="ltx_ref">73</a>]</cite> the model weight directly, some focus specifically on compressing the <em id="S6.SS0.SSS0.Px1.p1.1.1" class="ltx_emph ltx_font_italic">task vectors</em> <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib42" title="" class="ltx_ref">42</a>, <a href="#bib.bib74" title="" class="ltx_ref">74</a>, <a href="#bib.bib75" title="" class="ltx_ref">75</a>]</cite> through post-training sparsification or quantization, assuming that the base model is worth the storage cost since it is being used frequently for many tasks. Our proposed PEFT method, LoTA, builds on this observation that the task vectors are highly compressible through sparsification and imposes this sparsity constraint at the beginning of fine-tuning to train <em id="S6.SS0.SSS0.Px1.p1.1.2" class="ltx_emph ltx_font_italic">sparse task vectors</em>.</p>
</div>
</section>
<section id="S6.SS0.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Lottery Ticket Hypothesis.</h4>

<div id="S6.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px2.p1.1" class="ltx_p">Motivated by the success of pruning methods at extreme sparsity ratios <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib76" title="" class="ltx_ref">76</a>]</cite>, <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib77" title="" class="ltx_ref">77</a>]</cite> proposed the lottery ticket hypothesis (LTH), claiming the existence of sparse subnetworks (or lottery tickets) that could be trained from scratch to a performance comparably to training the dense model from scratch. While this could potentially provide a way to train models sparsely more efficiently rather than training them densely and pruning them later, finding the lottery tickets, i.e., the sparsity masks, is costly. Initially, <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib77" title="" class="ltx_ref">77</a>]</cite> proposed first training the models densely and then extracting the sparsity mask based on the magnitude of the trained dense model’s weights. Later, a number of more efficient methods were proposed to find the sparsity masks more efficiently, earlier in the dense training stage <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib78" title="" class="ltx_ref">78</a>, <a href="#bib.bib79" title="" class="ltx_ref">79</a>, <a href="#bib.bib80" title="" class="ltx_ref">80</a>, <a href="#bib.bib81" title="" class="ltx_ref">81</a>]</cite>. Our work shows that LTH works successfully for fine-tuning LLMs as well–giving us a sparse adaptation tool, LoTA. We extensively study the tradeoff between the cost of finding the sparsity masks and the performance of the sparsely fine-tuned model. Unlike other studies on LTH for LLM adaptation <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib82" title="" class="ltx_ref">82</a>, <a href="#bib.bib83" title="" class="ltx_ref">83</a>]</cite>, our main focus and motivation is to mitigate destructive interference in multi-task adaptation.</p>
</div>
</section>
<section id="S6.SS0.SSS0.Px3" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Parameter-Efficient Fine-Tuning (PEFT).</h4>

<div id="S6.SS0.SSS0.Px3.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px3.p1.1" class="ltx_p">Many practitioners fine-tune already pre-trained LLMs with less data and compute instead of training them from scratch <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib84" title="" class="ltx_ref">84</a>, <a href="#bib.bib85" title="" class="ltx_ref">85</a>, <a href="#bib.bib5" title="" class="ltx_ref">5</a>]</cite>. While this reduces the cost of LLM training significantly, fine-tuning each and every parameter of these large models for each (or a few) task is still very costly. This has led to a number of parameter-efficient fine-tuning (PEFT) methods reducing the number of trainable parameters during fine-tuning <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib25" title="" class="ltx_ref">25</a>, <a href="#bib.bib26" title="" class="ltx_ref">26</a>, <a href="#bib.bib27" title="" class="ltx_ref">27</a>, <a href="#bib.bib28" title="" class="ltx_ref">28</a>, <a href="#bib.bib86" title="" class="ltx_ref">86</a>, <a href="#bib.bib87" title="" class="ltx_ref">87</a>, <a href="#bib.bib34" title="" class="ltx_ref">34</a>, <a href="#bib.bib88" title="" class="ltx_ref">88</a>]</cite>. Among different PEFT methods, low-rank adaptation (LoRA) <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib6" title="" class="ltx_ref">6</a>]</cite> and its variants <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib89" title="" class="ltx_ref">89</a>, <a href="#bib.bib90" title="" class="ltx_ref">90</a>, <a href="#bib.bib91" title="" class="ltx_ref">91</a>, <a href="#bib.bib92" title="" class="ltx_ref">92</a>]</cite> have shown similar performance to full fine-tuning in many tasks while reducing the number of trainable parameters through low-rank approximation to model updates during fine-tuning. Our PEFT method, LoTA, while reducing the number of trainable parameters significantly via sparsity, has various other benefits in different applications, such as avoiding catastrophic forgetting (of especially safety alignment), enabling fine-tuning on new tasks more successfully, model merging using sparse task vectors, unlearning, and communication-efficient federated learning (FL). We demonstrate that full fine-tuning and the existing PEFT methods fall short in these applications and significantly underperform LoTA.</p>
</div>
</section>
<section id="S6.SS0.SSS0.Px4" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Catastrophic Forgetting.</h4>

<div id="S6.SS0.SSS0.Px4.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px4.p1.1" class="ltx_p">When LLMs go through sequential (or continual) multitask learning, i.e., fine-tuned on different tasks sequentially, they often suffer from performance loss on earlier tasks–known as catastrophic forgetting <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib9" title="" class="ltx_ref">9</a>, <a href="#bib.bib10" title="" class="ltx_ref">10</a>, <a href="#bib.bib11" title="" class="ltx_ref">11</a>, <a href="#bib.bib12" title="" class="ltx_ref">12</a>, <a href="#bib.bib13" title="" class="ltx_ref">13</a>]</cite>. To mitigate this, a number of data-centric and architectural solutions have been proposed for language and other domains. Replay-based methods <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib93" title="" class="ltx_ref">93</a>, <a href="#bib.bib94" title="" class="ltx_ref">94</a>]</cite> add a portion of the previously learned data during fine-tuning on a new task, which raises privacy concerns as it requires constant access to previously learned data. Regularization-based approaches <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib95" title="" class="ltx_ref">95</a>, <a href="#bib.bib96" title="" class="ltx_ref">96</a>]</cite> tend to have poor adaptability to specific tasks. An architecture-based approach, “progressive prompts” <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib97" title="" class="ltx_ref">97</a>]</cite>, sequentially concatenates soft prompts as they are being learned for each task–showing some resistance against forgetting. However, they require access to task identifiers at inference for each task, which is not always feasible. Other architecture-based approaches add additional modules to learn task-specific abilities <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib98" title="" class="ltx_ref">98</a>, <a href="#bib.bib99" title="" class="ltx_ref">99</a>]</cite>–requiring customized deployment due to architecture change. Closest to our work, <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib100" title="" class="ltx_ref">100</a>]</cite> updates a randomly selected subset of the parameters at each iteration of fine-tuning to preserve the earlier tasks in the not-updated parameters of that iteration. Despite similarities, our work LoTA (1) uses a fixed sparsity mask throughout fine-tuning instead of a new mask at every iteration, which yields sparse task vectors that are useful for other applications such as model merging and communication-efficient FL, and (2) finds data-dependent masks rather than the randomly selected masks in <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib100" title="" class="ltx_ref">100</a>]</cite>. Furthermore, unlike <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib100" title="" class="ltx_ref">100</a>]</cite>, LoTA not only preserves the earlier tasks on frozen parameters but also constraints the new tasks on a highly sparse subnetwork–providing resistance to catastrophic forgetting even when malicious users attempt to overwrite the earlier tasks via FFT.</p>
</div>
</section>
<section id="S6.SS0.SSS0.Px5" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Model Merging.</h4>

<div id="S6.SS0.SSS0.Px5.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px5.p1.1" class="ltx_p">Merging multiple task-specific models into a single model with multitask abilities <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib17" title="" class="ltx_ref">17</a>, <a href="#bib.bib18" title="" class="ltx_ref">18</a>, <a href="#bib.bib19" title="" class="ltx_ref">19</a>]</cite> has been an appealing alternative to sequential multitask learning, which suffers from catastrophic forgetting and could be inefficient, especially if task vectors are already available. The existing model merging methods include averaging weights of task-specific models <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib17" title="" class="ltx_ref">17</a>]</cite>, task arithmetic through combining task vectors <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib8" title="" class="ltx_ref">8</a>]</cite>, weighted aggregation of parameters <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib40" title="" class="ltx_ref">40</a>, <a href="#bib.bib41" title="" class="ltx_ref">41</a>]</cite>, combining task vectors after some post-processing such as trimming low-magnitude deltas <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib20" title="" class="ltx_ref">20</a>]</cite> or sparsifying the deltas <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib38" title="" class="ltx_ref">38</a>, <a href="#bib.bib39" title="" class="ltx_ref">39</a>]</cite>. Our method, LoTA, directly learns <em id="S6.SS0.SSS0.Px5.p1.1.1" class="ltx_emph ltx_font_italic">sparse task vectors</em>, obviating the need to post-process the task vectors, and outperforms existing model merging methods. Most importantly, LoTA enables merging task vectors trained on heterogeneous datasets, while the other model merging methods are often limited to similar datasets. This advancement is an important step towards scalable FL <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib101" title="" class="ltx_ref">101</a>]</cite> with LLMs as it enables merging <em id="S6.SS0.SSS0.Px5.p1.1.2" class="ltx_emph ltx_font_italic">sparse task vectors</em>, which brings communication efficiency, trained over heterogeneous datasets (of each edge device).</p>
</div>
<div id="S6.SS0.SSS0.Px5.p2" class="ltx_para">
<p id="S6.SS0.SSS0.Px5.p2.1" class="ltx_p">We note that we test model merging with LoTA specifically for highly dissimilar datasets to show its compatibility with FL, which considers edge devices with heterogeneous datasets. When used in FL, LoTA can reduce communication and memory costs significantly, which is a main bottleneck when scaling FL to large models. The successful use of LoTA for model merging and arithmetic further shows its promise for unlearning <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib8" title="" class="ltx_ref">8</a>]</cite> as well.</p>
</div>
</section>
</section>
<section id="S7" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">7 </span>Discussion</h2>

<div id="S7.p1" class="ltx_para">
<p id="S7.p1.1" class="ltx_p">We propose Lottery Ticket Adaptation (LoTA), a sparse alignment framework that fine-tunes only a sparse subnetwork of the base model, leaving the rest of the parameters frozen. LoTA successfully mitigates destructive interference (a problem with existing fine-tuning methods including full fine-tuning and low-rank adaptation (LoRA)) in many multi-task adaptation paradigms, prevents catastrophic forgetting of earlier tasks, including safety, and allows for successful model merging of even dramatically different tasks.</p>
</div>
<section id="S7.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Limitations.</h4>

<div id="S7.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="S7.SS0.SSS0.Px1.p1.1" class="ltx_p">As mentioned in Section <a href="#S5.SS1" title="5.1 LoTA Design Choices ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5.1</span></a>, LoTA does not provide the compute efficiency of LoRA. If the adapter needs to be compressed by more than <math id="S7.SS0.SSS0.Px1.p1.1.m1.1" class="ltx_math_unparsed" alttext="100\times" display="inline"><semantics id="S7.SS0.SSS0.Px1.p1.1.m1.1a"><mrow id="S7.SS0.SSS0.Px1.p1.1.m1.1b"><mn id="S7.SS0.SSS0.Px1.p1.1.m1.1.1">100</mn><mo lspace="0.222em" id="S7.SS0.SSS0.Px1.p1.1.m1.1.2">×</mo></mrow><annotation encoding="application/x-tex" id="S7.SS0.SSS0.Px1.p1.1.m1.1c">100\times</annotation></semantics></math>, LoTA may not provide sufficient compression. We evaluate on instruction following, reasoning, math, SQL generation, and summarization, yet even more tasks exist such as Python code generation, classification, or long-context question answering. We compare LoTA to baselines (LoRA, TIES) but other PEFT and merging methods exist. In a future revision of this paper, we plan to provide comparisons to a broader range of PEFT <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib31" title="" class="ltx_ref">31</a>]</cite> and merging <cite class="ltx_cite ltx_citemacro_citep">[<a href="#bib.bib38" title="" class="ltx_ref">38</a>]</cite> methods.</p>
</div>
</section>
<section id="S7.SS0.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Broader Impact.</h4>

<div id="S7.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="S7.SS0.SSS0.Px2.p1.1" class="ltx_p">LoTA emerges as a simple and efficient adaptation method to prevent destructive interference. We hope LoTA can enable compression of task adapters without losing performance, mitigate catastrophic forgetting (particularly to improve the durability of safety alignment in future model releases), and provide performant model merging across heterogeneous tasks.</p>
</div>
</section>
<section id="S7.SS0.SSS0.Px3" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Acknowledgements.</h4>

<div id="S7.SS0.SSS0.Px3.p1" class="ltx_para">
<p id="S7.SS0.SSS0.Px3.p1.1" class="ltx_p">This work was funded in part by the OpenAI Superalignment Grant and the Google PhD Fellowship.</p>
</div>
</section>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Panda et al. [2024]</span>
<span class="ltx_bibblock">
Ashwinee Panda, Berivan Isik, Xiangyu Qi, Sanmi Koyejo, Tsachy Weissman, and Prateek Mittal.

</span>
<span class="ltx_bibblock">Lottery ticket adaptation: Mitigating destructive interference in LLMs.

</span>
<span class="ltx_bibblock">In <em id="bib.bib1.1.1" class="ltx_emph ltx_font_italic">2nd Workshop on Advancing Neural Network Training: Computational Efficiency, Scalability, and Resource Optimization (WANT@ICML 2024)</em>, 2024.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=qD2eFNvtw4" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=qD2eFNvtw4</a>.

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Brown et al. [2020]</span>
<span class="ltx_bibblock">
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei.

</span>
<span class="ltx_bibblock">Language models are few-shot learners.

</span>
<span class="ltx_bibblock">In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, <em id="bib.bib2.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, volume 33, pages 1877–1901. Curran Associates, Inc., 2020.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf</a>.

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">OpenAI [2023]</span>
<span class="ltx_bibblock">
OpenAI.

</span>
<span class="ltx_bibblock">Gpt-4 technical report, 2023.

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Team et al. [2023]</span>
<span class="ltx_bibblock">
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al.

</span>
<span class="ltx_bibblock">Gemini: a family of highly capable multimodal models.

</span>
<span class="ltx_bibblock"><em id="bib.bib4.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2312.11805</em>, 2023.

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ouyang et al. [2022]</span>
<span class="ltx_bibblock">
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.

</span>
<span class="ltx_bibblock">Training language models to follow instructions with human feedback.

</span>
<span class="ltx_bibblock"><em id="bib.bib5.1.1" class="ltx_emph ltx_font_italic">Advances in neural information processing systems</em>, 35:27730–27744, 2022.

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hu et al. [2022]</span>
<span class="ltx_bibblock">
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.

</span>
<span class="ltx_bibblock">LoRA: Low-rank adaptation of large language models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib6.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>, 2022.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=nZeVKeeFYf9" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=nZeVKeeFYf9</a>.

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Beck et al. [2021]</span>
<span class="ltx_bibblock">
Tilman Beck, Bela Bohlender, Christina Viehmann, Vincent Hane, Yanik Adamson, Jaber Khuri, Jonas Brossmann, Jonas Pfeiffer, and Iryna Gurevych.

</span>
<span class="ltx_bibblock">Adapterhub playground: Simple and flexible few-shot learning with adapters.

</span>
<span class="ltx_bibblock"><em id="bib.bib7.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2108.08103</em>, 2021.

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ilharco et al. [2022]</span>
<span class="ltx_bibblock">
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi.

</span>
<span class="ltx_bibblock">Editing models with task arithmetic.

</span>
<span class="ltx_bibblock"><em id="bib.bib8.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2212.04089</em>, 2022.

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">McCloskey and Cohen [1989]</span>
<span class="ltx_bibblock">
Michael McCloskey and Neal J Cohen.

</span>
<span class="ltx_bibblock">Catastrophic interference in connectionist networks: The sequential learning problem.

</span>
<span class="ltx_bibblock">In <em id="bib.bib9.1.1" class="ltx_emph ltx_font_italic">Psychology of learning and motivation</em>, volume 24, pages 109–165. Elsevier, 1989.

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dong et al. [2023]</span>
<span class="ltx_bibblock">
Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou.

</span>
<span class="ltx_bibblock">How abilities in large language models are affected by supervised fine-tuning data composition.

</span>
<span class="ltx_bibblock"><em id="bib.bib10.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2310.05492</em>, 2023.

</span>
</li>
<li id="bib.bib11" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ramasesh et al. [2022]</span>
<span class="ltx_bibblock">
Vinay Venkatesh Ramasesh, Aitor Lewkowycz, and Ethan Dyer.

</span>
<span class="ltx_bibblock">Effect of scale on catastrophic forgetting in neural networks.

</span>
<span class="ltx_bibblock">In <em id="bib.bib11.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>, 2022.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=GhVS8_yPeEa" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=GhVS8_yPeEa</a>.

</span>
</li>
<li id="bib.bib12" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Luo et al. [2023]</span>
<span class="ltx_bibblock">
Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang.

</span>
<span class="ltx_bibblock">An empirical study of catastrophic forgetting in large language models during continual fine-tuning.

</span>
<span class="ltx_bibblock"><em id="bib.bib12.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2308.08747</em>, 2023.

</span>
</li>
<li id="bib.bib13" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. [2024]</span>
<span class="ltx_bibblock">
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu.

</span>
<span class="ltx_bibblock">A comprehensive survey of continual learning: Theory, method and application.

</span>
<span class="ltx_bibblock"><em id="bib.bib13.1.1" class="ltx_emph ltx_font_italic">IEEE Transactions on Pattern Analysis and Machine Intelligence</em>, 2024.

</span>
</li>
<li id="bib.bib14" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Crawshaw [2020]</span>
<span class="ltx_bibblock">
Michael Crawshaw.

</span>
<span class="ltx_bibblock">Multi-task learning with deep neural networks: A survey.

</span>
<span class="ltx_bibblock"><em id="bib.bib14.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2009.09796</em>, 2020.

</span>
</li>
<li id="bib.bib15" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhang and Yang [2021]</span>
<span class="ltx_bibblock">
Yu Zhang and Qiang Yang.

</span>
<span class="ltx_bibblock">A survey on multi-task learning.

</span>
<span class="ltx_bibblock"><em id="bib.bib15.1.1" class="ltx_emph ltx_font_italic">IEEE Transactions on Knowledge and Data Engineering</em>, 34(12):5586–5609, 2021.

</span>
</li>
<li id="bib.bib16" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dubois et al. [2024]</span>
<span class="ltx_bibblock">
Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto.

</span>
<span class="ltx_bibblock">Length-controlled alpacaeval: A simple way to debias automatic evaluators.

</span>
<span class="ltx_bibblock"><em id="bib.bib16.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2404.04475</em>, 2024.

</span>
</li>
<li id="bib.bib17" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wortsman et al. [2022]</span>
<span class="ltx_bibblock">
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al.

</span>
<span class="ltx_bibblock">Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time.

</span>
<span class="ltx_bibblock">In <em id="bib.bib17.1.1" class="ltx_emph ltx_font_italic">International conference on machine learning</em>, pages 23965–23998. PMLR, 2022.

</span>
</li>
<li id="bib.bib18" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Jin et al. [2022]</span>
<span class="ltx_bibblock">
Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng.

</span>
<span class="ltx_bibblock">Dataless knowledge fusion by merging weights of language models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib18.1.1" class="ltx_emph ltx_font_italic">The Eleventh International Conference on Learning Representations</em>, 2022.

</span>
</li>
<li id="bib.bib19" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhang et al. [2023a]</span>
<span class="ltx_bibblock">
Jinghan Zhang, Junteng Liu, Junxian He, et al.

</span>
<span class="ltx_bibblock">Composing parameter-efficient modules with arithmetic operation.

</span>
<span class="ltx_bibblock"><em id="bib.bib19.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 36:12589–12610, 2023a.

</span>
</li>
<li id="bib.bib20" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yadav et al. [2023]</span>
<span class="ltx_bibblock">
Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal.

</span>
<span class="ltx_bibblock">TIES-merging: Resolving interference when merging models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib20.1.1" class="ltx_emph ltx_font_italic">Thirty-seventh Conference on Neural Information Processing Systems</em>, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=xtaX3WyCj1" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=xtaX3WyCj1</a>.

</span>
</li>
<li id="bib.bib21" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ostapenko et al. [2024]</span>
<span class="ltx_bibblock">
Oleksiy Ostapenko, Zhan Su, Edoardo Maria Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, and Alessandro Sordoni.

</span>
<span class="ltx_bibblock">Towards modular llms by building and reusing a library of loras, 2024.

</span>
</li>
<li id="bib.bib22" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Mangrulkar et al. [2022]</span>
<span class="ltx_bibblock">
Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan.

</span>
<span class="ltx_bibblock">Peft: State-of-the-art parameter-efficient fine-tuning methods.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://github.com/huggingface/peft" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/huggingface/peft</a>, 2022.

</span>
</li>
<li id="bib.bib23" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Houlsby et al. [2019]</span>
<span class="ltx_bibblock">
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly.

</span>
<span class="ltx_bibblock">Parameter-efficient transfer learning for nlp.

</span>
<span class="ltx_bibblock">In <em id="bib.bib23.1.1" class="ltx_emph ltx_font_italic">International conference on machine learning</em>, pages 2790–2799. PMLR, 2019.

</span>
</li>
<li id="bib.bib24" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Apple [2024]</span>
<span class="ltx_bibblock">
Apple.

</span>
<span class="ltx_bibblock">Apple adapters, 2024.

</span>
<span class="ltx_bibblock">https://machinelearning.apple.com/research/introducing-apple-foundation-models.

</span>
</li>
<li id="bib.bib25" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhang et al. [2023b]</span>
<span class="ltx_bibblock">
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao.

</span>
<span class="ltx_bibblock">Adaptive budget allocation for parameter-efficient fine-tuning.

</span>
<span class="ltx_bibblock">In <em id="bib.bib25.1.1" class="ltx_emph ltx_font_italic">The Eleventh International Conference on Learning Representations</em>, 2023b.

</span>
</li>
<li id="bib.bib26" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li and Liang [2021]</span>
<span class="ltx_bibblock">
Xiang Lisa Li and Percy Liang.

</span>
<span class="ltx_bibblock">Prefix-tuning: Optimizing continuous prompts for generation.

</span>
<span class="ltx_bibblock">In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, <em id="bib.bib26.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)</em>, pages 4582–4597, Online, August 2021. Association for Computational Linguistics.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.18653/v1/2021.acl-long.353</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://aclanthology.org/2021.acl-long.353" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://aclanthology.org/2021.acl-long.353</a>.

</span>
</li>
<li id="bib.bib27" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Liu et al. [2022a]</span>
<span class="ltx_bibblock">
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang.

</span>
<span class="ltx_bibblock">P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks.

</span>
<span class="ltx_bibblock">In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, <em id="bib.bib27.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)</em>, pages 61–68, Dublin, Ireland, May 2022a. Association for Computational Linguistics.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.18653/v1/2022.acl-short.8</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://aclanthology.org/2022.acl-short.8" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://aclanthology.org/2022.acl-short.8</a>.

</span>
</li>
<li id="bib.bib28" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Lester et al. [2021]</span>
<span class="ltx_bibblock">
Brian Lester, Rami Al-Rfou, and Noah Constant.

</span>
<span class="ltx_bibblock">The power of scale for parameter-efficient prompt tuning.

</span>
<span class="ltx_bibblock">In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, <em id="bib.bib28.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing</em>, pages 3045–3059, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.18653/v1/2021.emnlp-main.243</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://aclanthology.org/2021.emnlp-main.243" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://aclanthology.org/2021.emnlp-main.243</a>.

</span>
</li>
<li id="bib.bib29" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. [2023]</span>
<span class="ltx_bibblock">
Lequn Chen, Zihao Ye, Yongji Wu, Danyang Zhuo, Luis Ceze, and Arvind Krishnamurthy.

</span>
<span class="ltx_bibblock">Punica: Multi-tenant lora serving, 2023.

</span>
</li>
<li id="bib.bib30" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Sheng et al. [2023]</span>
<span class="ltx_bibblock">
Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, Joseph E. Gonzalez, and Ion Stoica.

</span>
<span class="ltx_bibblock">S-lora: Serving thousands of concurrent lora adapters, 2023.

</span>
</li>
<li id="bib.bib31" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Liu et al. [2024a]</span>
<span class="ltx_bibblock">
Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen.

</span>
<span class="ltx_bibblock">Dora: Weight-decomposed low-rank adaptation.

</span>
<span class="ltx_bibblock"><em id="bib.bib31.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2402.09353</em>, 2024a.

</span>
</li>
<li id="bib.bib32" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kopiczko et al. [2023]</span>
<span class="ltx_bibblock">
Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki Markus Asano.

</span>
<span class="ltx_bibblock">Vera: Vector-based random matrix adaptation.

</span>
<span class="ltx_bibblock"><em id="bib.bib32.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2310.11454</em>, 2023.

</span>
</li>
<li id="bib.bib33" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Biderman et al. [2024]</span>
<span class="ltx_bibblock">
Dan Biderman, Jose Gonzalez Ortiz, Jacob Portes, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John P. Cunningham.

</span>
<span class="ltx_bibblock">Lora learns less and forgets less, 2024.

</span>
</li>
<li id="bib.bib34" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Nikdan et al. [2024]</span>
<span class="ltx_bibblock">
Mahdi Nikdan, Soroush Tabesh, and Dan Alistarh.

</span>
<span class="ltx_bibblock">Rosa: Accurate parameter-efficient fine-tuning via robust adaptation.

</span>
<span class="ltx_bibblock"><em id="bib.bib34.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2401.04679</em>, 2024.

</span>
</li>
<li id="bib.bib35" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ruder [2017]</span>
<span class="ltx_bibblock">
Sebastian Ruder.

</span>
<span class="ltx_bibblock">An overview of multi-task learning in deep neural networks.

</span>
<span class="ltx_bibblock"><em id="bib.bib35.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1706.05098</em>, 2017.

</span>
</li>
<li id="bib.bib36" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Çağatay Yıldız et al. [2024]</span>
<span class="ltx_bibblock">
Çağatay Yıldız, Nishaanth Kanna Ravichandran, Prishruit Punia, Matthias Bethge, and Beyza Ermis.

</span>
<span class="ltx_bibblock">Investigating continual pretraining in large language models: Insights and implications, 2024.

</span>
</li>
<li id="bib.bib37" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Lermen et al. [2023]</span>
<span class="ltx_bibblock">
Simon Lermen, Charlie Rogers-Smith, and Jeffrey Ladish.

</span>
<span class="ltx_bibblock">Lora fine-tuning efficiently undoes safety training in llama 2-chat 70b.

</span>
<span class="ltx_bibblock"><em id="bib.bib37.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2310.20624</em>, 2023.

</span>
</li>
<li id="bib.bib38" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yu et al. [2023]</span>
<span class="ltx_bibblock">
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li.

</span>
<span class="ltx_bibblock">Language models are super mario: Absorbing abilities from homologous models as a free lunch.

</span>
<span class="ltx_bibblock"><em id="bib.bib38.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2311.03099</em>, 2023.

</span>
</li>
<li id="bib.bib39" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Davari and Belilovsky [2023]</span>
<span class="ltx_bibblock">
MohammadReza Davari and Eugene Belilovsky.

</span>
<span class="ltx_bibblock">Model breadcrumbs: Scaling multi-task model merging with sparse masks.

</span>
<span class="ltx_bibblock"><em id="bib.bib39.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2312.06795</em>, 2023.

</span>
</li>
<li id="bib.bib40" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Matena and Raffel [2022]</span>
<span class="ltx_bibblock">
Michael S Matena and Colin A Raffel.

</span>
<span class="ltx_bibblock">Merging models with fisher-weighted averaging.

</span>
<span class="ltx_bibblock"><em id="bib.bib40.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 35:17703–17716, 2022.

</span>
</li>
<li id="bib.bib41" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Xiao et al. [2023a]</span>
<span class="ltx_bibblock">
Shitao Xiao, Zheng Liu, Peitian Zhang, and Xingrun Xing.

</span>
<span class="ltx_bibblock">Lm-cocktail: Resilient tuning of language models via model merging.

</span>
<span class="ltx_bibblock"><em id="bib.bib41.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2311.13534</em>, 2023a.

</span>
</li>
<li id="bib.bib42" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Isik et al. [2023]</span>
<span class="ltx_bibblock">
Berivan Isik, Hermann Kumbong, Wanyi Ning, Xiaozhe Yao, Sanmi Koyejo, and Ce Zhang.

</span>
<span class="ltx_bibblock">Gpt-zip: Deep compression of finetuned large language models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib42.1.1" class="ltx_emph ltx_font_italic">Workshop on Efficient Systems for Foundation Models@ ICML2023</em>, 2023.

</span>
</li>
<li id="bib.bib43" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Jiang et al. [2023]</span>
<span class="ltx_bibblock">
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al.

</span>
<span class="ltx_bibblock">Mistral 7b.

</span>
<span class="ltx_bibblock"><em id="bib.bib43.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2310.06825</em>, 2023.

</span>
</li>
<li id="bib.bib44" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">AI@Meta [2024]</span>
<span class="ltx_bibblock">
AI@Meta.

</span>
<span class="ltx_bibblock">Llama 3 model card.

</span>
<span class="ltx_bibblock">2024.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md</a>.

</span>
</li>
<li id="bib.bib45" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Touvron et al. [2023]</span>
<span class="ltx_bibblock">
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas
Scialom.

</span>
<span class="ltx_bibblock">Llama 2: Open foundation and fine-tuned chat models, 2023.

</span>
</li>
<li id="bib.bib46" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Cui et al. [2023]</span>
<span class="ltx_bibblock">
Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun.

</span>
<span class="ltx_bibblock">Ultrafeedback: Boosting language models with high-quality feedback, 2023.

</span>
</li>
<li id="bib.bib47" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. [2023]</span>
<span class="ltx_bibblock">
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto.

</span>
<span class="ltx_bibblock">Alpacaeval: An automatic evaluator of instruction-following models.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://github.com/tatsu-lab/alpaca_eval" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/tatsu-lab/alpaca_eval</a>, 2023.

</span>
</li>
<li id="bib.bib48" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zheng et al. [2023]</span>
<span class="ltx_bibblock">
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica.

</span>
<span class="ltx_bibblock">Judging llm-as-a-judge with mt-bench and chatbot arena, 2023.

</span>
</li>
<li id="bib.bib49" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yu et al. [2024]</span>
<span class="ltx_bibblock">
Zhuohao Yu, Chang Gao, Wenjin Yao, Yidong Wang, Wei Ye, Jindong Wang, Xing Xie, Yue Zhang, and Shikun Zhang.

</span>
<span class="ltx_bibblock">Kieval: A knowledge-grounded interactive evaluation framework for large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib49.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2402.15043</em>, 2024.

</span>
</li>
<li id="bib.bib50" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Christopher et al. [2019]</span>
<span class="ltx_bibblock">
Clark Christopher, Lee Kenton, Chang Ming-Wei, Kwiatkowski Tom, Collins Michael, and Toutanova Kristina.

</span>
<span class="ltx_bibblock">Boolq: Exploring the surprising difficulty of natural yes/no questions.

</span>
<span class="ltx_bibblock">In <em id="bib.bib50.1.1" class="ltx_emph ltx_font_italic">NAACL</em>, 2019.

</span>
</li>
<li id="bib.bib51" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bisk et al. [2019]</span>
<span class="ltx_bibblock">
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi.

</span>
<span class="ltx_bibblock">Piqa: Reasoning about physical commonsense in natural language, 2019.

</span>
</li>
<li id="bib.bib52" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Sap et al. [2019]</span>
<span class="ltx_bibblock">
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi.

</span>
<span class="ltx_bibblock">Socialiqa: Commonsense reasoning about social interactions, 2019.

</span>
</li>
<li id="bib.bib53" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zellers et al. [2019]</span>
<span class="ltx_bibblock">
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi.

</span>
<span class="ltx_bibblock">Hellaswag: Can a machine really finish your sentence?

</span>
<span class="ltx_bibblock">In <em id="bib.bib53.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</em>, 2019.

</span>
</li>
<li id="bib.bib54" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">ai2 [2019]</span>
<span class="ltx_bibblock">
Winogrande: An adversarial winograd schema challenge at scale.

</span>
<span class="ltx_bibblock">2019.

</span>
</li>
<li id="bib.bib55" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Clark et al. [2018]</span>
<span class="ltx_bibblock">
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord.

</span>
<span class="ltx_bibblock">Think you have solved question answering? try arc, the ai2 reasoning challenge.

</span>
<span class="ltx_bibblock"><em id="bib.bib55.1.1" class="ltx_emph ltx_font_italic">arXiv:1803.05457v1</em>, 2018.

</span>
</li>
<li id="bib.bib56" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Mihaylov et al. [2018]</span>
<span class="ltx_bibblock">
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal.

</span>
<span class="ltx_bibblock">Can a suit of armor conduct electricity? a new dataset for open book question answering.

</span>
<span class="ltx_bibblock">In <em id="bib.bib56.1.1" class="ltx_emph ltx_font_italic">EMNLP</em>, 2018.

</span>
</li>
<li id="bib.bib57" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yue et al. [2023]</span>
<span class="ltx_bibblock">
Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, and Wenhu Chen Yu Su.

</span>
<span class="ltx_bibblock">Mammoth: Building math generalist models through hybrid instruction tuning.

</span>
<span class="ltx_bibblock"><em id="bib.bib57.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2309.05653</em>, 2023.

</span>
</li>
<li id="bib.bib58" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Cobbe et al. [2021]</span>
<span class="ltx_bibblock">
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman.

</span>
<span class="ltx_bibblock">Training verifiers to solve math word problems.

</span>
<span class="ltx_bibblock"><em id="bib.bib58.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2110.14168</em>, 2021.

</span>
</li>
<li id="bib.bib59" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">b mc2 [2023]</span>
<span class="ltx_bibblock">
b mc2.

</span>
<span class="ltx_bibblock">sql-create-context dataset, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://huggingface.co/datasets/b-mc2/sql-create-context" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://huggingface.co/datasets/b-mc2/sql-create-context</a>.

</span>
<span class="ltx_bibblock">This dataset was created by modifying data from the following sources: <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib102" title="" class="ltx_ref">102</a>, <a href="#bib.bib103" title="" class="ltx_ref">103</a>]</cite>.

</span>
</li>
<li id="bib.bib60" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Lin [2004]</span>
<span class="ltx_bibblock">
Chin-Yew Lin.

</span>
<span class="ltx_bibblock">Rouge: A package for automatic evaluation of summaries.

</span>
<span class="ltx_bibblock">In <em id="bib.bib60.1.1" class="ltx_emph ltx_font_italic">Text summarization branches out</em>, pages 74–81, 2004.

</span>
</li>
<li id="bib.bib61" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gliwa et al. [2019]</span>
<span class="ltx_bibblock">
Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer.

</span>
<span class="ltx_bibblock">SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization.

</span>
<span class="ltx_bibblock">In <em id="bib.bib61.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2nd Workshop on New Frontiers in Summarization</em>, pages 70–79, Hong Kong, China, November 2019. Association for Computational Linguistics.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.18653/v1/D19-5409</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://www.aclweb.org/anthology/D19-5409" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://www.aclweb.org/anthology/D19-5409</a>.

</span>
</li>
<li id="bib.bib62" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Qi et al. [2023]</span>
<span class="ltx_bibblock">
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson.

</span>
<span class="ltx_bibblock">Fine-tuning aligned language models compromises safety, even when users do not intend to!, 2023.

</span>
</li>
<li id="bib.bib63" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhan et al. [2024]</span>
<span class="ltx_bibblock">
Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori Hashimoto, and Daniel Kang.

</span>
<span class="ltx_bibblock">Removing rlhf protections in gpt-4 via fine-tuning, 2024.

</span>
</li>
<li id="bib.bib64" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Scott Weiner [2024]</span>
<span class="ltx_bibblock">
Scott Weiner.

</span>
<span class="ltx_bibblock">California sb 1047, 2024.

</span>
<span class="ltx_bibblock">
<br class="ltx_break"><a target="_blank" href="https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=202320240SB1047" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=202320240SB1047</a>.

</span>
</li>
<li id="bib.bib65" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ghosh et al. [2024]</span>
<span class="ltx_bibblock">
Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, Dinesh Manocha, et al.

</span>
<span class="ltx_bibblock">A closer look at the limitations of instruction tuning.

</span>
<span class="ltx_bibblock"><em id="bib.bib65.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2402.05119</em>, 2024.

</span>
</li>
<li id="bib.bib66" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Daheim et al. [2024]</span>
<span class="ltx_bibblock">
Nico Daheim, Thomas Möllenhoff, Edoardo Ponti, Iryna Gurevych, and Mohammad Emtiyaz Khan.

</span>
<span class="ltx_bibblock">Model merging by uncertainty-based gradient matching.

</span>
<span class="ltx_bibblock">In <em id="bib.bib66.1.1" class="ltx_emph ltx_font_italic">The Twelfth International Conference on Learning Representations</em>, 2024.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=D7KJmfEDQP" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=D7KJmfEDQP</a>.

</span>
</li>
<li id="bib.bib67" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Frantar and Alistarh [2023]</span>
<span class="ltx_bibblock">
Elias Frantar and Dan Alistarh.

</span>
<span class="ltx_bibblock">Sparsegpt: Massive language models can be accurately pruned in one-shot.

</span>
<span class="ltx_bibblock">In <em id="bib.bib67.1.1" class="ltx_emph ltx_font_italic">International Conference on Machine Learning</em>, pages 10323–10337. PMLR, 2023.

</span>
</li>
<li id="bib.bib68" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dettmers et al. [2023a]</span>
<span class="ltx_bibblock">
Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh.

</span>
<span class="ltx_bibblock">Spqr: A sparse-quantized representation for near-lossless llm weight compression.

</span>
<span class="ltx_bibblock"><em id="bib.bib68.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2306.03078</em>, 2023a.

</span>
</li>
<li id="bib.bib69" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kim et al. [2023]</span>
<span class="ltx_bibblock">
Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W Mahoney, and Kurt Keutzer.

</span>
<span class="ltx_bibblock">Squeezellm: Dense-and-sparse quantization.

</span>
<span class="ltx_bibblock"><em id="bib.bib69.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2306.07629</em>, 2023.

</span>
</li>
<li id="bib.bib70" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Frantar et al. [2023]</span>
<span class="ltx_bibblock">
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh.

</span>
<span class="ltx_bibblock">OPTQ: Accurate quantization for generative pre-trained transformers.

</span>
<span class="ltx_bibblock">In <em id="bib.bib70.1.1" class="ltx_emph ltx_font_italic">The Eleventh International Conference on Learning Representations</em>, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=tcbBPnfwxS" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=tcbBPnfwxS</a>.

</span>
</li>
<li id="bib.bib71" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dettmers et al. [2022]</span>
<span class="ltx_bibblock">
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer.

</span>
<span class="ltx_bibblock">Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale.

</span>
<span class="ltx_bibblock"><em id="bib.bib71.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 35:30318–30332, 2022.

</span>
</li>
<li id="bib.bib72" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Xiao et al. [2023b]</span>
<span class="ltx_bibblock">
Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han.

</span>
<span class="ltx_bibblock">Smoothquant: Accurate and efficient post-training quantization for large language models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib72.1.1" class="ltx_emph ltx_font_italic">International Conference on Machine Learning</em>, pages 38087–38099. PMLR, 2023b.

</span>
</li>
<li id="bib.bib73" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Lin et al. [2023]</span>
<span class="ltx_bibblock">
Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han.

</span>
<span class="ltx_bibblock">Awq: Activation-aware weight quantization for llm compression and acceleration.

</span>
<span class="ltx_bibblock"><em id="bib.bib73.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2306.00978</em>, 2023.

</span>
</li>
<li id="bib.bib74" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Liu et al. [2024b]</span>
<span class="ltx_bibblock">
James Liu, Guangxuan Xiao, Kai Li, Jason D Lee, Song Han, Tri Dao, and Tianle Cai.

</span>
<span class="ltx_bibblock">Bitdelta: Your fine-tune may only be worth one bit.

</span>
<span class="ltx_bibblock"><em id="bib.bib74.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2402.10193</em>, 2024b.

</span>
</li>
<li id="bib.bib75" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yao and Klimovic [2023]</span>
<span class="ltx_bibblock">
Xiaozhe Yao and Ana Klimovic.

</span>
<span class="ltx_bibblock">Deltazip: Multi-tenant language model serving via delta compression.

</span>
<span class="ltx_bibblock"><em id="bib.bib75.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2312.05215</em>, 2023.

</span>
</li>
<li id="bib.bib76" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Han et al. [2015]</span>
<span class="ltx_bibblock">
Song Han, Huizi Mao, and William J Dally.

</span>
<span class="ltx_bibblock">Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.

</span>
<span class="ltx_bibblock"><em id="bib.bib76.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1510.00149</em>, 2015.

</span>
</li>
<li id="bib.bib77" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Frankle and Carbin [2019]</span>
<span class="ltx_bibblock">
Jonathan Frankle and Michael Carbin.

</span>
<span class="ltx_bibblock">The lottery ticket hypothesis: Finding sparse, trainable neural networks.

</span>
<span class="ltx_bibblock">In <em id="bib.bib77.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>, 2019.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=rJl-b3RcF7" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=rJl-b3RcF7</a>.

</span>
</li>
<li id="bib.bib78" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Frankle et al. [2021]</span>
<span class="ltx_bibblock">
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin.

</span>
<span class="ltx_bibblock">Pruning neural networks at initialization: Why are we missing the mark?

</span>
<span class="ltx_bibblock">In <em id="bib.bib78.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>, 2021.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=Ig-VyQc-MLK" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=Ig-VyQc-MLK</a>.

</span>
</li>
<li id="bib.bib79" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Lee et al. [2019]</span>
<span class="ltx_bibblock">
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip Torr.

</span>
<span class="ltx_bibblock">SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY.

</span>
<span class="ltx_bibblock">In <em id="bib.bib79.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>, 2019.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=B1VZqjAcYX" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=B1VZqjAcYX</a>.

</span>
</li>
<li id="bib.bib80" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. [2020]</span>
<span class="ltx_bibblock">
Chaoqi Wang, Guodong Zhang, and Roger Grosse.

</span>
<span class="ltx_bibblock">Picking winning tickets before training by preserving gradient flow.

</span>
<span class="ltx_bibblock">In <em id="bib.bib80.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>, 2020.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=SkgsACVKPH" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=SkgsACVKPH</a>.

</span>
</li>
<li id="bib.bib81" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Tanaka et al. [2020]</span>
<span class="ltx_bibblock">
Hidenori Tanaka, Daniel Kunin, Daniel L Yamins, and Surya Ganguli.

</span>
<span class="ltx_bibblock">Pruning neural networks without any data by iteratively conserving synaptic flow.

</span>
<span class="ltx_bibblock"><em id="bib.bib81.1.1" class="ltx_emph ltx_font_italic">Advances in neural information processing systems</em>, 33:6377–6389, 2020.

</span>
</li>
<li id="bib.bib82" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yuan et al. [2024]</span>
<span class="ltx_bibblock">
Fei Yuan, Chang Ma, Shuai Yuan, Qiushi Sun, and Lei Li.

</span>
<span class="ltx_bibblock">Ks-lottery: Finding certified lottery tickets for multilingual language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib82.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2402.02801</em>, 2024.

</span>
</li>
<li id="bib.bib83" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Xu and Zhang [2024]</span>
<span class="ltx_bibblock">
Jing Xu and Jingzhao Zhang.

</span>
<span class="ltx_bibblock">Random masking finds winning tickets for parameter efficient fine-tuning.

</span>
<span class="ltx_bibblock"><em id="bib.bib83.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2405.02596</em>, 2024.

</span>
</li>
<li id="bib.bib84" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Liu et al. [2022b]</span>
<span class="ltx_bibblock">
Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel.

</span>
<span class="ltx_bibblock">Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning.

</span>
<span class="ltx_bibblock"><em id="bib.bib84.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 35:1950–1965, 2022b.

</span>
</li>
<li id="bib.bib85" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. [2022]</span>
<span class="ltx_bibblock">
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A, Sumanta Patro, Tanay Dixit, and Xudong Shen.

</span>
<span class="ltx_bibblock">Super-NaturalInstructions: Generalization via declarative instructions on 1600+ NLP tasks.

</span>
<span class="ltx_bibblock">In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, <em id="bib.bib85.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing</em>, pages 5085–5109, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.18653/v1/2022.emnlp-main.340</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://aclanthology.org/2022.emnlp-main.340" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://aclanthology.org/2022.emnlp-main.340</a>.

</span>
</li>
<li id="bib.bib86" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Edalati et al. [2022]</span>
<span class="ltx_bibblock">
Ali Edalati, Marzieh Tahaei, Ivan Kobyzev, Vahid Partovi Nia, James J Clark, and Mehdi Rezagholizadeh.

</span>
<span class="ltx_bibblock">Krona: Parameter efficient tuning with kronecker adapter.

</span>
<span class="ltx_bibblock"><em id="bib.bib86.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2212.10650</em>, 2022.

</span>
</li>
<li id="bib.bib87" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hu et al. [2023]</span>
<span class="ltx_bibblock">
Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, and Roy Lee.

</span>
<span class="ltx_bibblock">LLM-adapters: An adapter family for parameter-efficient fine-tuning of large language models.

</span>
<span class="ltx_bibblock">In Houda Bouamor, Juan Pino, and Kalika Bali, editors, <em id="bib.bib87.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</em>, pages 5254–5276, Singapore, December 2023. Association for Computational Linguistics.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.18653/v1/2023.emnlp-main.319</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://aclanthology.org/2023.emnlp-main.319" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://aclanthology.org/2023.emnlp-main.319</a>.

</span>
</li>
<li id="bib.bib88" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Guo et al. [2020]</span>
<span class="ltx_bibblock">
Demi Guo, Alexander M Rush, and Yoon Kim.

</span>
<span class="ltx_bibblock">Parameter-efficient transfer learning with diff pruning.

</span>
<span class="ltx_bibblock"><em id="bib.bib88.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2012.07463</em>, 2020.

</span>
</li>
<li id="bib.bib89" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dettmers et al. [2023b]</span>
<span class="ltx_bibblock">
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer.

</span>
<span class="ltx_bibblock">Qlora: Efficient finetuning of quantized llms.

</span>
<span class="ltx_bibblock"><em id="bib.bib89.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 36, 2023b.

</span>
</li>
<li id="bib.bib90" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Guo et al. [2024]</span>
<span class="ltx_bibblock">
Han Guo, Philip Greengard, Eric Xing, and Yoon Kim.

</span>
<span class="ltx_bibblock">LQ-loRA: Low-rank plus quantized matrix decomposition for efficient language model finetuning.

</span>
<span class="ltx_bibblock">In <em id="bib.bib90.1.1" class="ltx_emph ltx_font_italic">The Twelfth International Conference on Learning Representations</em>, 2024.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=xw29VvOMmU" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=xw29VvOMmU</a>.

</span>
</li>
<li id="bib.bib91" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. [2024]</span>
<span class="ltx_bibblock">
Yixiao Li, Yifan Yu, Chen Liang, Nikos Karampatziakis, Pengcheng He, Weizhu Chen, and Tuo Zhao.

</span>
<span class="ltx_bibblock">Loftq: LoRA-fine-tuning-aware quantization for large language models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib91.1.1" class="ltx_emph ltx_font_italic">The Twelfth International Conference on Learning Representations</em>, 2024.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=LzPWWPAdY4" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=LzPWWPAdY4</a>.

</span>
</li>
<li id="bib.bib92" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kopiczko et al. [2024]</span>
<span class="ltx_bibblock">
Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki M Asano.

</span>
<span class="ltx_bibblock">VeRA: Vector-based random matrix adaptation.

</span>
<span class="ltx_bibblock">In <em id="bib.bib92.1.1" class="ltx_emph ltx_font_italic">The Twelfth International Conference on Learning Representations</em>, 2024.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=NjNfLdxr3A" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=NjNfLdxr3A</a>.

</span>
</li>
<li id="bib.bib93" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Rebuffi et al. [2017]</span>
<span class="ltx_bibblock">
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert.

</span>
<span class="ltx_bibblock">icarl: Incremental classifier and representation learning.

</span>
<span class="ltx_bibblock">In <em id="bib.bib93.1.1" class="ltx_emph ltx_font_italic">Proceedings of the IEEE conference on Computer Vision and Pattern Recognition</em>, pages 2001–2010, 2017.

</span>
</li>
<li id="bib.bib94" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Romanov et al. [2018]</span>
<span class="ltx_bibblock">
Alexey Romanov, Anna Rumshisky, Anna Rogers, and David Donahue.

</span>
<span class="ltx_bibblock">Adversarial decomposition of text representation.

</span>
<span class="ltx_bibblock"><em id="bib.bib94.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1808.09042</em>, 2018.

</span>
</li>
<li id="bib.bib95" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Huang et al. [2021]</span>
<span class="ltx_bibblock">
Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang, and Diyi Yang.

</span>
<span class="ltx_bibblock">Continual learning for text classification with information disentanglement based regularization.

</span>
<span class="ltx_bibblock">In <em id="bib.bib95.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies</em>, pages 2736–2746, 2021.

</span>
</li>
<li id="bib.bib96" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Aljundi et al. [2018]</span>
<span class="ltx_bibblock">
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars.

</span>
<span class="ltx_bibblock">Memory aware synapses: Learning what (not) to forget.

</span>
<span class="ltx_bibblock">In <em id="bib.bib96.1.1" class="ltx_emph ltx_font_italic">Proceedings of the European conference on computer vision (ECCV)</em>, pages 139–154, 2018.

</span>
</li>
<li id="bib.bib97" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Razdaibiedina et al. [2023]</span>
<span class="ltx_bibblock">
Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Almahairi.

</span>
<span class="ltx_bibblock">Progressive prompts: Continual learning for language models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib97.1.1" class="ltx_emph ltx_font_italic">The Eleventh International Conference on Learning Representations</em>, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=UJTgQBc91_" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=UJTgQBc91_</a>.

</span>
</li>
<li id="bib.bib98" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dou et al. [2023]</span>
<span class="ltx_bibblock">
Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Jun Zhao, Wei Shen, Yuhao Zhou, Zhiheng Xi, Xiao Wang, Xiaoran Fan, et al.

</span>
<span class="ltx_bibblock">Loramoe: Revolutionizing mixture of experts for maintaining world knowledge in language model alignment.

</span>
<span class="ltx_bibblock"><em id="bib.bib98.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2312.09979</em>, 2023.

</span>
</li>
<li id="bib.bib99" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wu et al. [2024]</span>
<span class="ltx_bibblock">
Chengyue Wu, Yukang Gan, Yixiao Ge, Zeyu Lu, Jiahao Wang, Ye Feng, Ping Luo, and Ying Shan.

</span>
<span class="ltx_bibblock">Llama pro: Progressive llama with block expansion.

</span>
<span class="ltx_bibblock"><em id="bib.bib99.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2401.02415</em>, 2024.

</span>
</li>
<li id="bib.bib100" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hui et al. [2024]</span>
<span class="ltx_bibblock">
Tingfeng Hui, Zhenyu Zhang, Shuohuan Wang, Weiran Xu, Yu Sun, and Hua Wu.

</span>
<span class="ltx_bibblock">Hft: Half fine-tuning for large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib100.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2404.18466</em>, 2024.

</span>
</li>
<li id="bib.bib101" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kairouz et al. [2021]</span>
<span class="ltx_bibblock">
Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D’Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao.

</span>
<span class="ltx_bibblock">Advances and open problems in federated learning, 2021.

</span>
</li>
<li id="bib.bib102" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhong et al. [2017]</span>
<span class="ltx_bibblock">
Victor Zhong, Caiming Xiong, and Richard Socher.

</span>
<span class="ltx_bibblock">Seq2sql: Generating structured queries from natural language using reinforcement learning.

</span>
<span class="ltx_bibblock"><em id="bib.bib102.1.1" class="ltx_emph ltx_font_italic">CoRR</em>, abs/1709.00103, 2017.

</span>
</li>
<li id="bib.bib103" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yu et al. [2018]</span>
<span class="ltx_bibblock">
Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al.

</span>
<span class="ltx_bibblock">Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task.

</span>
<span class="ltx_bibblock"><em id="bib.bib103.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1809.08887</em>, 2018.

</span>
</li>
<li id="bib.bib104" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Sharma et al. [2024]</span>
<span class="ltx_bibblock">
Archit Sharma, Sedrick Keh, Eric Mitchell, Chelsea Finn, Kushal Arora, and Thomas Kollar.

</span>
<span class="ltx_bibblock">A critical evaluation of ai feedback for aligning large language models, 2024.

</span>
</li>
</ul>
</section>
<div class="ltx_pagination ltx_role_newpage"></div>
<section id="A1" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix A </span>Additional Experimental Details</h2>

<section id="A1.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">A.1 </span>Code</h3>

<div id="A1.SS1.p1" class="ltx_para">
<p id="A1.SS1.p1.1" class="ltx_p">Because we evaluate multiple methods on a wide range of tasks, training on <math id="A1.SS1.p1.1.m1.1" class="ltx_Math" alttext="&gt;20" display="inline"><semantics id="A1.SS1.p1.1.m1.1a"><mrow id="A1.SS1.p1.1.m1.1.1" xref="A1.SS1.p1.1.m1.1.1.cmml"><mi id="A1.SS1.p1.1.m1.1.1.2" xref="A1.SS1.p1.1.m1.1.1.2.cmml"></mi><mo id="A1.SS1.p1.1.m1.1.1.1" xref="A1.SS1.p1.1.m1.1.1.1.cmml">&gt;</mo><mn id="A1.SS1.p1.1.m1.1.1.3" xref="A1.SS1.p1.1.m1.1.1.3.cmml">20</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.SS1.p1.1.m1.1b"><apply id="A1.SS1.p1.1.m1.1.1.cmml" xref="A1.SS1.p1.1.m1.1.1"><gt id="A1.SS1.p1.1.m1.1.1.1.cmml" xref="A1.SS1.p1.1.m1.1.1.1"></gt><csymbol cd="latexml" id="A1.SS1.p1.1.m1.1.1.2.cmml" xref="A1.SS1.p1.1.m1.1.1.2">absent</csymbol><cn type="integer" id="A1.SS1.p1.1.m1.1.1.3.cmml" xref="A1.SS1.p1.1.m1.1.1.3">20</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p1.1.m1.1c">&gt;20</annotation></semantics></math> datasets, we defer all the details on the prompts, exact dataset format, etc. to our <a target="_blank" href="https://github.com/kiddyboots216/lottery-ticket-adaptation" title="" class="ltx_ref ltx_href">open-source code repository</a>.</p>
</div>
</section>
<section id="A1.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">A.2 </span>Hyperparameter Ranges</h3>

<div id="A1.SS2.p1" class="ltx_para">
<p id="A1.SS2.p1.3" class="ltx_p"><span id="A1.SS2.p1.3.1" class="ltx_text ltx_font_bold">FFT.</span> We tune the learning rate in the range <math id="A1.SS2.p1.1.m1.2" class="ltx_Math" alttext="5e-7,1e-5" display="inline"><semantics id="A1.SS2.p1.1.m1.2a"><mrow id="A1.SS2.p1.1.m1.2.2.2" xref="A1.SS2.p1.1.m1.2.2.3.cmml"><mrow id="A1.SS2.p1.1.m1.1.1.1.1" xref="A1.SS2.p1.1.m1.1.1.1.1.cmml"><mrow id="A1.SS2.p1.1.m1.1.1.1.1.2" xref="A1.SS2.p1.1.m1.1.1.1.1.2.cmml"><mn id="A1.SS2.p1.1.m1.1.1.1.1.2.2" xref="A1.SS2.p1.1.m1.1.1.1.1.2.2.cmml">5</mn><mo lspace="0em" rspace="0em" id="A1.SS2.p1.1.m1.1.1.1.1.2.1" xref="A1.SS2.p1.1.m1.1.1.1.1.2.1.cmml">​</mo><mi id="A1.SS2.p1.1.m1.1.1.1.1.2.3" xref="A1.SS2.p1.1.m1.1.1.1.1.2.3.cmml">e</mi></mrow><mo id="A1.SS2.p1.1.m1.1.1.1.1.1" xref="A1.SS2.p1.1.m1.1.1.1.1.1.cmml">−</mo><mn id="A1.SS2.p1.1.m1.1.1.1.1.3" xref="A1.SS2.p1.1.m1.1.1.1.1.3.cmml">7</mn></mrow><mo id="A1.SS2.p1.1.m1.2.2.2.3" xref="A1.SS2.p1.1.m1.2.2.3.cmml">,</mo><mrow id="A1.SS2.p1.1.m1.2.2.2.2" xref="A1.SS2.p1.1.m1.2.2.2.2.cmml"><mrow id="A1.SS2.p1.1.m1.2.2.2.2.2" xref="A1.SS2.p1.1.m1.2.2.2.2.2.cmml"><mn id="A1.SS2.p1.1.m1.2.2.2.2.2.2" xref="A1.SS2.p1.1.m1.2.2.2.2.2.2.cmml">1</mn><mo lspace="0em" rspace="0em" id="A1.SS2.p1.1.m1.2.2.2.2.2.1" xref="A1.SS2.p1.1.m1.2.2.2.2.2.1.cmml">​</mo><mi id="A1.SS2.p1.1.m1.2.2.2.2.2.3" xref="A1.SS2.p1.1.m1.2.2.2.2.2.3.cmml">e</mi></mrow><mo id="A1.SS2.p1.1.m1.2.2.2.2.1" xref="A1.SS2.p1.1.m1.2.2.2.2.1.cmml">−</mo><mn id="A1.SS2.p1.1.m1.2.2.2.2.3" xref="A1.SS2.p1.1.m1.2.2.2.2.3.cmml">5</mn></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.p1.1.m1.2b"><list id="A1.SS2.p1.1.m1.2.2.3.cmml" xref="A1.SS2.p1.1.m1.2.2.2"><apply id="A1.SS2.p1.1.m1.1.1.1.1.cmml" xref="A1.SS2.p1.1.m1.1.1.1.1"><minus id="A1.SS2.p1.1.m1.1.1.1.1.1.cmml" xref="A1.SS2.p1.1.m1.1.1.1.1.1"></minus><apply id="A1.SS2.p1.1.m1.1.1.1.1.2.cmml" xref="A1.SS2.p1.1.m1.1.1.1.1.2"><times id="A1.SS2.p1.1.m1.1.1.1.1.2.1.cmml" xref="A1.SS2.p1.1.m1.1.1.1.1.2.1"></times><cn type="integer" id="A1.SS2.p1.1.m1.1.1.1.1.2.2.cmml" xref="A1.SS2.p1.1.m1.1.1.1.1.2.2">5</cn><ci id="A1.SS2.p1.1.m1.1.1.1.1.2.3.cmml" xref="A1.SS2.p1.1.m1.1.1.1.1.2.3">𝑒</ci></apply><cn type="integer" id="A1.SS2.p1.1.m1.1.1.1.1.3.cmml" xref="A1.SS2.p1.1.m1.1.1.1.1.3">7</cn></apply><apply id="A1.SS2.p1.1.m1.2.2.2.2.cmml" xref="A1.SS2.p1.1.m1.2.2.2.2"><minus id="A1.SS2.p1.1.m1.2.2.2.2.1.cmml" xref="A1.SS2.p1.1.m1.2.2.2.2.1"></minus><apply id="A1.SS2.p1.1.m1.2.2.2.2.2.cmml" xref="A1.SS2.p1.1.m1.2.2.2.2.2"><times id="A1.SS2.p1.1.m1.2.2.2.2.2.1.cmml" xref="A1.SS2.p1.1.m1.2.2.2.2.2.1"></times><cn type="integer" id="A1.SS2.p1.1.m1.2.2.2.2.2.2.cmml" xref="A1.SS2.p1.1.m1.2.2.2.2.2.2">1</cn><ci id="A1.SS2.p1.1.m1.2.2.2.2.2.3.cmml" xref="A1.SS2.p1.1.m1.2.2.2.2.2.3">𝑒</ci></apply><cn type="integer" id="A1.SS2.p1.1.m1.2.2.2.2.3.cmml" xref="A1.SS2.p1.1.m1.2.2.2.2.3">5</cn></apply></list></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p1.1.m1.2c">5e-7,1e-5</annotation></semantics></math> and the best-performing learning rate for the downstream finetuning tasks is <math id="A1.SS2.p1.2.m2.1" class="ltx_Math" alttext="1e-6" display="inline"><semantics id="A1.SS2.p1.2.m2.1a"><mrow id="A1.SS2.p1.2.m2.1.1" xref="A1.SS2.p1.2.m2.1.1.cmml"><mrow id="A1.SS2.p1.2.m2.1.1.2" xref="A1.SS2.p1.2.m2.1.1.2.cmml"><mn id="A1.SS2.p1.2.m2.1.1.2.2" xref="A1.SS2.p1.2.m2.1.1.2.2.cmml">1</mn><mo lspace="0em" rspace="0em" id="A1.SS2.p1.2.m2.1.1.2.1" xref="A1.SS2.p1.2.m2.1.1.2.1.cmml">​</mo><mi id="A1.SS2.p1.2.m2.1.1.2.3" xref="A1.SS2.p1.2.m2.1.1.2.3.cmml">e</mi></mrow><mo id="A1.SS2.p1.2.m2.1.1.1" xref="A1.SS2.p1.2.m2.1.1.1.cmml">−</mo><mn id="A1.SS2.p1.2.m2.1.1.3" xref="A1.SS2.p1.2.m2.1.1.3.cmml">6</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.p1.2.m2.1b"><apply id="A1.SS2.p1.2.m2.1.1.cmml" xref="A1.SS2.p1.2.m2.1.1"><minus id="A1.SS2.p1.2.m2.1.1.1.cmml" xref="A1.SS2.p1.2.m2.1.1.1"></minus><apply id="A1.SS2.p1.2.m2.1.1.2.cmml" xref="A1.SS2.p1.2.m2.1.1.2"><times id="A1.SS2.p1.2.m2.1.1.2.1.cmml" xref="A1.SS2.p1.2.m2.1.1.2.1"></times><cn type="integer" id="A1.SS2.p1.2.m2.1.1.2.2.cmml" xref="A1.SS2.p1.2.m2.1.1.2.2">1</cn><ci id="A1.SS2.p1.2.m2.1.1.2.3.cmml" xref="A1.SS2.p1.2.m2.1.1.2.3">𝑒</ci></apply><cn type="integer" id="A1.SS2.p1.2.m2.1.1.3.cmml" xref="A1.SS2.p1.2.m2.1.1.3">6</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p1.2.m2.1c">1e-6</annotation></semantics></math>, with the exception of instruction finetuning which we detail further below. We use a batch size of 32. We clip the gradient norm of each parameter group to <math id="A1.SS2.p1.3.m3.1" class="ltx_Math" alttext="1" display="inline"><semantics id="A1.SS2.p1.3.m3.1a"><mn id="A1.SS2.p1.3.m3.1.1" xref="A1.SS2.p1.3.m3.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="A1.SS2.p1.3.m3.1b"><cn type="integer" id="A1.SS2.p1.3.m3.1.1.cmml" xref="A1.SS2.p1.3.m3.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p1.3.m3.1c">1</annotation></semantics></math>. We clip the gradient norm of the parameters as opposed to the full gradient because we fuse the optimizer with the backward pass to enable FFT on a single A100.</p>
</div>
<div id="A1.SS2.p2" class="ltx_para">
<p id="A1.SS2.p2.1" class="ltx_p"><span id="A1.SS2.p2.1.1" class="ltx_text ltx_font_bold">LoTA.</span> We do not tune any hyperparameters for LoTA and merely use the same hyperparameters as FFT. Unless stated otherwise, we report the performance of LoTA with a sparsity ratio of <math id="A1.SS2.p2.1.m1.1" class="ltx_Math" alttext="0.9" display="inline"><semantics id="A1.SS2.p2.1.m1.1a"><mn id="A1.SS2.p2.1.m1.1.1" xref="A1.SS2.p2.1.m1.1.1.cmml">0.9</mn><annotation-xml encoding="MathML-Content" id="A1.SS2.p2.1.m1.1b"><cn type="float" id="A1.SS2.p2.1.m1.1.1.cmml" xref="A1.SS2.p2.1.m1.1.1">0.9</cn></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p2.1.m1.1c">0.9</annotation></semantics></math> that was calibrated on the entire dataset.</p>
</div>
<div id="A1.SS2.p3" class="ltx_para">
<p id="A1.SS2.p3.4" class="ltx_p"><span id="A1.SS2.p3.4.1" class="ltx_text ltx_font_bold">LoRA.</span> LoRA introduces the additional rank hyperparameter, which we tune <span id="A1.SS2.p3.4.2" class="ltx_text ltx_font_bold">jointly</span> with the “lora alpha” term and the learning rate. We tune the rank and alpha in the range <math id="A1.SS2.p3.1.m1.2" class="ltx_Math" alttext="(8,512)" display="inline"><semantics id="A1.SS2.p3.1.m1.2a"><mrow id="A1.SS2.p3.1.m1.2.3.2" xref="A1.SS2.p3.1.m1.2.3.1.cmml"><mo stretchy="false" id="A1.SS2.p3.1.m1.2.3.2.1" xref="A1.SS2.p3.1.m1.2.3.1.cmml">(</mo><mn id="A1.SS2.p3.1.m1.1.1" xref="A1.SS2.p3.1.m1.1.1.cmml">8</mn><mo id="A1.SS2.p3.1.m1.2.3.2.2" xref="A1.SS2.p3.1.m1.2.3.1.cmml">,</mo><mn id="A1.SS2.p3.1.m1.2.2" xref="A1.SS2.p3.1.m1.2.2.cmml">512</mn><mo stretchy="false" id="A1.SS2.p3.1.m1.2.3.2.3" xref="A1.SS2.p3.1.m1.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.p3.1.m1.2b"><interval closure="open" id="A1.SS2.p3.1.m1.2.3.1.cmml" xref="A1.SS2.p3.1.m1.2.3.2"><cn type="integer" id="A1.SS2.p3.1.m1.1.1.cmml" xref="A1.SS2.p3.1.m1.1.1">8</cn><cn type="integer" id="A1.SS2.p3.1.m1.2.2.cmml" xref="A1.SS2.p3.1.m1.2.2">512</cn></interval></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p3.1.m1.2c">(8,512)</annotation></semantics></math>. Common wisdom seems to dictate the use of larger learning rates for LoRA, so we expand the upper edge of the LoRA learning rate range to <math id="A1.SS2.p3.2.m2.1" class="ltx_Math" alttext="1e-4" display="inline"><semantics id="A1.SS2.p3.2.m2.1a"><mrow id="A1.SS2.p3.2.m2.1.1" xref="A1.SS2.p3.2.m2.1.1.cmml"><mrow id="A1.SS2.p3.2.m2.1.1.2" xref="A1.SS2.p3.2.m2.1.1.2.cmml"><mn id="A1.SS2.p3.2.m2.1.1.2.2" xref="A1.SS2.p3.2.m2.1.1.2.2.cmml">1</mn><mo lspace="0em" rspace="0em" id="A1.SS2.p3.2.m2.1.1.2.1" xref="A1.SS2.p3.2.m2.1.1.2.1.cmml">​</mo><mi id="A1.SS2.p3.2.m2.1.1.2.3" xref="A1.SS2.p3.2.m2.1.1.2.3.cmml">e</mi></mrow><mo id="A1.SS2.p3.2.m2.1.1.1" xref="A1.SS2.p3.2.m2.1.1.1.cmml">−</mo><mn id="A1.SS2.p3.2.m2.1.1.3" xref="A1.SS2.p3.2.m2.1.1.3.cmml">4</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.p3.2.m2.1b"><apply id="A1.SS2.p3.2.m2.1.1.cmml" xref="A1.SS2.p3.2.m2.1.1"><minus id="A1.SS2.p3.2.m2.1.1.1.cmml" xref="A1.SS2.p3.2.m2.1.1.1"></minus><apply id="A1.SS2.p3.2.m2.1.1.2.cmml" xref="A1.SS2.p3.2.m2.1.1.2"><times id="A1.SS2.p3.2.m2.1.1.2.1.cmml" xref="A1.SS2.p3.2.m2.1.1.2.1"></times><cn type="integer" id="A1.SS2.p3.2.m2.1.1.2.2.cmml" xref="A1.SS2.p3.2.m2.1.1.2.2">1</cn><ci id="A1.SS2.p3.2.m2.1.1.2.3.cmml" xref="A1.SS2.p3.2.m2.1.1.2.3">𝑒</ci></apply><cn type="integer" id="A1.SS2.p3.2.m2.1.1.3.cmml" xref="A1.SS2.p3.2.m2.1.1.3">4</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p3.2.m2.1c">1e-4</annotation></semantics></math> and indeed find that LoRA typically benefits from a larger learning rate. We use LoRA on all linear layers (q, k, v, gate, up, down). On the commonsense reasoning tasks, the best hyperparameters are: <math id="A1.SS2.p3.3.m3.2" class="ltx_Math" alttext="r=512,\alpha=32,\eta=1e-4" display="inline"><semantics id="A1.SS2.p3.3.m3.2a"><mrow id="A1.SS2.p3.3.m3.2.2.2" xref="A1.SS2.p3.3.m3.2.2.3.cmml"><mrow id="A1.SS2.p3.3.m3.1.1.1.1" xref="A1.SS2.p3.3.m3.1.1.1.1.cmml"><mi id="A1.SS2.p3.3.m3.1.1.1.1.2" xref="A1.SS2.p3.3.m3.1.1.1.1.2.cmml">r</mi><mo id="A1.SS2.p3.3.m3.1.1.1.1.1" xref="A1.SS2.p3.3.m3.1.1.1.1.1.cmml">=</mo><mn id="A1.SS2.p3.3.m3.1.1.1.1.3" xref="A1.SS2.p3.3.m3.1.1.1.1.3.cmml">512</mn></mrow><mo id="A1.SS2.p3.3.m3.2.2.2.3" xref="A1.SS2.p3.3.m3.2.2.3a.cmml">,</mo><mrow id="A1.SS2.p3.3.m3.2.2.2.2.2" xref="A1.SS2.p3.3.m3.2.2.2.2.3.cmml"><mrow id="A1.SS2.p3.3.m3.2.2.2.2.1.1" xref="A1.SS2.p3.3.m3.2.2.2.2.1.1.cmml"><mi id="A1.SS2.p3.3.m3.2.2.2.2.1.1.2" xref="A1.SS2.p3.3.m3.2.2.2.2.1.1.2.cmml">α</mi><mo id="A1.SS2.p3.3.m3.2.2.2.2.1.1.1" xref="A1.SS2.p3.3.m3.2.2.2.2.1.1.1.cmml">=</mo><mn id="A1.SS2.p3.3.m3.2.2.2.2.1.1.3" xref="A1.SS2.p3.3.m3.2.2.2.2.1.1.3.cmml">32</mn></mrow><mo id="A1.SS2.p3.3.m3.2.2.2.2.2.3" xref="A1.SS2.p3.3.m3.2.2.2.2.3a.cmml">,</mo><mrow id="A1.SS2.p3.3.m3.2.2.2.2.2.2" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.cmml"><mi id="A1.SS2.p3.3.m3.2.2.2.2.2.2.2" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.2.cmml">η</mi><mo id="A1.SS2.p3.3.m3.2.2.2.2.2.2.1" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.1.cmml">=</mo><mrow id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.cmml"><mrow id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.cmml"><mn id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.2" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.2.cmml">1</mn><mo lspace="0em" rspace="0em" id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.1" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.1.cmml">​</mo><mi id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.3" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.3.cmml">e</mi></mrow><mo id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.1" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.1.cmml">−</mo><mn id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.3" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.3.cmml">4</mn></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.p3.3.m3.2b"><apply id="A1.SS2.p3.3.m3.2.2.3.cmml" xref="A1.SS2.p3.3.m3.2.2.2"><csymbol cd="ambiguous" id="A1.SS2.p3.3.m3.2.2.3a.cmml" xref="A1.SS2.p3.3.m3.2.2.2.3">formulae-sequence</csymbol><apply id="A1.SS2.p3.3.m3.1.1.1.1.cmml" xref="A1.SS2.p3.3.m3.1.1.1.1"><eq id="A1.SS2.p3.3.m3.1.1.1.1.1.cmml" xref="A1.SS2.p3.3.m3.1.1.1.1.1"></eq><ci id="A1.SS2.p3.3.m3.1.1.1.1.2.cmml" xref="A1.SS2.p3.3.m3.1.1.1.1.2">𝑟</ci><cn type="integer" id="A1.SS2.p3.3.m3.1.1.1.1.3.cmml" xref="A1.SS2.p3.3.m3.1.1.1.1.3">512</cn></apply><apply id="A1.SS2.p3.3.m3.2.2.2.2.3.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2"><csymbol cd="ambiguous" id="A1.SS2.p3.3.m3.2.2.2.2.3a.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.3">formulae-sequence</csymbol><apply id="A1.SS2.p3.3.m3.2.2.2.2.1.1.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.1.1"><eq id="A1.SS2.p3.3.m3.2.2.2.2.1.1.1.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.1.1.1"></eq><ci id="A1.SS2.p3.3.m3.2.2.2.2.1.1.2.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.1.1.2">𝛼</ci><cn type="integer" id="A1.SS2.p3.3.m3.2.2.2.2.1.1.3.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.1.1.3">32</cn></apply><apply id="A1.SS2.p3.3.m3.2.2.2.2.2.2.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2"><eq id="A1.SS2.p3.3.m3.2.2.2.2.2.2.1.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.1"></eq><ci id="A1.SS2.p3.3.m3.2.2.2.2.2.2.2.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.2">𝜂</ci><apply id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3"><minus id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.1.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.1"></minus><apply id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2"><times id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.1.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.1"></times><cn type="integer" id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.2.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.2">1</cn><ci id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.3.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.2.3">𝑒</ci></apply><cn type="integer" id="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.3.cmml" xref="A1.SS2.p3.3.m3.2.2.2.2.2.2.3.3">4</cn></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p3.3.m3.2c">r=512,\alpha=32,\eta=1e-4</annotation></semantics></math>. On GSM8k, the best hyperparameters are: <math id="A1.SS2.p3.4.m4.2" class="ltx_Math" alttext="r=64,\alpha=32,\eta=1e-4" display="inline"><semantics id="A1.SS2.p3.4.m4.2a"><mrow id="A1.SS2.p3.4.m4.2.2.2" xref="A1.SS2.p3.4.m4.2.2.3.cmml"><mrow id="A1.SS2.p3.4.m4.1.1.1.1" xref="A1.SS2.p3.4.m4.1.1.1.1.cmml"><mi id="A1.SS2.p3.4.m4.1.1.1.1.2" xref="A1.SS2.p3.4.m4.1.1.1.1.2.cmml">r</mi><mo id="A1.SS2.p3.4.m4.1.1.1.1.1" xref="A1.SS2.p3.4.m4.1.1.1.1.1.cmml">=</mo><mn id="A1.SS2.p3.4.m4.1.1.1.1.3" xref="A1.SS2.p3.4.m4.1.1.1.1.3.cmml">64</mn></mrow><mo id="A1.SS2.p3.4.m4.2.2.2.3" xref="A1.SS2.p3.4.m4.2.2.3a.cmml">,</mo><mrow id="A1.SS2.p3.4.m4.2.2.2.2.2" xref="A1.SS2.p3.4.m4.2.2.2.2.3.cmml"><mrow id="A1.SS2.p3.4.m4.2.2.2.2.1.1" xref="A1.SS2.p3.4.m4.2.2.2.2.1.1.cmml"><mi id="A1.SS2.p3.4.m4.2.2.2.2.1.1.2" xref="A1.SS2.p3.4.m4.2.2.2.2.1.1.2.cmml">α</mi><mo id="A1.SS2.p3.4.m4.2.2.2.2.1.1.1" xref="A1.SS2.p3.4.m4.2.2.2.2.1.1.1.cmml">=</mo><mn id="A1.SS2.p3.4.m4.2.2.2.2.1.1.3" xref="A1.SS2.p3.4.m4.2.2.2.2.1.1.3.cmml">32</mn></mrow><mo id="A1.SS2.p3.4.m4.2.2.2.2.2.3" xref="A1.SS2.p3.4.m4.2.2.2.2.3a.cmml">,</mo><mrow id="A1.SS2.p3.4.m4.2.2.2.2.2.2" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.cmml"><mi id="A1.SS2.p3.4.m4.2.2.2.2.2.2.2" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.2.cmml">η</mi><mo id="A1.SS2.p3.4.m4.2.2.2.2.2.2.1" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.1.cmml">=</mo><mrow id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.cmml"><mrow id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.cmml"><mn id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.2" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.2.cmml">1</mn><mo lspace="0em" rspace="0em" id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.1" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.1.cmml">​</mo><mi id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.3" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.3.cmml">e</mi></mrow><mo id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.1" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.1.cmml">−</mo><mn id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.3" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.3.cmml">4</mn></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.p3.4.m4.2b"><apply id="A1.SS2.p3.4.m4.2.2.3.cmml" xref="A1.SS2.p3.4.m4.2.2.2"><csymbol cd="ambiguous" id="A1.SS2.p3.4.m4.2.2.3a.cmml" xref="A1.SS2.p3.4.m4.2.2.2.3">formulae-sequence</csymbol><apply id="A1.SS2.p3.4.m4.1.1.1.1.cmml" xref="A1.SS2.p3.4.m4.1.1.1.1"><eq id="A1.SS2.p3.4.m4.1.1.1.1.1.cmml" xref="A1.SS2.p3.4.m4.1.1.1.1.1"></eq><ci id="A1.SS2.p3.4.m4.1.1.1.1.2.cmml" xref="A1.SS2.p3.4.m4.1.1.1.1.2">𝑟</ci><cn type="integer" id="A1.SS2.p3.4.m4.1.1.1.1.3.cmml" xref="A1.SS2.p3.4.m4.1.1.1.1.3">64</cn></apply><apply id="A1.SS2.p3.4.m4.2.2.2.2.3.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2"><csymbol cd="ambiguous" id="A1.SS2.p3.4.m4.2.2.2.2.3a.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.3">formulae-sequence</csymbol><apply id="A1.SS2.p3.4.m4.2.2.2.2.1.1.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.1.1"><eq id="A1.SS2.p3.4.m4.2.2.2.2.1.1.1.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.1.1.1"></eq><ci id="A1.SS2.p3.4.m4.2.2.2.2.1.1.2.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.1.1.2">𝛼</ci><cn type="integer" id="A1.SS2.p3.4.m4.2.2.2.2.1.1.3.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.1.1.3">32</cn></apply><apply id="A1.SS2.p3.4.m4.2.2.2.2.2.2.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2"><eq id="A1.SS2.p3.4.m4.2.2.2.2.2.2.1.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.1"></eq><ci id="A1.SS2.p3.4.m4.2.2.2.2.2.2.2.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.2">𝜂</ci><apply id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3"><minus id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.1.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.1"></minus><apply id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2"><times id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.1.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.1"></times><cn type="integer" id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.2.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.2">1</cn><ci id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.3.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.2.3">𝑒</ci></apply><cn type="integer" id="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.3.cmml" xref="A1.SS2.p3.4.m4.2.2.2.2.2.2.3.3">4</cn></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p3.4.m4.2c">r=64,\alpha=32,\eta=1e-4</annotation></semantics></math>.</p>
</div>
<div id="A1.SS2.p4" class="ltx_para">
<p id="A1.SS2.p4.3" class="ltx_p"><span id="A1.SS2.p4.3.1" class="ltx_text ltx_font_bold">Instruction Tuning.</span> We use the hyperparameters of <cite class="ltx_cite ltx_citemacro_citet">Sharma et al. [<a href="#bib.bib104" title="" class="ltx_ref">104</a>]</cite> to avoid the expense of hyperparameter tuning on winrates. This means that we use a batch size of <math id="A1.SS2.p4.1.m1.1" class="ltx_Math" alttext="8" display="inline"><semantics id="A1.SS2.p4.1.m1.1a"><mn id="A1.SS2.p4.1.m1.1.1" xref="A1.SS2.p4.1.m1.1.1.cmml">8</mn><annotation-xml encoding="MathML-Content" id="A1.SS2.p4.1.m1.1b"><cn type="integer" id="A1.SS2.p4.1.m1.1.1.cmml" xref="A1.SS2.p4.1.m1.1.1">8</cn></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p4.1.m1.1c">8</annotation></semantics></math> and a learning rate of <math id="A1.SS2.p4.2.m2.1" class="ltx_Math" alttext="5e-7" display="inline"><semantics id="A1.SS2.p4.2.m2.1a"><mrow id="A1.SS2.p4.2.m2.1.1" xref="A1.SS2.p4.2.m2.1.1.cmml"><mrow id="A1.SS2.p4.2.m2.1.1.2" xref="A1.SS2.p4.2.m2.1.1.2.cmml"><mn id="A1.SS2.p4.2.m2.1.1.2.2" xref="A1.SS2.p4.2.m2.1.1.2.2.cmml">5</mn><mo lspace="0em" rspace="0em" id="A1.SS2.p4.2.m2.1.1.2.1" xref="A1.SS2.p4.2.m2.1.1.2.1.cmml">​</mo><mi id="A1.SS2.p4.2.m2.1.1.2.3" xref="A1.SS2.p4.2.m2.1.1.2.3.cmml">e</mi></mrow><mo id="A1.SS2.p4.2.m2.1.1.1" xref="A1.SS2.p4.2.m2.1.1.1.cmml">−</mo><mn id="A1.SS2.p4.2.m2.1.1.3" xref="A1.SS2.p4.2.m2.1.1.3.cmml">7</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.p4.2.m2.1b"><apply id="A1.SS2.p4.2.m2.1.1.cmml" xref="A1.SS2.p4.2.m2.1.1"><minus id="A1.SS2.p4.2.m2.1.1.1.cmml" xref="A1.SS2.p4.2.m2.1.1.1"></minus><apply id="A1.SS2.p4.2.m2.1.1.2.cmml" xref="A1.SS2.p4.2.m2.1.1.2"><times id="A1.SS2.p4.2.m2.1.1.2.1.cmml" xref="A1.SS2.p4.2.m2.1.1.2.1"></times><cn type="integer" id="A1.SS2.p4.2.m2.1.1.2.2.cmml" xref="A1.SS2.p4.2.m2.1.1.2.2">5</cn><ci id="A1.SS2.p4.2.m2.1.1.2.3.cmml" xref="A1.SS2.p4.2.m2.1.1.2.3">𝑒</ci></apply><cn type="integer" id="A1.SS2.p4.2.m2.1.1.3.cmml" xref="A1.SS2.p4.2.m2.1.1.3">7</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p4.2.m2.1c">5e-7</annotation></semantics></math> for both FFT and LoTA. We use the best performing hyperparameters for LoRA transferred from the commonsense reasoning tasks of <math id="A1.SS2.p4.3.m3.2" class="ltx_Math" alttext="r=64,\alpha=32,\eta=1e-4" display="inline"><semantics id="A1.SS2.p4.3.m3.2a"><mrow id="A1.SS2.p4.3.m3.2.2.2" xref="A1.SS2.p4.3.m3.2.2.3.cmml"><mrow id="A1.SS2.p4.3.m3.1.1.1.1" xref="A1.SS2.p4.3.m3.1.1.1.1.cmml"><mi id="A1.SS2.p4.3.m3.1.1.1.1.2" xref="A1.SS2.p4.3.m3.1.1.1.1.2.cmml">r</mi><mo id="A1.SS2.p4.3.m3.1.1.1.1.1" xref="A1.SS2.p4.3.m3.1.1.1.1.1.cmml">=</mo><mn id="A1.SS2.p4.3.m3.1.1.1.1.3" xref="A1.SS2.p4.3.m3.1.1.1.1.3.cmml">64</mn></mrow><mo id="A1.SS2.p4.3.m3.2.2.2.3" xref="A1.SS2.p4.3.m3.2.2.3a.cmml">,</mo><mrow id="A1.SS2.p4.3.m3.2.2.2.2.2" xref="A1.SS2.p4.3.m3.2.2.2.2.3.cmml"><mrow id="A1.SS2.p4.3.m3.2.2.2.2.1.1" xref="A1.SS2.p4.3.m3.2.2.2.2.1.1.cmml"><mi id="A1.SS2.p4.3.m3.2.2.2.2.1.1.2" xref="A1.SS2.p4.3.m3.2.2.2.2.1.1.2.cmml">α</mi><mo id="A1.SS2.p4.3.m3.2.2.2.2.1.1.1" xref="A1.SS2.p4.3.m3.2.2.2.2.1.1.1.cmml">=</mo><mn id="A1.SS2.p4.3.m3.2.2.2.2.1.1.3" xref="A1.SS2.p4.3.m3.2.2.2.2.1.1.3.cmml">32</mn></mrow><mo id="A1.SS2.p4.3.m3.2.2.2.2.2.3" xref="A1.SS2.p4.3.m3.2.2.2.2.3a.cmml">,</mo><mrow id="A1.SS2.p4.3.m3.2.2.2.2.2.2" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.cmml"><mi id="A1.SS2.p4.3.m3.2.2.2.2.2.2.2" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.2.cmml">η</mi><mo id="A1.SS2.p4.3.m3.2.2.2.2.2.2.1" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.1.cmml">=</mo><mrow id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.cmml"><mrow id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.cmml"><mn id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.2" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.2.cmml">1</mn><mo lspace="0em" rspace="0em" id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.1" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.1.cmml">​</mo><mi id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.3" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.3.cmml">e</mi></mrow><mo id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.1" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.1.cmml">−</mo><mn id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.3" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.3.cmml">4</mn></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.p4.3.m3.2b"><apply id="A1.SS2.p4.3.m3.2.2.3.cmml" xref="A1.SS2.p4.3.m3.2.2.2"><csymbol cd="ambiguous" id="A1.SS2.p4.3.m3.2.2.3a.cmml" xref="A1.SS2.p4.3.m3.2.2.2.3">formulae-sequence</csymbol><apply id="A1.SS2.p4.3.m3.1.1.1.1.cmml" xref="A1.SS2.p4.3.m3.1.1.1.1"><eq id="A1.SS2.p4.3.m3.1.1.1.1.1.cmml" xref="A1.SS2.p4.3.m3.1.1.1.1.1"></eq><ci id="A1.SS2.p4.3.m3.1.1.1.1.2.cmml" xref="A1.SS2.p4.3.m3.1.1.1.1.2">𝑟</ci><cn type="integer" id="A1.SS2.p4.3.m3.1.1.1.1.3.cmml" xref="A1.SS2.p4.3.m3.1.1.1.1.3">64</cn></apply><apply id="A1.SS2.p4.3.m3.2.2.2.2.3.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2"><csymbol cd="ambiguous" id="A1.SS2.p4.3.m3.2.2.2.2.3a.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.3">formulae-sequence</csymbol><apply id="A1.SS2.p4.3.m3.2.2.2.2.1.1.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.1.1"><eq id="A1.SS2.p4.3.m3.2.2.2.2.1.1.1.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.1.1.1"></eq><ci id="A1.SS2.p4.3.m3.2.2.2.2.1.1.2.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.1.1.2">𝛼</ci><cn type="integer" id="A1.SS2.p4.3.m3.2.2.2.2.1.1.3.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.1.1.3">32</cn></apply><apply id="A1.SS2.p4.3.m3.2.2.2.2.2.2.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2"><eq id="A1.SS2.p4.3.m3.2.2.2.2.2.2.1.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.1"></eq><ci id="A1.SS2.p4.3.m3.2.2.2.2.2.2.2.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.2">𝜂</ci><apply id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3"><minus id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.1.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.1"></minus><apply id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2"><times id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.1.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.1"></times><cn type="integer" id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.2.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.2">1</cn><ci id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.3.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.2.3">𝑒</ci></apply><cn type="integer" id="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.3.cmml" xref="A1.SS2.p4.3.m3.2.2.2.2.2.2.3.3">4</cn></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p4.3.m3.2c">r=64,\alpha=32,\eta=1e-4</annotation></semantics></math>.</p>
</div>
<div id="A1.SS2.p5" class="ltx_para">
<p id="A1.SS2.p5.3" class="ltx_p"><span id="A1.SS2.p5.3.1" class="ltx_text ltx_font_bold">TIES-Merging.</span> We consider post-hoc sparsification factors of <math id="A1.SS2.p5.1.m1.3" class="ltx_Math" alttext="0.1,0.2,0.3" display="inline"><semantics id="A1.SS2.p5.1.m1.3a"><mrow id="A1.SS2.p5.1.m1.3.4.2" xref="A1.SS2.p5.1.m1.3.4.1.cmml"><mn id="A1.SS2.p5.1.m1.1.1" xref="A1.SS2.p5.1.m1.1.1.cmml">0.1</mn><mo id="A1.SS2.p5.1.m1.3.4.2.1" xref="A1.SS2.p5.1.m1.3.4.1.cmml">,</mo><mn id="A1.SS2.p5.1.m1.2.2" xref="A1.SS2.p5.1.m1.2.2.cmml">0.2</mn><mo id="A1.SS2.p5.1.m1.3.4.2.2" xref="A1.SS2.p5.1.m1.3.4.1.cmml">,</mo><mn id="A1.SS2.p5.1.m1.3.3" xref="A1.SS2.p5.1.m1.3.3.cmml">0.3</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.p5.1.m1.3b"><list id="A1.SS2.p5.1.m1.3.4.1.cmml" xref="A1.SS2.p5.1.m1.3.4.2"><cn type="float" id="A1.SS2.p5.1.m1.1.1.cmml" xref="A1.SS2.p5.1.m1.1.1">0.1</cn><cn type="float" id="A1.SS2.p5.1.m1.2.2.cmml" xref="A1.SS2.p5.1.m1.2.2">0.2</cn><cn type="float" id="A1.SS2.p5.1.m1.3.3.cmml" xref="A1.SS2.p5.1.m1.3.3">0.3</cn></list></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p5.1.m1.3c">0.1,0.2,0.3</annotation></semantics></math>; the best performance is at either <math id="A1.SS2.p5.2.m2.1" class="ltx_Math" alttext="0.1" display="inline"><semantics id="A1.SS2.p5.2.m2.1a"><mn id="A1.SS2.p5.2.m2.1.1" xref="A1.SS2.p5.2.m2.1.1.cmml">0.1</mn><annotation-xml encoding="MathML-Content" id="A1.SS2.p5.2.m2.1b"><cn type="float" id="A1.SS2.p5.2.m2.1.1.cmml" xref="A1.SS2.p5.2.m2.1.1">0.1</cn></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p5.2.m2.1c">0.1</annotation></semantics></math> or <math id="A1.SS2.p5.3.m3.1" class="ltx_Math" alttext="0.2" display="inline"><semantics id="A1.SS2.p5.3.m3.1a"><mn id="A1.SS2.p5.3.m3.1.1" xref="A1.SS2.p5.3.m3.1.1.cmml">0.2</mn><annotation-xml encoding="MathML-Content" id="A1.SS2.p5.3.m3.1b"><cn type="float" id="A1.SS2.p5.3.m3.1.1.cmml" xref="A1.SS2.p5.3.m3.1.1">0.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.p5.3.m3.1c">0.2</annotation></semantics></math>.</p>
</div>
</section>
<section id="A1.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">A.3 </span>Individual Task Results for Averaged Experiments</h3>

<div id="A1.SS3.p1" class="ltx_para">
<p id="A1.SS3.p1.1" class="ltx_p">In the main body we present a number of experiments where we have to report the average performance over a number of tasks for space constraints. We now present the individual task results in Table <a href="#A1.T7" title="Table 7 ‣ A.3 Individual Task Results for Averaged Experiments ‣ Appendix A Additional Experimental Details ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a> and <a href="#A1.T8" title="In A.3 Individual Task Results for Averaged Experiments ‣ Appendix A Additional Experimental Details ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">8</span></a>.</p>
</div>
<figure id="A1.T7" class="ltx_table">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table"><span id="A1.T7.15.1.1" class="ltx_text" style="font-size:90%;">Table 7</span>: </span><span id="A1.T7.16.2" class="ltx_text" style="font-size:90%;">Individual task results for the “averaged” results in Table <a href="#S5.T5" title="Table 5 ‣ 5.3 Sequential Training ‣ 5 Experimental Results ‣ Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>. <span id="A1.T7.16.2.1" class="ltx_text ltx_font_bold">bold</span>: best method.</span></figcaption>
<div id="A1.T7.12" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:433.6pt;height:68pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(-12.6pt,2.0pt) scale(0.945030494975988,0.945030494975988) ;">
<table id="A1.T7.12.12" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A1.T7.12.12.13.1" class="ltx_tr">
<th id="A1.T7.12.12.13.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_th_row ltx_border_tt">FT Method on Task A</th>
<th id="A1.T7.12.12.13.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_th_row ltx_border_r ltx_border_tt">FT Method on Task B</th>
<th id="A1.T7.12.12.13.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">Instruction Following</th>
<th id="A1.T7.12.12.13.1.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">Arc</th>
<th id="A1.T7.12.12.13.1.5" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">GSM8k</th>
<th id="A1.T7.12.12.13.1.6" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">SQL</th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A1.T7.4.4.4" class="ltx_tr">
<th id="A1.T7.4.4.4.5" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t">FFT</th>
<th id="A1.T7.4.4.4.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_r ltx_border_t">FFT</th>
<td id="A1.T7.1.1.1.1" class="ltx_td ltx_align_center ltx_border_t"><math id="A1.T7.1.1.1.1.m1.1" class="ltx_Math" alttext="0.45_{0.21}" display="inline"><semantics id="A1.T7.1.1.1.1.m1.1a"><msub id="A1.T7.1.1.1.1.m1.1.1" xref="A1.T7.1.1.1.1.m1.1.1.cmml"><mn id="A1.T7.1.1.1.1.m1.1.1.2" xref="A1.T7.1.1.1.1.m1.1.1.2.cmml">0.45</mn><mn id="A1.T7.1.1.1.1.m1.1.1.3" xref="A1.T7.1.1.1.1.m1.1.1.3.cmml">0.21</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.1.1.1.1.m1.1b"><apply id="A1.T7.1.1.1.1.m1.1.1.cmml" xref="A1.T7.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.1.1.1.1.m1.1.1.1.cmml" xref="A1.T7.1.1.1.1.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.1.1.1.1.m1.1.1.2.cmml" xref="A1.T7.1.1.1.1.m1.1.1.2">0.45</cn><cn type="float" id="A1.T7.1.1.1.1.m1.1.1.3.cmml" xref="A1.T7.1.1.1.1.m1.1.1.3">0.21</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.1.1.1.1.m1.1c">0.45_{0.21}</annotation></semantics></math></td>
<td id="A1.T7.2.2.2.2" class="ltx_td ltx_align_center ltx_border_t"><math id="A1.T7.2.2.2.2.m1.1" class="ltx_Math" alttext="74.1_{.09}" display="inline"><semantics id="A1.T7.2.2.2.2.m1.1a"><msub id="A1.T7.2.2.2.2.m1.1.1" xref="A1.T7.2.2.2.2.m1.1.1.cmml"><mn id="A1.T7.2.2.2.2.m1.1.1.2" xref="A1.T7.2.2.2.2.m1.1.1.2.cmml">74.1</mn><mn id="A1.T7.2.2.2.2.m1.1.1.3" xref="A1.T7.2.2.2.2.m1.1.1.3.cmml">.09</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.2.2.2.2.m1.1b"><apply id="A1.T7.2.2.2.2.m1.1.1.cmml" xref="A1.T7.2.2.2.2.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.2.2.2.2.m1.1.1.1.cmml" xref="A1.T7.2.2.2.2.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.2.2.2.2.m1.1.1.2.cmml" xref="A1.T7.2.2.2.2.m1.1.1.2">74.1</cn><cn type="float" id="A1.T7.2.2.2.2.m1.1.1.3.cmml" xref="A1.T7.2.2.2.2.m1.1.1.3">.09</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.2.2.2.2.m1.1c">74.1_{.09}</annotation></semantics></math></td>
<td id="A1.T7.3.3.3.3" class="ltx_td ltx_align_center ltx_border_t"><math id="A1.T7.3.3.3.3.m1.1" class="ltx_Math" alttext="51.9_{0.09}" display="inline"><semantics id="A1.T7.3.3.3.3.m1.1a"><msub id="A1.T7.3.3.3.3.m1.1.1" xref="A1.T7.3.3.3.3.m1.1.1.cmml"><mn id="A1.T7.3.3.3.3.m1.1.1.2" xref="A1.T7.3.3.3.3.m1.1.1.2.cmml">51.9</mn><mn id="A1.T7.3.3.3.3.m1.1.1.3" xref="A1.T7.3.3.3.3.m1.1.1.3.cmml">0.09</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.3.3.3.3.m1.1b"><apply id="A1.T7.3.3.3.3.m1.1.1.cmml" xref="A1.T7.3.3.3.3.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.3.3.3.3.m1.1.1.1.cmml" xref="A1.T7.3.3.3.3.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.3.3.3.3.m1.1.1.2.cmml" xref="A1.T7.3.3.3.3.m1.1.1.2">51.9</cn><cn type="float" id="A1.T7.3.3.3.3.m1.1.1.3.cmml" xref="A1.T7.3.3.3.3.m1.1.1.3">0.09</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.3.3.3.3.m1.1c">51.9_{0.09}</annotation></semantics></math></td>
<td id="A1.T7.4.4.4.4" class="ltx_td ltx_align_center ltx_border_t"><math id="A1.T7.4.4.4.4.m1.1" class="ltx_Math" alttext="98.9_{0.01}" display="inline"><semantics id="A1.T7.4.4.4.4.m1.1a"><msub id="A1.T7.4.4.4.4.m1.1.1" xref="A1.T7.4.4.4.4.m1.1.1.cmml"><mn id="A1.T7.4.4.4.4.m1.1.1.2" xref="A1.T7.4.4.4.4.m1.1.1.2.cmml">98.9</mn><mn id="A1.T7.4.4.4.4.m1.1.1.3" xref="A1.T7.4.4.4.4.m1.1.1.3.cmml">0.01</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.4.4.4.4.m1.1b"><apply id="A1.T7.4.4.4.4.m1.1.1.cmml" xref="A1.T7.4.4.4.4.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.4.4.4.4.m1.1.1.1.cmml" xref="A1.T7.4.4.4.4.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.4.4.4.4.m1.1.1.2.cmml" xref="A1.T7.4.4.4.4.m1.1.1.2">98.9</cn><cn type="float" id="A1.T7.4.4.4.4.m1.1.1.3.cmml" xref="A1.T7.4.4.4.4.m1.1.1.3">0.01</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.4.4.4.4.m1.1c">98.9_{0.01}</annotation></semantics></math></td>
</tr>
<tr id="A1.T7.8.8.8" class="ltx_tr">
<th id="A1.T7.8.8.8.5" class="ltx_td ltx_align_center ltx_th ltx_th_row">LoTA (ours)</th>
<th id="A1.T7.8.8.8.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_r">FFT</th>
<td id="A1.T7.5.5.5.1" class="ltx_td ltx_align_center"><math id="A1.T7.5.5.5.1.m1.1" class="ltx_Math" alttext="\underline{11.48_{0.73}}" display="inline"><semantics id="A1.T7.5.5.5.1.m1.1a"><munder accentunder="true" id="A1.T7.5.5.5.1.m1.1.1" xref="A1.T7.5.5.5.1.m1.1.1.cmml"><msub id="A1.T7.5.5.5.1.m1.1.1.2" xref="A1.T7.5.5.5.1.m1.1.1.2.cmml"><mn id="A1.T7.5.5.5.1.m1.1.1.2.2" xref="A1.T7.5.5.5.1.m1.1.1.2.2.cmml">11.48</mn><mn id="A1.T7.5.5.5.1.m1.1.1.2.3" xref="A1.T7.5.5.5.1.m1.1.1.2.3.cmml">0.73</mn></msub><mo id="A1.T7.5.5.5.1.m1.1.1.1" xref="A1.T7.5.5.5.1.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="A1.T7.5.5.5.1.m1.1b"><apply id="A1.T7.5.5.5.1.m1.1.1.cmml" xref="A1.T7.5.5.5.1.m1.1.1"><ci id="A1.T7.5.5.5.1.m1.1.1.1.cmml" xref="A1.T7.5.5.5.1.m1.1.1.1">¯</ci><apply id="A1.T7.5.5.5.1.m1.1.1.2.cmml" xref="A1.T7.5.5.5.1.m1.1.1.2"><csymbol cd="ambiguous" id="A1.T7.5.5.5.1.m1.1.1.2.1.cmml" xref="A1.T7.5.5.5.1.m1.1.1.2">subscript</csymbol><cn type="float" id="A1.T7.5.5.5.1.m1.1.1.2.2.cmml" xref="A1.T7.5.5.5.1.m1.1.1.2.2">11.48</cn><cn type="float" id="A1.T7.5.5.5.1.m1.1.1.2.3.cmml" xref="A1.T7.5.5.5.1.m1.1.1.2.3">0.73</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.5.5.5.1.m1.1c">\underline{11.48_{0.73}}</annotation></semantics></math></td>
<td id="A1.T7.6.6.6.2" class="ltx_td ltx_align_center"><math id="A1.T7.6.6.6.2.m1.1" class="ltx_Math" alttext="73.3_{0.02}" display="inline"><semantics id="A1.T7.6.6.6.2.m1.1a"><msub id="A1.T7.6.6.6.2.m1.1.1" xref="A1.T7.6.6.6.2.m1.1.1.cmml"><mn id="A1.T7.6.6.6.2.m1.1.1.2" xref="A1.T7.6.6.6.2.m1.1.1.2.cmml">73.3</mn><mn id="A1.T7.6.6.6.2.m1.1.1.3" xref="A1.T7.6.6.6.2.m1.1.1.3.cmml">0.02</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.6.6.6.2.m1.1b"><apply id="A1.T7.6.6.6.2.m1.1.1.cmml" xref="A1.T7.6.6.6.2.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.6.6.6.2.m1.1.1.1.cmml" xref="A1.T7.6.6.6.2.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.6.6.6.2.m1.1.1.2.cmml" xref="A1.T7.6.6.6.2.m1.1.1.2">73.3</cn><cn type="float" id="A1.T7.6.6.6.2.m1.1.1.3.cmml" xref="A1.T7.6.6.6.2.m1.1.1.3">0.02</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.6.6.6.2.m1.1c">73.3_{0.02}</annotation></semantics></math></td>
<td id="A1.T7.7.7.7.3" class="ltx_td ltx_align_center"><math id="A1.T7.7.7.7.3.m1.1" class="ltx_Math" alttext="53.9_{0.09}" display="inline"><semantics id="A1.T7.7.7.7.3.m1.1a"><msub id="A1.T7.7.7.7.3.m1.1.1" xref="A1.T7.7.7.7.3.m1.1.1.cmml"><mn id="A1.T7.7.7.7.3.m1.1.1.2" xref="A1.T7.7.7.7.3.m1.1.1.2.cmml">53.9</mn><mn id="A1.T7.7.7.7.3.m1.1.1.3" xref="A1.T7.7.7.7.3.m1.1.1.3.cmml">0.09</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.7.7.7.3.m1.1b"><apply id="A1.T7.7.7.7.3.m1.1.1.cmml" xref="A1.T7.7.7.7.3.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.7.7.7.3.m1.1.1.1.cmml" xref="A1.T7.7.7.7.3.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.7.7.7.3.m1.1.1.2.cmml" xref="A1.T7.7.7.7.3.m1.1.1.2">53.9</cn><cn type="float" id="A1.T7.7.7.7.3.m1.1.1.3.cmml" xref="A1.T7.7.7.7.3.m1.1.1.3">0.09</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.7.7.7.3.m1.1c">53.9_{0.09}</annotation></semantics></math></td>
<td id="A1.T7.8.8.8.4" class="ltx_td ltx_align_center"><math id="A1.T7.8.8.8.4.m1.1" class="ltx_Math" alttext="98.9_{0.01}" display="inline"><semantics id="A1.T7.8.8.8.4.m1.1a"><msub id="A1.T7.8.8.8.4.m1.1.1" xref="A1.T7.8.8.8.4.m1.1.1.cmml"><mn id="A1.T7.8.8.8.4.m1.1.1.2" xref="A1.T7.8.8.8.4.m1.1.1.2.cmml">98.9</mn><mn id="A1.T7.8.8.8.4.m1.1.1.3" xref="A1.T7.8.8.8.4.m1.1.1.3.cmml">0.01</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.8.8.8.4.m1.1b"><apply id="A1.T7.8.8.8.4.m1.1.1.cmml" xref="A1.T7.8.8.8.4.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.8.8.8.4.m1.1.1.1.cmml" xref="A1.T7.8.8.8.4.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.8.8.8.4.m1.1.1.2.cmml" xref="A1.T7.8.8.8.4.m1.1.1.2">98.9</cn><cn type="float" id="A1.T7.8.8.8.4.m1.1.1.3.cmml" xref="A1.T7.8.8.8.4.m1.1.1.3">0.01</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.8.8.8.4.m1.1c">98.9_{0.01}</annotation></semantics></math></td>
</tr>
<tr id="A1.T7.12.12.12" class="ltx_tr">
<th id="A1.T7.12.12.12.5" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_bb">LoTA (ours)</th>
<th id="A1.T7.12.12.12.6" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_bb ltx_border_r">LoTTO (ours)</th>
<td id="A1.T7.9.9.9.1" class="ltx_td ltx_align_center ltx_border_bb"><math id="A1.T7.9.9.9.1.m1.1" class="ltx_Math" alttext="\mathbf{15.88_{0.88}}" display="inline"><semantics id="A1.T7.9.9.9.1.m1.1a"><msub id="A1.T7.9.9.9.1.m1.1.1" xref="A1.T7.9.9.9.1.m1.1.1.cmml"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="A1.T7.9.9.9.1.m1.1.1.2" xref="A1.T7.9.9.9.1.m1.1.1.2.cmml">15.88</mn><mn class="ltx_mathvariant_bold" mathvariant="bold" id="A1.T7.9.9.9.1.m1.1.1.3" xref="A1.T7.9.9.9.1.m1.1.1.3.cmml">0.88</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.9.9.9.1.m1.1b"><apply id="A1.T7.9.9.9.1.m1.1.1.cmml" xref="A1.T7.9.9.9.1.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.9.9.9.1.m1.1.1.1.cmml" xref="A1.T7.9.9.9.1.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.9.9.9.1.m1.1.1.2.cmml" xref="A1.T7.9.9.9.1.m1.1.1.2">15.88</cn><cn type="float" id="A1.T7.9.9.9.1.m1.1.1.3.cmml" xref="A1.T7.9.9.9.1.m1.1.1.3">0.88</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.9.9.9.1.m1.1c">\mathbf{15.88_{0.88}}</annotation></semantics></math></td>
<td id="A1.T7.10.10.10.2" class="ltx_td ltx_align_center ltx_border_bb"><math id="A1.T7.10.10.10.2.m1.1" class="ltx_Math" alttext="70.3_{0.01}" display="inline"><semantics id="A1.T7.10.10.10.2.m1.1a"><msub id="A1.T7.10.10.10.2.m1.1.1" xref="A1.T7.10.10.10.2.m1.1.1.cmml"><mn id="A1.T7.10.10.10.2.m1.1.1.2" xref="A1.T7.10.10.10.2.m1.1.1.2.cmml">70.3</mn><mn id="A1.T7.10.10.10.2.m1.1.1.3" xref="A1.T7.10.10.10.2.m1.1.1.3.cmml">0.01</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.10.10.10.2.m1.1b"><apply id="A1.T7.10.10.10.2.m1.1.1.cmml" xref="A1.T7.10.10.10.2.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.10.10.10.2.m1.1.1.1.cmml" xref="A1.T7.10.10.10.2.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.10.10.10.2.m1.1.1.2.cmml" xref="A1.T7.10.10.10.2.m1.1.1.2">70.3</cn><cn type="float" id="A1.T7.10.10.10.2.m1.1.1.3.cmml" xref="A1.T7.10.10.10.2.m1.1.1.3">0.01</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.10.10.10.2.m1.1c">70.3_{0.01}</annotation></semantics></math></td>
<td id="A1.T7.11.11.11.3" class="ltx_td ltx_align_center ltx_border_bb"><math id="A1.T7.11.11.11.3.m1.1" class="ltx_Math" alttext="52.5_{0.01}" display="inline"><semantics id="A1.T7.11.11.11.3.m1.1a"><msub id="A1.T7.11.11.11.3.m1.1.1" xref="A1.T7.11.11.11.3.m1.1.1.cmml"><mn id="A1.T7.11.11.11.3.m1.1.1.2" xref="A1.T7.11.11.11.3.m1.1.1.2.cmml">52.5</mn><mn id="A1.T7.11.11.11.3.m1.1.1.3" xref="A1.T7.11.11.11.3.m1.1.1.3.cmml">0.01</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.11.11.11.3.m1.1b"><apply id="A1.T7.11.11.11.3.m1.1.1.cmml" xref="A1.T7.11.11.11.3.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.11.11.11.3.m1.1.1.1.cmml" xref="A1.T7.11.11.11.3.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.11.11.11.3.m1.1.1.2.cmml" xref="A1.T7.11.11.11.3.m1.1.1.2">52.5</cn><cn type="float" id="A1.T7.11.11.11.3.m1.1.1.3.cmml" xref="A1.T7.11.11.11.3.m1.1.1.3">0.01</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.11.11.11.3.m1.1c">52.5_{0.01}</annotation></semantics></math></td>
<td id="A1.T7.12.12.12.4" class="ltx_td ltx_align_center ltx_border_bb"><math id="A1.T7.12.12.12.4.m1.1" class="ltx_Math" alttext="98.6_{0.01}" display="inline"><semantics id="A1.T7.12.12.12.4.m1.1a"><msub id="A1.T7.12.12.12.4.m1.1.1" xref="A1.T7.12.12.12.4.m1.1.1.cmml"><mn id="A1.T7.12.12.12.4.m1.1.1.2" xref="A1.T7.12.12.12.4.m1.1.1.2.cmml">98.6</mn><mn id="A1.T7.12.12.12.4.m1.1.1.3" xref="A1.T7.12.12.12.4.m1.1.1.3.cmml">0.01</mn></msub><annotation-xml encoding="MathML-Content" id="A1.T7.12.12.12.4.m1.1b"><apply id="A1.T7.12.12.12.4.m1.1.1.cmml" xref="A1.T7.12.12.12.4.m1.1.1"><csymbol cd="ambiguous" id="A1.T7.12.12.12.4.m1.1.1.1.cmml" xref="A1.T7.12.12.12.4.m1.1.1">subscript</csymbol><cn type="float" id="A1.T7.12.12.12.4.m1.1.1.2.cmml" xref="A1.T7.12.12.12.4.m1.1.1.2">98.6</cn><cn type="float" id="A1.T7.12.12.12.4.m1.1.1.3.cmml" xref="A1.T7.12.12.12.4.m1.1.1.3">0.01</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.T7.12.12.12.4.m1.1c">98.6_{0.01}</annotation></semantics></math></td>
</tr>
</tbody>
</table>
</span></div>
</figure>
<figure id="A1.T8" class="ltx_table">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table"><span id="A1.T8.2.1.1" class="ltx_text" style="font-size:90%;">Table 8</span>: </span><span id="A1.T8.3.2" class="ltx_text" style="font-size:90%;">Full results on commonsense reasoning.
</span></figcaption>
<div id="A1.T8.4" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:433.6pt;height:46.7pt;vertical-align:-0.8pt;"><span class="ltx_transformed_inner" style="transform:translate(-38.4pt,4.1pt) scale(0.849701795624941,0.849701795624941) ;">
<table id="A1.T8.4.1" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A1.T8.4.1.1.1" class="ltx_tr">
<th id="A1.T8.4.1.1.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_th_row ltx_border_tt">Model</th>
<th id="A1.T8.4.1.1.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_th_row ltx_border_r ltx_border_tt">Method</th>
<th id="A1.T8.4.1.1.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">BoolQ</th>
<th id="A1.T8.4.1.1.1.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">PIQA</th>
<th id="A1.T8.4.1.1.1.5" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">SIQA</th>
<th id="A1.T8.4.1.1.1.6" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">HellaSwag</th>
<th id="A1.T8.4.1.1.1.7" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">WinoGrande</th>
<th id="A1.T8.4.1.1.1.8" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">ARC-e</th>
<th id="A1.T8.4.1.1.1.9" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">ARC-c</th>
<th id="A1.T8.4.1.1.1.10" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_r ltx_border_tt">OBQA</th>
<th id="A1.T8.4.1.1.1.11" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt">Avg.</th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A1.T8.4.1.2.1" class="ltx_tr">
<th id="A1.T8.4.1.2.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_bb ltx_border_t" rowspan="3"><span id="A1.T8.4.1.2.1.1.1" class="ltx_text">Mistral</span></th>
<th id="A1.T8.4.1.2.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_r ltx_border_t">LoRA</th>
<td id="A1.T8.4.1.2.1.3" class="ltx_td ltx_align_center ltx_border_t">76.4</td>
<td id="A1.T8.4.1.2.1.4" class="ltx_td ltx_align_center ltx_border_t">90.7</td>
<td id="A1.T8.4.1.2.1.5" class="ltx_td ltx_align_center ltx_border_t">81.2</td>
<td id="A1.T8.4.1.2.1.6" class="ltx_td ltx_align_center ltx_border_t">91.4</td>
<td id="A1.T8.4.1.2.1.7" class="ltx_td ltx_align_center ltx_border_t">87.0</td>
<td id="A1.T8.4.1.2.1.8" class="ltx_td ltx_align_center ltx_border_t">94.1</td>
<td id="A1.T8.4.1.2.1.9" class="ltx_td ltx_align_center ltx_border_t">83.0</td>
<td id="A1.T8.4.1.2.1.10" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">90.4</td>
<td id="A1.T8.4.1.2.1.11" class="ltx_td ltx_align_center ltx_border_t">86.8</td>
</tr>
<tr id="A1.T8.4.1.3.2" class="ltx_tr">
<th id="A1.T8.4.1.3.2.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_bb ltx_border_r">LoTA (ours)</th>
<td id="A1.T8.4.1.3.2.2" class="ltx_td ltx_align_center ltx_border_bb">75.3</td>
<td id="A1.T8.4.1.3.2.3" class="ltx_td ltx_align_center ltx_border_bb">90.0</td>
<td id="A1.T8.4.1.3.2.4" class="ltx_td ltx_align_center ltx_border_bb">83.3</td>
<td id="A1.T8.4.1.3.2.5" class="ltx_td ltx_align_center ltx_border_bb">91.4</td>
<td id="A1.T8.4.1.3.2.6" class="ltx_td ltx_align_center ltx_border_bb">89.5</td>
<td id="A1.T8.4.1.3.2.7" class="ltx_td ltx_align_center ltx_border_bb">95.1</td>
<td id="A1.T8.4.1.3.2.8" class="ltx_td ltx_align_center ltx_border_bb">85.0</td>
<td id="A1.T8.4.1.3.2.9" class="ltx_td ltx_align_center ltx_border_bb ltx_border_r">90.2</td>
<td id="A1.T8.4.1.3.2.10" class="ltx_td ltx_align_center ltx_border_bb">87.5</td>
</tr>
</tbody>
</table>
</span></div>
</figure>
<div class="ltx_pagination ltx_role_newpage"></div>
</section>
</section>
</article>
</div>
<div class="ar5iv-footer"><a href="/html/2406.16796" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2406.16797" class="ar5iv-text-button ar5iv-severity-warning">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2406.16797">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2406.16797" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2406.16798" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Fri Jul  5 22:24:41 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2406.16797
Existing methods for adapting large language models (LLMs)  to new tasks are not suited to multi-task adaptation because they modify all the model weights–causing destructive interference between tasks. The resulting effects, such as catastrophic forgetting of earlier tasks, make it challenging to obtain good performance on multiple tasks at the same time.
To mitigate this, we propose Lottery Ticket Adaptation (LoTA) , a sparse adaptation method that identifies and optimizes only a sparse subnetwork of the model. We evaluate LoTA on a wide range of challenging tasks such as instruction following, reasoning, math, and summarization. LoTA obtains better performance than full fine-tuning and low-rank adaptation (LoRA) , and maintains good performance even after training on other tasks – thus, avoiding catastrophic forgetting. By extracting and fine-tuning overlottery tickets(orsparse task vectors) , LoTA also enables model merging over highly dissimilar tasks. Our code is madepublicly available.111A preliminary version of this work has appeared in.
abstract
Lottery Ticket Adaptation:Mitigating Destructive Interference in LLMs
Abstract
2406.16797
1IntroductionLarge language models (LLMs) have seen an explosion of applications to real-world problemsvia adaptationto new tasks. Three majormulti-task adaptationparadigms have emerged: storing and loading task-specificadapters, continuing to train instruction-tuned models on new tasks in serial viasequential training, and combining the adaptations to tasks learned in parallel viamodel merging.
Each paradigm has its own associated challenges, such as catastrophic forgetting during sequential training, and methods that have been proposed to mitigate these challenges. In this work, we propose a new LLM adaptation method, calledLottery Ticket Adaptation (LoTA) , that (1)  provides sparse adaptation by freezing a majority of the parameters and updating only a sparse subnetwork of the base model and (2)  resolves the challenges in commonmulti-task adaptationparadigms. (More details in Section3.)  We summarize our contributions:•We trainlottery tickets(orsparse task vectors)  that can be stored efficiently and obtain performance similar to full fine-tuning (FFT)  and higher than LoRA across a range of tasks spanning reasoning, math, code generation, and instruction following. When adapting Mistral for instruction following, FFT and LoTA both get a length-controlled AlpacaEval 2 winrate(how often GPT-4 prefers the outputs of our model over its own)  of, but LoRA only gets a winrate of.•We apply LoTA to mitigatecatastrophic forgettingof earlier tasks, enabling sequential adaptation to new tasks. When adapting an instruction tuned model to a mix of new tasks, the winrate of the FFT model drops fromto, but by using LoTA we can limit the drop to.•We can use LoTA tomerge models in parallelacross dramatically different tasks, achieving better performance than existing merging methods that rely on post hoc sparsification(which degrades performance for FFT models)  because it naturally trainssparse task vectors. When merging instruction following and math models with LoTA, we get a task-average performance ofwhere a merge of FFT models obtains.
introduction
Lottery Ticket Adaptation:Mitigating Destructive Interference in LLMs
1Introduction
2406.16797
2Background: Multi-Task AdaptationIn this section, we go over common multi-task adaptation paradigms and discuss the challenges existing fine-tuning methods, such as FFT and LoRA, bring in each paradigm.Figure 1:Multi-task adaptation: storing and loading adapters, sequential training, model merging.Storing and Loading Adapters.As illustrated in the first row of Figure1, an emerging paradigm for multi-task adaptation is to store an adapter for each desired task and load a particular adapter at inference time, depending on the need. Notably, Apple stores and loads adapters to enable their on-device models.
This approach, while avoiding any interference between tasks, increases the memory and compute cost as it requires storing and loading an additional adapter per task. To mitigate these costs, a number of PEFT methods have been developed.
Among them, LoRA(training adapters in the low-rank space)  has received notable attention due to its simplicity. For instance, services such as Punica and S-LoRA allow developers to use this approach to serve large numbers of task-specific adapters for specific requests.
However, a persistent gap in capacity between PEFT methods and FFT has presented a tradeoff between adapter overhead and performance.Sequential Training.When it is desired to have a single model with multi-task abilities (as opposed to storing and loading adapters per task) , one common approach is to fine-tune the model on different tasks sequentially, e.g., first fine-tune on task A, then fine-tune on task B. This is summarized in the second row of Figure1. Note that sequential training is distinct from continual pre-training, because sequential training uses the instruction tuning objective whereas continual pre-training typically uses the pre-training objective of next-word-prediction.Fine-tuning the LLM for new tasks with FFT or existing PEFT methods leads to catastrophic forgetting of earlier tasks. This is problematic, especially forsafetyalignment, since we can fine-tune an LLM to be safe but later get this feature erased during future fine-tuning on new tasks. In fact, a number of works have aimed to mitigate this vulnerability, leaving an open research question:Can model developers allow users to finetune their aligned models on custom datasets while retaining safety?Model Merging.There has been a recent interest in model arithmetic and editing methods, including merging multiple models adapted to different individual tasks to have a single model adapted to multiple tasks simultaneously. As the third row of Figure1shows, this is typically done via aggregatingtask vectors(or adapters)  of different tasks. The existing model merging techniques either require post-processing the task vectors through sparsification, degrading the performance on the task, and/or require extensive hyperparameter tuning for a weighted aggregation of task vectors.Each paradigm of multi-task learning poses different challenges, and different methods have been proposed to address these challenges. We defer the more in-depth analysis of these proposed methods inSection6because of the sheer quantity of related work that must be covered. The number of methods itself poses challenges for studying their drawbacks, especially when these methods are adopted in settings orthogonal to those they were originally developed for (i.e., LoRA leading to catastrophic forgetting of safety alignment) .Rather than proposing tailored solutions for each paradigm of multi-task adaptation, we want to propose a simple algorithm that can serve as an effective foundation across all three paradigms and evaluate it on a wide range of challenging tasks.
body
Lottery Ticket Adaptation:Mitigating Destructive Interference in LLMs
2Background: Multi-Task Adaptation
2406.16797
3Lottery Ticket Adaptation (LoTA) Algorithm 1Lottery Ticket Adaptation (LoTA) 0:Adaptation algorithm, alignment dataset, pre-trained weights, sparsity ratio, learning rate, number of calibration iterations, number of sparse training iterations.1:Mask Calibration:2:3:fordo4:{Compute gradient for weights}5:{Update the model}6:endfor7:Mask Extraction:8:{Find the task vector}9:{Create the sparsity mask by thresholding the task vector based on magnitude}10:Sparse Adaptation:11:12:fordo13:{Compute gradient for weights}14:{Apply sparse mask to gradient}15:{Update the model}16:endfor17:17:The desiderata for each multi-task adaptation paradigm motivates the design of our method. Foradapters, we want a representation that can be easily compressed for memory efficiency. Forsequential training, we want a representation that minimizes destructive interference between the previously learned tasks and tasks to be learned in the future.
Formodel merging, we want representations that are mutually sparse with each other in parameter space to again prevent destructive interference. We now propose LoTA, a single method that enjoys all these features. We first describe the workflow of LoTA, then revisit the problems each multi-task paradigm faces and discuss how and why LoTA successfully mitigates them.Lottery Ticket Adaptation (LoTA) .LoTA works in three phases as summarized in Figure2: (1)  mask calibration, (2)  mask extraction, (3)  sparse adaptation. In the mask calibration phase of LoTA, a base model with parametersis fine-tuned foriterations, yielding a fine-tuned model with parameters. Then, in the mask extraction phase, LoTA extracts a sparsity maskfrom the task vectorbased on the magnitude of the updates in.could be as small as one iteration. In the sparse adaptation phase of LoTA, the model is first reset to its original state with weights. Then the subnetworkis fine-tuned foriterations, while leaving the remaining parametersfrozen at their initial values. We summarize the workflow of LoTA in Figure2and Algorithm1further.By confining the adaptation updates within subnetworks (identified by) ,LoTA is able to mitigate destructive interference, e.g., adaptation loss during fine-tuning on future datasets or model merging, that FFT and LoRA suffer from. We discuss this in more detail under three multi-task adaptation paradigms below and provide empirical comparisons with FFT and LoRA in Section5.Figure 2:Lottery Ticket Adaptation (LoTA) : (1)  Mask calibration via FFT foriterations, (2)  Extracting the sparsity maskfrom the task vector, (3)  Sparse fine-tuning with sparsity maskforiterations.(1)  Storing & Loading Adapters.As mentioned before, PEFT methods have emerged to reduce the memory cost of storing and loading adapters. However, the most popular and commonly used PEFT method, LoRA, restricts the adaptation updates to have a low rank, which may be insufficient for complex downstream tasks. In parallel, recent workon compressing the delta between the fine-tuned and pre-trained modelsuggests that FFT updates are highly compressible through a simple magnitude-based sparsification. LoTA exploits this underlying sparsity during fine-tuning and obtains better performance than LoRA, while requiring fewer parameters to be trained. As we will show, LoTA adapters can be stored efficiently because of the small number of parameters that need to be updated. LoTA also requires fewer parameters to be updated in the model when the adapter is applied to the model, so loading a LoTA adapter can be faster than loading a LoRA adapter.(2)  Sequential Training.Existing fine-tuning methods, such as FFT and LoRA, are known to cause catastrophic forgetting. This is particularly concerning for safety alignment as any safety measure the model developers add could be erased by further fine-tuning. By restricting the task vectors to be sparse, LoTA provides robustness against catastrophic forgetting. This is because it will be less likely that an adaptation to new data requires modifying the weights needed for the previous task when fewer weights are updated. As LoTA prevents destructive interference between sequential tasks via sparse and disjoint task vectors, the same phenomenon is also helpful in adapting to new tasks. To further enhance the robustness against destructive interference in sequential training, we proposeLottery Ticket Together Optimization (LoTTO) which learns mutually sparse (i.e., non-overlapping)  masks for sequentially learned tasks.Algorithm 2Lottery Ticket Together Optimization (LoTTO) 0:Alignment dataset sequence, pre-trained weights, LoTA hyperparameter set.1:{Initialize constraint set}2:3:fordo4:{Create mask from constraint set}5:{Train model with constraint mask}6:{Extract sparse mask from trained model}7:{Train model with sparse mask}8:{Update constraints}9:endfor9:Further Enhancing Sequential Training via Lottery Ticket Together Optimization (LoTTO) .Without loss of generality, suppose we have two tasks, Task A and Task B, and that we have already learned Task A with LoTA. LoTTO calibrates a sparsity mask for Task B by first training a model where the only weights that can be updated are those that arenotupdated when running LoTA on Task A, and then using a sparse set of those weights to train the final model. LoTTO enables learning new tasks without forgetting old ones, by restricting weight adaptations to not interfere with the weights that were important for previous adaptations. This procedure can be applied inductively to enable sequential adaptation to multiple tasks, so that a model developer seeking to adapt a model adapted with LoTA (potentially on several tasks) , just needs to ensure that they do not update the task vector with respect to the base model. We envision that LoTTO will be applied to capabilities one at a time; first the model learns math, then coding, then reasoning, then planning, etc. Ifof the weights are updated for each capability, then LoTTO can fit at mostcapabilities. We believe this is sufficient for nearly all practical applications, but when the subnetworks for different tasks overlap, we do not expect that LoTTO will result in any worse forgetting than LoTA, which we will show itself has better robustness to forgetting than FFT.(3)  Model Merging.Existing model merging methods typically aim to merge task vectors of relatively similar language datasets, and they do so after a post hoc sparsificationof the task vectors. The post hoc sparsification aids in ensuring the task vectors are disjoint – hence can prevent destructive interference – but, in return, degrades the performance of each individual task. LoTA, on the other hand, enforces sparsity during fine-tuning and directly trainssparse task vectors, obviating the need for post hoc sparsification.
body
Lottery Ticket Adaptation:Mitigating Destructive Interference in LLMs
3Lottery Ticket Adaptation (LoTA)
2406.16797
4Experimental SetupIn this section, we provide details of the experimental setup, including the baselines, model, dataset, and metric selection. We present the results later inSection5. We share ouropen-source code. We are limited to an academic computing budget, and all results are conducted with a single A100 GPU. We typically doepochs of training for each dataset as this is a standard choice in LLM fine-tuning and indicate it specifically for single-epoch fine-tuning. We use the RMSProp optimizer with default hyperparameters.Baselines & Hyperparameters.Across all three multi-task adaptation paradigms, we compare LoTA against FFT and LoRA. We extensively tune the hyperparameters for FFT and LoRA to ensure that we are comparing against strong baselines. We do not tune hyperparameters for LoTA and directly transfer the hyperparameters from FFT. We fix the sparsity ratio hyperparameter in LoTA to, so that the number of parameters updated roughly matches that of the best-performing LoRA rank of. We provide an ablation study with higher sparsity levels in Section5.1. We report all ranges for hyperparameters inAppendixA.Models.We use the best performing open-weights model families, Mistraland Llama 3, specifically Mistral-7B and Llama-3-8B – the largest models we can adapt with FFT on a single GPU.TasksWe consider six capabilities: instruction following, safety, math, coding, summarization, and reasoning.
We now briefly discuss each capability, the datasets we use to fine-tune and evaluate the presented methods, and the motivation behind the choices.Instruction Following.The most widely-used instruction-tuned LLMs are the “Instruct” or “chat” versions of base models, such as Llama-3-8B-Instruct. This is because the process of tuning models on human instructions aligns models to human preferences across a range of tasks. For this, we adapt models to data from UltraFeedback, which contains a mixture of datasets covering truthfulness, honesty, and helpfulness in addition to instruction-following. We measure the instruction following ability by length-controlled AlpacaEval2 Win Rate, which we refer to as “winrate”. A high winrate means that GPT-4prefers the responses of our model on a set of representative prompts over its own responses. Winrate is the metric most closely correlated with human rating preference.
Another common benchmark for “chat” models is MT-Bench, but there is a significant degree of data contamination between MT-Bench and other task-specific training datasets–hence, we do not evaluate on MT-Bench.Reasoning.We train on the standard set ofcommonsense reasoning tasks(Boolq, PIQA, SocialIQA, Hellaswag, Winograde, ARC-easy, ARC-challenge, OpenBookQA)  and report the exact-match accuracy on the test set. As a representative task, we use ARC-easy.Math.We use the set ofmath instruction datasets fromfor fine-tuning and report performance on the test set of GSM8k. When only considering a single task, we choose GSM8k as the representative task as it is commonly used as a single training and test task by other papers.Code Generation.We use data that instructs the model to write SQL queries given some context(SQL-create-context)  and report the ROUGE-1 F1 scoreon the test set.Summarization.We use data from Samsumand report the ROUGE-1 F1 score on the test set.Safety.We define safety as a latent capability generated by instruction tuning.
A recent concern in AI policy is that, while frontier models such as GPT-3.5/GPT-4 are aligned, they can also be fine-tuned and this presents an opportunity to misalign them. Recently,show that by fine-tuning GPT-3.5 on justharmful examples for a few epochs, they can ask it to answer harmful queries that it ordinarily would refuse, andshow the same for GPT-4.show that this can be done with LoRA rather than the fine-tuning method OpenAI are using in their fine-tuning API (presumably FFT) . This is more than an academic concern; SB-1047, recently passed in California, requires model developers providing access to frontier models to implement best effort mitigations to prevent users from misaligning those models. We evaluate the safety of our models on HEx-Phi, a dataset of 330 questions spanning multiple categories such as malware, fraud, etc. The safety score (higher is better)  is the percentage of queries from the test set where the model refuses to respond. Aligned models such as Llama-3-Instruct will scoreon this task, but because we are doing the alignment ourselves starting from a base model, our baseline Instruct model only getson this task. Given that we are primarily interested in measuring theforgettingof safety alignment, we do not see this as a major limitation.
body
Lottery Ticket Adaptation:Mitigating Destructive Interference in LLMs
4Experimental Setup
2406.16797
5Experimental Results5.1LoTA Design ChoicesBefore comparing LoTA to other baselines, we first evaluate the impact of varying the two hyperparameters in our method: the sparsity level and how many iterations we calibrate the mask for.Sparsity.We vary the sparsity parameter in LoTA inTable1. Multiple sparsity thresholds work well; a small amount of sparsity (i.e.,)  seems to have a negative impact on the model, but this is within the error bars and may just be variance. Attempting to achievesparsity from the task vector of an FFT model does not yield good results. Instead, we can first obtain a mask withsparsity, then train a LoTA model with this mask, and then use the task vector from the-sparse LoTA model to calibrate the mask for our-sparsity model, which we denote as.
Composing multiple steps of calibration trades off performance in the high-compression setting with compute overhead.Table 1:The impact of varying the sparsity ratio on the performance of LoTA.denotes the model calibrated with iterative LoTA.SparsityPerformanceCompute Overhead.Arguably, the main reason why PEFT methods such as LoRA are commonly used is because they reduce the memory consumption in the backward pass, thus enabling the use of a larger pass.recently critically analyzed this and found that to fit tasks such as math and code generation, LoRA needs to use high ranks, which in turn reduces the speedup to at most. We nevertheless acknowledge that any and all PEFT methods will be faster than LoTA duringtrainingbecause LoTA requires an initial pass over the dataset to calibrate the sparsity mask.Table 2:LoTA performance degrades gracefully as a smaller fraction of the data is used on the most challenging task (instruction following) . 0 data usage corresponds to just creating a random mask.Data Used0Performance Drop (from) InTable2, we now consider how the performance drops if we calibrate the mask for a fraction of the overall dataset, including the baseline where we use a random mask which corresponds todata used. LoTA can be efficiently calibrated, and even training on a small fraction of the dataset can provideof the performance of the fully calibrated mask.
If we take just a single iteration to calibrate the mask for LoTA, we can obtainon GSM8k, higher than any result LoRA can obtain.
Furthermore, instruction tuning datasets are generally quite small (we completed all experiments in a few hours on just a single GPU) . Given that the mask can be efficiently calibrated, transferred from other datasets, or in the worst case, calibrated on the entire dataset in a few hours, we do not anticipate that the compute overhead of LoTA will be a major limitation of the method.Memory Overhead.LoTA has a minor overhead when compared to FFT, because a bit-mask must be loaded for each parameter group. We provide an efficient implementation for this in our open-source code. repository. We can finetune Llama-3-8B with LoTA on a single 80GB A100, so we do not anticipate this minor memory overhead being a major limitation of our method.We now evaluate LoTA and the baseline methods, FFT and LoRA, across all three paradigms of multi-task adaptation. In all experiments, we use LoTA withsparsity where the mask is calibrated by training fora single epoch(is one epoch)  on the adaptation dataset. We choosesparsity because this is a nontrivial sparsity level that we find achieves good performance across the range of tasks we consider. We ablate the level of sparsity and amount of calibration data inSection5.1.5.2Adapting to a Single TaskWe first consider the simplest setting, in which we fit an adapter to each dataset of interest starting from a pre-trained base model, i.e., one adapter per task. InTable3, we find that LoTA outperforms LoRA and performs similarly to FFT. Although LoRA is able to achieve similar performance to LoTA on the easier tasks, such as SQL, Samsum, there is a clear gap in performance on the more challenging tasks, such as Instruction Following and GSM8k. LoTA consistently recovers the performance of FFT.Table 3:Performance comparison of Full Finetuning (FFT) , LoRA and LoTA on single-task datasets for 3 epochs. We report the winrate on instruction following, the accuracy of exact match on the reasoning and math tasks, and the ROUGE-1 score on the SQL generation and summarization tasks. LoTA outperforms LoRA on the challenging tasks of instruction following, reasoning, and math, obtaining comparable performance to FFT.bold: best method,underline: second best method.ModelMethodInstruction FollowingReasoningGSM8kSQLSummarizationMistralFFTLoRALoTA (ours) Llama 3FFT84.8LoRALoTA (ours) High-Rank Adaptation Beats Low-Rank Adaptation.LoRA has a heavy regularizing effect on training. On the commonsense reasoning task, because the base model can get nontrivial zero-shot performance, LoRA actually improves the performance over that of FFT for Mistral. Note here that we do a grid search over learning rate and rank for LoRA, and the best performance is at. However, sparsity also has a regularizing effect, and LoTA is even more successful on the reasoning task when using the same learning rate as we searched for the base model () . On GSM8k, the regularization hurts performance significantly;report a similar-sized gap between LoRA and FFT on this dataset when training LLama-2-7B. If LoRA is underfitting the data, it may be better suited for settings where we only train for a single epoch, or on smaller datasets. InTable4, we make a side-by-side comparison of LoRA and LoTA when training for a single epoch and find that LoTA outperforms LoRA significantly across all tasks; in fact, the difference is even more pronounced after a single epoch.Table 4:Performance comparison on single-task datasets for 1 epoch.bold: best methodMethodModelArcGSM8kSQLSummarizationLoRAMistralLoTA (ours) MistralStoring LoTA Adapters Efficiently.Although FFT generally performs better than PEFT methods, it is typically infeasible to store a full copy of the model weights for each task. Practitioners, therefore, consider a tradeoff between memory and adaptation performance when loading task-specific adapters. We now discuss the memory consumption of LoTA and LoRA.
Storing the sparse task vector from LoTA requiresbits per parameter;from the parameter, andfor the delta encoding of the sparse indices. If we use-sparse LoTA, our task vector is compressed; if we use-sparse LoTA, our task vector is compressed. The memory-utility tradeoff between LoTA and LoRA can be quantified in terms of each method’s performance at a given level of compression, which translates into the sparsity level for LoTA and the rankfor LoRA. Comparing the compression-utility tradeoff between LoRA and LoTA is challenging because the size of the saved LoRA adapter is determined by the rank parameter, and more is not always better (which is why we have to tunefor every task for LoRA) .
As a single point of comparison, we can look at the performance on instruction following, where-sparse LoTA ()  outperforms LoRA for any rank (best performance ofachieved at, increasing rank reduces performance) , and they both achieve a similar compression factor.5.3Sequential TrainingDuring sequential training, a model is first adapted to one capability (Task A)  and then to another capability (Task B)  (such as when creating a specialized model for math)  or a set of capabilities (the most common setting for open-sourced fine-tunes of frontier models) . The main challenges we seek to mitigate are (1)  catastrophic forgetting of Task A and (2)  the inability to adapt to Task B.
We set instruction following as Task A in all settings because this maps to the enterprise adaptation setting, where user queries need to be answered with a combination of instruction following and domain-specific knowledge. In particular, OpenAI offers a fine-tuning API for their aligned models (GPT-3.5 and GPT-4) .Table 5:Sequential learning first on Task A (Instruction Following)  then on Task B (varied) . Fine-tuning on Tasks A and B is performed using both FFT and LoTA. The utility of each task is computed after fine-tuning on Task B is completed. Note that there is no utility when we fine-tune on harmful data to evaluate the catastrophic forgetting of Safety, and the baseline is the safety score of the Instruct model. FT=Fine-tuning. We reproduce the baseline of doing FFT on each method independently as reported inTable3for convenience; note that on the reasoning task LoTA outperforms FFT.bold: best method,underlinesecond-best method; we do not report second-best when only two methods are presented. All results are with Mistral. We reuse the same mask for LoTTO calibrated on GSM8k for MathInstruct, Reasoning, GSM8k+Arc+SQL and Safety.Task BMethod on Task AMethod on Task BUtility of Task A (Drop) Utility of Task B (Drop) Instruction FollowingBaseline-19.0 (-) -GSM8k-Baseline-59.8 (-) FFTFFT15.2 (3.8) 58.3 (1.5) LoTA (ours) FFT17.7 (1.3) 58.7 (1.1) FFTLoTA (ours) 15.9 (3.1) 54.2 (5.6) LoTA (ours) LoTTO (ours) 17.8 (1.2) 59.1 (0.7) FFTLoRA14.1 (4.2) 55.5 (4.9) FFTFFT (Mixed) 16.3 (2.7) 55.5 (4.3) MathInstruct-Baseline-56.7 (-) FFTFFTLoTA (ours) LoTA (ours) Reasoning-Baseline-83.5 (-) FFTFFT82.3 (1.2) LoTA (ours) LoTTO (ours) 83.7(-) GSM8k+Arc+SQL-Baseline-FFTFFTLoTA (ours) FFT75.4 (1.6) LoTA (ours) LoTTO (ours) SafetyBaseline-93.1 (-) -FFTFFT-LoTA (ours) LoTTO (ours) -Mitigating Catastrophic Forgetting While Enabling Adaptation to Downstream Datasets.InTable5we consider a range of method combinations for a simplified setting where we seek to adapt an Instruct model to Math data without catastrophic forgetting. We will go row-by-row through the table and analyze each set of results. Even when training on just a single, relatively small dataset, FFT in both phases suffers a significant drop in winrate. An easy way to mitigate this is to simply train the initial Instruct model with LoTA. Following this, FFT on GSM8k does not significantly reduce winrate. However, it does present a potentially unwelcome tradeoff in task accuracy. For this, we turn to LoTTO, which achieves the best performance across both tasks.Does LoRA Really Forget Less, or Does It Just Learn Less?Recent work evaluates the performance of pre-trained models before and after fine-tuning on domain-specific tasks with LoRAand concludes that LoRA is less prone to catastrophic forgetting than FFT. InTable5we find that adapting the FFT Instruct model with LoRA does not lead to less degradation in winrate than adapting the FFT Instruct model with FFT, but it does lead to worse performance on the downstream task. Doing LoRA and then LoRA, or LoRA and then FFT, also does not match LoTA’s ability to prevent forgetting while enabling adaptation to downstream data.How Much Does Data Reuse Help?The simplest and arguably most performant method from prior work that we found for mitigating catastrophic forgetting is simply to mix in some in-distribution data from Task A. This is the line marked with “FFT (Mixed) ”, and it does mitigate forgetting on Task A, but at the cost of performance on Task B. We ablate the amount of data from Task A to be used betweenof the dataset size of the data from Task B, and this is the best result in terms of mitigating forgetting on Task A. As we mix in less and less data from Task A, this method approaches just doing FFT sequentially in performance, so we omit those results for brevity.Adapting Sequentially to Multiple Tasks with LoTTO.We now consider the more challenging setting when we need to adapt tomultipletasks without catastrophic forgetting while still obtaining good performance on those tasks.
InTable5we adapt an Instruct model to a mix of reasoning, math, and SQL data and find a surprising result; the FFT Instruct model collapses almost completely to a winrate of less than half a percent. As we showed inTable5this can be mitigated by mixing in more instruction following data, albeit at a cost. The LoTA Instruct model still degrades in performance ()  but nowhere near as much. In the line marked “LoTTO”, we instead adapt the LoTA Instruct model to the downstream tasks using the mask calibrated from GSM8k with LoTTO. Applying LoTTO to make the downstream adaptation be mutually sparse with the initial LoTA Instruct model increases performance significantly on instruction following and math. Performance on Arc and SQL suffers somewhat because our mask does not consider those tasks, but this means that our mask is much cheaper to calibrate and is, in fact, generalizable not only across tasks within a domain (as shown inTable5)  but alsoacrossdomains.Mitigating Catastrophic Forgetting of Safety Alignment.In the “Safety” row ofTable5, we consider fine-tuning the Mistral Instruct model we trained ourselves on theharmful instructions from. The baseline model gets a score ofand training with FFT quickly degrades safety. Adapting the LoTA Instruct with LoTTO (again, with the LoTTO mask calibrated from GSM8k)  mitigates this safety drop significantly, even though our LoTTO mask was calibrated on an extremely different dataset. Therefore, a potential mitigation would be for an entity providing a fine-tuning API such as OpenAI to do the safety training with LoTA, calibrate the LoTTO mask on a utility dataset, and then do fine-tuning on their client’s dataset with LoTTO. We do not intend to present our method as an active defense against fine-tuning attacks; although finetuning with LoTTO improves safety, given sufficient data and access to the model weights, any attacker can of course undo safety tuning entirely. However, catastrophic forgetting of safety alignment is an important problem with real-world applications, and we find it compelling that our method can mitigate this.5.4LoTA for Model MergingWe now consider the setting of model merging, where we train models on disjoint datasets fully in parallel and then merge together the task vectors with the goal of producing a model with good performance on multiple tasks. Prior work in model merging mostly considers merging similar datasets, such as the commonsense reasoning datasets, but it is relatively easy to merge models when the datasets are similar and becomes increasingly hard as the datasets become more heterogeneous due to the gradient mismatch. We consider merging models trained on heterogeneous datasets.Merging Models.We use TIES-Mergingto merge models trained on heterogeneous tasks. TIES performs post-hoc sparsification on each task vector and requires a 2-D hyperparameter search for this quantity, which we perform for the merge of FFT models. Naturally, we could optimize the performance of Task A by fully sparsifying Task B, and vice versa; we report the result that achieves good performance on Task B while maintaining some performance on Task A, and report the full range of hyperparameters inAppendixA. LoTA is inherently sparse, so when we merge a LoTA model with an FFT model we do not need to perform hyperparameter search on the LoTA model, and we use the same level of sparsity for the FFT model that we obtained when merging together two FFT models. When merging two LoTA models together, no hyperparameter search is required at all as both models are inherently sparse. We could in theory sparsify the LoTA models beyond their existing levels with post-hoc sparsification, but we do not tune this hyperparameter.Challenge of Overlapping Sparsity in Model Merging.In the sequential training paradigm, we exploited the fact that masks for different tasks have a significant overlap in order to generalize our LoTTO mask calibrated on GSM8k to provide robustness to forgetting across a range of other tasks. However, this same phenomenon of overlapping sparsity presents a challenge in the model merging setting.
The challenge is that because the merging is parallel, we cannot use LoTTO to calibrate the masks to be disjointly sparse as we did in the sequential training setting. One alternative is to enforce random disjoint masks for each task, corresponding to “Data Used” inTable2.Table 6:Model merging using TIES of Task A, Instruction Following, and Task B, (varied) . Fine-tuning on Tasks A and B is performed using both full fine-tuning (FFT)  and LoTA. The utility of each task is computed after merging the two task vectors.bold: best result,underline: second best result. We reproduce the baseline for each task on FFT fromTable3for convenience; note again that for some tasks the LoTA baseline outperforms the FFT baseline. All results are with Mistral.Task BMethod on Task AMethod on Task BUtility of Task A (Drop) Utility of Task B (Drop) Instruction FollowingBaseline-19.0 (-) -GSM8k-Baseline-59.8 (-) FFTFFTLoTA (ours) FFTFFTLoTA (ours) LoTA (ours) LoTA (ours) Samsum-Baseline-52.0 (-) FFTFFTLoTA (ours) FFTFFTLoTA (ours) LoTA (ours) LoTA (ours) GSM8k+Samsum-Baseline-55.9 (-) FFTFFTLoTA (ours) FFTFFTLoTA (ours) LoTA (ours) LoTA (ours) InTable6we merge together models trained on GSM8k and Samsum with the Mistral model that we trained for instruction following. We consider the full combination of merging FFT and LoTA models. The merge of two FFT models performs poorly in all settings on both tasks, indicating that post-hoc sparsification does not perform well for heterogeneous tasks, which is in line with recent model merging theory. The merges that contain LoTA models have better performance across all tasks, but there is no combination that is pareto-optimal across all tasks.
body
Lottery Ticket Adaptation:Mitigating Destructive Interference in LLMs
5Experimental Results
2406.16797
6Related WorkModel Pruning & Quantization.Model pruning and quantization have been receiving increased attention for efficient storage and/or inference of large models. While most of the existing methods pruneor quantizethe model weight directly, some focus specifically on compressing thetask vectorsthrough post-training sparsification or quantization, assuming that the base model is worth the storage cost since it is being used frequently for many tasks. Our proposed PEFT method, LoTA, builds on this observation that the task vectors are highly compressible through sparsification and imposes this sparsity constraint at the beginning of fine-tuning to trainsparse task vectors.Lottery Ticket Hypothesis.Motivated by the success of pruning methods at extreme sparsity ratios,proposed the lottery ticket hypothesis (LTH) , claiming the existence of sparse subnetworks (or lottery tickets)  that could be trained from scratch to a performance comparably to training the dense model from scratch. While this could potentially provide a way to train models sparsely more efficiently rather than training them densely and pruning them later, finding the lottery tickets, i.e., the sparsity masks, is costly. Initially,proposed first training the models densely and then extracting the sparsity mask based on the magnitude of the trained dense model’s weights. Later, a number of more efficient methods were proposed to find the sparsity masks more efficiently, earlier in the dense training stage. Our work shows that LTH works successfully for fine-tuning LLMs as well–giving us a sparse adaptation tool, LoTA. We extensively study the tradeoff between the cost of finding the sparsity masks and the performance of the sparsely fine-tuned model. Unlike other studies on LTH for LLM adaptation, our main focus and motivation is to mitigate destructive interference in multi-task adaptation.Parameter-Efficient Fine-Tuning (PEFT) .Many practitioners fine-tune already pre-trained LLMs with less data and compute instead of training them from scratch. While this reduces the cost of LLM training significantly, fine-tuning each and every parameter of these large models for each (or a few)  task is still very costly. This has led to a number of parameter-efficient fine-tuning (PEFT)  methods reducing the number of trainable parameters during fine-tuning. Among different PEFT methods, low-rank adaptation (LoRA) and its variantshave shown similar performance to full fine-tuning in many tasks while reducing the number of trainable parameters through low-rank approximation to model updates during fine-tuning. Our PEFT method, LoTA, while reducing the number of trainable parameters significantly via sparsity, has various other benefits in different applications, such as avoiding catastrophic forgetting (of especially safety alignment) , enabling fine-tuning on new tasks more successfully, model merging using sparse task vectors, unlearning, and communication-efficient federated learning (FL) . We demonstrate that full fine-tuning and the existing PEFT methods fall short in these applications and significantly underperform LoTA.Catastrophic Forgetting.When LLMs go through sequential (or continual)  multitask learning, i.e., fine-tuned on different tasks sequentially, they often suffer from performance loss on earlier tasks–known as catastrophic forgetting. To mitigate this, a number of data-centric and architectural solutions have been proposed for language and other domains. Replay-based methodsadd a portion of the previously learned data during fine-tuning on a new task, which raises privacy concerns as it requires constant access to previously learned data. Regularization-based approachestend to have poor adaptability to specific tasks. An architecture-based approach, “progressive prompts”, sequentially concatenates soft prompts as they are being learned for each task–showing some resistance against forgetting. However, they require access to task identifiers at inference for each task, which is not always feasible. Other architecture-based approaches add additional modules to learn task-specific abilities–requiring customized deployment due to architecture change. Closest to our work,updates a randomly selected subset of the parameters at each iteration of fine-tuning to preserve the earlier tasks in the not-updated parameters of that iteration. Despite similarities, our work LoTA (1)  uses a fixed sparsity mask throughout fine-tuning instead of a new mask at every iteration, which yields sparse task vectors that are useful for other applications such as model merging and communication-efficient FL, and (2)  finds data-dependent masks rather than the randomly selected masks in. Furthermore, unlike, LoTA not only preserves the earlier tasks on frozen parameters but also constraints the new tasks on a highly sparse subnetwork–providing resistance to catastrophic forgetting even when malicious users attempt to overwrite the earlier tasks via FFT.Model Merging.Merging multiple task-specific models into a single model with multitask abilitieshas been an appealing alternative to sequential multitask learning, which suffers from catastrophic forgetting and could be inefficient, especially if task vectors are already available. The existing model merging methods include averaging weights of task-specific models, task arithmetic through combining task vectors, weighted aggregation of parameters, combining task vectors after some post-processing such as trimming low-magnitude deltasor sparsifying the deltas. Our method, LoTA, directly learnssparse task vectors, obviating the need to post-process the task vectors, and outperforms existing model merging methods. Most importantly, LoTA enables merging task vectors trained on heterogeneous datasets, while the other model merging methods are often limited to similar datasets. This advancement is an important step towards scalable FLwith LLMs as it enables mergingsparse task vectors, which brings communication efficiency, trained over heterogeneous datasets (of each edge device) .We note that we test model merging with LoTA specifically for highly dissimilar datasets to show its compatibility with FL, which considers edge devices with heterogeneous datasets. When used in FL, LoTA can reduce communication and memory costs significantly, which is a main bottleneck when scaling FL to large models. The successful use of LoTA for model merging and arithmetic further shows its promise for unlearningas well.
body
Lottery Ticket Adaptation:Mitigating Destructive Interference in LLMs
6Related Work
2406.16797
7DiscussionWe propose Lottery Ticket Adaptation (LoTA) , a sparse alignment framework that fine-tunes only a sparse subnetwork of the base model, leaving the rest of the parameters frozen. LoTA successfully mitigates destructive interference (a problem with existing fine-tuning methods including full fine-tuning and low-rank adaptation (LoRA) )  in many multi-task adaptation paradigms, prevents catastrophic forgetting of earlier tasks, including safety, and allows for successful model merging of even dramatically different tasks.Limitations.As mentioned in Section5.1, LoTA does not provide the compute efficiency of LoRA. If the adapter needs to be compressed by more than, LoTA may not provide sufficient compression. We evaluate on instruction following, reasoning, math, SQL generation, and summarization, yet even more tasks exist such as Python code generation, classification, or long-context question answering. We compare LoTA to baselines (LoRA, TIES)  but other PEFT and merging methods exist. In a future revision of this paper, we plan to provide comparisons to a broader range of PEFTand mergingmethods.Broader Impact.LoTA emerges as a simple and efficient adaptation method to prevent destructive interference. We hope LoTA can enable compression of task adapters without losing performance, mitigate catastrophic forgetting (particularly to improve the durability of safety alignment in future model releases) , and provide performant model merging across heterogeneous tasks.Acknowledgements.This work was funded in part by the OpenAI Superalignment Grant and the Google PhD Fellowship.
conclusion
Lottery Ticket Adaptation:Mitigating Destructive Interference in LLMs
7Discussion